ON BOUNDS OF SOME DYNAMIC INFORMATION DIVERGENCE MEASURES

STATISTICA, anno LXXII, n., 0 ON BOUNDS OF SOME DYNAMIC INFORMATION DIVERGENCE MEASURES. INTRODUCTION Discriminaion and inaccuracy measures play a key role in informaion heory, reliabiliy and oher relaed fields. There are many discriminaion measures (relaive enropy) available in lieraure and are used as a measure of he disance beeen o disribuions or funcions. Renyi s informaion divergence of order is one such popular discriminaion measures used by many researchers (see Asadi e al., 005a, 005b and references herein). Le X and Y be o absoluely coninuous non negaive random variables (rv s) ha describe he lifeimes of o iems. Denoe by f, F and F, he probabiliy densiy funcion (pdf), cumulaive disribuion funcion (cdf) and survival funcion (sf) of X respecively and g, G and G, he corresponding funcions of Y. Also, le h f / F and hy g/ G be he hazard (failure) raes and X f / F and Y g / G be he reversed hazard raes of X and Y respecively. Then Renyi s informaion divergence of order beeen o disribuions f and g is defined by ( ) f( X) IXY, ln f ( x) g ( x) dx lne 0 f gx ( ) for such ha 0. Hoever, in many applied problems viz., reliabiliy, survival analysis, economics, business, acuary ec. one has informaion only abou he curren age of he sysems, and hus are dynamic. Then he discriminaion informaion funcion beeen o residual lifeime disribuions based on Renyi s informaion divergence of order is given by ( ) XY, () ln ( ) f ( x) g ( x) I dx () F () G () X

4 for such ha 0. Noe ha IX, Y() IX, Y, here X ( X X ) and Y ( Y Y ) are residual lifeimes associaed o X and Y. Anoher se of ineres ha leads o he dynamic informaion measures is he pas lifeime of he individual. In he conex of pas lifeimes, (Asadi e al., 005b) defined Renyi s discriminaion implied by F and G beeen he pas lives ( X X ) and ( Y Y ) as ( ) XY, () ln 0 ( ) f ( x) g ( x) I dx () F () G () for such ha 0. Given ha a ime, o iems have been found o be failing, equaion () measures he dispariy beeen heir pas lives. Recenly, he inaccuracy measure due o (Kerridge, 96) is also idely used as a useful ool o measure he inaccuracy beeen o disribuions f and g. I is given by K f( x)ln g( x) dx. XY, I can be expressed as 0 K, D( X, Y) H( X) XY 0 here DXY (, ) f( x)ln( f( x)/ gx ( )) dx is he Kullback-Leibler (KL) divergence beeen X and Y and H( X) f( x)ln f( x) dx is Shannon measure 0 of informaion of X. (Taneja e al., 009) inroduced a dynamic version of Kerridge measure, given by f( x) g( x) KXY, () ln dx (3) F () G () Noe ha KX, Y() KX, Y. Clearly, hen X Y, equaion (3) becomes he popular dynamic measure of uncerainy (residual enropy) due o (Ebrahimi, 996). A similar expression for he inaciviy imes is available in (Vikas Kumar e al., 0) and given by f( x) g( x) KXY, () ln dx 0 F () G () (4) For a ide variey of research for he sudy of hese dynamic informaion measures, e refer o (Ebrahimi and Kirmani, 996a, 996b; Di Crescenzo and Lon-

On bounds of some dynamic informaion divergence measures 5 gobardi, 00, 004; Asadi e al., 005a, 005b; Taneja e al., 009; Vikas Kumar e al., 0) and references herein. The concep of eighed disribuions as inroduced by (Rao, 965) in connecion ih modeling saisical daa and in siuaions here he usual pracice of employing sandard disribuions for he purpose as no found appropriae. If he pdf of X is f and (.) is a non-negaive funcion saisfying EX ( ( )), hen he pdf f, df F sf F of he corresponding eighed rv X are respecively ( ) ( ) ( ) x f x f x, ( ( ) ) ( ) Ex X F x F ( x ) and ( ( ) ) ( ) Ex X F x F ( x ). An imporan disribuion hich arises as a special case of eighed disribuions is he equilibrium models, obained hen (.) F/ f. I is also arises naurally in reneal heory as he disribuion of he backard or forard recurrence ime in he limiing case. Associaed ih X, a equilibrium rv X E can be defined as ih pdf fe F/, ih sf and failure rae funcion are given respecively by FE( x) r( x) F( x)/ and he( x) / r( x), here E( X ) and rx ( ) EX ( x X x). For various applicaions and recen orks on eighed and equilibrium disribuions, e refer o (Gupa and Kirmani, 990; Navarro e al., 00; Di Crescenzo and Longobardi, 006; Gupa, 007; Maya and Sunoj, 008; Navarro e al., 0; Sunoj and Sreejih, press). Alhough a ide variey of research has been carried ou for sudying hese dynamic informaion measures () o (4) in he conex of modeling and analysis, hoever, very lile has been sudied o obain is bounds ih regard o some sochasic ordering. Accordingly in he presen paper, e obain cerain bounds/inequaliies on hese dynamic discriminaion measures () o (4) for rv s X and Y and subsequenly beeen X and X, using likelihood ordering. More imporanly, some close relaionships beeen hese dynamic discriminaion measures, reliabiliy measures and residual informaion measures are obained in erms of bounds.. RENYI S DISCRIMINATION MEASURE OF ORDER Renyi s discriminaion measure for he residual lives of he original and eighed rv s is given by ( ) f ( x) f XX, () ln ( ) ( x) I dx, (5) F () F () for such ha 0, and ha for pas lives is given by ( ) f ( x) f XX, () ln 0 ( ) ( x) I dx. (6) F () F ()

6 for such ha 0. Equaions (5) and (6) measures he discrepancy beeen he residual (pas) lives of original rv X and eighed rv X. More imporanly, IXX, () may be a useful ool for measuring ho far he rue densiy is disan from a eighed densiy. On he oher hand, hen he original and eighed densiy funcions are equal hen, IXX, () 0 ae.. Remark. Equaions (5) and (6) may be useful in he deerminaion of a eigh funcion and herefore for he selecions of a suiable eigh funcion in an observed mechanism, e can choose a eigh funcion for hich (5) or (6) are small. Moreover, (5) and (6) are asymmeric in f and f, herefore, for reversing he roles of f and f in (5), say IX, () X and equae ih (5) for a symmeric measure implies he eigh funcion is uniy, i.e., hen f = f (see Maya and Sunoj, 008). In many insances in applicaions, sochasic orders and inequaliies are very useful for he comparison of o disribuions. In he univariae case, several noions of sochasic orders are popular in lieraure. I is ell knon ha likelihood raio order is more imporan han he oher orders such as usual sochasic order or he hazard rae order (see Shaked and Shanhikumar, 007), as i implies he oher o. Accordingly, in he folloing heorems, e use he likelihood raio ordering o obain some bounds and inequaliies on Renyi s discriminaion measure of order beeen X and Y and subsequenly beeen X and X. We say X is said o be smaller han Y in likelihood raio ( X LR Y ) if f(x)/g(x) is decreasing in x over he union of he suppors of X and Y. The resuls are quie similar o KL informaion divergence given in (Di Crescenzo and Longobardi, 004). In a similar ay for he Renyi s informaion divergence of order, likelihood raio ordering provides some simple upper or loer bounds hich are funcions of imporan reliabiliy measures and/or Shannon informaion measure. The folloing heorem provides a simple upper bound for Renyi informaion of order ih bounds are funcions of hazard raes of X and Y. Theorem. If X Y LR, hen IXY, hx () () ( ) ln hy ( ) if (0 ). Proof. Since X Y, LR f ( x ) g( x ) is decreasing in x, i.e., f ( x) f( ) for all x >. g( x) g( ) ( ) f ( x ) g ( x ) f ( x ) g ( x I ) XY, () ln dx ln dx ( ) ( ) F () G () g ( x) F () G () hen,

On bounds of some dynamic informaion divergence measures 7 f () g( x) IXY, () ( ) ln dx ( ), for (0 ). g () F () G () h () () ln X h ln X. hy () hy ( ) Corollary. If X LR X, hen I, (0 ). XX E( X ( ) X ) () ( ) ln ( ) if Proof: Using he relaionship for hazard rae beeen X and X, e have hx () E( ( X) X ), from hich he corollary follos. h () () X Example. The Pareo disribuion has played a very imporan role in he invesigaion of ciy populaion, occurrence of naural resources, insurance risk and business failures and has been a useful model in many socio economic sudies (see Abdul Sahar e al., 005). Accordingly, e consider Pareo I disribuion ih c c pdf f( x) ck x, x k, k0, c o illusrae he above heorem. Using he eigh funcion x ( ) x, e have X LR X and X also follos a Pareo disribuion. c ( c ) c c IXX, ( ) ln ln ln. ( c ) ( c ) c c c E( X ( ) X ) So IXX, () ( ) ln ln according as c ( ) (0 ). Corollary. If X LR XE, hen IXX, E () ( ) ln[ r()] (0 ). if hx () Proof: Since r( ) for he equilibrium rv, he corollary follos. hx () E Even if he pas lifeime informaion measures appears o be a dual of is residual version, hoever, (Di Crescenzo and Longobardi, 004) has shon he imporance of pas lifeime discriminaion measures in comparison ih residual lifeime and hus a separae sudy of hese discriminaion measures for pas lifeime is quie orhhile. Accordingly, in he res of paper, e include he bounds

8 for hese discriminaion measures for he pas lifeimes as ell. The folloing heorem provides a loer (upper) bound for I, () using likelihood ordering. XY X () Theorem. If X LR Y, hen IXY, () ( ) ln Y ( ) No e exend he above heorem o eighed models. if (0 ). Corollary 3. If X LR X, hen I, (0 ). XX E( X ( ) X ) () ( ) ln ( ) if Example. Suppose X is a finie range rv ih pdf f ( x) cx,0x, c 0, and aking x ( ) x f( x) c ( 0) e have x is decreasing in x (i.e., f( x) c X X ). I is easy o sho ha I LR XX, c c c ( ) ln ln ( ) ln c c c E( X ( ) X ) ln ( ) according as (0 ), provided c. In he sudy of relaive enropies, i is quie useful if e find some close relaionships beeen is differen measures and oher imporan reliabiliy/informaion measures. Therefore, in he folloing heorem e derive a loer bounds for IXY, (), hich are funcions of boh hazard rae and Shannon informaion measure. Theorem 3. If g(x) is decreasing in x hen I, () ln h () I (), here IX () XY Y X c f ( x) ln dx, he residual Renyi s enropy funcion. F () Corollary 4. If f( x ) is decreasing in x, hen h () X () IXX, () ln () IX,. EX ( ( ) X )

On bounds of some dynamic informaion divergence measures 9 Example 3. Applying he same pdf and eigh funcion used in example 3, e can easily illusrae corollary 4. The analogous resuls are sraighforard for he pas life imes, he saemens are as follos: Theorem 4. If g(x) is increasing in x hen IXY, () ln Y() IX(),, 0 here f () ln ( x I ) X dx, he pas Renyi s enropy funcion. 0 F () Corollary 5. If f( x ) is increasing in x, hen () X () IXX, () ln () IX,, 0. EX ( ( ) X ) Example 4. I is easy o sho ha for he poer funcion rv ih pdf c c f ( x) cx,0x, c and aking x ( ) x, e have f ( x) ( c ) x increasing in x and hence corollary 5 follos. In he folloing heorems, e esablish an upper (loer) bound for IXY, () for more han o rv s. Theorem 5. Le X, X and Y be 3 non negaive absoluely coninuous rv s ih densiies f, f and g, sf s F, F and G and hazard raes h X, h X and h Y re- hx () specively. If X LR X, hen IX, Y() ( ) ln IX, Y() hx ( ) (0 ). if Example 5. Le X and X be o independen exponenial rv s ih parameers 0 and 0 respecively such ha, hen f( x) exp[ ( ) x ] is decreasing in x. Le Y min( X, X), hen f ( x) I X, Y () ln ln ln ln

30 ( ) ln ln h () ( ) X ln IX, Y( ) hx according as (0 ), provided i j i 0, i j, i, j,. Theorem 6. Le X, X and Y be 3 non negaive absoluely coninuous rv s ih densiies f, f and g, disribuion funcions F, F and G and reversed hazard raes X, X and Y respecively. If X LR X, hen X () if (0 ). X ( ) I () ( ) ln I () X, Y X, Y Example 6. Le X and X be o independen Poer funcion rv s ih densiies given by c f( x) cx ;0x, c 0 and f( x) cx ;0x, c 0 c f( x) c cc respecively such ha c c, so x is decreasing in x. Leing f ( x) c Y max( X, X ), hen i is easy o sho ha heorem 6 follos. Theorem 7. Le X, Y and Y be 3 non negaive absoluely coninuous rv s ih pdf s f, g and g, sf s F, G and G and hazard raes h X, h Y and h Y respec- hy () ively. If Y LR Y, hen IXY, () ln I XY, () for, 0. hy () Example 7. Le Y and Y be o independen Pareo I rv s ih densiies given c c by c c g( x) ck x ; x k 0, c 0 and g( x) ck x ; x k 0, c 0 c g( x) ck ( c c) respecively such ha c c, hen c x is decreasing in x. Consider X min( Y, Y), hen he heorem g( x) ck follos. Theorem 8. Le X, Y and Y be 3 non negaive absoluely coninuous rv s ih densiies f, g and g, disribuion funcions F, G and G and reversed hazard

On bounds of some dynamic informaion divergence measures 3 raes X, Y and () () XY I () ln I () Y XY,, Y respecively. If Y LR Y, hen Y for, 0. Example 8. Le Y and Y be o independen poer funcion rv s ih densiies c given by c g( x) cx ;0x, c 0 and g( x) cx ;0x, c 0 respecively such ha c c, so x is decreasing in x. Using g( x) c cc g ( x) c X max( Y, Y ), hen e can illusrae he heorem. 3. DYNAMIC INACCURACY MEASURE In his secion e obain bounds similar o ha given in secion for he Kerridge inaccuracy measures. Le X and Y be he rv s defined in secion. Then he dynamic inaccuracy measure for residual and pas lives of he original and eighed disribuions are given by and f( x) ( x) f( x) KXX, () ln dx F () EX ( ( ) X F ) (), (7) f( x) ( x) f( x) KXX, () ln 0 dx F () EX ( ( ) X F ) (). (8) Remark. From he above definiion, i is easy o obain, KXX, E () ln() r. The folloing heorem gives a simple loer bound for Kerridge inaccuracy measures using likelihood ordering. Theorem 9. If g(x) is decreasing in x, hen K, () ln h (). Proof. Since g(x) is decreasing in x, e have g( x) g( ) for all x >. Then, gx ( ) g ( ) KXY, ( ) f( x)ln dx f( x)ln dx ln hy ( ) F () G () F () G () XY. Y

3 E( X ( ) X ) Corollary 6. If f( x ) is decreasing in x, hen KXX, () ln. h () X () Analogues resuls are obained for pas lifeimes in he folloing heorems. Theorem 0. If g(x) is increasing in x, hen K, () ln (). XY Y Corollary 7. If f( x ) is increasing in x, hen K, XX E( X ( ) X ) () ln. h () X () Example 0. Suppose X is a Uniform rv ih pdf f( x) ;0 x a, a 0. Taking he eigh funcion x ( ) x, ( ) x a f x is increasing in x. Then, a E( X ( ) X ) KXY, ( ) ln( /) ln( /) ln. h () X () In he folloing heorem e have a simple bound for Kerridge inaccuracy measure beeen X and X hich are funcions of hazard raes of he same rv s and residual enropy of X. Theorem. If he eigh funcion (x) is increasing in x, hen EX ( ( ) X ) f( x) KXX, () ln H () X, here HX () f( x)ln dx () F () F () is he residual enropy funcion. Proof. Since (x) is increasing in x, e have g( x) g( ) for all x >. No using equaion (7) e have () f( x) EX ( ( ) X ) KXX, ( ) f( x)ln dx ln H ( ) X F (). EX ( ( ) X F ) () () c Example. Le X be a Pareo I rv ih pdf f( x) ck x ; c, x k 0. Take he eigh funcion as x ( ) x, hich is an increasing funcion in x. Then c

On bounds of some dynamic informaion divergence measures 33 KXX, () ln c c c c c ln ln ln ln c c c c c c c E( X ( ) X ) ln () HX (). The folloing heorem is an analogous resul of heorem for pas lifeime. Theorem. If he eigh funcion (x) is decreasing in x, hen E( X ( ) X ) KXX, () ln HX () (), here f( x) HX () f( x)ln dx F () 0 F () is he pas enropy funcion. Example. Consider a finie range rv X ih densiy funcion given by c f ( x) cx ;0x, c. Le x ( ), a decreasing funcion in x, hen using x equaion (8) e have K XX, c c c ( ) ln ln ln c c c c c c c c EX ( ( ) X ) ln ln ln HX ( ) c c c ( ) Theorem 3. If X LR Y, hen XY, hx () K () HX() ln. hy () Proof. From he definiion (3), e have f( x) f( x) g( x) F( ) KXY, () ln dx F () F () f( x) G () f( x) g( ) F( ) HX () ln dx F () f() G () f( x) g( x) F( ) HX () ln dx F () f( x) G () g () F () HX () ln f () G() here he inequaliy is obained by using ha g(x)/f(x) is increasing. A similar saemen exiss for he pas lifeime.

34 X () Theorem 4. If X LR Y, hen KXY, () HX() ln. Y () Similar o heorems 5 o 8, in he folloing heorems e obain some bounds for Kerridge s inaccuracy for more han o rv s. Theorem 5. Le X, Y and Y be 3 non negaive absoluely coninuous rv s ih pdf s f, g and g, sf s F, G and G and hazard raes h X, h Y and h Y respecively. If Y LR Y, hen KXY, () K XY, () ln. hy () hy () Proof. From he definiion (3), e have f( x) g ( x) g ( x) G ( ) KXY, () ln dx K XY, () ln F () G() g( x) G() dx F () g( x) G() f( x) g ( x) G ( ) f( x) g () G () KXY, () ln dx F () g() G() hy () KXY, () ln. hy () Example 3. Le Y and Y be o independen Pareo II rv s ih pdf s c c g( x) ac( ax) ; x 0, a, c 0 and g( x) ac( ax) ; x 0, a, c 0 g( x) c ( cc) such ha c c, hen ( ax ) is decreasing in x. Le g ( x) c X min( Y, Y ), hen K XY, a c c a c c c ( ) ln ln ln ac c c c ac c c c c c a c hy () ln ln KXY, () ln. c ac c c hy () Nex e obain an analogous resul for he pas lifeime. Theorem 6. Le X, Y and Y be 3 non negaive rv s ih pdf s f, g and g, df s F, G and G and reversed hazard raes X, Y and Y respecively. If Y LR Y Y () hen KXY, () K XY, () ln. Y ()

On bounds of some dynamic informaion divergence measures 35 Example 4. Le Y and Y be independen finie range rv s ih pdf s given by c ( ) c g x c x ;0x, c 0 and g ( x) c x ;0x, c 0 such ha c g( x) c cc c, hen x is decreasing in x. Le X max( Y, Y), e ge g ( x) c K XY, c c c c c ( ) ln ln ln c c c c c c c c c c c Y () ln ln KXY, () ln. c c c c Y () CONCLUSIONS Bounds are derived for some imporan discriminaion measures viz. Renyi informaion divergence of order and Kerridge s inaccuracy for measuring disance beeen a rue disribuion and an arbirary disribuion, using likelihood raio ordering. Furher, likelihood ordering provides some simple upper or loer bounds o hese discriminaion measures, here he bounds are funcions of cerain imporan reliabiliy measures such as hazard (reversed hazard) raes and residual (pas) Shannon informaion measure(s). These bounds are also exended o he eighed case by assuming arbirary disribuion as a eighed one, useful for comparison of observed and original disribuions. Examples are also given for hese bounds for heoreical validaion. Deparmen of Saisics Cochin Universiy of Science and Technology Cochin 68 0, Kerala, India S.M. SUNOJ M.N. LINU ACKNOWLEDGEMENTS Auhors are hankful o he referees for heir commens and suggesions ha helped a subsanial improvemen of he earlier version of he paper. REFERENCES A. DI CRESCENZO, M. LONGOBARDI, (00), Enropy based measures of uncerainy in pas life ime disribuions, Journal of Applied Probabiliy, 39, pp. 434-440. A. DI CRESCENZO, M. LONGOBARDI, (004), A measure of discriminaion beeen pas lifeime disribuions, Saisics and Probabiliy Leers, 67, pp. 73-8. A. DI CRESCENZO, M. LONGOBARDI, (006), On eighed residual and pas enropies, Scieniae Mahemaicae Japonicae, 64(), pp. 55-66.

36 C.R. RAO, (965), On discree disribuions arising ou of mehods of ascerainmen, Sankhya, A, 7, pp. 3-34. D.R. KERRIDGE, (96), Inaccuracy and inference, Journal of Royal Saisical Sociey, Ser. B, 3, pp. 84-94. E.I. ABDUL-SATHAR, E.S. JEEVANAND, K.R.M. MURALEEDHARAN NAIR, (005), Bayesian esimaion of Lorenz curve, Gini index and variance of logarihms in a Pareo disribuion, Saisica, anno LXV,, pp. 93-05. H.C. TANEJA, V. KUMAR, R. SRIVASTAVA, (009), A dynamic measure of inaccuracy beeen o residual lifeime disribuions, Inernaional Mahemaical Forum, 5, pp. 3-0. J. NAVARRO, Y. DEL AGUILA, J.M. RUIZ, (00), Characerizaions hrough reliabiliy measures from eighed disribuions, Saisical Papers, 4, pp. 395-40. J. NAVARRO, S. M. SUNOJ, M.N. LINU, (0), Characerizaions of bivariae models using dynamic Kullback-Leibler discriminaion measures, Saisics and Probabiliy Leers, 8, pp. 594-598. K. VIKAS, H.C. TANEJA, R. SRIVASTAVA, (00), A dynamic measure of inaccuracy beeen o pas lifeime disribuions, Merika, 74(), pp. -0. M. ASADI, N. EBRAHIMI, E. SOOFI, (005a), Dynamic generalized informaion measures, Saisics and Probabiliy Leers, 7, pp. 89-98. M. ASADI, N. EBRAHIMI, G.H. HAMEDANI, E. SOOFI, (005b), Minimum dynamic discriminaion informaion models, Journal of Applied Probabiliy, 4(3), pp. 643-660. M. SHAKED, J.G. SHANTHIKUMAR, (007), Sochasic orders, Springer Series in Saisics, Ne York. N. EBRAHIMI, (996), Ho o measure uncerainy abou residual life ime, Sankhya, A, 58, pp. 48-57. N. EBRAHIMI, S.N.U.A. KIRMANI, (996a), A characerizaion of he proporional hazards model hrough a measure of discriminaion beeen o residual life disribuions, Biomerika, 83, pp. 33-35. N. EBRAHIMI, S.N.U.A. KIRMANI, (996b), A measure of discriminaion beeen o residual lifeime disribuions and is applicaions, Annals of he Insiue of Saisical Mahemaics, 48, pp. 57-65. R.C. GUPTA, (007), Role of equilibrium disribuions in reliabiliy sudies, Probabiliy in he Engineering and Informaion Sciences,, pp. 35-334. R.C. GUPTA, S.N.U.A. KIRMANI, (990), The role of eighed disribuion in sochasic modeling, Communicaions in Saisics - Theory and Mehods, 9(9), pp. 347-36. S.M. SUNOJ, T.B.SREEJITH, (in press), Some resuls of reciprocal subangen in he conex of eighed models, Communicaions in Saisics - Theory and Mehods. S.S. MAYA, S.M. SUNOJ, (008), Some dynamic generalized informaion measures in he conex of eighed models, Saisica, anno LXVIII, pp. 7-84. SUMMARY On bounds of some dynamic informaion divergence measures In his paper, e obain cerain bounds for some dynamic informaion divergences measures viz. Renyi s divergence of order and Kerridge s inaccuracy, using likelihood raio ordering. The resuls are also exended o eighed models and heoreical examples are given o supplemen he resuls.