CUMULATIVE ENTROPIES: A SURVEY

Transcription

1 Revie of he Air orce Academy No 34/7 CUMULATIVE ENTROPIES: A SURVEY Vasile PREDA * Cosel BĂLCĂU * Doru CONSTANTIN * Ioana Ileana PANAIT ** * Deparmen of Mahemaics Universiy of Buchares Romania preda@fmiunibucro coselbalcau@upiro doruconsanin@upiro ** Docoral School of Mahemaics Universiy of Buchares Romania yoana_panai@yahoocom DOI: 96/ Absrac: We presen a revie of cumulaive enropies from reliabiliy heory Mahemaics Subjec Classificaion: Primary 94A7 Keyords: cumulaive enropy residual enropy pas enropy inerval enropy INTRODUCTION Consider a non-negaive absoluely coninuous random variable ih he probabiliy densiy funcion PD f he cumulaive disribuion funcion CD P and he reliabiliy funcion R P > Le [ and a real nonnegaive measurable funcion defined on [ Le D { < < } or any given pair D he condiional random variable has he PD f he CD and he R given by f f for all The Shannon enropy proposed by Shannon [45] 948 is H f f d Remark In all definiion from his paper e assume ha he inegrals eis and e use he noaional convenion and / The concep of enropy as proposed as a measure of he amoun of informaion supplied by a random variable or a probabilisic eperimen I has numerous eensions o oher enropy-ype measures some of hese ill be presened belo 3

2 Cumulaive Enropies: a Survey In he ne secions e presen he residual and pas enropy-ypes measures based on CD and R he cumulaive enropies he cumulaive relaive enropies and inaccuracy measures WEIHTED RESIDUAL AND PAST ENTROPIES The eighed enropy referred by Di Crescenzo and Longobardi [9] 6 in agreemen ih Beli s and uia s u [4] 968 is H as f f d 3 The residual enropy RE proposed by Ebrahimi and Pellerey [3] 996 is defined f f RE f f d d 4 The pas enropy PE proposed by Di Crescenzo and Longobardi [7] is f f PE f f d d 5 The eighed residual enropy WRE proposed by Di Crescenzo and Longobardi [9] 6 is f f WRE f f d d 6 The eighed pas enropy WPE proposed by Di Crescenzo and Longobardi [9] 6 is f f WPE f f d d 7 These eighed enropies are suiable o describe dynamic informaion of random lifeimes in analogy ih he enropies of residual and pas lifeimes inroduced in [3] and [7] respecively 3 CUMULATIVE ENTROPIES The cumulaive residual enropy CRE proposed by Rao Chen Vemuri Wang [4] 4 is CRE d 8 The CRE is an alernaive measure of uncerainy in he random variable ha enjoys many of he properies of Shannon enropy and has some advanages such us is alays non-negaive can be easily compued from sample daa and hese compuaions asympoically converge o he rue values 4

3 Revie of he Air orce Academy No 34/7 Like as he Shannon enropy he CRE can be used o consruc probabiliy disribuions by applying he Maimum Enropy Principle inroduced in 957 by Jaynes [8 9] or eample Rao [4] 5 obains a general resul for characerizaion of MA-CRE disribuions and applies his resul o consruc he uniform disribuion and he Weibull disribuion Drissi Chonavel and Boucher [] 8 generalize he definiion of CRE o he case of random variables ih suppors ha are no resriced o posiive values The dynamic cumulaive residual enropy DCRE proposed by Asadi and Zohrevand [] 7 is DCRE d d 9 The DCRE is a measure of he informaion in he residual life disribuion The auhors sho ha he CRE and he DCRE is conneced ih some ell-knon reliabiliy measures such as he mean residual lifeime and he hazard rae Also hey prove ha if he DCRE is an non-decreasing funcion on hen i characerizes he underlying disribuion funcion uniquely The cumulaive pas enropy CPE proposed by Di Crescenzo and Longobardi [] 9 is CPE d The CPE is also non-negaive and i is useful o measure informaion on he inaciviy ime of a sysem being appropriae for he sysems hose uncerainy is relaed o he pas The dynamic cumulaive pas enropy DCPE proposed by Di Crescenzo and Longobardi [] 9 and by Navarro Del Aguila and Asadi [37] is DCPE d d The inerval enropy IH proposed by Sunoj Sankaran and Maya [47] 9 is IH f f d f f d The eighed cumulaive residual enropies WCRE proposed by Misagh Panahi Yari Shahi [33] is WCRE d 3 The eighed cumulaive pas enropies WCPE proposed by Misagh Panahi Yari Shahi [33] is WCPE d 4 5

4 Cumulaive Enropies: a Survey 6 The auhors presen various properies of his measure including is connecion ih eighed residual and pas enropies and obain some upper and loer bounds The inerval cumulaive residual enropies ICRE proposed by Khorashadizadeh Rezaei Roknabadi and Mohashami Borzadaran [5] 3 is d ICRE d 5 The inerval cumulaive pas enropies ICPE proposed by Khorashadizadeh Rezaei Roknabadi and Mohashami Borzadaran [5] 3 is d ICPE d 6 The auhors presen some properies and characerizaion of his measures including is connecions ih doubly runcaed Shannon enropy and mean residual life The eighed cumulaive residual enropies WCRE proposed by Suhov and Yasaei Sekeh [46] 5 is d WCRE 7 The eighed cumulaive pas enropies WCPE proposed by Suhov and Yasaei Sekeh [46] 5 is d WCPE 8 The inerval eighed cumulaive residual enropy IWCRE of he random variable a inerval ] [ ih he eigh funcion is defined by Sekeh Borzadran and Roknabadi [5] 5 d IWCRE d 9 The inerval eighed cumulaive pas enropy IWCE of he random variable a inerval ] [ ih he eigh funcion is defined by Sekeh Borzadran and Roknabadi [5] 5 d IWCE d

5 Revie of he Air orce Academy No 34/7 4 CUMULATIVE RELATIVE ENTROPIES AND INACCURACY MEASURES Consider and Y o non-negaive absoluely coninuous random variables ih he probabiliy densiy funcions PDs f and g y he cumulaive disribuion funcions CDs P and y P Y y and he reliabiliy funcions Rs P > and y P Y > y respecively Le [ and D { < < < } or any given pair D consider he condiional random variable ih he PD f he CD and he R defined in he firs secion and he condiional random variable Y Y has he PD g y he CD y and he R y given by g y y g y y y y y forall y The relaive enropy Kullback-Leibler divergence Kullback-Leibler discriminaion informaion proposed by Kullback and Leibler [7] 95 is D Y f f d g Developing he Shannon enropy he auhors have he idea o compare he enropy inside a family of probabiliy measures insead of considering he enropy corresponding o only one probabiliy measure The Kerridge measure of inaccuracy proposed by Kerridge [4] 96 is H Y f g d D Y H 3 The eighed inaccuracy measure proposed by Taneja and Tueja [48] 986 is WH Y f g d 4 The residual relaive enropy proposed by Ebrahimi and Kirmani [5] 996 is RD Y f f d g f f / d g / The pas relaive enropy proposed by Crescenzo and Longobardi [8] 4 is f f f / PD Y f d d g 6 g / 5 7

6 Cumulaive Enropies: a Survey The dynamic measure of inaccuracy proposed by Taneja Kumar and Srivasava [49] 9 is f g RI Y f g d d 7 RD Y RE The eighed residual inaccuracy measure proposed by Kumar and Taneja [48] is WRI Y f g d WRD Y WRE here WRD Y f f g d f f f / d d g g / represens he eighed residual relaive enropy The pas inaccuracy measure PI proposed by Kumar and Taneja [48] is PI 8 9 f g f g d d PD Y PE 3 The eighed pas inaccuracy measure WPI proposed by Kumar and Taneja [48] is WPI f g d WPD Y WPE here f g d 3 f f f / WPD Y f d d g 3 g / represens he eighed pas relaive enropy The cumulaive residual inaccuracy CRI proposed by Taneja and Kumar [5] is CRI Y d CRD Y CRE 33 here CRD Y d represens he cumulaive residual relaive enropy 34 8

7 Revie of he Air orce Academy No 34/7 The dynamic cumulaive residual inaccuracy DCRI proposed by Taneja and Kumar [5] a version as also inroduced by Chamany and Barapour 4 is DCRI Y d DCRD Y DCRE here d 35 DCRD Y d / d / represens he dynamic cumulaive residual relaive enropy The inerval relaive enropy proposed by Misagh and Yari [35] is ID Y f f d g f f / [ ] d g / [ ] Here as proposed a measure of discrepancy beeen o lifeime disribuions a he inerval of ime in base of Kullback-Leibler discriminaion informaion They sudied various properies of his measure including is connecion ih residual and pas measures of discrepancy and inerval enropy and hey obained is upper and loer bounds The cumulaive pas inaccuracy CPI proposed by Kumar and Taneja [9] 5 is CPI Y d CPD Y CPE 38 here CPD Y d represens he cumulaive pas relaive enropy The dynamic cumulaive pas inaccuracy DCPI proposed by Kumar and Taneja [9] 5 received a 5 April 4 acceped a march 5 and as published in december 5 in JTSA and Kundu Di Crescenzo and Longobardi [3] 6 received a 8 March 4 acceped a Augus 5 and as published in 6 in Merika is DCPI Y d DCPD Y DCPE here d 4 9

8 Cumulaive Enropies: a Survey d Y DCPD d / / 4 represens he dynamic cumulaive pas relaive enropy The inerval inaccuracy measure II proposed by Kundu [3] 5 is d g f Y II d g f 4 The eighed inerval inaccuracy measure WII proposed by Kundu [3] 5 is d g f Y WII d g f 43 The inerval cumulaive residual inaccuracy ICRI proposed by Kundu Di Crescenzo and Longobardi [3] 6 is d Y ICRI d 44 The inerval cumulaive pas inaccuracy ICPI proposed by Kundu Di Crescenzo and Longobardi [3] 6 is d Y ICPI d 45 CONCLUSIONS In his paper e presen a revie of cumulaive enropies from reliabiliy heory irs e presen he Shannon enropy concep proposed by Shannon [45] and hen e presen he residual and pas enropy-ypes measures based on CD and R he cumulaive enropies he cumulaive relaive enropies and inaccuracy measures

9 Revie of he Air orce Academy No 34/7 REERENCES [] J Ahmadi A Di Crescenzo M Longobardi On dynamic muual informaion for bivariae lifeimes Advances in Applied Probabiliy 47 6 no [] M Asadi Y Zohrevand On he dynamic cumulaive residual enropy Journal of Saisical Planning and Inference 37 7 no [3] S Barapour Z Bami On he Discree Cumulaive Residual Enropy JIRSS 3 5 [4] M Beliș S uiașu A Quaniaive and qualiaive measure of informaion in cyberneic sysems IEEE Trans on Inf Theory [5] ND Cahill JA Schnabela JA Noblea and DJ Hakesc Overlap Invariance of Cumulaive Residual Enropy Measures for Mulimodal Image Alignmen Medical Imaging 9: Image Processing Proc of SPIE vol 759 [6] A Chamany S Barapour A Dynamic Discriminaion Informaion Based On Cumulaive Residual Enropy And Is Properies Commun Sa Theory Mehods 43 4 no [7] A Di Crescenzo M Longobardi Enropy-based measure of uncerainy in pas life ime disribuions Journal of Applied Probabiliy 39 no [8] A Di Crescenzo MA Longobardi A measure of discriminaion beeen pas lifeime disribuions Sa Prob Le 67 4 no 73 8 [9] A Di Crescenzo MA Longobardi On eighed residual and pas enropies Scieniae Mah Japon [] A Di Crescenzo M Longobardi On cumulaive enropies Journal of Saisical Planning and Inference 39 9 no [] S Das On Weighed eneralized Enropy Theoreical Saisics and Mahemaics Uni Indian Saisical Insiue 3 [] N Drissi T Chonavel JM Boucher eneralized Cumulaive Residual Enropy for Disribuions ih Unresriced Suppors Hindai Publishing Corporaion Research Leers in Signal Processing 8 [3] N Ebrahimi Pellerey Ne parial ordering of survival funcions based on noion of uncerainy Journal of Applied Probabiliy no [4] N Ebrahimi Ho o measure uncerainy abou residual lifeime Sankhya Ser A [5] N Ebrahimi SNUA Kirmani Some resuls on ordering of survival funcions hrough uncerainy Sais Prob Le no [6] S uiașu Weighed enropy Repor on Mah Physics [7] S uiașu M Malița Coaliion and Connecion in ames Pergamon Press Oford 98 [8] ET Jaynes Informaion heory and saisical mechanics Phys Rev [9] ET Jaynes Informaion heory and saisical mechanics II Phys Rev [] O Kamari SOME PROPERTIES O THE CUMULATIVE RESIDUAL ENTROPY Inernaional Journal of Mulidisciplinary Academic Research 3 5 no [] O Kamari On Dynamic Cumulaive Residual Enropy of Order Saisics Journal of Saisics Applicaions & Probabiliy 5 6 no [] DT Kang urher resuls on closure properies of LPQE order Saisical Mehodology [3] DT Kang Some Resuls on DDCRE Class of Life Disribuions The Indian Journal of Saisics Indian Saisical Insiue Par 4 vol 77-A [4] D Kerridge Inaccuracy and inference Journal of Royal Saisical Sociey Series B [5] [5] M Khorashadizadeh AH Rezaei Roknabadi R Mohashami Borzadaran Doubly runcaed inerval cumulaive residual and pas enropy Saisics and Probabiliy Leers 83 3 no [6] I Klein B Mangold M Doll Cumulaive Paired -Enropy Enropy 8 6 [7] S Kullback RA Leibler On informaion and sufficiency Ann Mah Sais [8] V Kumar HC Taneja On lengh biased dynamic measure of pas inaccuracy Merika [9] V Kumar HC Taneja Dynamic Cumulaive Residual and Pas Inaccuracy Measures Alanis Press Journal of Saisical Theory and Applicaions 4 5 no [3] C Kundu On Weighed Measure of Inaccuracy for Doubly Truncaed Random Variables Communicaions in Saisics-Theory & Mehods 5 [3] C Kundu A Di Crescenzo MA Longobardi On cumulaive residual pas inaccuracy for runcaed random variables Springer-Verlag Berlin Heidelberg 6

10 Cumulaive Enropies: a Survey [3] M Mirali S Barapourb CV akoorc On eighed cumulaive residual enropy Communicaions in Saisics - Theory and Mehods 6 [33] Misagh Y Panahi H Yari R Shahi Weighed cumulaive enropy and is esimaion Proceedings of he IEEE Inernaional Conference on Qualiy and Reliabiliy Bangkok [34] Misagh H Yari On eighed inerval enropy Saisics and Probabiliy Leers 8 no [35] Misagh H Yari Inerval enropy and informaive disance Enropy 4 no [36] Misagh On Shif-Dependen Cumulaive Enropy Measures Hindai Publishing Corporaion Inernaional Journal of Mahemaics and Mahemaical Sciences 6 [37] J Navarro Y del Aguila M Asadi Some ne resuls on he cumulaive residual enropy Journal of Saisical Planning and Inference 4 no 3 3 [38] M Nourbakhsh Yari Y Mehrali Weighed Enropies and Their Esimaions Communicaions in Saisics - Simulaion and Compuaion 6 [39] V Preda C Bălcău Enropy Opimizaion ih Applicaions Ed Academiei Române Buchares [4] Rajesh EI Abdul-Sahar KV Reshmi KR Muraleedharan-Nair Bivariae eneralized Cumulaive Residual Enropy The Indian Journal of Saisics Indian Saisical Insiue Par 4 vol 76-A [4] M Rao Y Chen BC Vemuri Wang Cumulaive residual enropy: a ne measure of informaion IEEE Transacions Informaions Theory 5 4 no 6 8 [4] M Rao More on a Ne Concep of Enropy and Informaion Journal of Theoreical Probabiliy 8 5 no [43] S Ravi A Saeb On informaion heory and is applicaions ProbSa orum 4 no [44] M Serrurier H Prade Enropy evaluaion based on confidence inervals of frequency esimaes: Applicaion o he learning of decision rees Proceedings of he 3nd Inernaional Conference on Machine Learning Lille 5 [45] CE Shannon A mahemaical heory of communicaion Bell Sysem Technical Journal [46] Y Suhov S Yasaei Sekeh Weighed cumulaive enropies: An eension of CRE and CE hp://arivorg/abs/5775v 5 [47] SM Sunoj P Sankaran SS Maya Characerizaions of life disribuions using condiional epecaions of doubly inerval runcaed random variables Communicaions in Saisics-Theory and Mehods 38 9 no [48] HC Taneja RK Tueja Characerizaion of a quaniaive-qualiaive measure of inaccuracy Kyberneika [49] HC Taneja V Kumar R Srivasava A dynamic measure of inaccuracy beeen o residual lifeime disribuions Inernaional Mahemaical orum 4 9 no [5] HC Taneja V Kumar On Dynamic Cumulaive Residual Inaccuracy Measure Proceedings of he World Congress on Engineering London [5] S Yasaei Sekeh R Mohashami Borzadran AH Rezaei Roknabadi A noe on double runcaed inerval eighed cumulaive enropies hp://arivorg/abs/5846v3 5 [5] V Zardash S Parsi M Mousazadeh On empirical cumulaive residual enropy and a goodness-offi es for eponenialiy Springerlinkcom 4