A new belief entropy: possible generalization of Deng entropy, Tsallis entropy and Shannon entropy

Transcription

1 A new belef entropy: possble generalzaton of Deng entropy, Tsalls entropy and Shannon entropy Bngy Kang a, Yong Deng a,b, a School of Computer and Informaton Scence, Southwest Unversty, Chongqng, , Chna b Insttute of Integrated Automaton, School of Electronc and Informaton Engneerng, Xan Jaotong Unversty, X an, Shaanx, , Chna Abstract Shannon entropy s the mathematcal foundaton of nformaton theory, Tsalls entropy s the roots of nonextensve statstcal mechancs, Deng entropy was proposed to measure the uncertanty degree of belef functon very recently. In ths paper, A new entropy H was proposed to generalze Deng entropy, Tsalls entropy and Shannon entropy. The new entropy H can be degenerated to Deng entropy, Tsalls entropy, and Shannon entropy under dfferent condtons, and also can mantans the mathematcal property of Deng entropy, Tsalls entropy and Shannon entropy. Keywords: Uncertanty measure, Entropy, Belef entropy, Deng entropy, Tsalls entropy, Shannnon entropy, Dempster-Shafer evdence theory Correspondng author: School of Computer and Informaton Scence, Southwest Unversty, Chongqng, , Chna. Emal address: ydeng@swu.edu.cn,prof.deng@hotmal.com (Yong Deng) Preprnt submtted to Physca A: Statstcal Mechancs and ts ApplcatonsOctober 2, 2016

2 1. Introducton How to measure the uncertanty has attracted much attenton [1, 2]. A lot of theores has been developed, such as probablty theory [3], fuzzy set theory [4], possblty theory [5], Dempster-Shafer evdence theory [6, 7], rough sets[8], DSmT[9, 10], generalzed evdence theory [11] and D numbers[12, 13]. Snce frstly proposed by Clausus n 1865 for thermodynamcs [14], varous types of entropes are presented, such as nformaton entropy [15], Tsalls entropy [16], nonaddtve entropy [17, 18, 19]. Informaton entropy [15], derved from the Boltzmann-Gbbs (BG) entropy [20] n thermodynamcs and statstcal mechancs, has been an ndcator to measures uncertanty whch s assocated wth the probablty densty functon (PDF). Dempster-Shafer theory evdence theory[6, 7] s manly proposed to handle such uncertanty. In Dempster-Shafer evdence theory, the epstemc uncertanty smultaneously contans nonspecfcty and dscord [21] whch are coexstng n a basc probablty assgnment functon (BPA). Several uncertanty measures, such as AU [22, 23], AM [21], have been proposed to quantfy such uncertanty n Dempster-Shafer theory. What s more, fve axomatc requrements have been further bult n order to develop a justfable measure. These fve axomatc requrements are range, probablstc consstency, set consstency, addtvty, subaddtvty, respectvely [24]. Exstng methods are not effcent to measure uncertan degree of BPA. To address ths ssue, a new entropy, named as Deng entropy [25], s proposed to measure the uncertanty of basc probablty assgnment for the evdence theory. In ths paper, 2

3 a dscussn of the maxmal value of Deng entropy has been dscussed and proofed, whch s useful for the real applcaton of Deng entropy. The paper s organzed as follows. The prelmnares Dempster-Shafer evdence theory and Deng entropy and Tsalls entropy are brefly ntroduced n Secton 2. Secton 3 makes some dscussons about the new entropy. Fnally, ths paper s concluded n Secton Prelmnares In ths secton, some prelmnares are brefly ntroduced Dempster-Shafer evdence theory Dempster-Shafer theory (short for D-S theory) s presented by Dempster and Shafer [6, 7]. Ths theory s wdely appled to uncertanty modelng [26, 27, 28, 29, 30], decson makng [31, 32, 33, 34, 35, 36, 37, 38], nformaton fuson [39, 40] and uncertan nformaton processng [41, 42]. D-S theory has many advantages to handle uncertan nformaton. Frst, D-S theory can handle more uncertanty n real world. In contrast to the probablty theory n whch probablty masses can be only assgned to sngleton subsets, n D-S theory the belef can be assgned to both sngletons and compound sets. Second, n D-S theory, pror dstrbuton s not needed before the combnaton of nformaton from ndvdual nformaton sources. Thrd, D-S theory allows one to specfy a degree of gnorance n some stuatons nstead of beng forced to be assgned for probabltes. Some basc concepts n D-S theory are ntroduced. 3

4 Let X be a set of mutually exclusve and collectvely exhaustve events, ndcated by X = {θ 1, θ 2,, θ,, θ X } (1) where set X s called a frame of dscernment. The power set of X s ndcated by 2 X, namely 2 X = {, {θ 1 },, {θ X }, {θ 1, θ 2 },, {θ 1, θ 2,, θ },, X} (2) For a frame of dscernment X = {θ 1, θ 2,, θ X }, a mass functon s a mappng m from 2 X to [0, 1], formally defned by: m : 2 X [0, 1] (3) whch satsfes the followng condton: m( ) = 0 and m(a) = 1 (4) A 2 X In D-S theory, a mass functon s also called a basc probablty assgnment (BPA). Assume there are two BPAs ndcated by m 1 and m 2, the Dempster s rule of combnaton s used to combne them as follows: 1 m 1 K 1 (B)m 2 (C), A ; m(a) = B C=A 0, A =. (5) wth K = m 1 (B)m 2 (C) (6) B C= Note that the Dempster s rule of combnaton s only applcable to such two BPAs whch satsfy the condton K < 1. 4

5 2.2. Deng entropy Wth the range of uncertanty mentoned above, Deng entropy [25] can be presented as follows E d = m(f ) log m(f ) 2 F 1 (7) where, F s a proposton n mass functon m, and F s the cardnalty of F. As shown n the above defnton, Deng entropy, formally, s smlar wth the classcal Shannon entropy, but the belef for each proposton F s dvded by a term (2 F 1) whch represents the potental number of states n F (of course, the empty set s not ncluded). Specally, Deng entropy can defntely degenerate to the Shannon entropy f the belef s only assgned to sngle elements. Namely, E d = 2.3. Tsalls entropy m(θ ) log m(θ ) 2 θ 1 = m(θ ) log m(θ ) For a dscrete random varable X = {X, = 1, 2,..., N} that has a probablty dstrbuton P = {p, = 1, 2,..., N}[p s the probablty of X = x ]. Scalng p to p m, where m s any real number, Tsalls Entropy [16] H m can be denoted as H m = k 1 N =1 pm m 1 = k m 1 N [p p m ] (8) =1 where k s often taken as unty. For m 1, the Tsalls entropy reduces to Shannon entropy 5

6 3. Proposed new Entropy: possble generalzaton of Deng entropy, Tsalls Entropy, Shannon entropy Assume F s the focal element and m(f ) s the basc probablty assgnment for F, then the possble generalzaton of Deng entropy and Tsalls entropy can be defned as H H = 1 [m (F )] m m 1 + m (F ) log ( 2 F 1 ) (9) where = 1, 2,..., 2 X 1, and X s the scale of the frame of dscernment. (1) For m 1, the new entropy H reduces to Deng entropy, namely E d = m(f ) log m(f ) 2 F 1 (10) Proof. For m 1, the new entropy can be shown as 1 [m (F )] m H m 1 = lm + m (F m 1 ) log ( 2 F 1 ) m 1 (11) Then 1 [m (F )] m H m 1 = lm m 1 m 1 + m (F ) log ( 2 F 1 ) (12) Because lm m 1 = lm m 1 { 1 [m(f )] m m 1 { 1 m = lm = m 1 } [m(f )] m } m (m 1) [m (F )] m log [m (F )] m (F ) log [m (F )] (13) 6

7 Then H m 1 = m (F ) log [m (F )] + m (F ) log ( 2 F 1 ) = { m (F ) log [m (F )] m (F ) log ( 2 F 1 )} (14) = m (F ) log m(f ) 2 F 1 Hence H m 1 = E d = m (F ) log m (F ) 2 F 1 (15) End proof (2) For the belef s only assgned to sngle elements, the new entropy H reduces to Tsalls entropy, namely H m = k 1 N =1 pm m 1 = k m 1 N [p p m ] (16) Proof. For the belef s only assgned to sngle elements, the F = 1, we can =1 easly get that Hence H = = m (F ) log ( 2 F 1 ) = 0 (17) 1 [m(f )] m m [m(f )] m 1 [m(θ )] m = m 1 m (F ) log ( 2 F 1 ) m 1 = H s (18) End proof (3) For m 1 and the belef s only assgned to sngle elements, the new entropy H reduces to Shannon entropy, namely H = m(θ ) log m(θ ) 2 θ 1 = 7 m(θ ) log m(θ ) (19)

8 Proof. When m 1, from Eq. (15), we can get H m 1 = E d = m (F ) log m (F ) 2 F 1 (20) When the belef s only assgned to sngle elements, the F = 1, we can easly get that End proof H = m(f ) log m(f ) 2 F 1 = m(θ ) log m(θ ) (21) 8

9 4. Concluson Shannon entropy s the mathematcal foundaton of nformaton theory, Tsalls entropy s the roots of nonextensve statstcal mechancs, Deng entropy was proposed to measure the uncertanty degree of belef functon very recently. In ths paper, A new entropy H was proposed to generalze Deng entropy, Tsalls entropy and Shannon entropy. The new entropy H can be degenerated to Deng entropy, Tsalls entropy, and Shannon entropy under dfferent condtons, and also can mantans the mathematcal property of Deng entropy, Tsalls entropy and Shannon entropy. Acknowledgments Ths work s partally supported by Natonal Natural Scence Foundaton of Chna, Grant Nos , and , Chongqng Natural Scence Foundaton for dstngushed scentst, Grant No. CSCT, 2010BA2003, Program for New Century Excellent Talents n Unversty, Grant No. NCET , Shangha Rsng-Star Program Grant No.09QA , the Chenxng Scholarshp Youth Found of Shangha Jao Tong Unversty, Grant No. T , the open fundng project of State Key Laboratory of Vrtual Realty Technology and Systems, Behang Unversty (Grant No.BUAA-VR-14KF-02). References [1] G. J. Klr, Uncertanty and nformaton: foundatons of generalzed nformaton theory, John Wley & Sons,

10 [2] S. Sankararaman, S. Mahadevan, Model valdaton under epstemc uncertanty, Relablty Engneerng & System Safety 96 (9) (2011) [3] W. Feller, An ntroducton to probablty theory and ts applcatons, Vol. 2, John Wley & Sons, [4] L. A. Zadeh, Fuzzy sets, Informaton and control 8 (3) (1965) [5] D. Dubos, H. M. Prade, H. Farreny, R. Martn-Clouare, C. Testemale, Possblty theory: an approach to computerzed processng of uncertanty, Vol. 2, Plenum press New York, [6] A. P. Dempster, Upper and lower probabltes nduced by a multvalued mappng, Annals of Mathematcs and Statstcs 38 (2) (1967) [7] G. Shafer, A Mathematcal Theory of Evdence, Prnceton Unversty Press, Prnceton, [8] Z. Pawlak, Rough sets, Internatonal Journal of Computer & Informaton Scences 11 (5) (1982) [9] J. Dezert, Foundatons for a new theory of plausble and paradoxcal reasonng, Informaton and Securty 9 (2002) [10] F. Smarandache, J. Dezert, Advances and Applcatons of DSmT for Informaton Fuson (Collected works), second volume: Collected Works, Vol. 2, Infnte Study,

11 [11] Y. Deng, Generalzed evdence theory, Appled Intellgence (2015) do /s [12] Y. Deng, D numbers: Theory and applcatons, Internatonal Journal of Computer & Informaton Scences 9 (9) (2012) [13] X. Deng, Y. Hu, Y. Deng, S. Mahadevan, Suppler selecton usng AHP methodology extended by D numbers, Expert Systems wth Applcatons 41 (1) (2014) [14] R. Clausus, The mechancal theory of heat: wth ts applcatons to the steam-engne and to the physcal propertes of bodes, J. van Voorst, [15] C. E. Shannon, A mathematcal theory of communcaton, ACM SIG- MOBILE Moble Computng and Communcatons Revew 5 (1) (2001) [16] C. Tsalls, Possble generalzaton of Boltzmann-Gbbs statstcs, Journal of Statstcal Physcs 52 (1-2) (1988) [17] C. Tsalls, Nonaddtve entropy: the concept and ts use, The European Physcal Journal A 40 (3) (2009) [18] C. Tsalls, Nonaddtve entropy and nonextensve statstcal mechancsan overvew after 20 years, Brazlan Journal of Physcs 39 (2A) (2009)

12 [19] C. Tsalls, U. Trnakl, Nonaddtve entropy and nonextensve statstcal mechancs some central concepts and recent applcatons, n: Journal of Physcs: Conference Seres, Vol. 201, 2010, p [20] J. L. Lebowtz, Boltzmann s entropy and tme s arrow, Physcs today 46 (1993) [21] A.-L. Jousselme, C. Lu, D. Grener, E. Bosse, Measurng ambguty n the evdence theory, IEEE Transactons on Systems, Man and Cybernetcs, Part A: Systems and Humans 36 (5) (2006) [22] Y. Maeda, H. T. Nguyen, H. Ichhash, Maxmum entropy algorthms for uncertanty measures, Internatonal Journal of Uncertanty, Fuzzness and Knowledge-Based Systems 1 (01) (1993) [23] D. Harmanec, G. J. Klr, Measurng total uncertanty n dempster-shafer theory: A novel approach, Internatonal journal of general system 22 (4) (1994) [24] G. J. Klr, M. J. Werman, Uncertanty-based nformaton: elements of generalzed nformaton theory, Sprnger, [25] Y. Deng, Deng entropy, Chaos, Soltons & Fractals 91 (2016) [26] P. Smets, R. Kennes, The transferable belef model, Artfcal Intellgence 66 (2) (1994) [27] T. L. Wckramarathne, K. Premaratne, M. N. Murth, Toward effcent 12

13 computaton of the dempster shafer belef theoretc condtonals, Cybernetcs, IEEE Transactons on 43 (2) (2013) [28] J. Abellán, Élo Bossé, Drawbacks of uncertanty measures based on the pgnstc transformaton, IEEE Transactons on Systems, Man, and Cybernetcs: Systems (2016) 1 7do: /TSMC [29] J. Abellán, S. Moral, Upper entropy of credal sets. applcatons to credal classfcaton, Internatonal Journal of Approxmate Reasonng 39 (2) (2005) [30] J. Abellán, S. Moral, Dfference of entropes as a non-specfcty functon on credal sets, Internatonal journal of general systems 34 (3) (2005) [31] L. V. Utkn, A new rankng procedure by ncomplete parwse comparsons usng preference subsets, Intellgent Data Analyss 13 (2) (2009) [32] V.-N. Huynh, T. T. Nguyen, C. A. Le, Adaptvely entropy-based weghtng classfers n combnaton usng dempster shafer theory for word sense dsambguaton, Computer Speech & Language 24 (3) (2010) [33] Y. Deng, F. Chan, A new fuzzy dempster MCDM method and ts applcaton n suppler selecton, Expert Systems wth Applcatons 38 (8) (2011)

14 [34] W. Jang, B. We, C. Xe, D. Zhou, An evdental sensor fuson method n fault dagnoss, Advances n Mechancal Engneerng 8 (3) (2016) 1 7. [35] W. Jang, C. Xe, B. We, D. Zhou, A modfed method for rsk evaluaton n falure modes and effects analyss of arcraft turbne rotor blades, Advances n Mechancal Engneerng 8 (4) (2016) do: / [36] R. R. Yager, N. Alajlan, Decson makng wth ordnal payoffs under dempster shafer type uncertanty, Internatonal Journal of Intellgent Systems 28 (11) (2013) [37] J.-B. Yang, D.-L. Xu, Evdental reasonng rule for evdence combnaton, Artfcal Intellgence 205 (2013) [38] J. Abellán, S. Moral, Maxmum of entropy for credal sets, Internatonal Journal of Uncertanty, Fuzzness and Knowledge-Based Systems 11 (05) (2003) [39] F. Cuzzoln, A geometrc approach to the theory of evdence, Systems, Man, and Cybernetcs, Part C: Applcatons and Revews, IEEE Transactons on 38 (4) (2008) [40] J. Schubert, Conflct management n Dempster-Shafer theory usng the degree of falsty, Internatonal Journal of Approxmate Reasonng 52 (3) (2011)

15 [41] S. Rao, K. Annamdas, A comparatve study of evdence theores n the modelng, analyss, and desgn of engneerng systems, Journal of Mechancal Desgn 135 (6) (2013) [42] B. Kang, Y. Deng, R. Sadq, S. Mahadevan, Evdental cogntve maps, Knowledge-Based Systems 35 (2012)