SOME NOISELESS CODING THEOREM CONNECTED WITH HAVRDA AND CHARVAT AND TSALLIS S ENTROPY. 1. Introduction

Transcription

1 Kragujevac Journal of Mathematcs Volume 35 Number (20, Pages 7 SOME NOISELESS COING THEOREM CONNECTE WITH HAVRA AN CHARVAT AN TSALLIS S ENTROPY SATISH KUMAR AN RAJESH KUMAR 2 Abstract A new measure L, called average code word length of order and tye has been defned and ts relatonsh wth a result of generalzed Havrda and Charvat and Tsalls s entroy has been dscussed Usng L, some codng theorem for dscrete noseless channel has been roved Introducton Let n = {P = (, 2,, N, 0, = }, N 2 be the set of all fnte dscrete robablty dstrbutons, for any robablty dstrbuton (, 2,, N = P n Shannon [23] defned entroy as: ( H (P = log Throughout ths aer, wll stand for n = unless otherwse stated and logarthms are taken to the base ( > Let a fnte set of N nut symbols X = {x, x 2,, x N } be encoded usng alhabet of symbols, then t has been shown by Fensten [5] that there s unquely decherable nstantaneous code wth length n, n 2,, n N f and Key words and hrases Tsalls s Entroy, Codeword length, Kraft nequalty and Otmal code length and Power robabltes 200 Mathematcs Subject Classfcaton 94A5, 94A7, 94A24, 265 Receved: May 0, 200 Revsed: November 2, 200

2 2 SATISH KUMAR AN RAJESH KUMAR only f (2 n where s the sze of code alhabet If (3 L = n s the average codeword length then for a code whch satsfes (2 t has also been shown by Fensten [5], that (4 L H (P wth equalty f and only f (5 n = log for =, 2,, N and that by sutable encoded nto words of long sequences, the average length can be made arbtrary close to H (P Ths s Shannon s noseless codng theorem By consderng Reny s [20] entroy, a codng theorem and analogous to the above noseless codng theorem has been establshed by Cambell [4] and the authors obtaned bounds for t n terms of H (P = log P I, > 0 ( Keffer [3] defned a class rules and showed H (P s the best decson rule for decdng whch of the two sources can be coded wth exected cost of sequences of length n when n, where the cost of encodng a sequence s assumed to be a functon of length only Further Jelnek [9] showed that codng wth resect to Cambell [4] mean length s useful n mnmzng the roblem of buffer overflow whch occurs when the source symbol are beng roduced at a fxed rate and the code words are stored temorarly n a fnte buffer Hooda and Bhaker [8] consder the followng generalzaton of Cambell [4] mean length: and roved L (t = t log { tn }, H (P L (t < H (P +, > 0,, under the condton n

3 SOME NOISELESS COING THEOREM 3 where H (P s generalzed entroy of order = and tye studed by Aczel +t and aroczy [] and Kaur [0] It may be seen that the mean codeword length (3 had been generalzed arametrcally and ther bounds had been studed n terms of generalzed measures of entroes Here we gve another generalzaton of (3 and study ts bounds n terms of generalzed entroy of order and tye Longo [5], Gurdal and Pessoa [6], Sngh, Kumar and Tuteja [24], Parkash and Sharma [8], Hooda and Bhaker [8], Khan, Bhat and Przada [2], Arndt [2], Bag and Ahmad [3], Kerrdge [], Kraft [4], Mc-Mllan [6], Przada and Bhat [9], Roy [2] and Satsh Kumar [22] have studed generalzed codng theorems by consderng dfferent generalzed measure of ( and (3 under condton (2 of unque decherablty In ths aer we study some codng theorems by consderng a new functon deendng on arameters and Our motvaton for studyng ths new functon s that t generalzes some entroy functon already exstng n lterature Havrda and Charvat [7] and Tsalls [25] entroy whch s used n hyscs 2 Codng Theorem In ths secton, we defne nformaton measure as (2 H (P = [ ], where > 0 (, > 0, > 0, =, =, 2,, N ( When =, (2 reduces to Havrda and Charvat [7] and Tsalls s [25] entroy e, (22 H (P = [ ] ( When =, then (2 reduces to Shannon s [23] entroy (23 H (P = log ( When then (2 reduces to Mathur and Mtter s [7] entroy for the - ower dstrbuton, e, (24 H log (P =

4 4 SATISH KUMAR AN RAJESH KUMAR efnton 2 The mean length L wth resect to nformaton measure s defned as (25 L = ( n ( where > 0 (, > 0, > 0, =, =, 2,, N, ( When =, Then (25 reduces to new mean codeword length, e, (26 L = [ { n ( } ] ( When =,, then (25 reduces to mean code length defned by Shannon [23], e, L = n We establsh a result, that n a sense, rovdes a characterzaton of H (P under the condton of unque decherablty Theorem 2 For all ntegers > (27 L H (P under the condton (2 equalty holds f and only f ( (28 n = log Proof We use Holder s nequalty (29 x y ( x ( y q for all x 0, y 0, =, 2,, N when P < ( and + q =, wth equalty f and only f there exsts a ostve number c such that (20 x = cy q Settng = ( x = q n, ( y =, and q = n (29 and usng (2 we obtan the result (27 after smlfcaton for > 0 as >

5 SOME NOISELESS COING THEOREM 5 The equalty holds f and only f n =, =, 2,, N whch s equvalent to n = log ( Σ, =, 2,, N Theorem 22 For every code wth lengths {n }, =, 2,, N, L can be made to satsfy (2 L < H (P + [ ] Proof Let n be the ostve nteger satsfyng, the nequaltes ( ( (22 log n < log + Consder the ntervals ( ( ] (23 δ = [ log, log + of length In every δ, there les exactly one ostve number n such that ( (24 0 < log n < log ( It can be shown that the sequence {n }, =, 2,, N thus defned, satsfes (2 From (24 we have ( n < log n > (25 n ( multlyng both sdes of (25 by smlfcaton for ( ( > ( +, +, summng over =, 2,, N and as >, gves (2 Theorem 23 For every code wth length {n }, =, 2,, N of Theorem 2, L can be made to satsfy (26 L H (P > H (P + (

6 6 SATISH KUMAR AN RAJESH KUMAR Proof Suose ( P (27 n = log Clearly n and n + satsfy equalty n Holder s nequalty (29 Moreover, n satsfes Kraft s nequalty (2 Suose n s the unque nteger between n and n +, then obvously, n satsfes (2 Snce > 0 (, we have ( (28 Hence, snce (28 becomes whch gves (26 ( ( n ( ( n < ( n References ( = (, < ( n ( ( n [] J Aczel and Z aroczy, Uber Verallegemeneste quaslnare mttelveste de mt grewnebts functonen gebldet, Snd Pub Math ebrecan, 0 (963, 7 90 [2] C Arndt, Informaton Measure-Informaton and ts descrton n Scence and Engneerng, Srnger, Berln, 200 [3] M A K Bag and Rayees Ahmad ar, Some noseless codng theorems of naccuracy measure of order and tye, Sarajevo Journal of Mathematcs, 3 (5 (2007, [4] L L Cambell, A codng theorem and Reny s entroy, Informaton and Control, 8 (965, [5] A Fensten, Foundaton of Informaton Theory, McGraw Hll, New York, 956 [6] Gurdal and F Pessoa, On Useful Informaton of order, J Comb Informaton and Syst Sc, 2 (977, [7] Havrda and Charvat, Qualfcaton Method of Classfcaton Process, the concet of structural -entroy, Kybernetka, 3 (967, [8] S Hooda and U S Bhaker, A generalzed useful nformaton measure and codng theorems, Soochow J Math, 23 (997, [9] F Jelnek, Buffer overflow n varable lengths codng of fxed rate sources, IEEE, 3 (980, [0] J N Kaur, Generalzed entroy of order and tye, Maths Semnar, elh, 4 (967 [] F Kerrdge, Inaccuracy and nference, J R Stat Soc, Ser B 23 (96, 84 94

7 SOME NOISELESS COING THEOREM 7 [2] A B Khan, B A Bhat and S Przada, Some Results on a Generalzed Useful Informaton Measure, Journal of Inequaltes n Pure and Aled Mathematcs, 6 (4 7 (2005 [3] J C Keffer, Varable lengths source codng wth a cost deendng only on the codeword length, Informaton and Control, 4 (979, [4] L G Kraft, A devce for quantzng, groung and codng amltude modulated ulses, MS Thess, Electrcal Engneerng eartment, MIT, 949 [5] G Longo, A Noseless Codng Theorem for Sources Havng Utltes, Sam J Al Math, 30 (4 (976, [6] Mc-Mllan, Two nequaltes mled by unque decherablty IRE Trans Inform Theory IT-2, (956, 5 6 [7] J Mtter and Y Mathur, Comarson of entroes of ower dstrbuton, ZAMM, 52 (972, [8] Om Parkash and P K Sharma, Noseless Codng Theorems Corresondng to Fuzzy Entroes, Southeast Asan Bulletn of Mathematcs, 27 (2004, [9] S Przada and B A Bhat, Some more results n codng theory, J Ksam, 0 (2 (2006, 23 3 [20] A Reny, On Measure of entroy and nformaton, Proc 4th Berkeley Sym Maths Stat Prob, (96, [2] L K Roy, Comarson of Reny s entroy of ower dstrbuton, ZAMM, 56 (976, [22] Satsh Kumar, Some More Results on R-Norm Informaton Measure, Tamkang Journal of Mathematcs, 40 ( (2009, 4 58 [23] C E Shannon, A Mathematcal Theory of Communcaton, Bell System TechJ, 27 (948, [24] R P Sngh, R Kumar and R K Tuteja, Alcaton of Holder s Inequalty n Informaton Theory, Informaton Scences, 52 (2003, [25] C Tsalls, Possble Generalzaton of Boltzmann Gbbs Statstcs, J Stat Phy, 52 (988, 479 eartment of Mathematcs, GIMT (Kanla Kurukshetra, Haryana, Inda E-mal address: drsatsh74@redffmalcom 2 eartment of Mathematcs, Hndu College, Unversty of elh, elh-7, Inda E-mal address: rajeshhctm@redffmalcom