Certain Coding Theorems Based on Generalized Inaccuracy Measure of Order α and Type β and 1:1 Coding

Transcription

1 MATEMATIKA, 2013, Volume 29, umber 1, c Deparmen of Mahemacal Scences, UTM Ceran Codng Theorems Based on Generalzed Inaccuracy Measure of Order α and Type β and 1:1 Codng 1 Sash Kumar and 2 Arun Choudhary 1,2 Deparmen of Mahemacs, Geea Insue of Managemen & Technology Kanpla , Kurukshera, Haryana, Inda e-mal: 1 drsash74@redffmal.com, 2 arunchoudhary07@gmal.com Absrac In hs paper, A new mean codeword lengh L β(u) s defned. We have esablshed some noseless codng heorems based on generalzed naccuracy measure of order α and ype β. Furher, we have defned mean codeword lengh L β,1:1(u) for he bes one-o-one code. Also we have shown ha he mean codeword lenghs L β,1:1(u) for he bes one-o-one code (no necessarly unquely decodable) are shorer han he mean codeword lengh L β(u). Moreover, we have suded gher bounds of L β(u). Keywords Generalzed naccuracy measures; Codeword; mean codeword lengh; Kraf s nequaly; Holder s nequaly Mahemacs Subjec Classfcaon 94A15, 94A17, 94A24, 26D15. 1 Inroducon I s well known fac ha nformaon measures are mporan for praccal applcaons of nformaon processng. For measurng nformaon, a general approach s provded n a sascal framework based on nformaon enropy nroduced by Shannon [1]. As a measure of nformaon, he Shannon enropy sasfes some desrable axomac requremens and also can be assgned operaonal sgnfcance n mporan praccal problems, for nsance, n codng and elecommuncaon. In codng heory, usually we come across he problem of effcen codng of messages o be sen over a noseless channel where our concern s o maxmze he number of messages ha can be sen hrough a channel n a gven me. Thus, we fnd he mnmum value of a mean codeword lengh subjec o a gven consran on codeword lenghs. However, snce he codeword lenghs are negers, he mnmum value wll le beween wo bounds and a noseless codng heorem seeks o fnd hese wo lower bounds whch are n erms of some measure of enropy for a gven mean and a gven consran. For unquely decpherable codes, Shannon [1] found he lower bounds for he arhmec mean by usng hs own enropy. Campbell [2] defned hs own exponenaed mean and by applyng Kraf s [3] nequaly, found lower bounds for hs mean n erms of Reny s [4] measure of enropy. Guasu and Pcard [5], Longo [6], Gurdal and Pessoa [7], Taneja and Tueja [8], Auar and Khan [9], Jan and Tueja [10], Hooda and Bhaker [11], Bhaa [12] and Sngh, Kumar and Tueja [13] consdered he problem of useful nformaon measures and used sudyng he noseless codng heorems for sources nvolvng ules. Chapeau-Blondeau e al. [14] have presened an exenson o source codng heorem radonally based upon Shannon s enropy and laer generalzed o Reny s enropy. Chapeau- Blondeau e al. [15] have descrbed a praccal problem of source codng and nvesgaed an mporan relaon sressng ha Reny s enropy emerges a an order α dfferng from he radonal Shannon s enropy.

2 86 Sash Kumar and Arun Choudhary Ramamoorhy [16] consdered he problem of ransmng mulple compressble sources over a nework a mnmum cos wh he objecve o fnd he opmal raes a whch he sources should be compressed. Tu e al. [17] have presened a new scheme based on varable lengh codng, capable of provdng relable resoluons for flow meda daa ransmsson n spaal communcaon. Some neresng work for he consrucon of nformaon heorec source nework codng n he presence of eavesdroppers has been presened by Luo e al. [18]. Wu e al. [19] have consruced a space rells and desgn a low-complexy jon decodng algorhm wh a varable lengh symbol-a poseror probably algorhm n resource consraned deep space communcaon neworks. The mean lengh of a noseless unquely decodable code for a dscree random varable X sasfes H (X) L UD < H (X) + 1 (1) Where H (X) = p log p (2) =1 s he Shannon s enropy [1] of he random varable X and L UD = p n (3) be he unque and decodable mean codeword lengh suded by Shannon [1]. Shannon s resrcon of codng of X o prefx codes s hghly jusfed by he mplc assumpon ha he descrpon wll be concaenaed and hus mus be unquely decodable and nsananeous codes, cf. [20], [21], he expeced codeword lengh s he same for boh he se of codes. There are some communcaon suaons n whch a random varable X s beng ransmed raher han a sequence of random varables. For hs conex Leung-Yang-Cheong and Cover [22] consdered one o one codes.e., codes whch assgn a dsnc bnary code o each oucome of he random varable X whou regard o he condon ha concaenaons of he descrpons mus be unquely decpherable. Bhaa [23], [24], have exended he dea of he one o one code o he Kerrdge s naccuracy [25] and also derved lower bounds o he exponenal mean codeword lengh for he bes one o one codes n erms of generalzed naccuracy of order α. In Secon 2, we have esablshed a generalzed codng heorem for personal probably codes by consderng useful naccuracy of order α and ype β. In Secon 3, we generalzed he dea of he bes 1:1 code o useful naccuracy of order α and ype β. Throughou he paper denoes he se of he naural numbers and for we se { } = (p 1,..., p )/p 0, = 1,...,, p = 1. =1 In case here s no rse o msundersandng we wre P nsead of (p 1,..., p ). Throughou hs paper, wll sand for =1 unless oherwse saed and logarhms are aken o he base D (D > 1). Consder he model gven below for a fne random expermen scheme havng (x 1, x 2,..., x ) as he complee sysem of evens, happenng wh respecve probables P = (p 1, p 2,..., p ), =1

3 Ceran Codng Theorems Based on Generalzed Inaccuracy Measure 87 p 0, =1 p = 1 and creded wh ules U = (u 1, u 2,..., u ), u > 0, = 1, 2,...,. Denoe x 1 x 2... x S = p 1 p 2... p u 1 u 2... u. (4) We call (4) he uly nformaon scheme. Le Q = (q 1, q 2,..., q ) be he predced dsrbuon havng he uly dsrbuon U = (u 1, u 2,..., u ), u > 0, = 1, 2,...,. Taneja and Tueja [8] have suggesed and characerzed he useful naccuracy measure I(P, Q; U) = u p log q. (5) By consderng he weghed mean code word lengh [5] =1 L(U) = u p n =1 u (6) p where n 1, n 2,..., n are he code lenghs of x 1, x 2,..., x respecvely. Taneja and Tueja [8] derved he lower and upper bounds on L(U) n erms of I(P,Q;U). Bhaa [23] defned he useful average code lengh of order as =1 u+1 =1 L (U) = 1 log p D n ( ) +1, 1 < < (7) =1 u p where D s he sze of he code alphabe. He also derved he bounds for he useful average code lengh of order n erms of generalzed useful n accuracy measure gven by [ I α (P, Q; U) = 1 ] 1 α log =1 u p q α 1 =1 u, α > 0( 1) (8) p under he condon =1 p q 1 D n 1 (9) where D s he sze of he code alphabe. In hs paper, we sudy some codng heorems by consderng a new funcon dependng on he parameers α and β and a uly funcon. Our movaon for sudyng hs funcon s ha generalzes some nformaon measures already exsng n he leraure. 2 Codng Theorems Consder a funcon [ ] Iα β 1 (P, Q; U) = 1 α log q(α 1) =1 u, α > 0( 1), β 1. (10) p β

4 88 Sash Kumar and Arun Choudhary () When β = 1, (10) reduces o a measure of useful nformaon measure of order α due o Bhaa [23]. () When β = 1, u = 1, = 1, 2,...,. (10) reduces o he naccuracy measure gven by ah [26], furher reduces o Reny s [4] enropy by akng p = q, = 1, 2,...,. () When β = 1, u = 1, = 1, 2,..., and α 1. (10) reduces o he measure due o Kerrdge [25]. (v) When u = 1, = 1, 2,..., and p = q, = 1, 2,..., he measure (10) becomes Aczel and Daroczy [27] and Kapur [28] enropy. We call hs Iα(P, β Q; U) n (10) he generalzed useful naccuracy measure of order α and ype β. Furher consder [ L β(u) = 1 ] log Dn =1 u p β, 1 < <. (11) () For β = 1, L β (U) n (11) reduces o he useful mean lengh L (U) of he code gven by Hooda and Bhaker [11]. () When β = 1, u = 1, = 1, 2,...,, L β (U) n (11) reduces o he mean lengh gven by Campbell [2]. () When β = 1, u = 1, = 1, 2,..., and α 1, L β (U) n (11) reduces o he opmal code lengh dencal o Shannon [1]. (v) For u = 1, = 1, 2,...,, L β (U) n (11) reduces o he mean lengh gven by Khan and Ahmed [29]. ow we fnd he lower bounds of L β (U) n erms of Iβ α(p, Q; U) under he condon =1 u p β q 1 D n u p β ; β 1, (12) where D s he sze of he code alphabe. Inequales (9) and (12) are he generalzaon of Kraf s nequaly [30]. A code sasfyng generalzed Kraf s nequales (9) and (12) would be ermed as personal probably code. Theorem 1 For every code whose lenghs n 1, n 2,..., n sasfes (12), hen he average code lengh sasfes where α = 1 1+, he equaly occurs f and only f =1 L β(u) I β α(p, Q; U), (13) n = log q α. (14) =1 upβ q(α 1) =1 upβ

5 Ceran Codng Theorems Based on Generalzed Inaccuracy Measure 89 Proof By Holder s nequaly [31] ( n =1 x p ) 1 p ( n =1 y q )1 q n x y, (15) for all x, y > 0, = 1, 2,..., and 1 p + 1 q = 1, p < 1 ( 0), q < 0 or q < 1 ( 0), p < 0. We see ha equaly holds f and only f here exss a posve consan c such ha Makng he subsuons x = ( u p β =1 x p = cyq. (16) p =, q = + 1 ( ) 1 D n, y = p β(1+ n (15) and usng (12), we ge [ Dn ] 1 ) u [ q(α 1) ) ( 1+ ) q 1 ]1+ Takng logarhm of boh he sdes wh base D, we oban (13). ex, we oban a resul gvng an upper bound o he generalzed average useful codeword lengh. Theorem 2 By properly choosng he lenghs n 1, n 2,..., n n he code of Theorem 1, L β (U) can be made o sasfy he nequaly L β(u) < I β α(p, Q; U) + 1. (17) Proof Le n be he posve neger sasfyng, he nequales log Consder he nerval δ = q α =1 upβ q(α 1) =1 upβ log n < log q α, log =1 upβ q(α 1) =1 upβ q α =1 upβ q(α 1) =1 upβ q α =1 upβ q(α 1) =1 upβ + 1. (18) + 1 of lengh 1. In every δ, here s exacly one posve neger n such ha 0 < log q α =1 upβ q(α 1) =1 upβ n < log q α =1 upβ q(α 1) =1 upβ + 1. (19) I can be shown ha he sequence {n }, = 1, 2,..., hus defned, sasfes (12).

6 90 Sash Kumar and Arun Choudhary From he rgh nequaly of (19), we have n < log D n(1 α α ) < D ( 1 α q α q(α 1) α ) + 1 q α q(α 1) =1 upβ ( α 1 α ). (20) Mulplyng boh sdes of (20) by ( ) rasng o he power, we ge α 1 α [ Dn(1 α u p β =1, summng over = 1, 2,..., and afer ha u p β α ) ]( 1 α) α < D [ ]( 1 α) 1 q(α 1) Takng logarhms on boh sdes and usng he relaon α = 1 1+, we ge (17). Theorem 3 For arbrary, α > 0( 1), β 1 and for every code word lenghs n, = 1,..., of Theorem 1, L β (U) can be made o sasfy, L β(u) Iα(P, β Q; U) > Iα(P, β Q; U) + 1 LogD. (21) Proof Suppose n = log q α, α > 0( 1), β 1. =1 upβ q(α 1) =1 upβ Clearly n and n + 1 sasfy equaly n Holder s nequaly (15). Moreover, n sasfes (12). Suppose n s he unque neger beween n and n + 1, hen obvously, n sasfes (12). Snce α > 0( 1), β 1, we have [ ] [ =1 u p β ] [ Dn =1 =1 u p β u p β ] D n =1 =1 u p β < D u p β D n (22) [ ] =1 Snce upβd n =1 = u p β Hence (22) becomes [ P [ ] Dn =1 u p β =1 upβ q(α 1) ] 1+. [ ] =1 u p β 1+ q(α 1) =1 u p β < D [ ] =1 u p β 1+ q(α 1) Rasng o he power 1 and akng logarhms on boh sdes, we ge (21).

7 Ceran Codng Theorems Based on Generalzed Inaccuracy Measure 91 3 Lower Bound on he Exponenaed Average Useful Codeword Lengh for he Bes 1:1 Code Le X be a random varable akng on a fne number of values (x 1, x 2,..., x ) wh probables (p 1, p 2,..., p ) and ules (u 1, u 2,..., u ). Le n, = 1, 2,..., be he lenghs of he code words n he bes 1:1 bnary code (0, 1, 00, 10, 01, 11, 000,...), for encodng he random varable X, n s he lengh of he codeword assgned o he oupu x. I s clear ha n 1 n 2 n 3... n and n general n = [ ( log +2 )] 2 2, where [x] denoes he smalles neger greaer han or equal o x. Thus he average useful codeword lengh for he bes 1:1 code s gven by [ ( u p +2 )] log2 2 L 1:1 (U) = (23) u p When ules are gnored, (23) reduces o L 1:1, cf. [22]. From (11), he exponenaed average useful codeword lengh for bnary codes can be gven by [ L β(u) = 1 ] log 2n 2 =1 u p β (24) we may call (24) useful codeword lengh for he bes 1:1 code. Thus he exponenaed average useful codeword lengh for he bes 1:1 code s gven by [ L β,1:1(u) = 1 ] log 2[log 2( +2 2 )] 2 (25) We wll now prove he followng heorem, whch gves a lower bound on L β,1:1. Theorem 4 For Iα(P, β Q; U), L β (U) and L β,1:1 as gven n (10), (24) and (25) respecvely, he followng esmaes hold: ( ) L β,1:1(u) Iα(P, β =1 Q; U) log 2 u p β q , =1 u p β (26) and ( L β,1:1(u) L =1 β(u) log 2 u p β q ) 2. (27) Proof From (25), we have [ L β,1:1(u) 1 log 2 ( +2 2 ) ] (28)

8 92 Sash Kumar and Arun Choudhary ow ( ) + 1 Iα(P, β Q; U) L β,1:1(u) log 2 q ( +1) ( +2 2 [ 1 log 2 ) ] = log 2 q ( +1) ( +1 ) [ ( +2 2 ) ] 1 (29) Applyng Holder s nequaly o (29), we oban ( Iα(P, β Q; U) L =1 β,1:1(u) log 2 u p β q ), whch gves (26). ow from (17) So L β(u) < I β α(p, Q; U) + 1 L β(u) L β,1:1(u) < I β α(p, Q; U) L β,1:1(u) log 2 q 1 ( 1 +2 ), whch proves (27). 4 Concluson We sudy some codng heorems by consderng a new funcon dependng on he parameers α and β and a uly funcon. Our movaon for sudyng hs funcon s ha generalzes some nformaon measures already exsng n he leraure. We know ha opmal code s ha code for whch he value L β (U) s equal o s lower bound. From he resul of he Theorem 1, can be seen ha he mean codeword lengh of he opmal code s dependen on hree parameers, U and β, whle n he case of Shannon s heorem does no depend on any parameer. So can be reduced sgnfcanly by akng suable values of parameers. Acknowledgemen We are hankful o he anonymous referee for her valuable commens and suggesons.

9 Ceran Codng Theorems Based on Generalzed Inaccuracy Measure 93 References [1] Shannon, C. E. A mahemacal heory of communcaon. Bell Sysem Tech. J : , [2] Campbell, L. L. A codng heorem and Reny s enropy. Informaon and Conrol : [3] Kraf, L. G. A Devce for Quanzng Groupng and Codng Amplude Modulaed Pulses. Elecrcal Engneerng Deparmen MIT: Maser s Thess [4] Reny, A. On Measure of enropy and nformaon. Proc. 4h Berkeley Symp. Mahs. Sa. Prob : [5] Guasu, S. and Pcard, C. F. Borne nfercure de la longuerur ule de ceran codes. C.R. Acad. Sc. Ser. A-B : [6] Longo, G. Quanave-qualave measure of nformaon. ew York: Sprnger Verlag [7] Gurdal, S. and Pessoa, F. On useful nformaon of order α. J. Comb. nf. Sys. Sc : [8] Taneja, H. C. and Tueja, R. K. Characerzaon of quanave measure of naccuracy. Kyberneka (5): [9] Auar, R. and Khan, A. B. On generalzed useful nformaon for ncomplee dsrbuon. J. Comb. Inf. Sys. Sc (4): [10] Jan, P. and Tueja, R. K. On codng heorem conneced useful enropy of order β. J. Mah. Sc (1): [11] Hooda, D. S. and Bhaker, U. S. A generalzed useful nformaon measure and codng heorems. Soochow J. Mah : [12] Bhaa, P. K. On a generalzed useful naccuracy for ncomplee probably dsrbuon. Soochow J. Mah (2): [13] Sngh, R. P., Kumar, R. and Tueja, R. K. Applcaon of Holder s nequaly n nformaon heory. Informaon Scences : [14] Chapeau-Blondeau, F., Delahaes, A. and Rousseau, D. Source codng wh Tsalls enropy. Elecron. Le : [15] Chapeau-Blondeau, F., Rousseau, D. and Delahaes, A. Reny enropy measure of nose aded nformaon ransmsson n a bnary channel. Phys. Rev. E : (1) (10). [16] Ramamoorhy, A. Mnmum cos dsrbued source codng over a nework. IEEE Trans. Inf. Theory :

10 94 Sash Kumar and Arun Choudhary [17] Tu, G. F., Lu, J. J., Zhang, C. S., Gao, S. and L, S. D. Sudes and advances on jon source-channel encodng/decodng echnques n flow meda communcaon. Sc. Chna Inform. Sc : [18] Luo, M. X., Yang, Y. X., Wang, L. C. and u, X. X. Secure nework codng n he presence of eavesdroppers. Sc. Chna Inform. Sc : [19] Wu, W. R., Tu, J., Tu, G. F., Gao, S. S. and Zhang, C. Jon source channel VL codng/decodng for deep space communcaon neworks based on a space rells. Sc. Chna Inform. Sc : 117. [20] Abramson,. Informaon heory and codng. ew York: McGraw Hll [21] Ash, R. Informaon heory. ew York: Iner Scence Publsher [22] Leung-Yan, C. and Cover, T. Some equvalence beween Shannon enropy and Kolmogrov complexy. IEEE Trans. On Inform. Theory IT-24: [23] Bhaa, P. K. Useful naccuracy of order α and 1.1 codng. Soochow J. Mah (1): [24] Bhaa, P. K., Taneja, H. C. and Tueja, R. K. Inaccuracy and 1:1 code. Mcroelecroncs Relably (6): [25] Kerrdge, D. F. Inaccuracy and nference. J.R. Sa. Soc. Ser. B : [26] ah, P. An axomac characerzaon of naccuracy for dscree generalzed probably dsrbuon. Opsearch : [27] Aczel, J. and Daroczy, Z. Uber Verallgemenere quaslneare melwere, de m Gewchsfunkonen geblde snd. Publ. Mah. Debrecen : [28] Kapur, J.. Generalzed enropy of order α and ype β. Delh: Mahs. Semnar : [29] Khan, A. B. and Ahmad, H. Some noseless codng heorems of enropy of order α of he power dsrbuon P β. Meron (3-4): [30] Fensen, A. Foundaon of Informaon Theory. ew York: McGraw Hll [31] Shsha, O. Inequales. ew York: Academc Press