Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE

Size: px

Start display at page:

Download "Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE"

Miranda Merritt
5 years ago
Views:

1 Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2004 Parma, Italy, October 10 13, 2004 Vector Permutatio Code Desig Algorithm Dailo SILVA ad Weiler A. FINAMORE Cetro de Estudos em Telecomuicações Potifícia Uiversidade Católica do Rio de Jaeiro, Rua Marquês de São Vicete, Rio de Jaeiro, RJ - Brazil {dailo, weiler}@cetuc.puc-rio.br Abstract Permutatio codes are a class of vector quatizers with a codebook structure based o permutatios. Their mai advatage is to geerate a fixed-rate output, thus avoidig the eed of etropy codig ad bufferig strategies typical of etropy-costraied quatizers. This paper deals with the extesio of (scalar) permutatio codes to the vector case. We have show that vector permutatio codes with log block-legth are asymptotically equivalet to etropy-costraied vector quatizers, i the rate-distortio sese. Based o this coectio we propose a algorithm for geeratig good vector permutatio codes. We preset results for uiform ad Gaussia sources which cofirm that VPC s achieve performace similar to ECVQ; i particular, they are superior to optimum ECSQ ad SPC s. 1. INTRODUCTION A permutatio code for source compressio is a vector quatizer with a codebook cosistig of all the distict permutatios of a referece codeword. The mai advatage of a permutatio code is the geeratio of a fixed-rate output by meas of a simple ecodig operatio [1], thus avoidig the eed of etropy codig ad bufferig strategies. Sice its itroductio i 1965 [2], the theory of permutatio codes has always cosidered the codewords to be composed by scalar elemets, ad, although simple to ecode, these (scalar) permutatio codes have a performace that is tied to that of etropy-costraied scalar quatizers (ECSQ) [1]. A recet paper [3] exteds this idea of scalar permutatio codes (SPC) to a higher-dimesioal case, where the codewords are composed by vectors, ad also provides a umerical method for costructig the referece codeword (uder the assumptio that aother parameter called the compositio vector is give). This work was partially supported by the Brazilia Natioal Research Coucil (CNPq). I the preset paper we show that the performace of vector permutatio codes (VPC) asymptotically approaches that of etropy-costraied vector quatizers (ECVQ). Based o this coectio we propose a algorithm for desigig of compositio vector which, combied with the codebook desig algorithm, provides a complete VPC desig procedure. We also exhibit some results for uiform ad Gaussia sources evidecig that VPC s (desiged by the proposed method) ca achieve better performaces tha ECSQ ad SPC. This paper is orgaized as follows. Sectio 2 presets the defiitios of vector permutatio codes. I sectio 3, a theorem is prove o the asymptotically equivalece of VPC ad ECVQ. A procedure for VPC desig is proposed i Sectio 4. Our experimetal results are show i Sectio 5, ad Sectio 6 presets the coclusios of this work. 2. VECTOR PERMUTATION CODES A L-dimesioal vector permutatio code is a structured fixed-rate vector quatizer of block-legth L, with a codebook C[y 0 ] = {y l } M 1 l=0 cosistig of all the distict permutatios of a referece codeword y 0 = (µ 1,..., µ }{{} 1, µ 2,..., µ }{{} 2 1 2,..., µ K,..., µ K }{{} K ) (1) of legth whose elemets are L-dimesioal vectors i R L. The sets {µ i } K i=1 ad { i} K i=1 are called, respectively, the user alphabet ad the compositio vector. The rate of this code, i bits per scalar sample, is where R = 1 log M (2) L M = is the size of the codebook.! K i=1 i! (3) 753

2 With the optimal ecodig scheme, a source vector x = (x 1... x ), x m R L, is reproduced by the permutatio of y 0 that is closer to x, i.e., by the codeword π(y 0 : x) = arg mi d(x, y), (4) y C[y 0] where d(x, y) is the per-sample distortio betwee x ad y. The average per-sample distortio of the overall scheme is thus D = E[d(X, π(y 0 : X))]. (5) If the distortio measure is the squared-error criterio, the D = 1 L E[ X, π(y 0 : X) 2 ]. (6) 3. EQUIVALENCE OF VPC AND ECVQ Cosider a L-dimesioal ECVQ with a cardiality K codebook. Its ecodig rule maps the source vector x R L to the codeword that miimizes a weighted sum of distortio ad output etropy [4]. Thus a optimal ECVQ ca be characterized by a set of codewords {µ i } K i=1 ad their correspodig probabilities {p i} K i=1. Now the followig theorem establishes the asymptotical equivalece of VPC ad ECVQ. Theorem 1: Let X be a L-dimesioal radom vector, ad let {X m } be a sequece of i.i.d. radom vectors with the same distributio as X. Assume that the distortio measure is a metric. For ay give ECVQ ({µ i } K i=1, {p i} K i=1 ) such that both K ad max i,j d(µ i, µ j ) are fiite, ad for ay give ɛ > 0, if the ECVQ ecodes X with rate R ad distortio D, the there exists a sequece {C } of L-dimesioal VPC s of block legth, = 1, 2,..., that ecode X = (X 1,..., X ) with correspodig rates R ad per-sample average distortios D that satisfy both lim R = R ad lim D D + ɛ. Proof: For all, set the VPC user alphabet equal to the ECVQ codebook {µ i } K i=1, ad let the VPC compositio vector { i } K i=1 be chose i such a way that lim i = p i. By the Stirlig s formula, it follows easily that lim R = 1 L K p i log p i = R. (7) i=1 Let ow Y be the sequece of ECVQ codewords geerated by ecodig each compoet of X with ECVQ. Thus, Y is a sequece of symbols draw i.i.d from a alphabet Y = {µ i } K i=1 accordig to a distributio {p i } K i=1. Let N(µ i Y ) be the umber of occurreces of µ i i Y, ad defie the set of strogly typical sequeces [5, p.288] T () = {Y Y : N(µ i Y ) p i <, i = 1,..., K}. (8) Note that if Y T (), the it must differ from oe of the VPC codewords i at most r K( +1) symbols. Now cosider the followig approach to VPC ecodig: if for ay i, N(µ i Y ) > i, the a ew sequece is geerated from Y by arbitrarily replacig oe µ i by some µ j for which N(µ j Y ) < j. This procedure is repeatedly applied to the ew sequece util o further substitutio is required. By defiitio, the resultig sequece Z will be a VPC codeword. Next, we obtai the average distortio for this scheme. The per-sample distortio betwee sequeces is d(x, Z ) = 1 d(x j, Z j ) (9) j=1 where d(x j, Z j ) is the per-sample distortio betwee the symbols X j ad Z j. Also, sice d(, ) satisfies the triagle iequality, we have: D = E [d(x, Z )] E [d(x, Y )] + E [d(y, Z )] = D + D (10) where D is the pealty (icrease i distortio) paid for ecodig with VPC istead of ECVQ. To evaluate D, we first obtai a boud o d(y, Z ): d(y, Z ) = 1 d(y j, Z j ) j=1 r d max, d max, if Y T () if Y / T () (11) where d max = max i,j d(µ i, µ j ) is a determiistic costat. If we let A be the evet {Y T () }, the D ca be writte as D = E [ d(y, Z ) ] A P (A ) +E [ d(y, Z ) ] A P ( A ) (12) Substitutig (11) ad the boud o r, we obtai D K( + 1) d max P (A ) + d max P ( A ) (13) ad, by the law of large umbers, Fially, we have lim D Kd max = ɛ. (14) lim D D + lim D D + ɛ (15) ad the proof is complete. 754

3 4. VPC DESIGN ALGORITHM 4.1. Codebook Desig The user alphabet ad the compositio vector sets are the two parameters that specify a VPC. Cosiderig sources that ca be characterized by a traiig set T = {x (1),..., x (t),..., x (T ) }, (16) a Lloyd-like algorithm ca be used for (locally) optimum codebook desig, if the compositio vector is give. Such algorithm appeared i [3] ad is reproduced below: Codebook Desig Algorithm 1. Iput { i } K i=1 ad the iitial {µ i} K i=1. 2. Set S i = i j=1 j. 3. Calculate 1 : a) y 0 = (µ 1,..., µ 1,..., µ K,..., µ K ) b) z = (z 1... z ) = 1 T T t=1 π(x(t) : y 0 ) c) µ i = 1 i Si j=1+s i 1 z j 4. Repeat step 3 util {µ i } K i=1 coverges. 5. Doe Compositio Vector Desig Fidig good permutatio codes amouts thus to fidig good compositio vectors a problem which, due to the overwhelmig umber of such vectors, is ot practical to be solved with a exhaustive search. The followig algorithm makes use of the derived relatioship betwee VPC ad ECVQ to facilitate this task. VPC Desig Algorithm 1. Iput, L, K, ad a target rate R. 2. Desig a L-dimesioal ECVQ with K codewords for a rate R, thus obtaiig a codebook {µ i } K i=1 with codeword probabilities {p i} K i=1. 3. Set i p i with a appropriate roudig rule. 4. Set {µ i } K i=1 as the iitial VPC user alphabet. 5. Update the user alphabet with the Codebook Desig Algorithm. 6. Doe. 1 Oe should otice that π(x (t) : y 0 ) meas that x (t) is to be permuted with respect to y Complexity If is sufficietly large, the 5th step i the above algorithm ca be omitted without compromisig the attaiable distortio. With this modificatio, the complexity of the VPC desig equals that of a ECVQ desig. O the other had, if the algorithm of sectio 4.1 must be used, the overall complexity will deped o that of the ecodig operatio (4). Equatio (4) ca be recogized as the optimal solutio to a weighted bipartite matchig problem which, for L 2, has a complexity of O( 2 log ) [6]. This is substatially differet from the special case L = 1 (SPC), where the ecodig operatio is a simple sortig procedure, with complexity O( log ) [7]. If the optimal VPC ecodig is prohibitive for a applicatio, a sub-optimal but less complex procedure ca be pursued. Oe such possibility, as give i Theorem 1, is to perform the VPC ecodig by firstly geeratig the sequece Y with ECVQ ecodig, the arbitrarily replacig the exceedig symbols i order to covert it to a VPC codeword. The complexity of this scheme is oly liear with, ad the resultig icrease i distortio is egligible whe is large. 5. EXPERIMENTAL RESULTS This sectio presets some results obtaied with the proposed method of sectio 4.2 for VPC desig ad the optimal ecodig operatio whe assessig the performace. Fig. 1 shows a compariso betwee VPC s of various dimesios ad the optimum ECSQ for a uiform memoryless source with a squared-error distortio measure. I each curve, the VPC s were obtaied for various values of parameters K ad (up to = 100 for L = 10). The rate-distortio fuctio is also plotted i the figure. We use traiig ad test sequeces of samples each, ad the geeralized Lloyd algorithm for the ECVQ desig. Therefore, all VPC s i this figure were actually obtaied from full rate VQ s (or ECVQ s with λ = 0 i the otatio of [4]). Figs. 2, 3 ad 4 show aalogous comparisos for a Gaussia memoryless source. Here we used the Chou et al. s iterative algorithm for ECVQ desig [4]. The figures also show the performaces of the obtaied ECVQ s; for each of them, a correspodig VPC was desiged without codebook updatig, ad its performace was calculated. Traiig ad test sequeces of L-dimesioal samples were used i the simulatios. Performace of SPC is ot show i the figure, but, as reported by Berger [7], its curve is kow to lie above that of the optimum ECSQ. As it ca be see i Fig. 3, the performace of VPC improves whe block-legth is icreased. The fact that 755

4 0.07 ECVQ (L = 2) Distortio Fuctio 0.06 SPC Distortio 0.05 VPC (L = 3) Distortio = 500 = 100 = VPC (L = 10) 0.03 Distortio Fuctio Figure 1: Compariso betwee VPC s with L = 1, 2, 3 ad 10 ad the optimum ECSQ for a uiforme source. Figure 3: Performace of a 2-dimesioal VPC versus block-legth. ECVQ (L = 2) Distortio Fuctio ECVQ (L = 3) VPC (L = 3) Distortio Fuctio Distortio Distortio Figure 2: Compariso betwee 2-dimesioal VPC ad ECVQ for a Gaussia source. The performace of the optimum ECSQ is also show. for Gaussia sources we have actually obtaied VPC s with better performace tha ECVQ is probably 2 because the use of Chou et al. s algorithm results, i geeral, i ECVQ s which are ot globally optimal [8]. 6. CONCLUSIONS The theory of permutatio codes has bee exteded to the vector case. It is a kow fact that the perfor- 2 However, it should be emphasized that the Theorem costructio was based o sub-optimal VPC ecodig, ad o coverse was prove. Therefore, it remais a ope questio whether the performace of optimum VPC ca surpass that of optimum ECVQ or ot. Figure 4: Compariso betwee 3-dimesioal VPC ad ECVQ for a Gaussia source. The performace of the optimum ECSQ is also show. mace of scalar permutatio codes approaches that of ECSQ whe block-legth teds to ifiity. I this paper, we prove the vector couterpart: the asymptotical equivalece of vector permutatio codes ad ECVQ. This coectio is ot oly theoretical, but it provides us with a practical scheme for the desig of VPC s. By meas of the desig algorithm preseted, we have obtaied some rate-distortio poits for uiform ad Gaussia sources which show that VPC s with fiite block-legth ca be as good as ECVQ at low to medium rates. We are curretly ivestigatig techiques to improve the performace of VPC s for small block-legth. 756

5 Refereces [1] T. Berger, Optimum quatizers ad permutatio codes, IEEE Trasactios o Iformatio Theory, vol. 18, pp , [2] D. Slepia, Permutatio modulatio, Proceedigs of the IEEE, vol. 53, pp , [3] W. A. Fiamore, S. V. B. Bruo, ad D. Silva, Vector permutatio ecodig for the uiform sources, Proc. of the Data Compressio Coferece, p. 539, March [4] P. A. Chou, T. Lookabaugh, ad R. M. Gray, Etropy costraied vector quatizatio, IEEE Trasactios o Iformatio Theory, vol. 37, pp , [5] T. M. Cover ad J. A. Thomas, Elemets of Iformatio Theory. New York: Wiley, [6] A. V. Goldberg ad R. Keedy, A efficiet cost scalig algorithm for the assigemet problem, tech report, Staford uiversity, CS Departmet, Staford, CA, May [7] T. Berger, F. Jeliek, ad J. K. Wolf, Permutatio codes for sources, IEEE Trasactios o Iformatio Theory, vol. 18, pp , [8] J. C. Kieffer, A survey of the theory of source codig with a fidelity criterio, IEEE Trasactios o Iformatio Theory, vol. 39, pp ,

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE Joural of ELECTRICAL EGIEERIG, VOL. 56, O. 7-8, 2005, 200 204 OPTIMAL PIECEWISE UIFORM VECTOR QUATIZATIO OF THE MEMORYLESS LAPLACIA SOURCE Zora H. Perić Veljo Lj. Staović Alesadra Z. Jovaović Srdja M.