decoder t the coecient level, s opposed to the bit level, so it is fundmentlly dierent from schemes tht use informtion bout the trnsmitted dt to produ

Transcription

1 To pper in Proceedings of IEEE Dt Compression Conference 998 Lucent Technologies -- PROPRIETARY Optiml Multiple Description Trnsform Coding of Gussin Vectors Vivek K Goyl Dept. of Elec. Eng. & Comp. Sci. University of Cliforni, Berkeley v.goyl@ieee.org Jelen Kovcevic Bell Lbortories Murry Hill, NJ jelen@bell-lbs.com Abstrct Multiple description coding (MDC) is source coding for multiple chnnels such tht decoder which receives n rbitrry subset of the chnnels my produce useful reconstruction. Orchrd et l. [] proposed trnsform coding method for MDC of pirs of independent Gussin rndom vribles. This pper provides generl frmework which extends multiple description trnsform coding (MDTC) to ny number of vribles nd expnds the set of trnsforms which re considered. Anlysis of the generl cse is provided, which cn be used to numericlly design optiml MDTC systems. The cse of two vribles sent over two chnnels is nlyticlly optimized in the most generl setting where chnnel filures need not hve equl probbility or be independent. It is shown tht when chnnel filures re eqully probble nd independent, the trnsforms used in [] re in the optiml set, but mny other choices re possible. A cscde structure is presented which fcilittes low-complexity design, coding, nd decoding for system with lrge number of vribles. Introduction For decdes fter the inception of informtion theory, techniques for source nd chnnel coding developed seprtely. This ws motivted both by Shnnon's fmous \seprtion principle" nd by the conceptul simplicity of considering only one or the other. Recently, the limittions of seprte source nd chnnel coding hs led mny reserchers to the problem of designing joint source-chnnel (JSC) codes. An exmintion of Shnnon's result leds to the primry motivting fctor for constructing joint source-chnnel codes: The seprtion theorem is n symptotic result which requires innite block lengths (nd hence innite complexity nddely) t both source coder nd chnnel coder for prticulr nite complexity or dely, one cn often do better with JSC code. JSC codes hve lso drwn interest for being robust to chnnel vrition. Multiple description trnsform coding is technique which cn be considered JSC code for ersure chnnels. The bsic ide is to introduce correltion between trnsmitted coecients in known, controlled mnner so tht ersed coecients cn be sttisticlly estimted from received coecients. This correltion is used t the

2 decoder t the coecient level, s opposed to the bit level, so it is fundmentlly dierent from schemes tht use informtion bout the trnsmitted dt to produce likelihood informtion for the chnnel decoder. The ltter is common element of JSC coding systems. Our generl model for multiple description coding is s follows: A source sequence fx k g is input to coder, which outputs m strems t rtes R, R,...R m. These strems re sent onmseprte chnnels. There re mny receivers, nd ech receives subset of the chnnels nd uses decoding lgorithm bsed on which chnnels it receives. Speciclly, there re m ; receivers, one for ech distinct subset of strems except for the empty set, nd ech experiences some distortion. (This is equivlent to communicting with single receiver when ech chnnel my be working or broken, nd the sttus of the chnnel is known to the decoder but nottotheencoder.) This is resonble model for lossy pcket network. Ech \chnnel" corresponds to pcket or set of pckets. Some pckets my be lost, but becuse of heder informtion it is known which pckets re lost. An pproprite objective isto minimize weighted sum of the distortions subject to constrint on the totl rte. When m =, the sitution is tht studied in informtion theory s the multiple description problem [, 3, 4]. Denote the distortions when both chnnels re received, only chnnel is received, nd only chnnel is received by D 0, D,ndD, respectively. The clssicl problem is to determine the chievble (R R D 0 D D )- tuples. A complete chrcteriztion is known only for n i.i.d. Gussin source nd squred-error distortion [3]. This pper considers the cse where fx k g is n i.i.d. sequence of zero-men jointly Gussin vectors with known correltion mtrix R x = E[x k x T k ]. Distortion is mesured by the men-squred error (MSE). The technique wedevelop is bsed on squre, liner trnsforms nd simple sclr quntiztion, nd the design of the trnsform is prmount. Rther dissimilr methods hve been developed which use nonsqure trnsforms [5]. The problem could lso be ddressed with n emphsis on quntizer design [6, 7]. Proposed Coding Structure Since the source is jointly Gussin, we cn ssume without loss of generlity tht the components re independent. If not, one cn use Krhunen-Loeve trnsform of the source t the encoder nd the inverse t ech decoder. We propose the following steps for multiple description trnsform coding (MDTC) of source vector x:. x is quntized with uniform sclr quntizer with stepsize : x qi = [x i ], where [ ] denotes rounding to the nerest multiple of.. The vector x q =[x q x q ::: x qn ] T is trnsformed with n invertible, discrete trnsform ^T : Zn! Z n, y = ^T (xq ). The design nd implementtion of ^T re described below. For exmple, the internet, when UDP is used s opposed to TCP. The vectors cn be obtined by blocking sclr Gussin source.

3 3. The components of y re independently entropy coded. 4. If m<n, the components of y re grouped to be sent over the m chnnels. When ll the components of y re received, the reconstruction process is to (exctly) invert the trnsform ^T to get ^x = xq. The distortion is precisely the quntiztion error from Step. If some components of y re lost, they re estimted from the received components using the sttisticl correltion introduced by the trnsform ^T. The estimte ^x is then generted by inverting the trnsform s before. Strting with liner trnsform T with determinnt one, the rst step in deriving discrete version ^T is to fctor T into \lifting" steps [8]. This mens tht T is fctored into product of upper nd lower tringulr mtrices with unit digonls T = T T T k. The discrete version of the trnsform is then given by ^T (x q )= T T :::[T k x q ] : () The lifting structure ensures tht the inverse of ^T cn be implemented by reversing the clcultions in (): ^T ; (y) = h i T ; k ::: T ; T ; y : The fctoriztion of T is not unique for exmple, b 0 c +bc = 0 +bc; b b 0 ; b = ; c 0 0 c +bc; c 0 () Dierent fctoriztions yield dierent discrete trnsforms, except in the limit s pproches zero. The coding structure proposed here is generliztion of the method proposed by Orchrd, et l. []. In [], only trnsforms implemented in two lifting steps were considered. (By xing = in (), both fctoriztions reduce to hving two nonidentity fctors.) It is very importnt to note tht we rst quntize nd then use (discrete) trnsform. If we were to pply (continuous) trnsform rst nd then quntize, the use of nonorthogonl trnsform would led to noncubic prtition cells, which re inherently suboptiml mong the clss of prtition cells obtinble with sclr quntiztion [9]. The present congurtion llows one to use discrete trnsforms derived from nonorthogonl liner trnsforms, nd thus obtin better performnce []. 3 Anlysis of n MDTC System The nlysis nd optimiztions presented in this pper re bsed on ne quntiztion pproximtions. Speciclly, we mke three ssumptions which re vlid for smll : First, we ssume tht the sclr entropy of y = ^T ([x] ) is the sme s tht of [Tx]. Second, we ssume tht the correltion structure of y is unected by the : 3

4 quntiztion. Finlly, when t lest one component of y is lost, we ssume tht the distortion is dominted by the eect of the ersure, so quntiztion cn be ignored. Denote the vrinces of the components of x by,,..., n nd denote the correltion mtrix of x by R x = dig( ::: n). Let R y = TR x T T. In the bsence of quntiztion, R y would be exctly the correltion mtrix of y. Under our ne quntiztion pproximtions, we will use R y in the estimtion of rtes nd distortions. Estimting the rte is strightforwrd. Since the quntiztion is ne, y i is pproximtely the sme s [(Tx) i ], i.e., uniformly quntized Gussin rndom vrible. If we tret y i s Gussin rndom vrible with power y i =(R y ) ii quntized with bin width, we get for the entropy of the quntized coecient [0, Ch. 9] H(y i ) log e y i ; log = log y i + log e ; log = log y i + k where k 4 = (log e)= ; log nd ll logrithms re bse-two. Notice tht k depends only on. We thus estimte the totl rte s R = nx i= H(y i )=nk + log n Y i= y i : (3) The minimum rte occurs when Q n i= y i = Q n i= i nd t this rte the components of y re uncorrelted. Interestingly, T = I is not the only trnsform which chieves the minimum rte. In fct, n rbitrry split of the totl rte mong the dierent components of y is possible. This is justiction for using totl rte constrint in our following nlyses. However, we will py prticulr ttention to the cse where the rtes sent cross ech chnnel re equl. Wenow turn to the distortion, nd rst consider the verge distortion due only to quntiztion. Since the quntiztion noise is pproximtely uniform, this distortion is = for ech component. Thus the distortion when no components re ersed is given by D 0 = n (4) nd is independent of T. Now consider the cse when `>0 components re lost. We rst must determine how the reconstruction should proceed. By renumbering the vribles if necessry, ssume tht y, y, :::, y n;` re received nd y n;`+, :::, y n re lost. Prtition y into \received" nd \not received" portions s y =[~y r ~y nr ] T where ~y r =[y y ::: y n;`] T nd ~y nr = [y n;`+ ::: y n; y n ] T. The minimum MSE estimte of x given ~y r is E[xj~y r ], which hs simple closed form becuse x is jointly Gussin vector. Using the linerity of the expecttion opertor gives the following sequence of clcultions: ^x = E[xj~y r ] = E[T ; Txj~y r ] = T ; E[Txj~y r ] = T ; ~yr E ~y r = T ; ~y r : (5) ~y nr E[~y nr j~y r ] 4

5 If the correltion mtrix of y is prtitioned in wy comptible with the prtition of y s R y = TR x T T R B = B T R then it cn be shown tht ~y nr j~y r is Gussin with men B T R ; ~y r nd correltion mtrix A = 4 R ; B T R ; B. Thus E[~y nrj~y r ] = B T R ; ~y r, nd = 4 ~y nr ; E[~y nr j~y r ] is Gussin with zero men nd correltion mtrix A. is the error in predicting ~y nr from ~y r nd hence is the error cused by the ersure. However, becuse we hve used nonorthogonl trnsform, we must return to the originl coordintes using T ; in order to compute the distortion. Substituting ~y nr + for E[~y nr j~y r ] in (5) gives ^x = T ; ~y r ~y nr + = x + T ; 0 where U is the lst ` columns of T ;. Finlly, Ekx ; ^xk = so kx ; ^xk = `X i= `X j= 0 T ; = T U T U (U T U) ij A ij : (6) We denote the distortion with ` ersures by D`. To determine D` wemust nowverge (6) over the n ` possible ersures of ` components, weighted by their probbilities if necessry. Our nl distortion criterion is weighted sum of the distortions incurred with dierent numbers of chnnels vilble: D = nx `=0 `D`: For the cse where ech chnnel hs n outge probbility of p nd the chnnel outges re independent, the weighting ` = n p`( ; p) n;` mkes ` D the overll expected MSE. However, there re certinly other resonble choices for the weights. Consider n imge coding scenrio when n imge is split over ten pckets. One might wnt cceptble imge qulity s long s eight or more pckets re received. In this cse, one should set 3 = 4 = = 0 =0. For given rte R, our gol is to minimize D. The expressions given in this section cn be used to numericlly determine trnsforms to relize this gol. Anlyticl solutions re possible in certin specil cses. Some of these re given in the following section. 4 Sending Two Vribles Over Two Chnnels Generl cse Let us now pply the nlysis of the previous section to nd the best trnsforms for sending n = vribles over m = chnnels. In the most generl sitution, chnnel outges my hve unequl probbilities nd my be dependent. Suppose the probbilities of the combintions of chnnel sttes re given by the following tble: 5

6 Let T = c Chnnel broken working Chnnel broken ; p0 ; p ; p p working p p0, normlized so tht det T =. Then T ; = b d R y = TR x T T = + b c + bd c + bd c + d : d ;b ;c nd By (3), the totl rte is given by R =k + log(r y) (R y ) =k + log( + b )(c + d ): (7) Minimizing (7) over trnsforms with determinnt one gives minimum possible rte of R =k + log. We refer to = R ; R s the redundncy [], i.e., the price we py in rte in order to potentilly reduce the distortion when there re ersures. In order to evlute the overll verge distortion, we must form weighted verge of the distortions for ech of the four possible chnnel sttes. If both chnnels re working, the distortion (due to quntiztion only) is D 0 = =6. If neither chnnel is working, the distortion is D = +. The remining cses require the ppliction of the results of the previous section. We rst determine D, the MSE distortion when y is received but y is lost. Substituting in (6), ;b D = {z } (U T U) (R y ) ; (R y) (R y ) {z } A =( + b ) + b where we hve used det T = d;bc = in the simpliction. Similrly, the distortion when y is received but y is lost is D =(c + d ) =(c + d ). The overll verge distortion is D = p 0 D 0 + p D + p D +(; p 0 ; p ; p ) D = p 0 6 +(; p 0 ; p ; p )( + ) p + p D + D p {z } D 0 where the rst brcketed term is independent of T. Thus our optimiztion problem is to minimize D 0 for given redundncy. If the source hs circulrly symmetric probbility density, i.e., =, then D 0 =(+p =p ) independent of T. Henceforth we ssume >. After eliminting d through d =( + bc)=, one cn show tht the optiml trnsform will stisfy h jj = p p i ; + ; ; 4bc(bc +) : c 6

7 Furthermore, D 0 depends only on the product b c, not on the individul vlues of b nd c. The optiml vlue of bc is given by " (bc) optiml = ; + # ;= p p p ; + ; 4 ; : p p p It is esy to check tht (bc) optiml rnges from - to 0 s p =p rnges from 0 to. The limiting behvior cn be explined s follows: Suppose p p, i.e., chnnel is much more relible thn chnnel. Since (bc) optiml pproches 0, d must pproch, nd hence one optimlly sends x (the lrger vrince component) over chnnel (the more relible chnnel), nd vice-vers. This is the intuitive, lyered solution. The multiple description pproch is most useful when the chnnel filure probbilities re comprble, but this demonstrtes tht the multiple description frmework subsumes lyered coding. Equl chnnel filure probbilities If p = p, then (bc) optiml = ;=, independent of. The optiml set of trnsforms is described by 6= 0 (but otherwise rbitrry) c = ;=b b = ( ; p ; ) = d = = (8) nd using trnsform from this set gives D = (D + D )= ; ; ; p ; ( ; ): (9) This reltionship is plotted in Figure (). Notice tht, s expected, D strts t mximum vlue of ( + )= nd symptoticlly pproches minimum vlue of =. By combining (3), (4), nd (9), one cn nd the reltionship between R, D 0, nd D. For vrious vlues of R, the trde-o between D 0 nd D is plotted in Figure (b). The solution for the optiml set of trnsforms (8) hs n interesting property tht fter xing, there is n \extr" degree of freedom which does not ect the vs. D performnce. This degree of freedom cn be used to control the prtitioning of the rte between chnnels or to give simplied implementtion. Optimlity of Orchrd et l. trnsforms In [] it is suggested to use trnsforms b of the form. As result of our nlysis we conclude tht these ;=(b) = trnsforms in fct lie in the optiml set of trnsforms. The \extr" degree of freedom hs been used by xing =, which yields trnsform which cn be fctored into two lifting steps in the generl cse three lifting steps re needed. Optiml trnsforms tht give blnced rtes The trnsforms of [] do not give chnnels with equl rte (or, equivlently, power). In prctice, this cn be remedied 7

8 0.6.5 R = 8 R = 7 R = 6 R = 5 R = D (MSE) D (MSE in db) Redundncy (bits/vector) () D0 (MSE in db) (b) Figure : Optiml R-D 0 -D trde-os for =, =0:5: () Reltionship between redundncy nd D (b) Reltionship between D 0 nd D for vrious rtes. through time-multiplexing. An lterntive is to use the \extr" degree of freedom to mke R = R. Doing this is equivlent to requiring jj = jcj nd jbj = jdj, nd yields = s ; ; p ; b = = s ; ; p ; : In the next section, when wepplytwo-by-two correlting trnsform, we will ssume blnced-rte trnsform. Speciclly, we will use T 4 =() =. ; =() Geometric interprettion The trnsmitted representtion of x is given by y = hx ' i nd y = hx ' i, where ' = [ b] T nd ' = [c d] T. In order to gin some insight into the vectors ' nd ' tht result in n optiml trnsform, let us neglect the rte nd distortion tht re chieved, nd simply consider the trnsforms described by d = = nd bc = ;=. We cn show tht ' nd ' form the sme (bsolute) ngles with the positive x -xis (see Figure ()). For convenience, suppose > 0 nd b < 0. Then c d > 0. Let nd be the ngles by which ' nd ' re below nd bove the positive x -xis, respectively. Then tn = ;b= = d=c = tn. If we ssume >, then the mximum ngle (for = 0) is rctn( = ) nd the minimum ngle (for! ) is zero. This hs the nice interprettion of emphsizing x over x becuse it hs higher vrince s the coding rte is incresed (see Figure (b)). 5 Three or More Vribles Three vribles over three chnnels Applying the results of Section 3 to the design of 3 3 trnsforms is considerbly more complicted thn wht hs been 8

9 x x d = ;b ' d ' c x = c x b ' b ' () (b) Figure : Geometric interprettions. () When >, the optimlity condition (d = =, bc = ;=) is equivlent to = < mx = rctn( = ). (b) If in ddition to the optimlity condition we require the output strems to hve equl rte, the nlysis vectors re symmetriclly situted to cpture the dimension with gretest vrition. At = 0, = = mx s!, ' nd ' close on the x -xis. presented thus fr. Even in the cse of equl chnnel filures, closed form solution will be much more complicted tht (8). When > > 3 nd ersure probbilities re equl nd smll, set of trnsforms which gives ner optiml performnce is described by p ; 3 0 p 3 ; 6 p 3 6 p 3 ; 6 p 3 A derivtion of this set must be omitted for lck of spce. Four nd more vribles For sending ny even number of vribles over two chnnels, Orchrd et l. [] hve suggested the following: form pirs of vribles, dd correltion within ech pir, nd send one vrible from ech pir cross ech chnnel. A necessry condition for optimlity is tht ll the pirs re operting t the sme distortion-redundncy slope. If T is used to trnsform vribles with vrinces nd nd T is used to trnsform vribles with vrinces 3 nd 4, then the equl-slope condition implies tht we should hve p 4 = (68 4 ; )+ 4 (6 8 4 ; ) where = 3 4( 3 ; 4) ( ; ) : Finding the optiml trnsform under this piring constrint still requires nding the optiml piring. Cscde Structures In order to extend these schemes to n rbitrry number of chnnels while mintining resonble ese of design, we propose the cscde use of :

10 x y T T x y x 3 y 3 T T x 4 y 4 Figure 3: Cscde structure llows ecient MDTC of lrge vectors. piring trnsforms s shown in Figure 3. The cscde structure simplies the encoding, decoding (both with nd without ersures), nd design when compred to using generl n n trnsform. Empiricl evidence suggests tht for n = m = 4 nd considering up to one component ersure, there is no performnce penlty in restricting considertion to cscde structures. This phenomenon is under investigtion. As expected, there is gret improvement over simple piring of coecients piring cn not be expected to be ner optiml for m lrger thn two. References [] M. T. Orchrd, Y. Wng, V. Vishmpyn, nd A. R. Reibmn. Redundncy rte-distortion nlysis of multiple description coding using pirwise correlting trnsforms. In Proc. IEEE Int. Conf. Imge Proc [] J. K. Wolf, A. D. Wyner, nd J. Ziv. Source coding for multiple descriptions. Bell Syst. Tech. J., 59(8):47{46, 980. [3] L. Ozrow. On source-coding problem with two chnnels nd three receivers. Bell Syst. Tech. J., 59(0):909{9, 980. [4] A. A. El Gml nd T. M. Cover. Achievble rtes for multiple descriptions. IEEE Trns. Info. Theory, IT-8(6):85{857, 98. [5] V. K Goyl, J. Kovcevic, nd M. Vetterli. Multiple description trnsform coding: Robustness to ersures using tight frme expnsions. In Proc. IEEE Int. Symp. Info. Th., August 998. Submitted. [6] V. A. Vishmpyn. Design of multiple description sclr quntizers. IEEE Trns. Info. Theory, 39(3):8{84, 993. [7] J.-C. Btllo nd V. A. Vishmpyn. Asymptotic performnce of multiple description trnsform codes. IEEE Trns. Info. Theory, 43():703{707, 997. [8] I. Dubechies nd W. Sweldens. Fctoring wvelet trnsforms into lifting steps. Technicl report, Bell Lbortories, Lucent Technologies, September 996. [9] A. Gersho nd R. M. Gry. Vector Quntiztion nd Signl Compression. Kluwer Acd. Pub., Boston, MA, 99. [0] T. M. Cover nd J. A. Thoms. Elements of Informtion Theory. John Wiley & Sons, New York, 99. 0