Differential Coding 3.1 INTRODUCTION TO DPCM SIMPLE PIXEL-TO-PIXEL DPCM

Transcription

1 3 Dfferental Codng Instead of encodng a sgnal drectly, the dfferental codng technque codes the dfference between the sgnal tself and ts predcton. Therefore t s also known as predctve codng. By utlzng spatal and/or temporal nterpxel correlaton, dfferental codng s an effcent and yet computatonally smple codng technque. In ths chapter, we frst descrbe the dfferental technque n general. Two components of dfferental codng, predcton and quantzaton, are dscussed. There s an emphass on (optmum) predcton, snce quantzaton was dscussed n Chapter 2. When the dfference sgnal (also known as predcton error) s quantzed, the dfferental codng s called dfferental pulse code modulaton (DPCM). Some ssues n DPCM are dscussed, after whch delta modulaton (DM) as a specal case of DPCM s covered. The dea of dfferental codng nvolvng mage sequences s brefly dscussed n ths chapter. More detaled coverage s presented n Sectons III and IV, startng from Chapter 10. If quantzaton s not ncluded, the dfferental codng s referred to as nformaton-preservng dfferental codng. Ths s dscussed at the end of the chapter. 3.1 INTRODUCTION TO DPCM As depcted n Fgure 2.3, a source encoder conssts of the followng three components: transformaton, quantzaton, and codeword assgnment. The transformaton converts nput nto a format for quantzaton followed by codeword assgnment. In other words, the component of transformaton decdes whch format of nput s to be encoded. As mentoned n the prevous chapter, nput tself s not necessarly the most sutable format for encodng. Consder the case of monochrome mage encodng. The nput s usually a 2-D array of gray level values of an mage obtaned va PCM codng. The concept of spatal redundancy, dscussed n Secton , tells us that neghborng pxels of an mage are usually hghly correlated. Therefore, t s more effcent to encode the gray dfference between two neghborng pxels nstead of encodng the gray level values of each pxel. At the recever, the decoded dfference s added back to reconstruct the gray level value of the pxel. Snce neghborng pxels are hghly correlated, ther gray level values bear a great smlarty. Hence, we expect that the varance of the dfference sgnal wll be smaller than that of the orgnal sgnal. Assume unform quantzaton and natural bnary codng for the sake of smplcty. Then we see that for the same bt rate (bts per sample) the quantzaton error wll be smaller,.e., a hgher qualty of reconstructed sgnal can be acheved. Or, for the same qualty of reconstructed sgnal, we need a lower bt rate SIMPLE PIXEL-TO-PIXEL DPCM Denote the gray level values of pxels along a row of an mage as z, = 1,L,M, where M s the total number of pxels wthn the row. Usng the mmedately precedng pxel s gray level value, z 1, as a predcton of that of the present pxel, ẑ,.e., ẑ = z -1 (3.1) we then have the dfference sgnal ˆ 1 d = z - z = z -z - (3.2)

2 FIGURE 3.1 (a) Hstogram of the orgnal boy and grl mage. (b) Hstogram of the dfference mage obtaned by usng horzontal pxel-to-pxel dfferencng. (c) A close-up of the central porton of the hstogram of the dfference mage. Assume a bt rate of eght bts per sample n the quantzaton. We can see that although the dynamc range of the dfference sgnal s theoretcally doubled, from 256 to 512, the varance of the dfference sgnal s actually much smaller. Ths can be confrmed from the hstograms of the boy and grl mage (refer to Fgure 1.1) and ts dfference mage obtaned by horzontal pxel-to-pxel dfferencng, shown n Fgure 3.1(a) and (b), respectvely. Fgure 3.1(b) and ts close-up (c) ndcate that by a rate of 42.44% the dfference values fall nto the range of 1, 0, and +1. In other words, the hstogram of the dfference sgnal s much more narrowly concentrated than that of the orgnal sgnal.

3 FIGURE 3.2 Block dagram of a pxel-to-pxel dfferental codng system. A block dagram of the scheme descrbed above s shown n Fgure 3.2. There z denotes the sequence of pxels along a row, d s the correspondng dfference sgnal, and ˆd s the quantzed verson of the dfference,.e., ˆd = Q( d) = d + eq (3.3) where e q represents the quantzaton error. In the decoder, z represents the reconstructed pxel gray value, and we have z = z + dˆ -1 (3.4) Ths smple scheme, however, suffers from an accumulated quantzaton error. We can see ths clearly from the followng dervaton (Sayood, 1996), where we assume the ntal value z 0 s avalable for both the encoder and the decoder. as =, d = z -z dˆ = d + e 1 1 q, 1 (3.5) z = z + dˆ = z + d + e = z + e q, 1 1 q, 1 Smlarly, we can have as = z = z + e + e 2, 2 2 q, 1 q, 2 (3.6) and, n general, Â q, j j = 1 z = z + e (3.7) Ths problem can be remeded by the followng scheme, shown n Fgure 3.3. Now we see that n both the encoder and the decoder, the reconstructed sgnal s generated n the same way,.e., z = z + dˆ -1 (3.8) and n the encoder the dfference sgnal changes to d = z -z -1 (3.9)

4 FIGURE 3.3 Block dagram of a practcal pxel-to-pxel dfferental codng system. Thus, the prevously reconstructed z 1 s used as the predctor, ẑ,.e., zˆ z. = -1 (3.10) In ths way, we have as =, d = z -z dˆ = d + e 1 1 q, 1 (3.11) z = z + dˆ = z + d + e = z + e q, 1 1 q, 1 Smlarly, we have as =, d = z -z dˆ = d + e 2 2 q, 2 (3.12) z = z + dˆ = z + e q, 2 In general, z = z + eq, (3.13) Thus, we see that the problem of the quantzaton error accumulaton has been resolved by havng both the encoder and the decoder work n the same fashon, as ndcated n Fgure 3.3, or n Equatons 3.3, 3.9, and GENERAL DPCM SYSTEMS In the above dscusson, we can vew the reconstructed neghborng pxel s gray value as a predcton of that of the pxel beng coded. Now, we generalze ths smple pxel-to-pxel DPCM. In a general DPCM system, a pxel s gray level value s frst predcted from the precedng reconstructed pxels gray level values. The dfference between the pxel s gray level value and the predcted value s then quantzed. Fnally, the quantzed dfference s encoded and transmtted to the recever. A block

5 FIGURE 3.4 Block dagram of a general DPCM system. dagram of ths general dfferental codng scheme s shown n Fgure 3.4, where the codeword assgnment n the encoder and ts counterpart n decoder are not ncluded. It s noted that, nstead of usng the prevously reconstructed sample, z 1, as a predctor, we now have the predcted verson of z, ẑ, as a functon of the n prevously reconstructed samples, z 1, z 2, L, z n. That s, zˆ f z, z,, z = ( ) L n (3.14) Lnear predcton,.e., that the functon f n Equaton 3.14 s lnear, s of partcular nterest and s wdely used n dfferental codng. In lnear predcton, we have n = Âajz- j j = 1 ẑ (3.15) where a j are real parameters. Hence, we see that the smple pxel-to-pxel dfferental codng s a specal case of general dfferental codng wth lnear predcton,.e., n = 1 and a 1 = 1. In Fgure 3.4, d s the dfference sgnal and s equal to the dfference between the orgnal sgnal, z, and the predcton ẑ. That s, d = z -zˆ (3.16) The quantzed verson of d s denoted by ˆd. The reconstructed verson of z s represented by z, and z = zˆ + dˆ (3.17) Note that ths s true for both the encoder and the decoder. Recall that the accumulaton of the quantzaton error can be remeded by usng ths method. The dfference between the orgnal nput and the predcted nput s called predcton error, whch s denoted by e p. That s, ep = z -zˆ (3.18) where the e p s understood as the predcton error assocated wth the ndex. Quantzaton error, e q, s equal to the reconstructon error or codng error, e r, defned as the dfference between the orgnal sgnal, z, and the reconstructed sgnal, z, when the transmsson s error free:

6 e = d -dˆ q ( ) - ( - ) = z -zˆ z zˆ = z - z = e r (3.19) Ths ndcates that quantzaton error s the only source of nformaton loss wth an error-free transmsson channel. The DPCM system depcted n Fgure 3.4 s also called closed-loop DPCM wth feedback around the quantzer (Jayant, 1984). Ths term reflects the feature n DPCM structure. Before we leave ths secton, let us take a look at the hstory of the development of dfferental mage codng. Accordng to an excellent early artcle on dfferental mage codng (Musmann, 1979), the frst theoretcal and expermental approaches to mage codng nvolvng lnear predcton began n 1952 at the Bell Telephone Laboratores (Olver, 1952; Kretzmer, 1952; Harrson, 1952). The concepts of DPCM and DM were also developed n 1952 (Cutler, 1952; Dejager, 1952). Predctve codng capable of preservng nformaton for a PCM sgnal was establshed at the Massachusetts Insttute of Technology (Elas, 1955). The dfferental codng technque has played an mportant role n mage and vdeo codng. In the nternatonal codng standard for stll mages, JPEG (covered n Chapter 7), we can see that dfferental codng s used n the lossless mode and n the DCT-based mode for codng DC coeffcents. Moton-compensated (MC) codng has been a major development n vdeo codng snce the 1980s and has been adopted by all the nternatonal vdeo codng standards such as H.261 and H.263 (covered n Chapter 19), MPEG 1 and MPEG 2 (covered n Chapter 16). MC codng s essentally a predctve codng technque appled to vdeo sequences nvolvng dsplacement moton vectors. 3.2 OPTIMUM LINEAR PREDICTION Fgure 3.4 demonstrates that a dfferental codng system conssts of two major components: predcton and quantzaton. Quantzaton was dscussed n the prevous chapter. Hence, n ths chapter we emphasze predcton. Below, we formulate an optmum lnear predcton problem and then present a theoretcal soluton to the problem FORMULATION Optmum lnear predcton can be formulated as follows. Consder a dscrete-tme random process z. At a typcal moment, t s a random varable z. We have n prevous observatons z 1, z 2, L, z n avalable and would lke to form a predcton of z, denoted by ẑ. The output of the predctor, ẑ, s a lnear functon of the n prevous observatons. That s, n ẑ = Âajz- j j = 1 (3.20) wth a j, j = 1,2,L,n beng a set of real coeffcents. An llustraton of a lnear predctor s shown n Fgure 3.5. As defned above, the predcton error, e p, s ep = z -zˆ (3.21)

7 FIGURE 3.5 An llustraton of a lnear predctor. The mean square predcton error, MSE p, s MSE E È e p p E z z ÎÍ = - ˆ = ( ) [( ) ] 2 2 (3.22) The optmum predcton, then, refers to the determnaton of a set of coeffcents a j, j = 1,2,L,n such that the mean square predcton error, MSE p, s mnmzed. Ths optmzaton problem turns out to be computatonally ntractable for most practcal cases due to the feedback around the quantzer shown n Fgure 3.4, and the nonlnear nature of the quantzer. Therefore, the optmzaton problem s solved n two separate stages. That s, the best lnear predctor s frst desgned gnorng the quantzer. Then, the quantzer s optmzed for the dstrbuton of the dfference sgnal (Habb, 1971). Although the predctor thus desgned s suboptmal, gnorng the quantzer n the optmum predctor desgn allows us to substtute the reconstructed z j by z j for j = 1,2,L,n, accordng to Equaton Consequently, we can apply the theory of optmum lnear predcton to handle the desgn of the optmum predctor as shown below ORTHOGONALITY CONDITION AND MINIMUM MEAN SQUARE ERROR By takng the dfferentaton of MSE p wth respect to coeffcent a j s, one can derve the followng necessary condtons, whch are usually referred to as the orthogonalty condton: [ ] = = E ep z- j 0 for j 1, 2, L, n (3.23) The nterpretaton of Equaton 3.23 s that the predcton error, e p, must be orthogonal to all the observatons, whch are now the precedng samples: z j, j = 1,2,L,n accordng to our dscusson n Secton These are equvalent to

8 n z  j z j = 1 ( ) = ( - ) = R m a R m j for m 12,, L, n (3.24) where R z represents the autocorrelaton functon of z. In a vector-matrx format, the above orthogonal condtons can be wrtten as ( ) ( ) ( ) ( - ) È Rz 1 È Rz 0 Rz 1 L L Rz n 1 Èa1 Í Í Í Rz ( 2) Í Rz( 1) Rz( 0) L L Rz( n - 2) Í a Í 2 Í M = Í M M L L M Í M Í Í Í Í M Í M M L L M Í M Í ÎRz ( n) Í ÎRz( n -1) Rz( n) L L Rz( 0) Î Ía n (3.25) Equatons 3.24 and 3.25 are called Yule-Walker equatons. The mnmum mean square predcton error s then found to be n p z  j z j = 1 MSE R 0 a R j = ( ) - ( ) (3.26) These results can be found n texts dealng wth random processes, e.g., n (Leon-Garca, 1994) SOLUTION TO YULE-WALKER EQUATIONS Once autocorrelaton data are avalable, the Yule-Walker equaton can be solved by matrx nverson. A recursve procedure was developed by Levnson to solve the Yule-Walker equatons (Leon-Garca, 1993). When the number of prevous samples used n the lnear predctor s large,.e., the dmenson of the matrx s hgh, the Levnson recursve algorthm becomes more attractve. Note that n the feld of mage codng the autocorrelaton functon of varous types of vdeo frames s derved from measurements (O Neal, 1966; Habb, 1971). 3.3 SOME ISSUES IN THE IMPLEMENTATION OF DPCM Several related ssues n the mplementaton of DPCM are dscussed n ths secton OPTIMUM DPCM SYSTEM Snce DPCM conssts manly of two parts, predcton and quantzaton, ts optmzaton should not be carred out separately. The nteracton between the two parts s qute complcated, however, and thus combned optmzaton of the whole DPCM system s dffcult. Fortunately, wth the mean square error crteron, the relaton between quantzaton error and predcton error has been found: MSE q ª 9 N 2 2 MSE p (3.27) where N s the total number of reconstructon levels n the quantzer (O Neal, 1966; Musmann, 1979). That s, the mean square error of quantzaton s approxmately proportonal to the mean square error of predcton. Wth ths approxmaton, we can optmze the two parts separately, as mentoned n Secton Whle the optmzaton of quantzaton was addressed n Chapter 2, the

9 optmum predctor was dscussed n Secton 3.2. A large amount of work has been done on ths subject. For nstance, the optmum predctor for color mage codng was desgned and tested n (Prsch and Stenger, 1977) D, 2-D, AND 3-D DPCM In Secton 3.1.2, we expressed lnear predcton n Equaton However, so far we have not really dscussed how to predct a pxel s gray level value by usng ts neghborng pxels coded gray level values. Recall that a practcal pxel-to-pxel dfferental codng system was dscussed n Secton There, the reconstructed ntensty of the mmedately precedng pxel along the same scan lne s used as a predcton of the pxel ntensty beng coded. Ths type of dfferental codng s referred to as 1-D DPCM. In general, 1-D DPCM may use the reconstructed gray level values of more than one of the precedng pxels wthn the same scan lne to predct that of a pxel beng coded. By far, however, the mmedately precedng pxel n the same scan lne s most frequently used n 1-D DPCM. That s, pxel A n Fgure 3.6 s often used as a predcton of pxel Z, whch s beng DPCM coded. Sometmes n DPCM mage codng, both the decoded ntensty values of adjacent pxels wthn the same scan lne and the decoded ntensty values of neghborng pxels n dfferent scan lnes are nvolved n the predcton. Ths s called 2-D DPCM. A typcal pxel arrangement n 2-D predctve codng s shown n Fgure 3.6. Note that the pxels nvolved n the predcton are restrcted to be ether n the lnes above the lne where the pxel beng coded, Z, s located or on the lefthand sde of pxel Z f they are n the same lne. Tradtonally, a TV frame s scanned from top to bottom and from left to rght. Hence, the above restrcton ndcates that only those pxels, whch have been coded and avalable n both the transmtter and the recever, are used n the predcton. In 2-D system theory, ths support s referred to as recursvely computable (Bose, 1982). An oftenused 2-D predcton nvolves pxels A, D, and E. Obvously, 2-D predctve codng utlzes not only the spatal correlaton exstng wthn a scan lne but also that exstng n neghborng scan lnes. In other words, the spatal correlaton s utlzed both horzontally and vertcally. It was reported that 2-D predctve codng outperforms 1-D predctve codng by decreasng the predcton error by a factor of two, or equvalently, 3dB n SNR. The mprovement n subjectve assessment s even larger (Musmann, 1979). Furthermore, the transmsson error n 2-D predctve mage codng s much less severe than n 1-D predctve mage codng. Ths s dscussed n Secton 3.6. In the context of mage sequences, neghborng pxels may be located not only n the same mage frame but also n successve frames. That s, neghborng pxels along the tme dmenson are also nvolved. If the predcton of a DPCM system nvolves three types of neghborng pxels: those along the same scan lne, those n the dfferent scan lnes of the same mage frame, and those FIGURE 3.6 Pxel arrangement n 1-D and 2-D predcton.

10 n the dfferent frames, the DPCM s then called 3-D dfferental codng. It wll be dscussed n Secton ORDER OF PREDICTOR The number of coeffcents n the lnear predcton, n, s referred to as the order of the predctor. The relaton between the mean square predcton error, MSE p, and the order of the predctor, n, has been studed. As shown n Fgure 3.7, the MSE p decreases qute effectvely as n ncreases, but the performance mprovement becomes neglgble as n > 3 (Habb, 1971) ADAPTIVE PREDICTION Adaptve DPCM means adaptve predcton and adaptve quantzaton. As adaptve quantzaton was dscussed n Chapter 2, here we dscuss adaptve predcton only. Smlar to the dscusson on adaptve quantzaton, adaptve predcton can be done n two dfferent ways: forward adaptve and backward adaptve predcton. In the former, adaptaton s based on the nput of a DPCM system, whle n the latter, adaptaton s based on the output of the DPCM. Therefore, forward adaptve predcton s more senstve to changes n local statstcs. Predcton parameters (the coeffcents of the predctor), however, need to be transmtted as sde nformaton to the decoder. On the other hand, quantzaton error s nvolved n backward adaptve predcton. Hence, the adaptaton s less senstve to local changng statstcs. But, t does not need to transmt sde nformaton. FIGURE 3.7 Mean square predcton error vs. order of predctor. (Data from Habb, 1971.)

11 In ether case, the data (ether nput or output) have to be buffered. Autocorrelaton coeffcents are analyzed, based on whch predcton parameters are determned EFFECT OF TRANSMISSION ERRORS Transmsson errors caused by channel nose may reverse the bnary bt nformaton from 0 to 1 or 1 to 0 wth what s known as bt error probablty, or bt error rate. The effect of transmsson errors on reconstructed mages vares dependng on dfferent codng technques. In the case of the PCM-codng technque, each pxel s coded ndependently of the others. Therefore bt reversal n the transmsson only affects the gray level value of the correspondng pxel n the reconstructed mage. It does not affect other pxels n the reconstructed mage. In DPCM, however, the effect caused by transmsson errors becomes more severe. Consder a bt reversal occurrng n transmsson. It causes error n the correspondng pxel. But, ths s not the end of the effect. The affected pxel causes errors n reconstructng those pxels towards whch the erroneous gray level value was used n the predcton. In ths way, the transmsson error propagates. Interestngly, t s reported that the error propagaton s more severe n 1-D dfferental mage codng than n 2-D dfferental codng. Ths may be explaned as follows. In 1-D dfferental codng, usually only the mmedate precedng pxel n the same scan lne s nvolved n predcton. Therefore, an error wll be propagated along the scan lne untl the begnnng of the next lne, where the pxel gray level value s rentalzed. In 2-D dfferental codng, the predcton of a pxel gray level value depends not only on the reconstructed gray level values of pxels along the same scan lne but also on the reconstructed gray level values of the vertcal neghbors. Hence, the effect caused by a bt reversal transmsson error s less severe than n the 1-D dfferental codng. For ths reason, the bt error rate requred by DPCM codng s lower than that requred by PCM codng. For nstance, whle a bt error rate less than s normally requred for PCM to provde broadcast TV qualty, for the same applcaton a bt error rate less than 10 7 and 10 9 are requred for DPCM codng wth 2-D predcton and 1-D predcton, respectvely (Musmann, 1979). Channel encodng wth an error correcton capablty was appled to lower the bt error rate. For nstance, to lower the bt error rate from the order of 10 6 to the order of 10 9 for DPCM codng wth 1-D predcton, an error correcton by addng 3% redundancy n channel codng has been used (Bruders, 1978). 3.4 DELTA MODULATION Delta modulaton (DM) s an mportant, smple, specal case of DPCM, as dscussed above. It has been wdely appled and s thus an mportant codng technque n and of tself. The above dscusson and characterzaton of DPCM systems are applcable to DM systems. Ths s because DM s essentally a specal type of DPCM, wth the followng two features. 1. The lnear predctor s of the frst order, wth the coeffcent a 1 equal to The quantzer s a one-bt quantzer. That s, dependng on whether the dfference sgnal s postve or negatve, the output s ether +D/2 or D/2. To perceve these two features, let us take a look at the block dagram of a DM system and the nput-output characterstc of ts one-bt quantzer, shown n Fgures 3.8 and 3.9, respectvely. Due to the frst feature lsted above, we have: ẑ = z -1 (3.28)

12 FIGURE 3.8 Block dagram of DM systems. FIGURE 3.9 Input-output characterstc of two-level quantzaton n DM. Next, we see that there are only two reconstructon levels n quantzaton because of the second feature. That s, ˆd Ï f z z = + D 2 Ì > Ó- D 2 f z < z -1-1 (3.29) From the relaton between the reconstructed value and the predcted value of DPCM, dscussed above, and the fact that DM s a specal case of DPCM, we have z = zˆ + dˆ (3.30) Combnng Equatons 3.28, 3.29, and 3.30, we have z Ïz + D 2 f z > z = Ì Óz - D 2 f z < z (3.31)

13 FIGURE 3.10 DM wth fxed step sze. The above mathematcal relatonshps are of mportance n understandng DM systems. For nstance, Equaton 3.31 ndcates that the step sze D of DM s a crucal parameter. We notce that a large step sze compared wth the magntude of the dfference sgnal causes granular error, as shown n Fgure Therefore, n order to reduce the granular error we should choose a small step sze. On the other hand, a small step sze compared wth the magntude of the dfference sgnal wll lead to the overload error dscussed n Chapter 2 for quantzaton. Snce n DM systems t s the dfference sgnal that s quantzed, the overload error n DM becomes slope overload error, as shown n Fgure That s, t takes a whle (multple steps) for the reconstructed samples to catch up wth the sudden change n nput. Therefore, the step sze should be large n order to avod the slope overload. Consderng these two conflctng factors, a proper compromse n choosng the step sze s common practce n DM. To mprove the performance of DM, an oversamplng technque s often appled. That s, the nput s oversampled pror to the applcaton of DM. By oversamplng, we mean that the samplng frequency s hgher than the samplng frequency used n obtanng the orgnal nput sgnal. The ncreased sample densty caused by oversamplng decreases the magntude of the dfference sgnal. Consequently, a relatvely small step sze can be used so as to decrease the granular nose wthout ncreasng the slope overload error. Thus, the resoluton of the DM-coded mage s kept the same as that of the orgnal nput (Jayant, 1984; Lm, 1990). To acheve better performance for changng nputs, an adaptve technque can be appled n DM. That s, ether nput (forward adaptaton) or output (backward adaptaton) data are buffered and the data varaton s analyzed. The step sze s then chosen accordngly. If t s forward adaptaton, sde nformaton s requred for transmsson to the decoder. Fgure 3.11 demonstrates step sze adaptaton. We see the same nput as that shown n Fgure But, the step sze s now not fxed. Instead, the step sze s adapted accordng to the varyng nput. When the nput changes wth a large slope, the step sze ncreases to avod the slope overload error. On the other hand, when the nput changes slowly, the step sze decreases to reduce the granular error.

14 FIGURE 3.11 Adaptve DM. 3.5 INTERFRAME DIFFERENTIAL CODING As was mentoned n Secton 3.3.2, 3-D dfferental codng nvolves an mage sequence. Consder a sensor located n 3-D world space. For nstance, n applcatons such as vdeophony and vdeoconferencng, the sensor s fxed n poston for a whle and t takes pctures. As tme goes by, the mages form a temporal mage sequence. The codng of such an mage sequence s referred to as nterframe codng. The subject of mage sequence and vdeo codng s addressed n Sectons III and IV. In ths secton, we brefly dscuss how dfferental codng s appled to nterframe codng CONDITIONAL REPLENISHMENT Recognzng the great smlarty between consecutve TV frames, a condtonal replenshment codng technque was proposed and developed (Mounts, 1969). It was regarded as one of the frst real demonstratons of nterframe codng explotng nterframe redundancy (Netraval and Robbns, 1979). In ths scheme, the prevous frame s used as a reference for the present frame. Consder a par of pxels: one n the prevous frame, the other n the present frame both occupyng the same spatal poston n the frames. If the gray level dfference between the par of pxels exceeds a certan crteron, then the pxel s consdered a changng pxel. The present pxel gray level value and ts poston nformaton are transmtted to the recevng sde, where the pxel s replenshed. Otherwse, the pxel s consdered unchanged. At the recever ts prevous gray level s repeated. A block dagram of condtonal replenshment s shown n Fgure There, a frame memory unt n the transmtter s used to store frames. The dfferencng and thresholdng of correspondng pxels n two consecutve frames can then be conducted there. A buffer n the transmtter s used to smooth the transmsson data rate. Ths s necessary because the data rate vares from regon to regon wthn an mage frame and from frame to frame wthn an mage sequence. A buffer n the recever s needed for a smlar consderaton. In the frame memory unt, the replenshment s

15 FIGURE 3.12 Block dagram of condtonal replenshment. carred out for the changng pxels and the gray level values n the recever are repeated for the unchanged pxels. Wth condtonal replenshment, a consderable savngs n bt rate was acheved n applcatons such as vdeophony, vdeoconferencng, and TV broadcastng. Experments n real tme, usng the head-and-shoulder vew of a person n anmated conversaton as the vdeo source, demonstrated an average bt rate of 1 bt/pxel wth a qualty of reconstructed vdeo comparable wth standard 8 bt/pxel PCM transmsson (Mounts, 1969). Compared wth pxel-to-pxel 1-D DPCM, the most popularly used codng technque at the tme, the condtonal replenshment technque s more effcent due to the explotaton of hgh nterframe redundancy. As ponted n (Mounts, 1969), there s more correlaton between televson pxels along the frame-to-frame temporal dmenson than there s between adjacent pxels wthn a sgnal frame. That s, the temporal redundancy s normally hgher than spatal redundancy for TV sgnals. Tremendous efforts have been made to mprove the effcency of ths rudmentary technque. For an excellent revew, readers are referred to (Haskell et al., 1972, 1979). 3-D DPCM codng s among the mprovements and s dscussed next D DPCM It was soon realzed that t s more effcent to transmt the gray level dfference than to transmt the gray level tself, resultng n nterframe dfferental codng. Furthermore, nstead of treatng each pxel ndependently of ts neghborng pxels, t s more effcent to utlze spatal redundancy as well as temporal redundancy, resultng n 3-D DPCM. Consder two consecutve TV frames, each consstng of an odd and an even feld. Fgure 3.13 demonstrates the small neghborhood of a pxel, Z, n the context. As wth the 1-D and 2-D DPCM dscussed before, the predcton can only be based on the prevously encoded pxels. If the pxel under consderaton, Z, s located n the even feld of the present frame, then the odd feld of the present frame and both odd and even felds of the prevous frame are avalable. As mentoned n Secton 3.3.2, t s assumed that n the even feld of the present frame, only those pxels n the lnes above the lne where pxel Z les and those pxels left of the Z n the lne where Z les are used for predcton. Table 3.1 lsts several utlzed lnear predcton schemes. It s recognzed that the case of element dfference s a 1-D predctor snce the mmedately precedng pxel s used as the predctor. The

16 FIGURE 3.13 Pxel arrangement n two TV frames. (After Haskell, 1979.) feld dfference s defned as the arthmetc average of two mmedately vertcal neghborng pxels n the prevous odd feld. Snce the odd feld s generated frst, followed by the even feld, ths predctor cannot be regarded as a pure 2-D predctor. Instead, t should be consdered a 3-D predctor. The remanng cases are all 3-D predctors. One thng s common n all the cases: the gray levels of pxels used n the predcton have already been coded and thus are avalable n both the transmtter and the recever. The predcton error of each changng pxel Z dentfed n thresholdng process s then quantzed and coded. An analyss of the relatonshp between the entropy of movng areas (bts per changng pxel) and the speed of the moton (pxels per frame nterval) n an mage contanng a movng mannequn s head was studed wth dfferent lnear predctons, as lsted n Table 3.1 n Haskell (1979). It was found that the element dfference of feld dfference generally corresponds to the lowest entropy, meanng that ths predcton s the most effcent. The frame dfference and element dfference correspond to hgher entropy. It s recognzed that, n the crcumstances, transmsson error wll be propagated f the pxels n the prevous lne are used n predcton (Connor, 1973). Hence, the lnear predctor should use only pxels from the same lne or the same lne n the prevous frame when bt reversal error n transmsson needs to be consdered. Combnng these two factors, the element dfference of frame dfference predcton s preferred. TABLE 3.1 Some Lnear Predcton Schemes. (After Haskell, 1979). Orgnal sgnal (Z) Predcton sgnal (Ẑ) Dfferental sgnal (d z) Element dfference Z G Z-G Feld dfference Z E + J E + J Z Frame dfference Z T Z-T Element dfference of frame dfference Z T + G S (Z-G)-(T-S) Lne dfference of frame dfference Z T + B M (Z-B)-(T-M) Element dfference of feld dfference Z E + J Q + W Ê E + J ˆ Ê Q + W ˆ T + - Z - - T Ë 2 Ë 2

17 3.5.3 MOTION-COMPENSATED PREDICTIVE CODING When frames are taken densely enough, changes n successve frames can be attrbuted to the moton of objects durng the nterval between frames. Under ths assumpton, f we can analyze object moton from successve frames, then we should be able to predct objects n the next frame based on ther postons n the prevous frame and the estmated moton. The dfference between the orgnal frame and the predcted frame thus generated and the moton vectors are then quantzed and coded. If the moton estmaton s accurate enough, the moton-compensated predcton error can be smaller than 3-D DPCM. In other words, the varance of the predcton error wll be smaller, resultng n more effcent codng. Take moton nto consderaton ths dfferental technque s called moton compensated predctve codng. Ths has been a major development n mage sequence codng snce the 1980s. It has been adopted by all nternatonal vdeo codng standards. A more detaled dscusson s provded n Chapter INFORMATION-PRESERVING DIFFERENTIAL CODING As emphaszed n Chapter 2, quantzaton s not reversble n the sense that t causes permanent nformaton loss. The DPCM technque, dscussed above, ncludes quantzaton, and hence s lossy codng. In applcatons such as those nvolvng scentfc measurements, nformaton preservaton s requred. In ths secton, the followng queston s addressed: under these crcumstances, how should we apply dfferental codng n order to reduce the bt rate whle preservng nformaton? Fgure 3.14 shows a block dagram of nformaton-preservng dfferental codng. Frst, we see that there s no quantzer. Therefore, the rreversble nformaton loss assocated wth quantzaton does not exst n ths technque. Second, we observe that predcton and dfferencng are stll used. That s, the dfferental (predctve) technque stll apples. Hence t s expected that the varance of the dfference sgnal s smaller than that of the orgnal sgnal, as explaned n Secton 3.1. Consequently, the hgher-peaked hstograms make codng more effcent. Thrd, an effcent lossless coder s utlzed. Snce quantzers cannot be used here, PCM wth natural bnary codng s not used here. Snce the hstogram of the dfference sgnal s narrowly concentrated about ts mean, lossless codng technques such as an effcent Huffman coder (dscussed n Chapter 5) s naturally a sutable choce here. FIGURE 3.14 Block dagram of nformaton-preservng dfferental codng.

18 As mentoned before, nput mages are normally n a PCM coded format wth a bt rate of eght bts per pxel for monochrome pctures. The dfference sgnal s therefore nteger-valued. Havng no quantzaton and usng an effcent lossless coder, the codng system depcted n Fgure 3.14, therefore, s an nformaton-preservng dfferental codng technque. 3.7 SUMMARY Rather than codng the sgnal tself, dfferental codng, also known as predctve codng, encodes the dfference between the sgnal and ts predcton. Utlzng spatal and/or temporal correlaton between pxels n the predcton, the varance of the dfference sgnal can be much smaller than that of the orgnal sgnal, thus makng dfferental codng qute effcent. Among dfferental codng methods, dfferental pulse code modulaton (DPCM) s used most wdely. In DPCM codng, the dfference sgnal s quantzed and codewords are assgned to the quantzed dfference. Predcton and quantzaton are therefore two major components n the DPCM systems. Snce quantzaton was addressed n Chapter 2, ths chapter emphaszes predcton. The theory of optmum lnear predcton s ntroduced. Here, optmum means mnmzaton of the mean square predcton error. The formulaton of optmum lnear predcton, the orthogonalty condton, and the mnmum mean square predcton error are presented. The orthogonalty condton states that the predcton error must be orthogonal to each observaton,.e., to the reconstructed sample ntensty values used n the lnear predcton. By solvng the Yule-Walker equaton, the optmum predcton coeffcents may be determned. In addton, some fundamental ssues n mplementng the DPCM technque are dscussed. One ssue s the dmensonalty of the predctor n DPCM. We dscussed 1-D, 2-D, and 3-D predctors. DPCM wth a 2-D predctor demonstrates better performance than a 1-D predctor snce 2-D DPCM utlzes more spatal correlaton,.e., not only horzontally but also vertcally. As a result, a 3-dB mprovement n SNR was reported. 3-D predcton s encountered n what s known as nterframe codng. There, temporal correlaton exsts. 3-D DPCM utlzes both spatal and temporal correlaton between neghborng pxels n successve frames. Consequently, more redundancy can be removed. Moton-compensated predctve codng s a very powerful technque n vdeo codng related to dfferental codng. It uses a more advanced translatonal moton model n the predcton, however, and t s covered n Sectons III and IV. Another ssue s the order of predctors and ts effect on the performance of predcton n terms of mean square predcton error. Increasng the predcton order can lower the mean square predcton error effectvely, but the performance mprovement becomes nsgnfcant after the thrd order. Adaptve predcton s another ssue. Smlar to adaptve quantzaton, dscussed n Chapter 2, we can adapt the predcton coeffcents n the lnear predctor to varyng local statstcs. The last ssue s concerned wth the effect of transmsson error. Bt reversal n transmsson causes a dfferent effect on reconstructed mages dependng on the type of codng technque used. PCM s known to be bt-consumng. (An acceptable PCM representaton of monochrome mages requres sx to eght bts per pxel.) But a one-bt reversal only affects an ndvdual pxel. For the DPCM codng technque, however, a transmsson error may propagate from one pxel to the other. In partcular, DPCM wth a 1-D predctor suffers from error propagaton more severely than DPCM wth a 2-D predctor. Delta modulaton s an mportant specal case of DPCM n whch the predctor s of the frst order. Specfcally, the mmedately precedng coded sample s used as a predcton of the present nput sample. Furthermore, the quantzer has only two reconstructon levels. Fnally, an nformaton-preservng dfferental codng technque s dscussed. As mentoned n Chapter 2, quantzaton s an rreversble process: t causes nformaton loss. In order to preserve nformaton, there s no quantzer n ths type of system. To be effcent, lossless codes such as Huffman code or arthmetc code should be used for dfference sgnal encodng.

19 3.8 EXERCISES 3-1. Justfy the necessty of the closed-loop DPCM wth feedback around quantzers. That s, convnce yourself why the quantzaton error wll be accumulated f, nstead of usng the reconstructed precedng samples, we use the mmedately precedng sample as the predcton of the sample beng coded n DPCM Why does the overload error encountered n quantzaton appear to be the slope overload n DM? 3-3. What advantage does oversamplng brng up n the DM technque? 3-4. What are the two features of DM that make t a subclass of DPCM? 3-5. Explan why DPCM wth a 1-D predctor suffers from bt reversal transmsson error more severely than DPCM wth a 2-D predctor Explan why no quantzer can be used n nformaton-preservng dfferental codng, and why the dfferental system can work wthout a quantzer Why do all the pxels nvolved n predcton of dfferental codng have to be n a recursvely computable order from the pont of vew of the pxel beng coded? 3-8. Dscuss the smlarty and dssmlarty between DPCM and moton compensated predctve codng. REFERENCES Bose, N. K. Appled Multdmensonal System Theory, Van Nostrand Renhold, New York, Bruders, R., T. Kummerow, P. Neuhold, and P. Stamntz, En versuchssystem zur dgtalen ubertragung von fernsehsgnalen unter besonderer beruckschtgung von ubertragungsfehlern, Festschrft 50 Jahre Henrch-Hertz-Insttut, Berln, Connor, D. J. IEEE Trans. Commun., com-21, , Cutler, C. C. U. S. Patent 2,605,361, DeJager, F. Phlps Res. Rep., 7, , Elas, P. IRE Trans. Inf. Theory, t-1, 16-32, Habb, A. Comparson of nth-order DPCM encoder wth lnear transformatons and block quantzaton technques, IEEE Trans. Commun. Technol., COM-19(6), , Harrson, C. W. Bell Syst. Tech. J., 31, , Haskell, B. G., F. W. Mounts, and J. C. Candy, Interframe codng of vdeotelephone pctures, Proc. IEEE, 60, 7, , Haskell, B. G. Frame replenshment codng of televson, n Image Transmsson Technques, W. K. Pratt (Ed.), Academc Press, New York, Jayant, N. S. and P. Noll, Dgtal Codng of Waveforms, Prentce-Hall, Upper Saddle Rver, NJ, Kretzmer, E. R. Statstcs of televson sgnals, Bell Syst. Tech. J., 31, , Leon-Garca, A. Probablty and Random Processes for Electrcal Engneerng, 2nd ed., Addson-Wesley, Readng, MA, Lm, J. S. Two-Dmensonal Sgnal and Image Processng, Prentce-Hall, Englewood Clffs, NJ, Mounts, F. W. A vdeo encodng system wth condtonal pcture-element replenshment, Bell Syst. Tech. J., 48, 7, Musmann, H. G. Predctve Image Codng, n Image Transmsson Technques, W. K. Pratt (Ed.), Academc Press, New York, Netraval, A. N. and J. D. Robbns, Moton-compensated televson codng. Part I, Bell Syst. Tech. J., 58, 3, , Olver, B. M. Bell Syst. Tech. J., 31, , O Neal, J. B. Bell Syst. Tech. J., 45, , Prsch, P. and L. Stenger, Acta Electron., 19, , Sayood, K. Introducton to Data Compresson, Morgan Kaufmann, San Francsco, CA, 1996.