Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur
Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur
Instructonal Objectves At the end of ths lesson, the students should be able to: 1. Defne quantzaton.. Dstngush between scalar and vector quantzaton. 3. Defne quantzaton error and optmum scalar quantzer desgn crtera. 4. Desgn a Lloyd-Max quantzer. 5. Dstngush between unform and non-unform quantzaton. 6. Defne rate-dstorton functon. 7. State source codng theorem. 8. Determne the mnmum possble rate for a gven SNR to encode a quantzed Gaussan sgnal. 6.0 Introducton In lesson-3, lesson-4 and lesson-5, we have dscussed several lossless compresson schemes. Although the lossless compresson technques guarantee exact reconstructon of mages after decodng, ther compresson performance s very often lmted. We have seen that wth lossless codng schemes, our achevable compresson s restrcted by the source entropy, as gven by Shannon s noseless codng theorem. In lossless predctve codng, t s the predcton error that s encoded and snce the entropy of the predcton error s less due to spatal redundancy, better compresson ratos can be acheved. Even then, compresson ratos better than :1 s often not possble for most of the practcal mages. For sgnfcant bandwdth reductons, lossless technques are consdered to be nadequate and lossy compresson technques are employed, where psycho-vsual redundancy s exploted so that the loss n qualty s not vsually perceptble. The man dfference between the lossy and the lossless compresson schemes s the ntroducton of the quantzer. In mage compresson systems, dscussed n lesson-, we have seen that the quantzaton s usually appled to the transform-doman mage representatons. Before dscussng the transform codng technques or the lossy compresson technques n general, we need to have some basc background on the theory of quantzaton, whch s the scope of the present lesson. In ths lesson, we shall frst present the defntons of scalar and vector quantzaton and then consder the desgn ssues of optmum quantzer. In partcular, we shall dscuss Lloyd-Max quantzer desgn and then show the relatonshp between the rate-dstorton functon and the sgnal-to-nose rato. Verson ECE IIT, Kharagpur
6.1 Quantzaton Quantzaton s the process of mappng a set of contnuous-valued samples nto a smaller, fnte number of output levels. Quantzaton s of two basc types (a) scalar quantzaton and (b) vector quantzaton. In scalar quantzaton, each sample s quantzed ndependently. A scalar quantzer Q(.) s a functon that maps a contnuous-valued varable s havng a probablty densty functon p(s) nto a dscrete set of reconstructon levels r ( = 1,, L) by applyng a set of the decson levels d ( = 1,, L), appled on the contnuous-valued samples s, such that Q () s r f s ( d 1, d ], = 1,, L =.(6.1), where, L s the number of output level. In words, we can say that the output of the quantzer s the reconstructon level r, f the value of the sample les wthn the range ( d,. 1 d ] In vector quantzaton, each of the samples s not quantzed. Instead, a set of contnuous-valued samples, expressed collectvely as a vector s represented by a lmted number of vector states. In ths lesson, we shall restrct our dscussons to scalar quantzaton. In partcular, we shall concentrate on the scalar quantzer desgn,.e., how to desgn d and r n equaton (6.1). The performance of a quantzer s determned by ts dstorton measure. Let s = Q()be s the quantzed varable. Then, ε = s s s the quantzaton error and the dstorton D s measured n terms of the expectaton of the square of the quantzaton error (.e., the mean-square error) and s gven by D = E[ ( s s ) ]. We should desgn d and r so that the dstorton D s mnmzed. There are two dfferent approaches to the optmal quantzer desgn [ ] (a) Mnmze D E ( s s ) = wth respect to d and r ( 1,, L) =, subject to the constrant that L, the number of output states n the quantzer s fxed. These quantzers perform non-unform quantzaton n general and are known as Lloyd-Max quantzers. The desgn of Lloyd-Max quantzers s presented n the next secton. [ ] (b) Mnmze D = E ( s s ) wth respect to d and r ( 1,, L) the constrant that the source entropy H ( s) = C =, subject to s a constant and the number of output states L may vary. These quantzers are called entropyconstraned quantzers. Verson ECE IIT, Kharagpur
In case of fxed-length codng, the rate R for quantzers wth L states s gven by log R, whle R > H () s n case of varable-length codng. Thus, Lloyd-Max quantzers are more suted for use wth fxed-length codng, whle entropyconstraned quantzers are more sutable for use wth varable-length codng. 6. Desgn of Lloyd-Max Quantzers The desgn of Lloyd-Max quantzers requres the mnmzaton of D = E L d [( s r ) ] = ( s r ) p( s)ds = 1 d 1 (6.) Settng the partal dervatves of D wth respect to d and = 1,, L to zero and solvng, we obtan the necessary condtons for mnmzaton as r ( ) r = d d 1 d d 1 sp p () s () s ds ds, 1 L (6.3) d = r + r + 1, 1 L...(6.4) Mathematcally, the decson and the reconstructon levels are solutons to the above set of nonlnear equatons. In general, closed form solutons to equatons (6.3) and (6.4) do not exst and they need to be solved by numercal technques. Usng numercal technques, these equatons could be solved n an teratve way by frst assumng an ntal set of values for the decson levels{ d }. For smplcty, one can start wth decson levels correspondng to unform quantzaton, where decson levels are equally spaced. Based on the ntal set of decson levels, the reconstructon levels can be computed usng equaton (6.3) f the pdf of the nput varable to the quantzer s known. These reconstructon levels are used n equaton (6.4) to obtan the updated values of{ d }. Solutons of equatons (6.3) and (6.4) are teratvely repeated untl a convergence n the decson and reconstructon levels are acheved. In most of the cases, the convergence s acheved qute fast for a wde range of ntal values. Verson ECE IIT, Kharagpur
6.3 Unform and non-unform quantzaton Lloyd-Max quantzers descrbed above perform non-unform quantzaton f the pdf of the nput varable s not unform. Ths s expected, snce we should perform fner quantzaton (that s, the decson levels more closely packed and consequently more number of reconstructon levels) wherever the pdf s large and coarser quantzaton (that s, decson levels wdely spaced apart and hence, less number of reconstructon levels), wherever pdf s low. In contrast, the reconstructon levels are equally spaced n unform quantzaton,.e., r 1 + 1 r = θ L 1 where θ s a constant, that s defned as the quantzaton step-sze. In case, the pdf of the nput varable s s unform n the nterval [A, B],.e., p () s 1 = B A 0 A s B otherwse the desgn of Lloyd-Max quantzer leads to a unform quantzer, where θ = B A L d r = A + θ 0 L θ = d 1 + 1 L If the pdf exhbts even symmetrc propertes about ts mean, e.g., Gaussan and Laplacan dstrbutons, then the decson and the reconstructon levels have some symmetry relatons for both unform and non-unform quantzers, as shown n Fg.6.1 and Fg.6. for some typcal quantzer characterstcs (reconstructon vels vs. nput varable s) for L even and odd respectvely. Verson ECE IIT, Kharagpur
Verson ECE IIT, Kharagpur
When pdf s even symmetrc about ts mean, the quantzer s to be desgned for only L/ levels or (L-1)/ levels, dependng upon whether L s even or odd, respectvely. 6.4 Rate-Dstorton Functon and Source Codng Theorem Shannon s Codng Theorem on noseless channels consders the channel, as well as the encodng process to be lossless. Wth the ntroducton of quantzers, the encodng process becomes lossy, even f the channel remans as lossless. In most cases of lossy compressons, a lmt s generally specfed on the maxmum tolerable dstorton D from fdelty consderaton. The queston that arses s Gven a dstorton measure D, how to obtan the smallest possble rate? The answer s provded by a branch of nformaton theory that s known as the ratedstorton theory. The correspondng functon that relates the smallest possble rate to the dstorton, s called the rate-dstorton functon R(D). A typcal nature of rate-dstorton functon s shown n Fg.6.3. Verson ECE IIT, Kharagpur
At no dstorton (D=0),.e. for lossless encodng, the correspondng rate R(0) s equal to the entropy, as per Shannon s codng theorem on noseless channels. Rate-dstorton functons can be computed analytcally for smple sources and dstorton measures. Computer algorthms exst to compute R(D) when analytcal methods fal or are mpractcal. In terms of the rate-dstorton functon, the source codng theorem s presented below. Source Codng Theorem There exsts a mappng from the source symbols to codewords such that for a gven dstorton D, R(D) bts/symbol are suffcent to enable source reconstructon wth an average dstorton arbtrarly close to D. The actual bts R s gven by R R D ( ) Verson ECE IIT, Kharagpur