Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Transcription

1 Module 3 LOSSY IMAGE COMPRESSION SYSEMS Versio ECE II, Kharagpur

2 Lesso 8 rasform Codig & K-L rasforms Versio ECE II, Kharagpur

3 Istructioal Oectives At the ed of this lesso, the studets should e ale to:. Distiguish etwee spatial ad trasform-domai image compressio systems.. State the oectives of trasform codig. 3. Write the geeral expressios for forward ad iverse trasforms.. Defie separale ad symmetric trasforms. 5. Defie asis images. 6. Determie the covariace matrix of image loc. 7. Represet a covariace matrix i terms of its eigevectors ad eigevalues. 8. Defie K-L trasform. 9. Show that K-L trasform is optimal i terms of mea-square trucatio error.. State why K-L trasforms are difficult to implemet i practice. 8. Itroductio he lossy image compressio techiques discussed i lesso-7 wor i the spatial domai, sice we are predictig the pixel values ad the predictio errors also correspod to the pixels i the origial space. It is see that although the liear predictio mechaism essetially tries to exploit the iheret spatial redudacy, the compressio ratios of Differetial Pulse Code Modulatio (DPCM) ecoded images are ot always very high. his is primarily due to the fact that i presece of sharp chages i itesity values, which are always expected i ay atural image due to the presece of oects of varyig itesities, predictio suffers ad ecodig large predictio errors i those regios lead to high cosumptio of its. I terms of compressio, performace is see to e etter i trasform-domai approaches, i which the pixel itesities are first mapped ito a set of liear, reversile trasform coefficiets, which are susequetly quatized ad ecoded. he trasform coefficiets are de-correlated ad ted to pac most of the eergy withi few coefficiets oly. hus, it is possile to achieve sigificat compressio y either discardig the coefficiets which do ot carry much of the eergy or, at least coarsely quatizig them. Versio ECE II, Kharagpur

4 I this lesso, we shall first itroduce the asic cocepts of trasform codig techiques i a geeric sese. Susequetly, we are goig to discuss Karhue- Loeve trasforms (KL) which is a optimal trasformatio i terms of the retaied trasform coefficiets. We shall study that despite optimal performace, KL is ofte ot the preferred trasform codig techique, sice the process of trasformatio is heavily image depedet ad the computatioal cost is high. It is for this reaso that KL has ot ee recommeded i the iteratioal multimedia stadards for image or video compressio. 8. rasform Codig he asic priciple of trasform codig is to map the pixel values ito a set of liear trasform coefficiets, which are susequetly quatized ad ecoded. By applyig a iverse trasformatio o the decoded trasform coefficiets, it is possile to recostruct the image with some loss. It must e oted that the loss is ot due to the process of trasformatio ad iverse trasformatio, ut due to quatizatio aloe. Sice the details of a image ad hece it s spatial frequecy cotet vary from oe local regio to the other, it leads to a etter codig efficiecy if we apply the trasformatio o local areas of the image, rather tha applyig gloal trasformatio o the etire image. Such local trasformatios require maageale size of the hardware, which ca e replicated for parallel processig. For trasform codig, the first ad foremost step is to sudivide the image ito o-overlappig locs of fixed size. Without loss of geerality, we ca cosider a square image of size N x N pixels ad divide it ito umer of locs, each of size (N/) x (N/), where <<N ad is a factor of N. Fig 8.: Bloc Diagram of rasform Codig System Ecoder. Fig.8. shows the loc diagram of a trasform codig system ad fig.8. shows the correspodig decoder. Although trasformatio does ot directly achieve ay compressio, it prepares the iput sigal to compressio i the trasformed domai. Fig 8.: Bloc Diagram of rasform Codig System Decoder Versio ECE II, Kharagpur

5 A trasformatio must ecessarily fulfill the followig properties (i) (ii) he coefficiets i the trasformed space should e de-correlated. Oly a limited umer of trasform coefficiets should carry most of the sigal eergy (i other words, the trasformatio should possess eergy compactio capailities) ad most of the coefficiets should carry isigificat eergy. Oly the the quatizatio process ca coarsely quatize those coefficiets to achieve compressio, without much of perceptile degradatio. A umer of trasformatio techiques, such as Discrete Fourier rasforms (DF), Discrete Cosie rasforms (DC), Discrete Wavelet rasforms (DW), K-L rasforms (KL), Discrete Haar rasforms, ad Discrete Hadamard rasforms etc. exist that fulfill the aove properties, although their eergy pacig capailities vary. I terms of eergy pacig, KL is optimal ad we are goig to study KL i the latter part of this lesso. 8. Geeralized forward ad iverse trasforms Several trasformatio techiques are availale, ut the choice of the techique depeds o the amout of recostructio error that ca e availale ad the computatioal resources availale. Let us cosider a image loc of size x whose pixel itesities are represeted y s (, ) (,,, ) where ad are the row ad the colum idices of the array. Its geeral expressio for trasformatio is give y S, ( ) s(, ) g(,,, ),,, where ( )....(8.) S, (,,, represets the trasform coefficiets of the loc with ad as the row ad the colum idices i the trasformed array ad g (,,, ) is the trasformatio erel that maps the iput image pixels ito the trasform coefficiets. Give the trasform coefficiets S (, ), the iput image s (, ) may e otaied as s ), ( ) S(, ) h(,,, ),,,. (8.) I the aove equatio, erel. h (,,, ) represets the iverse trasformatio Versio ECE II, Kharagpur

6 8.. Separale erel A trasformatio erel is said to e separale if it ca e expressed as a product of two erels alog the row ad the colum, i.e. (,,, ) g(, ) g (, g ) (8.3) where g (.) ad g (.) represet the trasformatio erels alog the row ad the colum directios respectively. By a similar way, the iverse trasformatio erel too ca e separale. Separale trasforms are easier to implemet i hardware, sice the trasformatio ca first e applied alog the rows (or the colums) ad the alog the colums (or the rows). 8.. Symmetric erel A separale trasform is symmetric, if the erels alog the row ad the colum have the idetical fuctio, i.e. if (,,, ) g(, ) g(, ) g (8.) Most of the trasformatios that we deal with have separale, symmetric erels. For example, the forward ad the iverse trasformatio erels of Discrete Fourier rasform (DF) for x image loc is give y g (,,, ) ad ( ) π + exp...(8.5) h (,,, ) ( ) π + exp. (8.6) are separale ad symmetric. he studets ca easily derive the row ad the colum trasformatio erels y expressig the erel of equatio (8.5) as a product of two erels. his is left as a exercise Basis Images Equatio (8.) relates the pixel itesities of the image loc o a elemet y elemet asis to the trasformatio coefficiets S (, ) (,,, ) ad there are umer of similar equatios, defied for each pixel elemet. hese equatios ca e comied ad writte i the matrix form (, ), s S H (8.7) Versio ECE II, Kharagpur

7 where s is a x matrix cotaiig the pixels of s (, ) ad H, h h h (,,, ) h(,,, ) h(,,, ) (,,, ) ( ) ( ) (,,, h,,, h,,, ).(8.8) H is a x matrix defied for ( ),,. he image loc s ca therefore e realized y a weighted summatio of images, each of size x, defied y equatio (8.8) ad the weights are provided y the trasform coefficiets S (, ). he matrix H is ow as a asis image correspodig to ( ),,. here are such asis images, each of size x, correspodig to each (, ). Some examples of asis images for typical trasforms will e show later. 8.3 Covariace Matrix Sice trasforms are applied o a loc-y-loc asis, each loc of a image may e treated as a radom field. A loc may e represeted y a - dimesioal radom variale vector x, whose elemets are composed y the lexicographic orderig of pixel itesity values. We defie a vector, such that [] x x E (8.9) where, E[.] is the expectatio operator. he expectatio of x ca e otaied from the mea of the radom variale x over all the locs preset i the image. hus, x μ (8.) where, μ N B i x i,.(8.) i is the loc idex ad N B is the total umer of locs. he covariace matrix computed over locs of size x is defied y R Versio ECE II, Kharagpur

8 [( x μ )( x ) ] E[ R E μ ] (8.) where, as efore, the expectatio is calculated y averagig over all the locs. Sice is a -dimesioal vector, its outer product realizes a x dimesioal matrix, which is the size of R. he matrix R is real ad symmetric ad it is possile to fid a set of orthoormal eigevectors. Let e ad λ, i,,, e the eigevectors ad the correspodig i i eigevalues, arraged i o-icreasig order, such that,,,. By the asic defiitio of eigevectors, R λ for λ + e λ e.. (8.3) Pre-multiplyig oth the sides of equatio (8.3) y for orthoormal eigevectors, it follows that e ad otig that e e e R e λ.... (8.) We ow compose a matrix Γ of dimesio, whose rows are formed from the eigevectors of R, ordered such that the first row of Γ is the eigevector correspodig to the largest eigevalue ad the last row is the eigevector correspodig to the smallest eigevalue. Cosiderig all the eigevectors, we ca write equatio (8.) i matrix form as ΓR Γ Λ.. (8.5) where, Λ is a diagoal matrix of ordered eigevalues, defied as λ Λ λ λ.... (8.6) Pre-multiplyig equatio (8.5) y orthoormal properties of matrix Γ, i.e., Γ, post-multiplyig y Γ ad otig the Γ Γ, we otai R Γ ΛΓ (8.7) Versio ECE II, Kharagpur

9 8. K-L rasforms If we use the matrix Γ to map the loc of -dimesioal vector ito a trasformed loc of -dimesioal vector y, defied y y Γ (8.8) the trasformatio is called Karhue-Loeve trasforms (KL). he covariace matrix of the y s is give y R y R y E E ΓE ΓR [ yy ] [( Γ)( Γ) ] [ ] Γ Γ.. (8.9) he pre-multiplicatio of R y Γ ad post-multiplicatio y ito a diagoal matrix Λ of eigevectors ad the matrix R y Γ diagoalizes ca e writte as R R y Λ.. (8.) he covariace matrix R R y has the same eigevectors ad eigevalues as that of, ut its off-diagoal elemets are zero, which sigifies that the elemets of the trasform-domai vectors y are ucorrelated. Usig equatio (8.8), it is possile to recover vector as Γ y. (8.) Usig equatio (8.) ad orthoormality property of Γ, it is possile to recostruct the origial loc x as x Γ y + μ... (8.) he aove equatio leads to exact recostructio. Suppose that istead of usig all the eigevectors of, we use oly eigevectors correspodig to the largest R eigevalues ad form a trasformatio matrix Γ of order. he resultig trasformed vector ŷ therefore ecomes -dimesioal ad the recostructio Versio ECE II, Kharagpur

10 give i equatio (8.) will ot e exact. he recostructed vector give y xˆ is the xˆ Γ yˆ + μ... (8.3) 8.5 Optimality of K-L rasform o show that K-L rasform is optimal i the least square error sese, we first estalish a relatio etwee the variace of the origial data vector x ad the eigevalues. If we proect the mea-removed vector, defied i equatio (8.) ito ay of e,,,, the proectio is defied y the ier the eigevectors ( ) product of the vectors ad e is give y A e e... 8.) he variace σ E E e e σ [ A ] [ e e ] E[ ] e R e of the proectio is therefore give y ) By proectig the vector ito all the eigevectors ad usig equatio (8.), we otai the total variace as σ R e e λ.. 8.6) By cosiderig oly the first eigevectors out of approximatig sigal i the proected space is give y, the variace of the σ λ. (8.7) e ms hus, the mea-square error i the proected space y cosiderig oly the first compoets ca e otaied y sutractig equatio (8.7) from equatio (8.6) Versio ECE II, Kharagpur

11 e ms σ σ λ λ ) λ + Sice, the trasformatio is eergy-preservig, the same mea-square error exists etwee the origial vector x ad its approximatio x ˆ. It is evidet from the aove equatio that the mea square error is zero whe, i.e., if all the eigevectors are used i the trasformatio. Sice the λ s decrease mootoically, the error ca e miimized y selectig the first eigevectors are associated with the largest eigevalues. hus, K-L trasform is optimal i the sese that it miimizes the mea-square error etwee the origial iput vectors x ad their approximatios xˆ. 8.6 Practical limitatios of K-L rasforms Despite the optimal performace of K-L trasforms, it is rarely used i practice ecause of the followig limitatios: (i) (ii) (iii) he trasformatio matrix for a loc of image is derived from the covariace matrix, which eeds to e computed for every loc. his maes the trasformatio data depedet ad ivolves o-trivial computatios. Perfect de-correlatio i trasform domai is ot possile, sice rarely, the image locs ca e modeled as a radom field. No fast computatioal algorithms are availale for its implemetatio. Other trasform-domai approaches, such as DF, DC etc. o the other had are ot image depedet ad wor o fixed asis images. Moreover, fast computatioal algorithms ad efficiet VLSI architectures are availale for these trasforms. It is see that the siusoidal trasforms, such as the DF or the DC more closely approximate the iformatio pacig capaility of the optimal K-L trasforms. Versio ECE II, Kharagpur

12 Questios NOE: he studets are advised to thoroughly read this lesso first ad the aswer the followig questios. Oly after attemptig all the questios, they should clic to the solutio utto ad verify their aswers. PAR-A A.. Distiguish etwee spatial domai ad trasform-domai compressio approaches. A.. State the asic oectives of trasform codig. A.3. Write the geeral expressios for forward ad iverse trasforms. A.. Defie separale trasforms with a example. A.5. Defie symmetric trasforms with a example. A.6. Defie K-L trasform ad its iverse, applied o a loc of image. A.7. Express the covariace matrix of a loc i terms of its eigevalues ad eigevectors. A.8. Show that K-L trasforms are optimal i least square error sese whe a limited umer of o-decreasigly ordered eigevalues ad the correspodig eigevectors are cosidered. A.9. Why are trasforms lie DC, DF etc. are preferred over K-L trasforms from practical implemetatio cosideratios. PAR-B: Multiple Choice I the followig questios, clic the est out of the four choices. B. A trasformatio erel for a N x N image loc, as give y g(,,, ) cos π ( ) π ( + ) N + cos N is (A) either separale, or symmetric. (B) separale, ut ot symmetric. (C) ot separale, ut symmetric. (D) oth separale ad symmetric. Versio ECE II, Kharagpur

13 B. he x asis images i a image trasform are give y,,,, H H H H he trasform-domai coefficiets S(, ) are give y ( ) 3, S he spatial-domai image is (A) (B) 7 (C) (D) 3 3 B.3 he iverse trasformatio erel for a x image loc is give y ( ) ( ) + exp,,, h π (A) (B) (C) (D) B. A x loc image is represeted y the vector [ ] 5 5 x. he mea vector computed over the etire image is give y [ ] 3 μ. he covariace matrix for the give loc is Versio ECE II, Kharagpur

14 (A) (B) (C) (D) B.5 Which of the followig matrices ca qualify to e a Λ matrix? (i) (ii) 3 (iii) (iv) 3 3 (A) Oly matrix-(i) (B) Oly matrix-(iv) (C) Matrices (ii) ad (iv) (D) All four of them. B.6 he umer of eigevalues of the covariace matrix for a 8x8 image loc will e (A) 8 (B) 6 (C) 5 Versio ECE II, Kharagpur

15 (D) 96 B.7 For a 8x8 image loc, the umer of elemets i the Γ matrix will e (A) 8 (B) 6 (C) 5 (D) 96 B.8 he Γ matrix must ecessarily fulfill the followig coditio: (A) (B) (C) Γ Γ Γ Γ Γ Γ (D) It is a diagoal matrix. B.9 A x loc image has the followig eigevalues for its covariace matrix: λ 8, λ, λ3, λ he eigevector correspodig to the smallest eigevalue is dropped while performig K-L trasform. he ratio of mea-square recostructio error to the sigal variace is (A) :5 (B) :5 (C) 8:5 (D) :5. PAR-C: Prolems C-. (a) Write a computer program to lexicographically order a x loc ito a 6- dimesioal vector ad compute the mea vector ad covariace matrix y cosiderig o-overlappig x locs over a image from the archive. () Apply K-L trasformatio o the image after retaiig oly top eigevalues ad the correspodig eigevectors of the covariace matrix. (c) Apply iverse K-L trasformatio o the aove ad recostruct the image. Compute the PSNR of the recostructed image. C-. Versio ECE II, Kharagpur

16 (a) Cosider the first six frames of the video sequece Forema. Compose 6- elemet vectors y picig up pixel values at the same spatial positio over six cosecutive frames. () Determie the mea of these 6-elemet vectors, cosiderig all spatial positios. Compute the 6x6 covariace matrix ad determie its eigevalues ad the correspodig eigevectors. Retai oly top two of these eigevalues ad the correspodig eigevectors (c) Otai the top two pricipal compoet images y proectig the vectors (otaiig dot-products) o the two pricipal eigevectors ad display the results. (d) Apply iverse K-L trasformatio ad otai the recostructed frames. Compute the PSNR of each recostructed frame. SOLUIONS A. A. A.3 A. A.5 A.6 A.7 A.8 A.9 B. (D) B. (B) B.3 (A) B. (C) B.5 (C) B.6 (B) B.7 (D) B.8 (A) B.9 (A). C. C. Versio ECE II, Kharagpur

17 Versio ECE II, Kharagpur