WAVELET-BASED IMAGE COMPRESSION USING SUPPORT VECTOR MACHINE LEARNING AND ENCODING TECHNIQUES

Transcription

1 WAVELE-BASED IMAGE COMPRESSION USING SUPPOR VECOR MACHINE LEARNING AND ENCODING ECHNIQUES Rakb Ahmed Gppsand Schoo of Computng and Informaton echnoogy Monash Unversty, Gppsand Campus Austraa. ABSRAC hs paper presents a method of compressng st mages combnng the powerfu features of support vector machne (SVM) for machne earnng wth dscrete waveet transform (DW) n mage transformaton. DW, based on the haar waveet, has been used to transform the mage and the coeffcents acqured from DW are then traned wth SVM usng Gaussan kerne. SVM has the property that t seects a mnma number of coeffcents to mode the tranng data for a predefned eve of accuracy. he coeffcents are then quantzed and encoded usng the Huffman codng agorthm. he performance of the proposed method s asprng and comparabe wth the exstng mage compresson standards. KEY WORDS Image Compresson, Dscrete Waveet ransform, Support Vector Machne, Regresson. 1. Introducton Image compresson s one of the major technooges that enabes the revouton of mutmeda. Image compresson technques fnd severa appcatons n the areas ke, Internet, dgta photography, medca, wreess and document magng, mage archves and databases, securty and nvestgaton, prntng, scannng, and facsme. Machne earnng agorthms have been used often n mage compresson. A method usng the back-propagaton agorthm n a feed-forward network s descrbed n [1]. he compresson rato of the mage recovered usng ths agorthm was generay around 8:1 wth an mage quaty much ower than JPEG, one of the most we-known mage compresson standards. he compresson scheme presented by Amerjckx et a. [] based on vector quantzaton (VQ) of the dscrete cosne transform (DC) coeffcents by the Kohonen map, dfferenta codng by frst order predctor and entropc codng of the dfferences gave better performance than JPEG for compresson ratos greater than 30:1. Robnson and Kecman n [3] and [4] have used mage compresson agorthms based on SVM earnng of the DC coeffcents. he method has produced better mage quaty than JPEG n hgher compresson ratos. Compresson based on DC has some drawbacks as descrbed n the foowng secton. he atest standard of st mage compresson JPEG000 uses the state-of-theart dscrete waveet transform (DW) technoogy wth the vew of overcomng these mtatons. In ths paper an mage compresson agorthm based on waveet technoogy s proposed that uses the support vector machne earnng agorthm to acheve the goa. he resut of compresson s qute satsfactory and asprng.. Dscrete Waveet ransform Bock-based DC technques are usuay suffered from bockng artfacts at hgher compresson ratos (ow bt rates). On the other hand, compressons based on Waveet technques provde substanta mprovement n pcture quaty at ower bt rates [5]. If f(t) s any square ntegrabe functon satsfyng f(t) dt < (1) the contnuous tme waveet transform of f(t) wth respect to a waveet s defned as 1 t W Ψ τ a a ( a, τ ) f ( t) dt where the rea varabes a and τ are daton and transaton parameters, respectvey, and denotes compex conjugaton [6]. ()

2 he waveet may be defned as Ψ aτ 1 / t τ (3) () t a Ψ a and the nverse dscrete tme waveet transform as f ( m) / d( k, ) Ψ m ( m ) (9) he functon, referred to as the mother waveet, satsfes two condtons t ntegrates to zero and s square ntegrabe, or has fnte energy. In the waveet transform, the wndow sze n the tme doman vares wth frequency,.e., onger tme wndow for ower frequency and shorter tme wndow for hgher frequency. For mage data, tme-frequency pane concept becomes a space-frequency pane. he waveet transform aows the spata resouton and frequency bandwdth to vary n the space-frequency pane thereby resuts n achevng better bt aocaton for actve and smooth areas. For mage compresson usng DC one major dffcuty s to choose the bock sze. he choce of the bock sze s a trade off between handng actve areas and smooth areas of the mage. It s preferred to represent f(t) as a dscrete superposton sum rather than an ntegra for dgta mage compresson. Equaton (3) now becomes Ψ k, () t Ψ ( t ) / (4) he decomposton of an mage usng dscrete waveet transform comprses of a chosen ow pass and a hgh pass fter, known as Anayss fter par. he ow pass and hgh pass fters are apped to each row of data to separate the ow frequency and the hgh frequency components. hese data can be sub-samped by two. he fterng s then done for each coumn of the ntermedate data fnay resuts n a two dmensona array of coeffcents contanng four bands of data, known as owow (LL), hgh-ow (HL), ow-hgh (LH) and hgh-hgh (HH). Each coeffcent represents a spata area correspondng to one-quarter of the orgna mage sze. he ow frequences represent a bandwdth correspondng to 0< ω <π/, whe the hgh frequences represent the band π/< ω <π. It can be possbe to decompose the LL band n the same way up to any eve, resutng n pyramd-structured decomposton as shown Fg 1. he LL band at the top of the pyramd contanng approxmate coeffcents hods the most sgnfcant nformaton and the other bands contanng detas coeffcents have esser sgnfcance. hus the degree of sgnfcance s decreasng from the top of the pyramd to the bands at the bottom. where a k and τ k for dscrete space wth k and both ntegers. he correspondng waveet transform can be rewrtten as ( k, ) f ( t) ( t) dt W Ψ k (5) LL LH HL HH LL HL LH HH LH HL HH and the nverse transform as f ( t) / d( k, ) Ψ k ( t ) (6) a) 1st-eve decomposton b) nd-eve decomposton Fg. 1. wo-dmensona waveet transform. he vaues of the waveet transform at those a and τ are represented by d(k,) W(k,)/C (7) he d(k,) coeffcents are referred to as the dscrete waveet transform of the functon f(t). If the descretzaton s aso apped to the tme doman ettng t m, where m s an nteger and s the sampng nterva chosen accordng to Nyqust sampng theorem, then the dscrete tme waveet transform s defned as m wd ( k, ) f ( m) Ψ ( m) k (8). Support Vector Machne Learnng Support vector (SV) machnes deveoped by Vapnk [8] can be used not ony for cassfcaton probems but aso for regresson anayss,.e., functon estmaton [9]. he SV machne mpements the dea of mappng the nput vectors x nto a hgh-dmensona feature space Z through some chosen nonnear mappng. SV approxmaton to regresson takes pace f Regresson s estmated n the set of near functons f(x,w) (w.x) + b, the probem of regresson estmaton s defned as that of rsk mnmzaton wth respect to an ε-nsenstve

3 (ε 0) oss functon L where y f ( x, w ) 0, y f ( y f ( x w) ) y f ( x,w) ε ( x, w ), (10) ε f y f ( x, w ) ε (11) - ε, otherwse. tranng ponts above and beow an ε - tube. For data ponts nsde the tube, both mutpers equa zero. he constant C, whch nfuences a trade-off between an approxmaton error and the weghts vector norm w, s a desgn parameter chosen by the user. Insenstvty zone ε s the another most reevant earnng parameters that can be utzed n constructng SV machnes for regresson. Increase n ε decreases the number of SVs at the cost of accuracy of approxmaton. he constraned optmzaton probem can now be soved by formng a prma varabes Lagrangan L p (w, ξ, ξ) L P ( w, b, ξ, ξ, α, α, β, β ) 1 w w + C ξ + ξ 1 1 α [ y w x b + ε + ξ ] α [ w x + b y + ε + ξ ] ( β ξ + βξ ) (14) Fg.. Support Vector Machne Parameters. he ε -nsenstvty oss functon (Equaton 10) defnes an ε - tube (Fg. ). If the predcted vaue s wthn the tube, the oss s zero. In sovng regresson probems, SVM performs near regresson n n-dmensona feature space usng ε -nsenstvty oss functon. At the same tme, t tres to reduce mode capacty by mnmzng w, n order to ensure better generazaton. hese can be acheved by mnmzng rsk R 1 Rw, ξ, ξ w + C ξ + ξ 1 1 where C s a constant under constrants y w w x b ε + ξ, x + b y ξ 0, ξ 0, ε + ξ, 1,...., 1,...., 1,...., 1,...., (1) (13) hs probem can be soved n a dua space [3] and the souton may be gven by NSV f ( x, y) α 1 ( α ) G( x, x) subject to constrants 0 α C, 1,..., 0 α C, 1,..., (15) where N SV s the number of support vectors and G(x,x) s the kerne functon. 4. Proposed Method hs secton expans the proposed agorthm for compressng the coeffcents found by appyng dscrete waveet transform on an mage data. he expermenta Image D DW Approxmate coeffcents Detas coeffcents SVM SVs Weghts Quantzaton Encodng where ξ and ξ are sack varabes and postve vaued, shown n Fg. for measurements above and beow an ε - tube, respectvey. Lagrange mutpers α and α, correspondng to ξ and ξ, w be nonzero vaues for Fg. 3. Schematc dagram of the proposed mage compresson agorthm.

4 mage s ted nto bocks, te sze of mage-bock beng chosen by the user. he compete mage may aso be treated as a bock. he two-dmensona dscrete waveet transform s apped on each te treatng them as one snge mage. he haar waveet has been used n ths paper. he other waveets may aso be found sutabe. he resutng approxmate coeffcents and detas coeffcents are then stored for each sub-mage. Support vector machne agorthm for regresson anayss s then apped to each matrx of coeffcents. he SV machne produces a mnmum number of SVs requred to generaze the tranng data wthn a predefned error (shown n Fg as ε-nsenstvty tube). It s found from the experment that SVM performs better for not too arge sets of coeffcents as ts tranng data. he coeffcents are then quantzed n predefned eves and encoded usng Huffman codng prncpe. he proposed compresson agorthm s shown schematcay n Fg Smuaton Resuts he gray-scae peppers mage of sze 51 X 51 (shown n Fg. 4a) has been taken to test the compresson capabty of the proposed method. he mage to be compressed s frst ted nto some bocks, for exampe t may be dvded nto bocks of 18 X 18 sub-mages, or the whoe mage may be treated as one bock. Dvdng the mage nto bocks mproves tme compexty for compressng the mage. he two-dmensona dscrete waveet transform ss apped on the sub-mages treatng each of them as a compete ndvdua mage. It gves the approxmate coeffcents, CA and detas coeffcents, DH, DV and DD. For the experment, the whoe mage of sze 51 X 51 was treated as a bock and after appyng DW the coeffcent matrces generated were each of sze 56 X 56. he support vector regresson earnng agorthm was apped on each set of coeffcents. It was found that whe appyng SVM earnng, tme compexty ncreased for data sets more than a certan mt (such as for 56 X 56 coeffcent matrx of the mage n Fg. 4a) and the performance aso deterorates. o overcome ths, the coeffcent matrces were dvded nto bocks and then the SVM earnng was apped ndvduay on each bock. hs process has not ony mproved tme compexty but aso provded better generazaton. After the SVM regresson agorthm was apped to the coeffcent matrces for a predefned accuracy, two sets of coeffcents for each matrx, namey, support vectors (SVs) and correspondng weghts were acheved accordng to equaton (15). hese were then quantzed and encoded. a) Orgna (51 X 51) Compresson Rato 36:1 Compresson Rato 35:1 PSNR 8.34 PSNR 8.19 Compresson Rato 4:1 Compresson Rato 40:1 PSNR PSNR 7.1 b) Proposed Method c) JPEG Fg. 4. Resuts of Experments. In the reverse process, the mage was reconstructed foowng the decodng and dequantzaton process and by usng the weghts acheved thereby. hus greater compresson rato s possbe to acheve. Fg. 4 shows the orgna mage and the resut of compresson ndcatng compresson rato and peak sgna to nose rato (PSNR). Fg. 5 shows the three dmensona mesh pot of the orgna and the recovered mage wth 4:1 rato of compresson. 5. Concuson It may be concuded from the resuts of the experment that the proposed method of compressng st mages has shown asprng performances. he proposed method has been performed better than JPEG. However, the

5 comparson of compresson rato and pcture quaty wth the atest standard, such as JPEG000, s yet to be done. he method s very much n the eary stage and there are severa areas to mprove the method n order to acheve better pcture quaty and hgher rato of compresson, such as appcaton of approprate encodng technques and waveets, seecton of sutabe SV kerne. (a) (b) References: [1] M H Hassoun, Fundamentas of Artfca Neura Networks, Cambrdge, MA: MI Press, [] C. Amerjckx, M. Vereysen, P. hssen, and J. Legat, Image Compresson by sef-organzed Kohonen map, IEEE rans. Neura Networks, vo. 9, pp , [3] J. Robnson and V. Kecman, Combnng Support Vector Machne Learnng wth the Dscrete Cosne ransform n Image Compresson, IEEE ransactons on Neura Networks, Vo 14, No 4, Juy 003. [4] Jonathan Robnson, he Appcaton of Support Vector Machnes to Compresson of Dgta Images, PhD dssertaton, Schoo of Engneerng, Unversty of Auckand, New Zeaand, February 004. [5] D S aubman, M W Marcen, JPEG000: Image Compresson Fundamentas, Standards and Practce, Kuwer Academc Pubshers, 00. [6] Y Q Sh and H Sun, Image and Vdeo Compresson for Mutmeda Engneerng Fundamentas, Agorthms and Standards, CRC Press LLC, 000. [7] D. Cha and A. Bouzerdoum, JPEG000 Image Compresson: An Overvew, Austraan and New Zeaand Integent Informaton Systems Conference (ANZIIS'001), Perth, Austraa, pp Nov [8] V N Vapnk, he Nature of Statstca Learnng heory, Sprnger, 000. [9] V Kecman, Learnng and Soft Computng: Support Vector Machnes, Neura Networks, and Fuzzy Logc Modes, he MI Press, 001. Fg. 5. 3D mesh pot of the mage (a) Orgna Image (b) Recovered Image wth 4:1 compresson.