Introduton Ths work addresses the problem of eent 2-D lnear lterng n the dsrete Cosne transform (DCT) doman, whh s an mportant problem n the area of p

Transcription

1 Fast DCT Doman Flterng Usng the DCT and the DST Renato Kresh and Ner Merhav HP Israel Sene Center y Abstrat A method for eent spatal doman lterng, dretly n the DCT-IIe doman, s developed and proposed. It onssts of usng the dsrete sne transform (DST), together wth the dsrete osne transform (DCT), for transform doman proessng, based on the reently derved onvoluton-multplaton propertes of dsrete trgonometr transforms. The proposed sheme requres no zero paddng of the nput data, or kernel symmetry. It s demonstrated that, n typal applatons, the proposed algorthm s sgnantly more eent than the onventonal spatal doman method. The method s applable to any DCT based data ompresson standard, suh as JPEG, MPEG, and H.26. Keywords: DCT-doman lterng, dsrete sne transform, data ompresson. Whle on sabbatal leave at Hewlett-Pakard Laboratores, 5 Page Mll Road, Palo Alto, CA 9434, USA. y Address: HP Israel Sene Center, Tehnon Cty, Hafa 32, Israel. E-mal: [renato,merhav]@hp.tehnon.a.l

2 Introduton Ths work addresses the problem of eent 2-D lnear lterng n the dsrete Cosne transform (DCT) doman, whh s an mportant problem n the area of proessng and manpulaton of mages and vdeo streams ompressed n DCT-based methods, suh as JPEG, MPEG, H.26, and others (see, e.g., [-7]). More speally, suppose that an nput mage s gven n the format of a sequene of sets of DCT oeents of 2-D bloks. We assume the DCT to be of type II-e, whh s the type used n JPEG/MPEG/H.26 applatons. We are nterested n alulatng eently a ltered mage, n the same format, that orresponds to spatal onvoluton between the nput mage and a gven lter. The standard spatal approah onssts of alulatng the nverse DCT of the data, performng the onvoluton n the spatal data, and then transformng the result bak to the DCT doman. Our am s to operate dretly n the DCT doman, avodng explt transformatons from the DCT doman to the spatal doman and ve versa. One approah for dealng wth ths problem s by means of a onvoluton-multplaton property (CMP) for the DCT. Lke CMP's for the dsrete Fourer transform (DFT), the am of usng a CMP for the DCT s to turn the lterng operaton nto a smple elementby-element multplaton n the transform doman. Chen and Fralk [] derved one suh DCT-CMP, and Ngan and Clarke [2] have used t for low-pass lterng of mages. Later on, Chtprasert and Rao [3] derved another DCT-CMP, smpler than the prevous one. A thorough study of CMP's was performed by Martu n [4], where he derved CMP's nvolvng all types of DCT's and DST's (dsrete sne transforms), nludng the prevous CMP's as partular ases. Sanhez et al. [6] rewrote part of the CMP's (some those nvolvng DCT's only) n the form of matrx dagonalzaton propertes, and studed ther asymptot behavor. The problem wth almost all the shemes proposed n the above works s that they do not mplement lnear onvoluton, but rather a \folded" type of onvoluton, alled symmetr onvoluton. An exepton s the lnear onvoluton sheme proposed by Martu [4, 5]; he showed that one an smulate lnear onvolutons by means of symmetr onvolutons, through approprate zero-paddng of the spatal data. Unfortunately, n our applaton, the data s already provded n the transform doman wthout pror zero-paddng, and therefore Matu's sheme annot be used. Another dsadvantage of the above methods s that they are lmted to symmetr or antsymmetr lters. The works of Lee and Lee [7], Chang and Messershmtt [8], and Ner et al. [9] provde an alternatve to the CMP approah. They propose to preompute the produt of the operator matres orrespondng to nverse DCT, the onvoluton, and the DCT, and then to use the ombned operator matrx dretly n the DCT doman. The eeny of ths approah depends heavly on the sparseness of the DCT data; proessng of non-sparse DCT bloks an be of hgh omputatonal omplexty. In ths work, we follow the CMP approah, and propose an eent lterng sheme of the DCT data, not requrng pror zero paddng of the spatal data, and sutable for symmetr or non-symmetr lterng. The proposed sheme uses the DCT and the DST oeents of the data, based on the DCT/DST CMP's derved n [4]. Sne the DST oeents are not avalable n advane, they have to be omputed from the gven DCT data. To ths end, we develop fast algorthms that dretly transform from the DCT to the DST (denoted CST) and ve versa (SCT). Inorporatng these algorthms n the lterng sheme, we obtan an overall omplexty that, for symmetr lters, s 35-64% smaller than that of the standard spatal approah. In the non-symmetr ase, up to 64% of the omputatons are saved,

3 ompared to the spatal doman lterng method, dependng on the kernel sze. The outlne of ths doument s as follows. Seton 2 brey revews the DCT/DST onvolutonmultplaton propertes. Seton 3 presents the proposed lterng sheme, for -D sgnals. Ths s done for ddat purposes. Seton 4 extends the algorthm for 2-D mages. Frst, the ase of separable lters s onsdered, for whh ase the omplexty analyss s presented, and, then, the outlne of a nonseparable algorthm s provded. Seton 5 onludes the paper. 2 Convoluton-Multplaton Propertes of Dsrete Trgonometr Transforms There are 8 types of DCT's and 8 types of DST's dened by Wang, whh are alled generally dsrete trgonometr transforms (DTT) [4]. From the famly of DTT's, we are partularly nterested n dealng wth the DCT of type 2e, denoted DCT-IIe, sne t s the man buldng blok of the mage odng standards JPEG, MPEG, and others. Also of nterest n ths work are three other DTT's, whh are used together wth the DCT-IIe n the proposed lterng sheme. Speally, these are the DST of the same type, DST-IIe, and the DCT and DST of type e, denoted DCT-Ie and DST-Ie, respetvely. Thus, we revew here only the denton of these spe transforms, and the CMP's nvolvng them. The above transforms are dened n terms of matres that left-multply an nput vetor. The transform matres of order N of DCT-Ie, DST-Ie, DCT-IIe, and DST-IIe are denoted by C Ie, S Ie, C IIe, and C IIe, respetvely, and are dened by []: [C Ie ] mn = q 2 k N mk n os mn N ; m; n = ; ; : : : ; N () [S Ie ] mn = q 2 sn mn N N [C IIe ] mn = q 2 k N m os [S IIe ] mn = q 2 N k m sn ; m; n = ; 2; : : : ; N? (2) m(n+ 2) N m(n? 2) N ; m; n = ; ; : : : ; N? (3) ; m; n = ; 2; : : : ; N (4) where k =, for N?, and k = k N = = p 2. A laraton about the ndes s requred: For the osne matres, the ndes m and n relate to the (m + ) th row and (n + ) th olumn, respetvely, whereas for sne matres they relate to the m th row and n th olumn. Note that C IIe and S IIe are both of sze N N, but C Ie and S Ie are of sze (N + ) (N + ) and (N? ) (N? ), respetvely. All these matres are orthogonal and nvertble, the nverse matrx of eah one beng ts transpose. Martu has proposed a new formulaton of the DTT matres, alled onvoluton form [4], n whh matrx orthogonalty s lost for most DTT types. On the other hand, the onvoluton form s more approprate for presentng the DTT CMP's than the above orthogonal form derved by Wang, beause t avods the need for addng any salng fators or weghtng funtons to the CMP formul. The transform matres n onvoluton and orthogonal 2

4 nput ndex ranges output " a " b x(n) y(n) ndex range T a T b T HSHS WSWS! N?! N! N? C 2e C e C 2e HAHA WSWS! N?! N! N? S 2e C e S 2e HSHS WAWA! N?! N?! N? C 2e S e S 2e HAHA WAWA! N?! N?! N? S 2e S e?c 2e Table : Convoluton-multplaton propertes of DCT and DST of type Ie and IIe. forms are losely related; one an swth between them by left and rght-multplyng eah by non-sngular dagonal matres. The onvoluton form of the DCT-IIe, DST-IIe, DCT-Ie, and DST-Ie are denoted by C 2e, S 2e, C e, and S e, respetvely, and an be found n [4]. One last onept that we must revew, before onsderng the CMP's, s of symmetrextenson. It onssts of replatng a gven nput sequene n order to produe a symmetr output sequene. There are 6 types of symmetr-extenson operators dened n [4], four of whh are of nterest to us, denoted by HSHS, HAHA, WSWS, WAWA, aordng to the poston of the symmetry pont (`H' and `W' meanng `Half sample' and `Whole sample', respetvely), and the symmetr/antsymmetr nature of the replaton (`S' and `A' meanng `symmetr' and `antsymmetr', respetvely). The exat denton of the above operators an be found n [4]; we prefer to revew them by means of the followng example: where R performs perod replaton. HSHS(; 2; 3; 4) = R(; 2; 3; 4; 4; 3; 2; ); HAHA(; 2; 3; 4) = R(; 2; 3; 4;?4;?3;?2;?); WSWS(; 2; 3; 4) = R(; 2; 3; 4; 3; 2); WAWA(; 2; 3; 4) = R(; 2; 3; ;?3;?2); We now revew the CMP's nvolvng the above transforms, aordng to [4]. There are four of these propertes, and they assume the followng format: w n = " a fx n g " b fy n g = T? ft a fx n g T b fy n gg; (5) where fx n g and fy n g are two nput sequenes, and fw n g s the output onvolved sequene. In the above expresson, " a and " b are two symmetr extenson operators, and denotes rular onvoluton. Moreover, T a, T b, and T? are two DTT's and one nverse DTT, approprately seleted among the above spe DTT's, and denotes element-by-element multplaton. Table lsts the spe values for the four CMP's. Equaton (5) tells us that symmetr onvoluton (whh means rular onvoluton between symmetr-extended versons of the operands) an be obtaned by transformng the nput sgnals by DTT's, multplyng the results, element-by-element, and then performng an nverse DTT. Equaton (5) s a smpled verson of the equaton provded n [4], adapted for the spe four DTT's that we are nterested n. 3

5 3 The -D Flterng Sheme 3. Matrx Formulaton of the Convoluton-Multplaton Propertes Some of the DTT CMP's presented n [4] were onverted nto matrx form n [6]. Speally, part of the CMP's nvolvng only DCT's were wrtten n the format: Y a = C? a D (C b y) C a ; (6) where Y a s a matrx whh performs a symmetr onvoluton of an nput sgnal by the sgnal y, D(x) s the dagonal operator, whh returns a dagonal matrx whose dagonal s lled wth the elements of x, and C a and C b are two DCT's n onvoluton form. Note that the nverse DCT s equal to one of the forward DCT's, a property whh s not neessarly extended for smlar matrx formulaton for the rest of the DTT CMP's. The meanng of (6) s that the symmetr onvoluton matres Y a are dagonalzed by the DCT's C a, the egenvalues beng the oeents of the transform of y by C b. Ths s analogous to rular onvoluton matres beng dagonalzed by the DFT matrx, wth egenvalues gven by the DFT oeents of the kernel. In [6], the above matrx formulaton for DCT matres n onvoluton form was adapted for DCT matres n orthogonal form as well, where saled and weghted versons of Y a are the matres whh are dagonalzed. The rst out of the four CMP's revewed n Seton 2 nvolves DCT's only, and thus, t an be wrtten n matrx form aordng to (6) (ths s expltly presented n [6]). On the other hand, the remanng three CMP's do not satsfy ths requrement, but we wsh to wrte them n matrx form as well. We adapt now equaton (6), n order to summarze the matrx form (dagonalzaton property) of all four CMP's we are nterested n. Thus, the dagonalzaton propertes wll assume the followng format: Y ;a = T? D`(T b y) T a ; (7) where T a, T b, and T are three DTT's (not neessarly DCT's, and not neessarly n onvoluton form), and Y ;a s a symmetr onvoluton matrx, dependng on the ndes a and. In the sequel we detal the spe values and formats those matres assume for the four desred CMP's. The operator D` s a generalzaton of D; t returns a square matrx, havng ts `th dagonal lled wth the elements of the nput vetor. The `th dagonal of a square matres A j onssts of the elements for whh j = + `. In (7), we requre T a and T to express type II-e matres n orthogonal form, wth the purpose of applyng them dretly to gven DCT-IIe nput data. For ths reason, we smplfy the notaton for the DCT-IIe and DST-IIe matres, denotng them C and S, respetvely. That s, C = C IIe and S = S IIe, by denton. Moreover, T b shall express ether DCT or DST matrx of type Ie, assumng the followng format, respetvely: [C I ] mn = 2 os mn ; m; n = ; ; : : : ; N (8) N [S I ] mn = 2 sn mn N ; m; n = ; 2; : : : ; N? : (9) It turns out that C I s almost dental to C e, whereas S I s equal to S e. 4

6 As for the four symmetr onvoluton matres Y ;a, t turns out that eah an be wrtten as a ombnaton of the followng four trangular matres: Y = Y y y y.. y N?2 y y N? y N?2 y y y y y N?2 y N? y yn? y y... y C A C A ; Y 2 = ; Y y y 2 y N? y N y 2 y 3 y N.. y N? y N y N y N y N y N?.. y N y 2 y N y N? y 2 y C A C A ; () : () The reader may note that the matres Y, Y 2, Y 3, and Y 4 are upper-rght, lower-rght, lowerleft, and upper-left trangular, respetvely. Moreover, Y 3 s smply the transpose of Y, whereas Y 4 s also a transposton of Y 2, but wth the upper-left to lower-rght dagonal as symmetry axs, nstead of the usual upper-rght to lower-left dagonal. y arefully onvertng the CMP's gven n (5) and Table to the matrx form shown n (7), we obtan the desred four dagonalzaton propertes: nc o Y ; = Y + Y 3 + Y 2 + Y 4 = C t I D y C (2) ;:::;N? nc o Y s;s = Y + Y 3? Y 2 + Y 4 = S t I D y S (3) ;:::;N Y ;s = Y? Y 3? Y 2 + Y 4 = C t D? S I y S (4) Y s; = Y? Y 3 + Y 2? Y 4 =?S t D S I y C (5) These relatons are the bass of the lterng sheme developed n the next setons. 3.2 Spatal Flterng Sheme Suppose a -D sgnal x(n) s gven, and we wsh to perform ts lnear lterng by a kernel h(n), obtanng the output sgnal w(n), that s, w(n) = x(n) h(n). Suppose also that x(n) s gven as a set of vetors fx g, onsstng of non-overlappng N-pont segments of the nput sgnal, and that we wsh to obtan the output w(n) n the same format,.e. fw g. Assumng that the lter h(n) has regon of support wthn the nterval [?N; N], where t assumes the values h(n) = h n, for a gven sequene fh n g, one an express the lterng operaton as followng: w = H x? x x = h H + 2 H + + H? 2 H? x? x x (6) 5

7 where H +, H + 2, H? 2, and H? H + 2 = = H + 2 x? + H + x + H? x + H? 2 x + (7) h N h N? h 2 h h N. h 2. h N h N? h N are dened as follows. C A h h? h 2?N h?n h. h 2?N. h h?... h ; H + = C ; H? 2 = h h h.. h N?2 h h N? h N?2 h h?n h?n h?n.. h?2 h?n h? h?2 h?n h?n where s an arbtrary real number, and =?. In the general ase onsdered heren the hoe of s mmateral from the aspets of omputatonal eeny. In ertan speal ases that wll be studed later, however, the hoe of wll be mportant. Note that, f we set y = h n (), for 6=, and y = h, then we obtan Y = H + Y 2 = H 2 +, where s a \ppng matrx", gven by: C A C A ; (8) C A ; (9) and : (2) It s easy to verfy that rght-multplyng a matrx A by nverts the order of the olumns of A. Smlarly, left-multplyng a olumn vetor by nverts the order of ts elements. The above relatonshp between Y, Y 2, H +, and H 2 + permts us to use (2)-(5) to rewrte the latter two matres n terms of the DCT-IIe and DST-IIe transforms. Speally, we note that Y 2 = (Y?Y ss?y s +Y s )=4, and Y = (Y +Y ss +Y s +Y s )=4, and therefore: H + = (C t H + C? S t H + sss? C t H + ss? S t H + sc); (2) H 2 + = C t H + C + S t H + sss + C t H + ss? S t H + sc: (22) where the spe values of the lterng kernels H + p, p 2 f; ss; s; sg, are gven n Table 2. 6

8 p H + p H? p 4 D fc I [h ; h ; : : : ; h 8 ]g ;;7 ss 4 D fc I [h ; h ; : : : ; h 8 ]g ;;8 s s 4 D 4 D fc I [h ; h? ; : : : ; h?8 ]g ;;7 4 D fc I [h ; h? ; : : : ; h?8 ]g ;;8 4 D? SI [h ; : : : ; h 7 ] 4 D? SI [h? ; : : : ; h?7 ] SI [h ; : : : ; h 7 ] 4 D SI [h? ; : : : ; h?7 ] Table 2: Flter kernels. Smlarly, after settng y =?h n (), for 6=, and y = h, we obtan Y 3 = H? and Y 4 = H? 2, and by usng the relatons Y 3 = (Y? Y ss + Y s? Y s )=4 and Y 4 = (Y + Y ss? Y s? Y s )=4, we obtan: H? = (C t H? C? S t H? sss + C t H? ss + S t H? sc); (23) H 2 + = C t H? C + S t H? sss? C t H? ss + S t H? sc; (24) where the values of H? p, are also shown n Table 2. A lterng sheme s then obtaned by replang (2)-(24) nto (7), and rearrangng the terms: w = C t h H + C(x + x? ) + H + ss(x? x? ) + H? C(x + x + )? H? ss(x? x + ) + S t h H + sss(x? x? )? H + sc(x + x? ) + H? sss(x? x + ) + H? sc(x + x + ) : (25) The mplementaton steps of the above sheme, aordng to (25), are as follows. Frst, alulate x + x?, x + x +, x? x?, and x? x +, whh onsst of \foldng" the adjaent spatal segments x? and x + onto the urrent one, x. We all t \foldng" beause the operator smply nverts the order of the elements of the nput vetor. Next, alulate the DCT-II of the postvely folded data, and the DST-II of the negatvely folded data. Then, operate upon the transform data by approprately multplyng t, element-byelement, by the lter kernels. And nally, alulate the nverse DCT-II and nverse DST-II of some ombnatons of the multpled data. 3.3 Flterng Dretly n the DCT Doman The algorthm derved n the last seton s sutable to be used n applatons where the nput data s gven n the spatal (or tme) doman. However, n our spe applaton, the data s gven n the DCT-IIe doman n the form of segments (vetors) X = Cx, and therefore ths algorthm, n ts present form, s not approprate here. In ths seton, we adapt the above algorthm n order to apply t to the problem of lterng dretly n the DCT doman. Ths adaptaton onssts of three steps: To onvert the above algorthm to the DCT doman, to derve fast CST and SCT transforms, and to norporate 7

9 part of the CST and SCT transforms nto the kernel lters and the quantzaton and dequantzaton matres of the JPEG/MPEG odng algorthm to save multplatons. Those steps are desrbed below. Converson to the DCT Doman Frst, we pre-multply both sdes of (25) by C, n order to obtan the output n the DCT- IIe doman. Then, we use the lnearty of matrx multplaton to wrte C(x + x? ) = X + Cx?, and smlarly to all the nput terms. And, nally, we note that C = C and S = S, where = D (f(?) m g m=;:::;n? ) : (26) After norporatng all the above steps n (25), we obtan the followng DCT-doman sheme: W = H + (X + X?) + H + s(x s? X s?) + H? (X + X?)? H? s(x s? X s?) + T t h H + ss(x s? X s +)? H + s(x + X +) + H? ss(x s? X s +) + H? s(x + X +) (27) where T = SC t s nterpreted as the -D DCT-to-DST doman transform (CST) operator matrx (hene T t s the SCT operator matrx), and fx s g are the set of DST-IIe oeents of the nput data,.e., X s = Sx, for all. Equaton (27) yelds an eent lterng sheme provded that the SCT an be mplemented eently. Ths s true beause all the H-matres n (27) are dagonal. Note, that multplaton by s ostless. Nevertheless, unlke the DCT oeents, the DST oeents are not avalable n advane and therefore a fast CST s also requred for obtanng these matres. Ths s solved n the sequel. Eent CST and SCT Algorthms In ths seton we derve eent CST and SCT algorthms,.e., fast multplaton by T and T t. We wll assume N = 8, whh s the ase n JPEG/MPEG applatons. The man dea s to fatorze T nto produts of sparse matres. Frst, we shall use the followng property relatng the DST matrx to the DCT matrx: S = C (28) where and are dened n (2) and (26), respetvely. Ths equalty holds for any value of N. In addton, we shall use a fatorzaton of C that orresponds to the fastest exstng algorthm for 8-pont DCT due to Ara, Agu, and Nakajma [] (see also [2]). Aordng to ths fatorzaton, C s represented as follows. C = DP 2 MA A 2 A 3 (29) 8

10 where D s a dagonal matrx (whh an be absorbed n the quantzaton/dequantzaton assoated wth the ompresson standards), P s a permutaton matrx,, 2, A, A 2, and A 3 ontan only addtons, and M ontans 5 nontrval multplatons. Thus, we have T = SC t = CC t = DP 2 MA A 2 A 3 A t 3A t 2A t M t t 2 t P t D t (3) The proposed CST algorthm s based on the observaton that the produt s a farly sparse matrx, gven by: G G = MA A 2 A 3 A t 3A t 2A t M t (3) 2?5:2264 2:648 2?2 :442 :824 2:632 2?5:2264 :824?2 2:648 2: :442 2 Usng part of the nter-relatons between the elements of G, one an mplement the multplaton of a vetor by G wth 8 multplatons and 2 addtons. As a onsequene of the above, we an deompose T aordng to T = DP 2 G t 2 t P t D t, or equvalently, aordng to: T = ^D ~ T D; (32) where ~T = P 2 G t 2 t P t ; (33) and where ^D = D s a dagonal matrx havng the same elements as D but n reversed order. The matrx ~T represents a moded CST, where pre and post multplatons by ^D and D, respetvely, are stll requred n order to produe the orgnal CST (T ). The reason why we wrte T n terms of ~T n (32) s that one last modaton of the lterng sheme makes t possble to avod the multplatons by D and ^D,.e., we an use the moded CST nstead of the orgnal one. Ths s shown n below. The operaton by ~T takes 8 multplatons and 28 addtons, whh s 6 multplatons and 3 addtons less than what the operaton by T takes. The moded 8-pont SCT, orrespondng to T t, an be obtaned by ^T t = ~T, and, therefore, an be mplemented wth the same number of operatons as ~T. Inorporatng the Weghtng Fators nto the Flter Kernels and Quantzaton Tables As mentoned before, we present the algorthm here for -D sgnals for ddat reasons. Our purpose s to extend t for the 2-D ase, where t an be appled to JPEG/MPEG data 9 C A

11 (see Seton 4 below). In those odng applatons, the nput DCT data are the output of a dequantzaton proess, whh onssts of multplyng quantzed data elements by the elements of a look up table, alled dequantzaton table. Moreover, the output DCT data s supposed to be onverted bak nto quantzed data, by multplyng t by another look up table, alled quantzaton table. In ths seton, we show how to mplement the CST transform, avodng the multplatons by the matres ~ D and D, by absorbng them n the kernel matres H r, 2 f; ss; s; sg and r 2 f+;?g, and n quantzaton/dequantzaton look-up tables. We assume, therefore, that eah nput vetor X s the result of left-multplyng a ertan quantzed vetor by a dagonal dequantzaton matrx Q d. Moreover, the output W serves as an nput to to a quantzaton proess that onssts of left-multplyng t by a dagonal quantzaton matrx Q q. Suppose we alter the above tables n the followng way: Q d D Q d ; Q q Q q D; (34) thus, gven the same quantzed nput data as prevously, the dequantzed nput segments are now gven by X ~ = DX, nstead of X ; smlarly, the output ltered segments must now be gven by W ~ = D? W, n order to produe the same quantzed output data as prevously. In addton, we produed moded DST segments X ~ s, whh are alulated from s the moded DCT segments by: X ~ = ~T X ~. We now left-multply both sdes of eq. (27) by D?, and use (32) to obtan: ~W = ~ H + ( ~ X + ~ X?) + ~ H + s( ~ X s? ~ X s?) + ~H?? s s ( X ~ + X ~?)? H ~ s( X ~? X ~?) + ~T h t + s s + H ~ ss( X ~? X ~ +)? H ~ s( X ~ + X ~ +) + ~H? ss( ~ X s? ~ X s +) + ~ H? s( ~ X + ~ X +) (35) r where the H ~ are moded versons of the orgnal H lter kernels. Speally, H ~ D? H r ad? r, H ~ s = D? H r ^D, r ~ a H ss = ^DH r ^D, a and H ~ r s = ^DH r ad?, where r 2 f+;?g. The advantage of mplementng the sheme aordng to (35) nstead of (27) reles on the fat that the moded CST and SCT are omputatonally more eent than the orgnal transforms. 3.4 Dsusson and Partular Cases Equaton (35) represents the proposed lterng sheme, n the -D ase. Its mplementaton onssts of the followng steps, for every. Frst, the moded DST segments ~ X s?, ~ X s, and ~X s + must be alulated from the orrespondng nput moded DCT segments (we assume that the dequantzaton table has already been moded as speed n (34)). Atually, one =

12 s s an assume that the values of X ~? and X ~ have already been alulated and stored n the s prevous teratons? and? 2. Therefore, only X ~ + has to be atually alulated (and stored for two teratons). The moded DST segment s obtaned by the moded DCT segment by left multplyng t by ~T. The seond step onssts of reatng the \butteres" ndated by the expressons nsde the parentheses n (35); they are equvalent to the \data foldng" performed n the spatal verson of the algorthm (eq. (25)), but performed dretly n the transform doman. Note that ppng the order of elements n a spatal segment s obtaned dretly n both the DCT and the DST domans by a trval modulaton the transform data (left multplaton by ). Next, the approprate dagonal kernel lters operate upon the butteres, and nally, part of the ltered data s onverted from the DST doman to the DCT doman by left multplyng them by T t, whle the other part of the data s already n the DCT doman. These parts are then added. Important speal ases of the above algorthm are the symmetr/antsymmetr, ausal/antausal, and ausal-symmetr ases. They related to the nature of the lterng kernel. In the symmetr ase, h?n = h n, for n = ; : : : ; 8. Here, by settng = = =2 n (8) and (9), we obtan H ~ + p = H ~? p, for p 2 f; ss; s; sg. Therefore, n ths ase, by denng + ~H p = H ~ p, for all p, the sheme n (35) s redued to: ~W s s = H ~ [2 X ~ + ( X ~? + X ~ +)]? H ~ s ( X ~?? X ~ +) + ~T n t H ~ ss [2 X ~ s? ( X ~ s? + X ~ s +)]? H ~ s ( X ~?? X ~ +) o (36) y omparng (36) wth (35), one an note that about half of the omputatons are saved when usng a symmetr kernel. In the antsymmetr ase,.e., h?n =?h n, n = ; : : : ; 8, h =, a smlar sheme s obtaned, wth some sgn hanges. In the ausal ase, dened by h n = for n <, settng =? = n (8) and (9), gves H? = H? s = H? ss = H? s =, where s the 8 8 null matrx. Inorporatng ths fat nto the lterng equaton (35), one obtans ~W + + s s = H ~ ( X ~ + X ~?) + H ~ s( X ~? X ~ +) + ~T h t + s s + H ~ ss( X ~? X ~ +)? H ~ s( X ~ + X ~ +) ; (37) whh s also a sgnant smplaton. Note that, n ths ase, ~ X? s not needed n the omputaton; only ~ X and ~ X +. The antausal lterng sheme, where h n =, for n >, s obtaned smlarly, by settng the H + -type matres to zero, nstead of the H? -type ones. The best speal ase of the proposed sheme, n terms of omplexty, s obtaned wth a 4-pxel delayed ausal-symmetr lter, for whh both the ausalty and the symmetry propertes an be used to save omputatons. A k-pxel delayed ausal symmetr lter wll be dened as a ausal lter fh n g k n= wth h n = h 2k?n for all n 2k. Obvously, a ausal symmetr lter s a delayed verson of a non-ausal lter that s symmetr about the orgn. In the ausal symmetr ase, when k = 4, the omputatonal benets of both symmetry and ausalty are ombned. Speally, on the top of the smplaton due to ausalty for

13 =, we also have h C I [h ; h ; : : : ; h N ] t = ; k = 2; 4; 6; 8; (38) k h S I [h ; h ; : : : ; h N ] t = ; k = 2; 4; 6: (39) k Therefore, the matres ~ H + p, p 2 f; ss; s; sg, have only 4 nonzero elements eah. Ths ase an be of muh nterest for fast symmetr onvoluton wth a short kernel, when the user does not mnd a 4-pxel translaton of the resultng mage (see dsusson n Seton 4.2). 4 The 2-D Flterng Sheme The -D sheme derved n the last seton s extended here for 2-D sgnals. As mentoned before, the purpose of ths extenson s to apply the sheme to DCT bloks obtaned durng the deodng of JPEG or MPEG data. Therefore, we suppose now the nput and output data beng gven as a set of 8 8 DCT bloks fx ;jg and fw ;jg, respetvely. 4. Separable Flterng Let us onsder rst a separable approah, where the 2-D lter kernel s separable,.e., the orrespondng spatal lterng operaton an be wrtten n the form: w ;j = V x?;j? x?;j x?;j+ x ;j? x ;j x ;j+ x +;j? x +;j x +;j H t ; (4) where fx ;j g and fw ;j g are, respetvely, the nput and output data n the spatal doman, H s a 8 24 lter matrx, as dened n (6)-(9), for a gven sequene fh n g, and V s smlarly dened, for a gven fv n g. In ths ase, the 2-D lterng an be mplemented by means of the proposed -D sheme, by smply applyng the latter rst to the nput DCT blok olumns, and then to the rows of the resultng bloks, where the above olumn and row passes are performed usng the transform doman versons of the V and H kernels, respetvely. Spealy, the rst pass (olumn proessng) orresponds to the followng spatal proessng: z ;j = V x?;j x ;j x +;j whle the seond pass (row proessng) orresponds to: ; (4) w ;j z t ;j? z t ;j z t ;j+ 3 t 7C 5A : (42) 2

14 Therefore, the olumn proessng produes fz ;jg, whh are the 2-D DCT verson of the bloks fz ;j g, whle the row proessng produes the output fw ;jg. Although beng straghtforward, the separable 2-D extenson of the -D sheme has the followng few detals that requre areful onsderaton.. At a gven step (; j) of the algorthm, where the output blok W ;j s to be alulated, Z ;j+ has to be alulated, but not Z ;j and Z ;j?, sne they have been prevously alulated (and assumed stored) n the two prevous steps (? ; j) and (? 2; j). Therefore, durng the olumn pass orrespondng to the step (; j) we need to refer only to the nput bloks X?;j+, X ;j+, and X +;j+. 2. As shown n the prevous seton, at the begnnng of the -D algorthm, the data s transformed to the DST doman by means of the -D CST. As for the 2-D algorthm, at the begnnng of the olumn pass, only the blok olumns are operated upon by the -D CST, and not the rows; therefore, we do not get the 2-D DST oeents of those bloks, but a 2-D mxed DCT/DST doman blok X s ;j = T X ;j, orrespondng to a osne transformaton of the spatal rows and a sne transformaton of the olumns. The remanng steps of the olumn proessng s performed on these mxed bloks, besdes the orgnal DCT bloks. y the end of the olumn proessng, the resultng bloks are transformed bak to the 2-D DCT doman, gvng Z ;j. Smlarly, durng the row proessng, the algorthm operates on 2-D mxed DST/DCT doman bloks Z s ;j = Z ;jt t, besdes the orgnal 2-D DCT doman bloks Z ;j. 3. Smlarly to the stuaton n tem above, at a gven step (; j), only the mxed DCT/DST blok X s +;j+ need to be alulated at the begnnng of the rst pass, sne X s ;j+ and X s?;j+ were prevously alulated and are assumed stored. In the same way, Z s ;j and Z s ;j? are also avalable, so only the 2-D mxed DST/DCT blok Z s ;j+ has to be alulated at the begnnng of the row pass. 4. In the 2-D algorthm, lke n the -D one, the weghtng fators n D and ^D are norporated n the kernel matres and the quantzaton/dequantzaton tables, but wth a small modaton n relaton to the -D algorthm. Here, the dequantzaton (Q d ) and quantzaton (Q q ) tables are altered as follows: Q d D Q d D; Q q D Q q D; (43) and not as n (34). Moreover, the relaton between the moded kernels and the orgnal ones are gven now by Table Complexty Analyss of the Separable Sheme Data Sparseness An mportant fator to be taken nto aount n the mplementaton s that of typal sparseness of the quantzed DCT nput data bloks. We dene a DCT oeent blok as sparse 3

15 p ~V + p ~V p? ~H + p ~H p? D? V + D? D? V? D? D? H D +? D? H D?? s D? V + s ^D D? V? s ^D ^DV + s D? ^DV? s D? ss ^DV + ss ^D ^DV? ^D ss ^DH + ^D ss ^DH ss? ^D s ^DV + s D? ^DV s? D? D? V + ^D s D? V? Table 3: Relatonshps between the moded kernels and the orgnal kernels, for the 2-D separable lterng algorthm. s ^D f only ts 4 4 upper left quadrant (orrespondng to low frequenes n both dretons) ontans nonzero elements. A very hgh perentage of the DCT bloks usually satsfy ths requrement, whh s farly easy to hek dretly n the ompressed format. Inorporatng the sparseness assumpton n the above denton nto the moded CST algorthm proposed n seton 3.3, results n 7 multplatons and 7 addtons, as opposed to 8 multplatons and 28 addtons n the general ase. Complexty The omputatonal omplexty of the proposed 2-D separable lterng sheme has been alulated for the derent lterng ases, and are presented n Table 4. The dervaton detals for the ausal-symmetr ase an be found n the appendx; the other ases have smlar dervaton, whh the nterested reader an nd n [3]. In Table 4 the omplexty of the proposed sheme s also ompared to that of the straghtforward approah (alled Spatal sheme) of transformng the nput DCT bloks to the spatal doman, performng the onvoluton, and transformng bak to the DCT doman. The entres n the table are the number of salar operatons requred for proessng of eah 8 8 output blok, expressed n terms of the number of multplatons m and addtons a, and the overall operaton ount, assumng a proessor for whh eah multplaton s equvalent to 3 addtons. The lterng ases are gven n terms of data sparseness and kernel type. Non-sparse data refers to the ase where at least one of the 9 nput bloks X ~ u;v,? u; v +, s not sparse, whereas Sparse data means that all the 9 bloks are sparse. Note that the results for the spatal sheme ndepend on data sparseness. The struture of the kernel s general, symmetr, ausal, or ausal-symmetr; eah s related to a derent mplementaton of the proposed algorthm, as detaled n Seton 3.4. Note that the results for the spatal sheme are aeted only by the exstene of symmetry n the kernel, and not by whether t s ausal or not. Fnally, for eah ase, three kernel szes (n terms of L L samples) are onsdered: Small, medum, and large. For non-ausal kernels, the largest value of L s 7, whether for ausal kernels, t s 9. Note that the results for the proposed sheme do not depend on the lter sze. Stll n Table 4: ullets ndate the ases for whh the proposed sheme s preferable to the spatal one. Aordng to ths nformaton, and the other entres n the table, one an onlude the followng:. If the nput DCT data are sparse (whh s a typal stuaton), and one wshes to perform ausal or ausal-symmetr lterng (see dsusson below), then the proposed sheme s up to 64% more eent than the spatal one. 4

16 Flterng Case Sheme Kernel Type Spatal Proposed Data Struture Sze m a op. m a op. Non-Sparse General L = L = L = Symmetr L = L = L = Causal L = L = L = Causal-Symmetr L = L = L = Sparse General L = L = L = Symmetr L = L = L = Causal L = L = L = Causal-Symmetr L = L = L = Flterng shemes for whh the proposed sheme s more eent than the Spatal one,.e., the proessor operaton ount (op.) s lower for the former than the latter. Table 4: Computatonal eeny of the proposed 2-D separable sheme, as ompared to the spatal approah, gven n terms of multplatons and addtons requred for eah 8 8 output blok. The operaton ount (op.) s gven by the formula op = 3m + a, whh assumes eah multplaton to be equvalent to 3 addtons. 5

17 2. If the data are sparse, but the lterng n not ausal (or ausal-symmetr), then the proposed sheme s preferable to the spatal one for medum and large kernels. 3. If the data s not sparse, then the proposed sheme s preferable for large kernels, and also for medum kernels n the symmetr and ausal-symmetr ases. Causal Nonausal Kernels The proposed approah s muh more eent n the ausal/ausal-symmetr ases than n the general/symmetr ases, as seen n Table 4. Also, the ausal versons use half of the memory that s used n the nonausal versons of the approah, sne DCT and DST oeents are to be temporarly stored, nstead of only the spatal data. However, ausal kernels are seldom enountered n mage proessng applatons, whle the use of symmetr kernels s farly ommon. How ould one take advantage of the eeny of the ausal versons of the algorthm to mplement a nonausal lterng? If a gven symmetr 2-D lter s not longer than 9 taps n eah dmenson, then t an always be trvally transformed nto a ausal-symmetr kernel, smply by applyng a 4-pxel shft (n eah dmenson) to ts spatal oeents. In ths ase, the use of ths ausal-symmetr verson of an orgnal symmetr kernel, nstead of the symmetr kernel tself, results n a 4-pxel shft of the output mage, n the opposte dretons. Ths shft s tolerable n many applatons, espeally those nvolvng human vsualzaton, or an be ompensated n a later dsplay or prntng applaton. On the other hand, as mentoned above, ths drawbak, whh s often mnor or neglgble, provdes great savngs n omputatons (about 36%) and n memory requrements (5%). Smlar onsderatons apply also for usng ausal kernels nstead of general kernels. In ths ase, on the other hand, the shft s not restrted to 4 pxels; the smallest number of pxels than turns the kernel nto a ausal kernel an be used. 4.3 Nonseparable Sheme The DTT CMP's presented n [4] an be extended to 2-D sgnals. Although requrng labor, suh generalzaton s oneptually smple. Wth the rse n dmensonalty, however, the number of CMP's s also nreased. For nstane, the four -D CMP's presented n Seton 2 turn nto sxteen 2-D CMP's. A dervaton, smlar to the one presented n Setons 3 and 4, based now on the sxteen 2-D CMP's, leads to the 2-D Nonseparable lterng sheme. The eeny of the nonseparable approah s now under study. Although the number of CMP's was multpled by four, and therefore the number of lterng terms was nreased by the same proporton, the nonseparable sheme s not expeted to be less eent than a nonseparable spatal ounterpart. The opposte s true; the advantage of the nonseparable 2-D sheme over the spatal one s expeted to be even larger than n the separable ase. 6

18 5 Conluson In ths work, we propose an eent lterng sheme, to be appled dretly to DCT data bloks gven n JPEG/MPEG applatons. The sheme also outputs the ltered data n the same DCT format. It s based on the onvoluton-multplaton propertes of the dsrete trgonometr transforms, and t requres the alulaton of DST oeents, whh are used together wth the DCT oeents n the lterng proess. A fast CST (osne to sne transform) was thus derved, to redue the omputatonal overhead of ths operaton. Comparson between the proposed algorthm and the straghtforward approah, of onvertng bak to the unompressed doman, onvolvng n the spatal doman, and re-transformng to the DCT doman, was arred out, for separable 2-D kernels. It was demonstrated that, by takng nto aount the typal sparseness of the nput DCT-data, the proposed algorthm provdes better results for symmetr lterng, f a 4-pxel translaton of the mage n both dretons s allowed (ausal-symmetr lterng). In ths ase, 35-64% of the omputatons are saved, dependng on the kernel sze. The approah s also typally more eent for long or medum-length, non-symmetr, kernels. Twe as muh memory s requred by the proposed algorthm n omparson to the spatal sheme. A separable verson of the algorthm was also derved, and t s urrently under study. Appendx Implementaton of the Proposed Sheme for Causal-Symmetr Flterng The mplementaton of the 2-D separable verson of the proposed sheme, n the ausalsymmetr ase, an be done as follows (assume to be an 8 8 buer):. Compute ~ X s +;j+ = ~T ~ X +;j+ and store for future use as ~ X s ;j+. (Non-sparse data: 8 (8m + 28a) = 64m + 224a; sparse data: 4 (7m + 7a) = 28m + 68a) + s s 2. Compute: Odd rows of V ~ ss( X ~ ;j+? X ~ +;j+). Note that only the odd rows of the data vetors have to be summed. (Non-sparse: 32m + 32a; sparse: 6m + 6a) + 3. Compute: Even rows of? V ~ s( X ~ ;j+ + X ~ +;j+). Here only the even rows of the data vetors have to be summed. (Non-sparse: 32m + 32a; sparse: 8m + 8a) 4. Compute ~ T t. (Non-sparse: 8 (8m + 28a) = 64m + 224a; sparse: 4 (8m + 28a) = 32m + 2a) ;j+ + X ~ +;j+) + Odd rows of. (Non Compute: Odd rows of Z ~ ~ ;j+ V sparse: 32m + 64a; sparse: 8m + 6a) ( ~ X s ;j+? X ~ +;j+) + Even rows of. (Non Compute: Even rows of Z ~ ~ ;j+ V sparse: 32m + 64a; sparse: 6m + 32a) s( ~ X s 7. Compute ~ Z s ;j+ = ~ Z ;j+ ~ T t and store for future use as ~ Z s ;j. (Non-sparse: 8 (8m + 28a) = 64m + 224a; sparse: 8 (7m + 7a) = 28m + 68a) 7

19 s s 8. Compute: Odd olumns of ( Z ~ ;j? Z ~ ss. Note that only the odd olumns of the data vetors have to be summed. (Non-sparse and sparse: 32m + 32a) ;j+) ~ H + 9. Compute: Even olumns of?( Z ~ ;j+ Z ~ s) t. Here only the even olumns of the data vetors have to be summed. (Non-sparse: 32m + 32a; sparse: 6m + 6a) ;j+)( ~ H +. Compute ~T. (Non-sparse and sparse: 8 (8m + 28a) = 64m + 224a). (Non-. Compute: Odd olumns of W ~ ;j Odd olumns of + ( Z ~ ;j + Z ~ sparse: 32m + 64a; sparse: 6m + 32a) ;j+) ~ H + s s 2. Compute: Even olumns of W ~ ;j Even olumns of + ( Z ~ ;j? Z ~ (Non-sparse and sparse: 32m + 64a) + ;j+)( H ~ s) t. The total number of operatons s 52m + 28a, when sparseness s not assumed, and 296m + 688a, when sparseness s assumed. In terms of proessor operatons, the average omplexty s 286 n the non-sparse ase, and 576 n the sparse ase. 8

20 7 Referenes [] W. H. Chen and S. C. Fralk, \Image enhanement usng osne transform lterng," Image S. Math. Symp., Monterey, CA, November 976. [2] K. N. Ngan and R. J. Clarke, \Lowpass lterng n the osne transform doman," Int. Conf. on Commun., Seattle, WA, pp , June 98. [3]. Chtprasert and K. R. Rao, \Dsrete osne transform lterng," Sgnal Proessng, Vol. 9, pp , 99. [4] S. A. Martu, \Symmetr onvoluton and dsrete sne and osne transforms," IEEE Trans. on Sgnal Proessng, Vol. SP-42, no. 5, pp. 38-5, May 994. [5] S. A. Martu, \Dgtal lterng of mages usng the dsrete sne or osne transform," Optal Engneerng, Vol. 35, no., pp. 9-27, January 996. [6] V. Sanhez, P. Gara, A.M. Penado, J.C. Segura, and A.J. Rubo, \Dagonalzng Propertes of the Dsrete Cosne Transforms," IEEE Trans. on Sgnal Proessng, Vol. 43, no., pp , November 995. [7] J.. Lee and. G. Lee, \Transform doman lterng based on ppelnng struture," IEEE Trans. on Sgnal Proessng, Vol. SP-4, no. 8, pp , August 992. [8] S.-F. Chang and D. G. Messershmtt, \Manpulaton and ompostng of MC-DCT ompressed vdeo," IEEE J. Seleted Areas n Communatons, Vol. 3, no., pp. -, January 995. [9] A. Ner, G. Russo, and P. Talone, \Inter-blok lterng and downsamplng n DCT doman," Sgnal Proessng: Image Communaton, Vol. 6, pp , 994. [] K. R. Rao, and P. Yp, Dsrete Cosne Transform: Algorthms, Advantages, Applatons, Aadem Press 99. [] Y. Ara, T. Agu, and M. Nakajma, \A Fast DCT-SQ Sheme for Images," Trans. of the IEICE, E 7():95, November 988. [2] W.. Pennebaker and J. L. Mthell, JPEG Stll Image Data Compresson Standard, Van Nostrand Renhold, 993. [3] R. Kresh and N. Merhav, \Fast DCT doman lterng usng the DCT and the DST," HPL Tehnal Report #HPL-95-4, Deember