Direct Inter-Mode Selection for H.264 Video Coding using Phase Correlation

Transcription

1 Drect Inter-Mode Selecton for H.64 Vdeo Codng usng Phase Correlaton Manoranjan Paul, Member, IEEE, Wes Ln, Senor member, IEEE, Chew Tong Lau, Member, IEEE, and Bu-sung Lee, Member, IEEE Abstract The H.64 vdeo codng standard exhbts hgher performance compared to the other exstng standards such as H.63, MPEG-X. Ths mproved performance s acheved manly due to the multplemode moton estmaton and compensaton. Recent research tred to reduce the computatonal tme usng the predctve moton estmaton, early zero moton vector detecton, fast moton estmaton, and fast mode decson, etc. These approaches reduce the computatonal tme substantally, at the expense of degradng mage qualty and/or ncrease btrates to a certan extent. In ths paper we use phase correlaton to capture the moton nformaton between the current and reference blocs and then devse an algorthm for drect moton estmaton mode predcton, wthout excessve moton estmaton. A bgger amount of computatonal tme s reduced by the drect mode decson and explotaton of avalable moton vector nformaton from phase correlaton. The expermental results show that the proposed scheme outperforms the exstng relevant fast algorthms, n terms of both operatng effcency and vdeo codng qualty. To be more specfc, 8~9% of encodng tme s saved compared to the exhaustve mode selecton (aganst 58~74% n the relevant state-of-the-art), and ths s acheved wthout jeopardzng mage qualty (n fact, there s some mprovement over the exhaustve mode selecton at md to hgh bt rates) and for a wde range of vdeos and btrates (another advantages over the relevant state-of-the-art). Index Terms Moton estmaton, vdeo codng, H.64, mode selecton, phase correlaton, and moton compensaton. T I. INTRODUCTION HE VARIABLE bloc sze moton estmaton (ME) and moton compensaton (MC) n H.64/AVC [1] s a major reason for the mproved codng effcency when compared to the prevous vdeo codng standards. The bloc szes (popularly nown as nter-modes) are chosen from 16 16, 16 8, 8 16, 8 8, 8 4, 4 8, and 4 4 pxels to capture smple to complex moton respectvely. Choosng larger partton szes (16 16, 16 8, and 8 16) Ths wor s supported by the SINGAPORE MOE Academc Research Fund (AcRF) Ter, Grant Number: T08B118. M. Paul s wth the School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore (phone: ; Fax: ; e-mal: M_Paul@ntu.edu.sg). W. Ln s wth the School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore (phone: ; e-mal: WSLIN@ntu.edu.sg. C. T. Lau s wth the School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore (phone: ; e-mal: ASCTLAU@ntu.edu.sg. F. B-S Lee s wth the School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore (phone: Fax: ; e-mal: ebslee@ntu.edu.sg. requres a smaller number of bts to encode the moton vectors and the type of partton, at the expense of the number of bts n moton compensated resdual error n the areas of hgh moton detals. By contrast, choosng smaller partton szes (8 4, 4 8, and 4 4) may result n a lower number of bts for encodng the resdual error at the expense of the number of bts to encode the moton vectors and the type of parttons. Thus the choce of ntermodes has a crucal role on rate-dstorton optmzaton, snce most vsual data are nter-coded. Lagrangan Multpler (LM) [4] has been used for mode selecton to trade off between the qualty of the vdeo and the bt rate [1] [3] at a gven bt rate. To fnd the best mode, bts and dstorton are determned by performng ME and MC wth all modes for a pxel bloc whch s popularly nown as macrobloc (MB). In ths exhaustve mode selecton (EMS) process, the LM (λ) s frst calculated wth an emprcal formula usng the selected Quantzaton Parameter (QP) for each MB [4]: ( QP 1 ) λ = (1) Durng the ME&MC and encodng process, all modes for each MB are nvestgated and the resultant bts R( m ) and dstortons D ( m ) are determned, where m s the -th mode. Lagrangan cost functon s defned for mode selecton as: J m = D m + λ R m. () ( ) ( ) ( ) From the Lagrangan cost functon, the ultmate mode s selected as follows: m n = m n T argmn ( J ( m )) R( m ) R (3) m T where R s the target bt rate. It s reported that the above mentoned EMS technque usually taes 50% to 90% of the total computatonal load of the JM reference software [8][18]. Thus, any attempt to reduce ths huge computatonal tme wthout degradng mage qualty and eepng the same bt rates eventually extend the vdeo codng applcatons n real tme (especally mportant wth low-powered and low-processng electroncs devces such as hand-held PDA, moble-phone, etc.). A number of fast mode selecton algorthms ncludng [5]-[1] have been proposed so far. Ahmad et al. [5] proposed a fast mode selecton algorthm usng the moton vector cost and prevous frame nformaton. The expermental results showed that ths approach reduced encodng tme by around 3% whle ncreasng btstream sze by about 18% wthout degradaton of mage qualty. The man dsadvantages of ths approach are the extra memory requrement to store the prevous nformaton and the ncrease n btstream sze. Yang et al. [6]

2 proposed another fast mode selecton algorthm (VBBMD- varable bloc-sze best moton detecton) whch stopped moton searches early for some cases, thus spped a large number of search ponts. Early termnaton of the search process s very smple and able to reduce the encodng tme by 19~3%. Ths process suffers from large performance degradaton f there s a mstae n early termnaton due to the presence of a local mnmum n the performance surface. Moreover, the rate-dstorton performance deterorates at hgh btrates for most of the vdeo sequences. Km et al. [7] proposed a fast mode selecton technque by reducng the domnatng computaton cost ncurred by the nter 8 8 and ntra 4 4 modes by analyzng the costs of already avalable modes. The expermental results showed that around 53% of total encodng tme reducton s acheved at QP = 8. Zeng et al. [8] proposed a fast mode selecton algorthm (MAMD- moton actvty-based mode decson) usng fve dfferent moton classes for an MB. Ths represents the state-of-the-art n fast mode decson. The authors used two QP dependant thresholds and two moton vector dependant thresholds to classfy moton actvtes. The expermental results show that around 6% of encodng tme could be saved for QP = 4~40. We have observed that ths process suffers from rate-dstorton performance degradaton for QP > 40 and QP < 4. It s due to the lmtaton of the thresholds whch are prmarly desgned for the QP = 4~40. We have also observed that t does not perform well for smooth or complex vdeo sequences at hgh btrates due to the average nature of the thresholds and LM. The approaches [9][10] proposed to flter out a number of modes from ME and MC for an MB usng ether smoothness of the bloc compared to the reference blocs or the property of an all-zero coeffcents bloc. Moon et al. [11] and Paul et al. [1] reduced the computatonal tme of mode selecton usng only the dstorton and moton vector wthout determnng the actual btrate n mode selecton decson. However, they also reported that around 0.dB degradaton n mage qualty compared to the EMS whle 31% of operatons were reduced. There are three ways to reduce the computatonal tme n ME and mode selecton. The frst s reducng operatons n the mode selecton by smplfed Lagrangan cost functon [11][1], the second s reducng the searchng ponts n ME (e.g., fast search algorthms and so called early termnaton mode selecton algorthms) [5][6] and the thrd s reducng the number of canddate modes [7]-[10]. The proposed method n ths paper s the combnaton of the second and thrd approaches wthout the need of calculatng Lagrangan cost functon for large modes wth the MB characterstcs engendered from the phase correlaton whch s to be ntroduced n the followng paragraph and explaned n detals n Secton II. Phase correlaton [13]-[15] essentally gves moton nformaton wthout the computatonally expensve searchng, and to be more specfc, the shftng nformaton between two correlated mages va Fourer transformaton. To et al. [14] confrmed that phase nformaton ndcates relable moton. In ths paper we proposed a fast mode selecton technque usng phase correlaton based on the prelmnary study [16] whch reports 5~50% of computatonal reducton usng fxed threshold and wthout dervng moton vectors from phase correlaton. The major contrbuton of ths paper s the new mode selecton strategy, dynamc thresholdng and phase correlated moton vector explotaton, besdes more analyss and expermental results usng dfferent scenaros. Other new features of ths paper nclude analyss for motvaton, computatonal complexty analyss n theoretcal and expermental perspectves, and performance comparsons wth three state-of-art-methods usng dfferent vdeo sequences n wde range of bt rates. The proposed technque n ths paper drectly selects only one mode among the canddate large modes based on the moton nformaton between the current and reference blocs. It also explots phase correlaton derved moton vectors (PCDMVs) as predcted moton vectors so that a reduced moton search range can be used. It performs n wde range of btrates by usng QP dependent dynamc threshold whch adjusts the phase correlated moton nformaton nto mode selecton. Note that most of the exstng algorthms are effectve for a small range of QPs, and uses LM (le n [3]) to tradeoff between mage qualty and btrates for mode selecton. We observed that the LM, whch s a functon of QP, s not optmal n mode selecton especally n large bloc (16 6, 16 8, 8 16, and 8 8) for a wde range of moton actve vdeo sequences due to the lac of consderaton of vdeo-content nformaton. It only represents the average trend. In the proposed method, we do not use LM for large mode selecton, and as a result, we get better rate-dstorton performance at hgh btrates for both complex moton and smple moton vdeo sequences. The expermental results confrm the expected performance mprovement of the proposed scheme. The rest of the paper s organzed as follows. Secton II descrbes the method for phase correlaton and phasematched error (PME). Secton III devces the proposed drect mode selecton technque, whle the assocated computatonal complexty s analyzed and verfed n the Secton IV. The overall expermental set up and results for the proposed scheme are also presented n Secton IV, whle Secton V concludes the paper. II. PHASE CORRELATION AND PHASE-MATCHED ERROR The phase correlaton technque based on the dscrete Fourer transformaton (DFT) can estmate relatve dsplacement between co-located (.e., wth zero-moton vector) blocs [13]-[15]. The magntude of the transformed blocs tells how much of a frequency component s present and the phase tells where the frequency component s n the blocs. In the proposed scheme, the phase nformaton has been used for

3 (a) (b) 3 3-pxel Bloc at (X=161, Y=19) (c) 3 3-pxel Bloc at (X=193, Y=19) (d) 3 3-pxel Bloc at (X=33, Y=33) (e)8 8-pxel Bloc at (X=185, Y=137) (f) 8 8-pxel Bloc at (X=65, Y=11) (g)8 8-pxel Bloc at (X=33, Y=33) Fgure 1: Phase correlaton peas (n Z-axs) between two correspondng blocs from the reference frame and current frame where (a) current frame, (b) & (e) more than one peas generated usng phase correlaton that represent multple motons n 3 3 and 8 8-pxel blocs respectvely, (c) & (f) one small magntude pea representng sngle moton n 3 3 and 8 8-pxel blocs respectvely, and (d) & (g) one bg magntude pea exact at the mddle representng no moton n 3 3 and 8 8-pxel blocs respectvely. predcton of the ntal moton vector (.e., PCDMV), and the phase-matched error, PME, has been adopted for mode selecton. In ths secton, we dscuss the related concepts. A. Phase correlaton The technque correlates two blocs by frst performng DFT on each bloc, fndng the dfference of ther phases, and performng nverse DFT on the resultng phase dfference. Ths process provdes a two dmensonal array whch wll have a pea at the coordnates correspondng to the shft between the two blocs. For clear vsualzaton we shft zero-frequency component to the center of the spectrum. Fgure 1 shows phase correlaton pea usng dfferent blocs where dfferent motons are observed. In Matlab mplementaton, the above mentoned manpulatons can be expressed as D=fftshft(abs(fft(exp(j.*(angle(Fr)-angle(Fc)))))) where Fr and Fc are the DFTs of the reference and current blocs and D s the resultant two dmensonal array. In Fgure 1 t s clear that 3 3 blocs at postons (X = 161, Y = 19), (X = 193, Y = 19), and (X = 33, Y = 33) represents multple motons, sngle moton, and no moton respectvely (wth D beng plotted). The correspondng motons are represented by 8 8 blocs at postons (X = 185, Y = 137), (X = 65, Y = 11), and (X = 33, Y = 33). Usng phase correlaton we also observed that two peas (b) & (e), one pea (c) & (f), and one bg magntude (near unty) pea at the mddle (d) & (g) successfully represents the dfferent motons n correspondng blocs. The magntude of the pea value s a good moton ndcaton of the correspondng bloc. Note that here we have consdered 3 3-pxel bloc for clear vsualzaton n Fgure 1, but our processng unt s 8 8-pxel bloc n the actual wor. B. Phase-matched error(pme) To et al. [14] showed that better moton ndcaton can be found usng PME, whch s calculated va subtracton of the current bloc from ts best-matched reference bloc wth ts phase replaced by phase of the current bloc. Fgure shows the bloc dagram [14]. Applyng phase correlaton (as ntroduced above and further descrbed n Part C of ths secton) on the current bloc and the reference bloc (.e., the co-located bloc n the reference frame), the PCDMV (dx, dy) s consdered as the dstance

4 [ mddle between the largest pea poston n the D (see Secton II.A) and the mddle poston of the bloc,.e., the PCDMV (0, 0) means that the largest pea n D s n the exact mddle. If the mddle poston of D s (x mddle, y mddle ) then we can defne moton vector PCDMV (dx, dy) as dx, dy ] = [arg max( D( x, y)] [ x mddle, y ].From the obtaned moton vector, the moton compensated reference bloc s determned and then DFT s appled. A matched bloc s formed by the magntude of the moton compensated reference bloc and phase of the current bloc. Ths s because the matched bloc has very smlar phase wth the current bloc; to fnd the dfference between matched bloc and current bloc, we only use magntude. Fnally the PME bloc s formed by subtractng the matched bloc from the current bloc to see the dsplacement. Fgure : Bloc dagram of phase matched error generaton. A moton nconsstence metrc (MIM) can be derved from the rato of the PME e, and ts low-frequency verson e low. The low-frequency verson of PME s determned as follows. Frst fnd the transformaton of PME usng dscrete cosne transformaton (DCT), eep only the top-left trangle coeffcents (assumng that the matrx begns wth (1, 1) on the top-left poston) of ths transformed bloc, and then fll up the rest of the postons by zeros. Here DCT s used as t has better energy-compacton propertes and less dscontnuty at the boundares, compared to DFT. After that the nverse DCT s appled. Thus, the MIM, δ, s defned n [14] as follows: 3M 3M u 4 4 M M δ = e low ( u, v) e ( x, y) (4) u= 1 v= 1 x= 1 y= 1 where M = 8, beng the dmenson of the bloc, and. The hgher the MIM s, the less relable the detected moton wll be. The sgnfcance of δ can be justfed by theoretcal dervaton that follows. Let dv be the resdual moton (.e., after nteger moton estmaton by the phase correlaton) [14] between the current bloc and the reference -1 (.e, moton compensated) bloc respectvely. Phase correlaton yelds nteger ME, hence ths resdual can be assumed as beng of fractonal-pxel,.e., dv < 1. Wth Taylor s seres, we have: 1 ( v) = ( v + dv) ' dv " = ( v) + dv ( v) + ( v) +... (5)! ' ( v) + dv. ( v) where v s the moton vector [14]. Tang Fourer transform of both sdes of (5), we get I 1( w) = (1 + dv. j. w) I ( w) (6) I ( w) = I ( w) 1+ ( dv. w) 1 Accordng to the defnton used n [14], the PME s as follows: 1 jθ e = F ( I ( w). e ) = F = F 1 1 (( I ((1 F( e) = ((1 E = (1 1 ( w) I 1 ( w) ). e 1+ ( dv. w) ). I 1+ ( dv. w) ). I 1+ ( dv. w) ). I ( w). e ( w) jθ ) jθ ) ( w). e Equaton (7) s partcularly nterestng because t establshed a relatonshp between the transform of PME e, and the transform of the orgnal bloc. By plottng functon 1 1+ ( dv. w) wth respect to w, To et al. showed that low-frequency components of E are sgnfcantly suppressed as compared wth the orgnal bloc f moton s estmated correctly. Note that Equaton (7) s derved from the Taylor s seres n (5) under the assumpton that moton vector up to an nteger s accurately estmated. Thus, δ has low value when ME s relable by sgnfcant shftng of energy from the low frequences and a reverse stuaton happens when ME does not provde relable estmaton. In the proposed algorthm, we wll use MIM as a predctor of moton nformaton. C. PME and varable-bloc sze ME For an 8 8 bloc, we have also observed that δ has large value (.e., close to unty) n the bloc contanng object movements, on the other hand δ has small value (.e., close to zero) when the bloc has temporally smooth areas. Fgure 3 shows an example. A horzontal porton (8 300 pxels started from Row 49 to Row 56 and Column 1 to Column 300) of the current frame mared by dotted lnes n the second frame of Tenns vdeo sequence (see Fgure 3 (a)) s taen to see the relatonshp of the sum of square dfferences (SSD) (between the correspondng co-located areas n the current & reference frames) and MIM δ. We calculate PME by shftng an 8 8-pxel bloc by one pxel horzontally at a tme. Fgure 3(b) shows hgher SSD values when the ball appears compared to the other regons. The smlar tendency s also observed for δ. It s nterestng to note that unle the SSD the response of δ on moton does not depend on the sze of moton nconsstent areas,.e., rato of the object jθ ) (7)

5 and bacground (exhbtng an nverse U-shape for the entre area nstead of a sharp pea for SSD). Ths property of δ provdes an advantage over SSD or phase correlaton pea (see Fgure 1) to ndcate moton ntensty wthn a bloc. 0 to 1, we can use t for selectng dfferent modes at wde range of btrates stuatons. (a) (b) (c) Fgure 3: Relatonshp of moton nconsstence metrc, δ wth object movements; (a) a horzontal part (rows 49:56 and column 1:300 pxel postons) mared by dotted lnes n the second frame of Tenns vdeo sequence, (b) sum of square dfference (n 1000) between the frst frame (.e., decoded Intra frame) and the second frame, and (c) the correspondng moton nconsstence metrc, δ, va shftng an 8 8-pxel bloc by one pxel horzontally at a tme. Two dmensonal translatonal ME s a smplfed moton approxmaton of actual complex moton actvtes such as rotatng, pannng, zoomng, llumnaton changes, etc. Thus, hgh moton area can be better moton estmated wth D translatonal ME usng smaller blocs, and low moton/bacground area can be estmated usng larger blocs. For ths reason, varable-bloc sze ME&MC s ntroduced n the H.64 to capture dfferent motons wthn an MB. Fgure 4 shows the number of blocs regarded as moton ones wth respect to the total blocs aganst δ. A bloc s consdered as a moton bloc, for example, f ts δ s greater than 0.4. The number of moton blocs monotoncally decreases wth the ncrease of δ. It s also worthy of notng that the number of moton blocs depend on moton areas and moton ntensty of a vdeo sequence. For example, Football sequence has more moton ntensve blocs and areas compared to the Tenns sequence. There are more moton areas wth hgher number at the ntal poston (e.g., for δ = 0.1, Football has 98% and Tenns has 55% moton blocs; so ths ndcates the former havng more moton areas), and hgher moton ntensty can be ndcated by hgher number at fnal poston (e.g., for δ = 0.9, Football has 0% and Tenns has 5% moton blocs respectvely). Snce δ s consstent wth moton actvtes (both moton ntensty and moton areas) wthn a bloc and ts range vares from Fgure 4: Relatonshp between moton nconsstence metrc, δ and the rato of moton blocs wth respect to the total blocs for varous vdeo sequences. For example, when δ = 0.4, a bloc s defned as a moton bloc f ts δ s hgher than 0.4. D. Phase correlaton derved moton vector (PCDMV) In the proposed technque we le to use PCDMV as a predcted moton vector to reduce the ME complexty. As shown n Fgure 3 (c), there s relatonshp between moton and δ, and we further demonstrate that δ monotoncally ncreases wth the moton vector (see Fgure 5). Fgure 5 also shows that for a small moton vector, δ s very small (near zero) but for hgh moton vector, δ s very hgh (close to unty). Ths nspres us to mae a hypothess. Can PCDMV be an ntal startng pont for actual moton search nstead of (0, 0) or predcted (.e., the standard way n the H.64) moton vector for a bloc? If the PCDMV yelds better codng qualty than the predcted or (0, 0) moton vector, then we can use t as the startng pont of the actual ME process. Thus, we can reduce the moton search range wth less computatonal tme. Note that we don t need to do any extra operaton for that as PCDMV s already avalable when we processed PME (see Fgure ). Snce the avalable PCDMV s only for 8 8-pxel bloc, now another queston s how to explot ths moton vector for other modes? Dscusson about ths wll be provded n Secton III.D. Fgure 5: Relatonshp of moton nconsstence metrc, δ wth phase correlaton moton vector usng Football vdeo sequence where z-axs represents δ. The qualty of the PCDMV s tested usng the sum of absolute dfference (SAD) between the current and two reference blocs. The best-matched reference bloc of the current bloc wll provde lowest SAD. The two reference

6 blocs are generated wth PCDMV and zero-moton vector (.e., co-located bloc) from the reference frame respectvely (here we don t consder H.64 predcted moton vectors as a sgnfcant amount of them are zero). Fgure 6 shows more than 70% of blocs beng bestmatched wth the current bloc usng PCDMV for dfferent vdeo sequences, ncludng 15% of non-zero moton vectors. Thus, we can easly use PCDMV at a startng pont for actual moton vector estmaton. We note that for the exstng moton vector estmaton, LM s used to control moton vector length wth QPs so that at low btrates smaller moton vector s selected and at hgh btrates larger moton vector s selected. But we don t have that nd of control when we use PCDMV. Ths aspect of the proposed technque wll be also dscussed n the second last paragraph of Secton III.C. Sub-pxel phase correlaton ME approach [1] can be appled on top of the proposed method to further speedng up. Fgure 6: Percentages of blocs n whch the PCDMV provdes the best-matched current and reference blocs. III. PROPOSED MODE SELECTION SCHEME The proposed technque s based on the drect nter mode selecton (nstead of EMS) usng phase correlaton nformaton, so that sgnfcant computaton reducton s acheved whle mantanng smlar mage qualty and btrates. Pror to encodng, a pxel MB s dvded nto four 8 8- pxel blocs. Usng phase correlaton (descrbed n Secton II.A) and PME (descrbed n Secton II.B), we can determne the MIM and the ntal moton vector for an MB. Based on the dynamc threshold, T (to be dscussed n Secton III.C), we generate a bnary matrx (Secton III.A). Then, we devse an algorthm to fnd the fnal mode among canddate modes namely 16 16, 16 8, 8 16, and 8 8 (together wth other smaller modes and ntra modes) from the bnary matrx (see Secton III.B). Note that we have drectly selected only one mode among larger modes by ths strategy. We assume that sp mode s already decded. When an 8 8 mode s selected, we need to explore ntra modes and other smaller modes such as 8 4, 4 8, and 4 4, but we do not use phase correlaton method to predct them ndvdually. We process them usng LM. Ths s because a sgnfcant amount of these smaller modes are normally selected for extremely hgh btrates. When a devce has capablty to operate n such a hgh btrate, t usually has enough computatonal power so that computatonal complexty s not an ssue. A. Bnary Matrx Generaton Before encodng a pxel MB, we determne MIM usng phase correlaton after dvdng an MB nto four 8 8-pxel blocs. Let Ω and δ be bnary matrx and MIM of the th MB respectvely and both are sze to accommodate four data. An element of bnary matrx s assgned wth 1 f MIM of the correspondng bloc s greater than or equal to threshold, T; otherwse t s assgned wth 0. Here 1 ndcates moton and 0 ndcates no moton. 1 ) f δ T Ω =. (8) 0 otherwse Based on the bnary values and postons of Ω, we select the modes for the correspondng MB. Obvously, a toolow value of T ( 0) leads to 1 for almost all blocs, and on the other hand, a too-hgh value of T ( 1) results n 0 for almost all blocs. Thus, T plays a sgnfcant role n selectng mode and shapng the ultmate rate-dstorton performance. Table 1: Mode selecton by the proposed technque Sp The H.64 default technque Mode If any condton from () to () s satsfed () Ω (, l) 1 = 1 l= 1 L Ω 1, + Ω,1 == ) & Ω L 16 8 If any condton from (v) to (v) s satsfed () ( Ω ( 1,1 ) + Ω (,) == ) & Ω < 4 () ( ( ) ( ) < 4 (v) Ω ( 1,) + Ω (,) == (v) Ω ( 1,1) + Ω (,1) == (v) ( Ω ( 1,) + Ω (,1) ) == ) & ( Ω 4) L (v) ( Ω ( 1,1) + Ω (,)) == ) & ( Ω 4) L 8 16 If any condton from (v) to (x) s satsfed 8 8 (not allowng further subdvson nto small or ntra modes) 8 8 (allowng further subdvson nto small or ntra modes) (v) Ω ( 1,1) + Ω ( 1, ) == (x)( Ω (,1) + Ω (,) == (x) & = 1 l= 1 ( Ω (,) 0) (, ) 3 l == & ( Ω < 4) Ω If any condton from (x) to (x) s satsfed (x) Ω (, l) == 3 & ( Ω 4) = 1 l= 1 (x) (, ) 3 l == & ( Ω (,) == 0) Ω = 1 l= 1 If the followng condton s satsfed (x) Ω (, l) == 4 = 1 l= 1 L L

7 B. Mode Selecton from the Bnary Matrx Based on the relatve postons and numbers of 1 or 0 n the bnary matrx Ω defned n (8) and the smlarly defned bnary matrx ΩL for the L-shaped neghborhood (the shaded 5-poston areas n Fgure 7), we wll decde the fnal mode among the canddate modes (16 16, 16 8, 8 16, and 8 8) for the th MB. We have consdered Ω and ΩL to avod nose and explot the spatal correlaton. The detaled strategy of the drect mode selecton process s explaned below wth the ad of Fgure 7 and Table 1. Mode s selected f Ω has all 0. We also select ths mode f Ω has only one 1 or two 1 s dagonally (combned and mared as the don t cares X s n Fgure 7 (a)), provded that Ω L has less than four 1 s (see condtons (), (), and () n Table 1); otherwse we select the 16 8 mode (condtons (v) and (v) n the Table 1). The ratonalty of selectng mode n those cases s that ME&MC usng the whole MB should capture ths type wth no or less moton X 0 0 X X X (a) Mode (b) Mode (c) Mode (d) Mode 8 8 (not allowng further sub-dvson nto small or ntra modes) (e) Mode 8 8 (allowng further sub-dvson nto smaller and ntra modes) Fgure 7: Drect mode selecton from the bnary matrx. When we have two 1 s n Ω vertcally or horzontally, we assume that the 16 8 or 8 16 mode s sutable as a half has no moton and the other half has moton (the frst two cases n Fgure 7(b, c)). For vertcally postoned 1 s we select 16 8 by assumng that two motons are dentcal (condtons (v) and (v) n the Table 1); and for the same reason we select 8 16 for horzontally two 1 s (condtons (v) and (x) n the Table 1). We also select the 16 8 mode for the last two scenaros L consderng Ω n Fgure 7(b) (condtons (v) and (v) n the Table 1). If there s no vertcal or horzontal algnment of 1 s or 0 s and there are four or more 1 s n Ω L, we select 16 8 mode. Besdes horzontal algnment of 1 s or 0 s, we also select the 8 16 mode for the last three cases (condton (x) n the Table 1) where the total number of 1 s n Ω s three (except for 0 beng n the bottom-rght poston as the L-shaped upper part n Ω has already enough 1 s to be encoded usng 8 8 mode (see Fgure 7 (c,d)) and also there are less than four 1 s n Ω L. Actually, for model 16 8 (vertcal coherence n moton), condtons (v) and (v) are clear-cut, whle condtons (v) and (v) are not (.e., they may be also assumed to be mode 8 16); smlarly, for mode 8 16 (horzontal coherence n moton), condtons (v) and (x) are clear-cut, whle condton (x) s not (.e., t may be also assumed to be mode 16 8). Therefore, n prncples, we can assgn condtons (v), (v), and (x) to mode 16 8 or 8 16 arbtrarly; consderng that there s more horzontal moton than vertcal moton n a typcal vdeo, we use the stated choce n ths wor: mode 8 16 (horzontal coherence n moton) s chosen f more moton s n Ω ; otherwse mode 16 8 s selected. Mode 8 8 s selected (no ntra or further sub-dvson nto small sub-blocs such as 8 4, 4 8 or 4 4 are allowed n ths case) f there are three 1 s n Ω and also there are four or more 1 s n Ω L (see Fgure 7(d) and condtons (x) and (x) n the Table 1). We have restrcted the exploraton by the smaller blocs or ntra blocs due to the lac of moton (.e., total number of 1 s s three) wthn the bloc f condtons (x) and (x) are satsfed. We only allow ntra modes and further dvson nto subblocs by the 8 8 mode when all blocs have moton (.e., total number of 1 s s four) (see Fgure 7(e) and condton (x) n the Table 1) and we tae the ultmate decson n ths case usng LM, as descrbed n the second paragraph at the start of ths secton. C. Threshold Selecton for Dfferent Btrates Fgure 4 shows that the number of moton-blocs (.e., those mared wth 1 s n (8)) decreases f we ncrease T for all vdeo sequences. Of course the rate of ncrease may be dfferent for dfferent vdeo sequences but the smlar trend s clearly realzed. Fgure 8 shows moton and non-moton blocs for Mss Amerca and Tenns vdeo sequences usng the bnary matrx generated by PME where we used T {0.7,0.5,0.3}. It s clear from the fgure that only the moton domnatng areas are dentfed by the rectangular blocs. For example, tenns ball, bat, porton of hand, and some border lnes are dentfed as moton areas. Fgure 8 also demonstrates that the number of moton blocs s nversely proportonal to the threshold T. For relatvely hgh thresholds, largely statonary areas n an object are not classfed as moton-blocs but for a

8 relatve low threshold, those are classfed as motonblocs. For example, a porton of face n Mss Amerca or a porton of bat n Tenns s not classfed as motonblocs by T=0.7 (see Fgure 8 (b) & (f)), but they are classfed as moton-blocs by T=0.3 (see Fgure 8(d) & (h)). On the other hand, normal tendency s that the EMS employs more and more sub-blocs (smaller modes) to encode an MB resultng n ncrease n btrates (.e., hgh qualty). To cope wth ths tendency, we need to use large T at low btrates and small T at hgh btrates; so that large modes can be used at low btrates and small modes can be used at hgh btrates. Thus, we ntroduce dynamc thresholdng based on the dfferent QPs so that at dfferent btrates, dfferent number of moton blocs can be classfed by the proposed mode selecton strategy. Snce btrates and qualty of a vdeo sequence are trade-off n Lagrangan optmzaton functon, and the LM s an exponental functon of QP, we assume that the threshold T s an exponental functon of QP. (a) (b) (c) (d) Fgure 8: Moton domnated blocs dentfed by the bnary matrces usng dfferent T s where red blocs (8 8 pxels) ndcate moton and other areas ndcate no moton; (a) & (e) show the dfference between the frst (Intra decoded) and thrd (orgnal) frames, whch s multpled by 6 for better vsualzaton; (b) & (f) T=0.7; (c) & (g) T=0.5; and (d) & (h) T=0.3, for Mss Amerca (QCIF) and Tenns (SIF) sequences (the true sze s not reflected here) respectvely. Let T be defned as follows: bqp T = ae. (9) To determne parameters a and b, we tested a large (e) (f) (g) (h) number of standard and non-standard vdeo sequences, wth all possble thresholds, T = { t, = 1,..., n}, and quantzaton parameters QP, Q = { q j, j = 1,..., m} ; our am s to determne a and b so that the performance of the proposed drect nter-mode selecton approach s comparable wth the exhaustve Lagrangan optmzaton. Assume that the proposed algorthm and the exhaustve P optmzaton gve D, and H D as dstortons, j P H and R, and R as btrates wth j th QP after encodng a j j vdeo sequence. Note that we apply () usng the average dstorton and bts derved from the entre vdeo sequence rather than the MB level, thus, accordng to (), = D + R (10) and P P P J, j, j λ j, j J H j H H = D + λ R. (11) j j j For quantzaton parameter q j, a threshold t(j) s selected as follows: P H t( j) = arg ( J, j J j ). (1) mn t In ths way we wll get the correspondng new set Q T = t( q ), t( q ) Lt( q ), Lt( q )} usng a vdeo sequence { 1 j m for a gven set Q = q, q Lq j Lq } where two or more { 1 m elements n T Q can be the same. We wll collect T Q for other vdeo sequences. Then we approxmate all T Q by (9). Table : The proposed and the correspondng average threshold derved from the ten best thresholds usng ten vdeo sequences aganst QPs QP Proposed T Average value Followng the dscusson n the second paragraph at the start of Secton III, the values of QP below 0 have not been consdered as 0~51 are the range used for typcal vdeo codng. For each vdeo sequence, we have consdered 19 3 = 608 rate-dstorton data usng 19 dfferent T s (consderng 0.5 precson n T) and 3 dfferent QPs from 0 to 51. We have found that a = 0.13 and b = The threshold, T s shown n the Fgure 9. The average threshold derved from the best thresholds of 10 vdeo sequences usng dfferent QPs s also shown n Table. Snce the best threshold actually s the average from the ten best thresholds aganst a QP, t (and yelded values by Equaton (9)), gven n Table, s not wth the nterval of 0.5. A drect loo-up table may j

9 provde better results for some vdeo sequences but for generalzaton we have used an approxmaton functon. Moreover, to avod memory requrement for the loo up table we have formulated the threshold as an explct functon (rather than a loo up table) of QP. We have also observed that a relatvely hgh value of T s sutable for low moton vdeo sequences and a low value for hgh moton vdeo sequences Fgure 9: Proposed dynamc threshold as a to eep almost functon of QP. same ratedstorton performance compared to that of the EMS. When we used a low value for low moton vdeo sequences, a hgh number of blocs are classfed as moton blocs and hence encoded usng smaller modes compared to the EMS. As a consequence, we may get better rate-dstorton performance compared to the EMS process. But usng a small value of T also requres more modes per MB (.e., less computatonal gan) as we have consdered LM for selectng very small blocs (for example 4 4 blocs). On the other hand f we use a hgh value for hgh moton vdeo sequences, some less moton ntensve blocs may not be classfed as moton-blocs and encoded usng relatvely large mode blocs. Thus also better rate-dstorton can be acheved by the proposed system compared to the EMS approach. As we have stated n the second last paragraph n Secton I, we can get better rate-dstorton performance compared to the EMS approach at hgh btrates for both complex moton and smple moton vdeo sequences. Fgure 10 shows the comparatve results by the EMS technque and the drect mode selecton technque usng the proposed strategy where T=0.5. Our approach dentfes moton domnatng areas when there s an object n those sequences; on the contrary, the EMS approach dentfes only a porton of them. For example, the tenns ball & bat n Fgure 10 (b), players n Fgure 10 (d), and the face n Fgure 10 (f) are clearly dentfed by the proposed technque and wth smaller modes. On the other hand, the moton domnatng areas n ball and bat of Tenns sequences or players n Football sequences or face n Mss Amerca sequence s not well dentfed by the EMS technque and used large modes. Ths happens because the large bloc generates smaller Lagrangan cost functon compared to the other small blocs. Note that the cost functon s calculated by addng dstorton wth bts multpled by LM n the EMS. There s 40~70% of classfcaton beng matched among the proposed and the EMS method. The matchng percentages normally reduce wth the QP. (a) (c) (e) Fgure 10: Classfcaton of all MBs usng H.64 (a), (c), & (e) and proposed technque (b), (d), & (f) for three standard vdeo sequences where blac, cyan, green, blue, and red/magenta colour for sp, 16 16, 16 8, 8 16, 8 8 (wth others) respectvely. D. Intal Moton Vector usng PCDMV As dscussed earler phase correlated moton vector, PCDMV has potental to predct moton. In the phase correlaton step we only use 8 8-pxel bloc, thus generated moton vector can easly be used n the same or smaller blocs for the startng moton vector n further true moton vector search. For example, we can use them for 8 8, 8 4, 4 8, and 4 4. But we could not use them straght forward n any larger blocs such as 8 16, 16 8, or as they have two or four canddate moton vectors of each sub-bloc. For example, a bloc has four sets of moton vectors (.e., each for 8 8 bloc). We can fnd the best moton vector as the startng pont,.e., select the moton vector among the canddate moton vectors whch provde the least SAD. When we use PCDMV as a startng pont for further moton search we reduce search range. In our experment we use half of the search length assumng that t does not affect so much n rate-dstorton performance as PCDMV has already provded good predcton. Thus, theoretcally we can predct that a sgnfcant computatonal tme reducton wll be acheved compared to the EMS n ths regard. IV. OVERALL PERFORMANCE WITH EXPERIMENTAL RESULTS Overall expermental results are performed usng 10 standard vdeo sequences, comprsng of SIF (35 40), CIF (35 88), and QCIF ( ) dgtal vdeo format [9]. SIF and CIF sequences are encoded at 30 frames per second (fps) and QCIF sequences are encoded at 15 (b) (d) (f)

10 frames per second. Full-search fractonal ME wth ±15 as the search length and 15 as the group of pcture (GOP) sze are used n the proposed, VBBMD, MAMD, and EMS technques. We have used only Y component of vdeo sequence n our experment. Note that all the technques are mplemented based on the H.64 recommendatons. A. Computatonal Complexty Accordng to our strategy descrbed n Secton III, we only use one mode for ME&MC for larger (16 16, 16 8, and 8 16) blocs and f all the bnary values are 1 n Ω, we select 8 8 and also apply ME&MC n the smaller blocs. But f three out of four are 1 s and the L-shaped neghborng blocs n ΩL have also suffcent number of 1 s then ME&MC s appled usng only 8 8 mode (not apply on smaller blocs). For far comparson we need to fnd the overheads for calculatng phase-matched error and bnary matrx. We perform DFT on each bloc, replace the phase of the reference bloc wth the current bloc phase, calculate the rato of upper trangular error compared to the total error, and assgn 1 or 0 to a matrx. For an N N bloc we need 3N operatons for forward and nverse DFT, N operatons for replacng phase, 0.5 N operatons for rato calculatons, N operatons for assgnng 1 s or 0 s. Thus total 5.5 N operatons are needed for the manpulatons up to the bnary matrx formatons. For full search ME usng d search length, we need 3 N (d + 1) operatons for each mode as we need to calculate ( d + 1) search ponts where each pont needs 3 operatons for N N-pxel bloc. Zeng et al. [8] and Chen et al. [18] observed that ME requres 50% to 90% of overall computatonal tme usng JM reference software. We have also observed very smlar trend. Thus the EMS needs 4.3 H N (d + 1) operatons for overall encodng on average where H s the average number of modes for ME&MC per MB usng the EMS. On the other hand, the number of operatons requred by the proposed technque usng PCDMV wth full moton search length (Proposed- FMSL) and half moton search length (Proposed-HMSL) are: 1.43 P N (3( d + 1) + 5.5) and 1.43 P N (3( d + 1) + 5.5) respectvely where P s the average number of modes per MB for the proposed method. Theoretcally the Proposed- FMSL and the Proposed-HMSL technques reduce encodng tme approxmately 50% and 87.5% respectvely compared to the EMS technque f both the proposed technques requre a half number of mode per MB compared to the EMS technque. The expermental data shows that based upon 10 vdeo sequences and dfferent bt rates, K H = 5.3 for EMS, and K P =.14 and.47 for the proposed-fmsl and the Proposed-HMSL respectvely. From the lteratures we have found that fast mode selecton algorthm proposed by Zeng et al. [8] s the fastest one. The authors reported computatonal reducton s on average 6.96% (mnmum s 55% for Foreman sequence) wth losng db n PSNR and ncreasng 0.19% of btstream usng four GOP sze (.e., IPPP), and QPs={40,36,30,4} for a number of QCIF and CIF standard sequences. We have also found same performance usng same setup for those vdeo sequences. Normally the researchers use large GOP sze such as 1, 15, 4, or 30, and CIF/SIF format for relatvely hgh moton vdeo sequences and QCIF format for low moton vdeo sequences. Thus, we have nvestgated the performance of the MAMD technque usng 15 GOP sze and a wde range of CIF/SIF/QCIF vdeo sequences. We have found that the MAMD reduces on average 61% (smlar to the author s clam n [8]) computatonal tme but suffers rate-dstorton performance degradaton for very hgh moton vdeo sequences and low/smooth moton vdeo sequences at hgh btrates. Ths algorthm used four thresholds based on the emprcal results for moton actvty determnaton. Snce ths algorthm s desgned based on the Lagrangan cost functon, t suffers ratedstorton performance degradaton for those cases. We have compared computatonal complexty of our algorthm wth ths recent fast mode selecton algorthm. We have also compared our result wth the smple and well-nown early termnaton algorthm (VBBMD). (a) (b) Fgure 11: Total encodng tme reducton usng the proposed (Proposed-FMSL and Proposed-HMSL), the varable bloc-sze best moton detecton (VBBMD), and the moton actvty-based mode decson (MAMD) technques compared to the EMS technque; (a) ndvdual complexty reducton (average on sx QPs) for each of the ten vdeo sequences usng the proposed, MAMD, and VBBMD technques; and (b) average complexty reducton compared to the EMS usng 10 sequences wth QPs by the proposed (both FMSL and HMSL), the VBBMD, and the MAMD technques. Fgure 11 shows computatonal tme reducton usng the proposed (Proposed-FMSL and Proposed-HMSL), the VBBMD [6], and the MAMD [8] technques compared to the EMS based on the expermental data. Fgure 11 (a) demonstrates that the Proposed-FMSL and the Proposed-

11 HMSL technques reduce 47~71% and 84~90% computatonal complexty on average compared to the EMS respectvely, whereas the VBBMD and the MAMD technques reduce 5~50% and 4~80% compared to the EMS respectvely. Besdes the ndvdual vdeo sequence, we have also calculated average encodng tme reducton aganst the EMS wth dfferent QPs. Fgure 11 (b) reports 8~9% and 49~74% of encodng tme reducton by the proposed (Proposed-HMSL and Proposed-FMSL) technques respectvely whereas 58~74% and 19~3% of reducton by the MAMD and the VBBMD technques respectvely (n our expermental results) compared to the EMS technque. Table 3: BD-PSNRs (db) [19][0] usng EMS, VBBMD, MAMD, and the proposed methods usng full moton search length (FMSL) and half moton search length (HMSL) BD-PSNR (Method) BD-PSNR (EMS) Proposed Vdeos EMS VBBMD MAMD HMSL FMSL Mss Amerca Car phone News Suze Football Tenns Moble Foreman Hall Contaner Average Obvously the Proposed-HMSL technque outperforms all technques n terms of computatonal tme, however, the MAMD and the Proposed-FMSL technques outperform each other for dfferent cases (see Fgure 11(a)). In the followng secton, we wll show that the Proposed-FMSL and the Proposed-HMSL technques outperform the MAMD and VBBMD technques sgnfcantly n terms of rate-dstorton performance. Table 4: BD-Btrates [19][0] usng EMS, VBBMD, MAMD, and the proposed methods usng full moton search length (FMSL) and half moton search length (HMSL) BD-Btrate (Method) BD-Btrate (EMS) Proposed Vdeos EMS VBBMD MAMD HMSL FMSL Mss Amerca Car phone News Suze Football Tenns Moble Foreman Hall Contaner Average B. Overall Rate-Dstorton Performance To see the effects on a wde range of bt rates, we have calculated BD-Btrates and BD-PSNR [19][0] usng EMS, VBBMD [6], MAMD [8], and the proposed methods (Proposed-FMSL and Proposed-HMSL). Table 3 shows BD-PSNRs and Table 4 shows BD-Btrates respectvely. Negatve value n the tables ndcates nferor mage qualty for the BD-PSNR and reducton of bt-rates for the BD-Btrates compared to the EMS method. Obvously postve value for BD-PSNR and negatve value for the BD-Btrate ndcate superor performance compared to the EMS technque. The tables show that for all cases (except Tenns sequence) the proposed method outperforms not only the VBBMD and the MAMD technques but also the EMS technque. The tables also reveal that the VBBMD and the MAMD could not outperform the EMS technque at all. In contrast wth the exstng fast nter-mode decson algorthms that outperform the EMS n terms of requred computaton for ether a specfc QP or a small range of QPs, the proposed algorthm performs well not only n terms of much bgger computatonal savng (as presented n Secton IV.A) but also for a wde range of QPs,.e., a wde range of btrates. Table 3 and Table 4 show BD_PSNR and BD_Btrate performance for QP beng =0 to 51 whch covers 40 bps rates (typcally requred by Mss Amerca of pxel sze) to 8 mbps n btrates (typcally requred by Moble of pxel sze). (a) 0.75bpp, 38.88dB (b) 0.75bpp, 38.74dB (c) 1.4bpp, 37.63dB (d) 1.4bpp, 37.69dB (e) 0.19bpp, 43.6dB (f) 0.19bpp, 43.7dB Fgure 1: Decoded second frame by the EMS technque (a), (c), & (e), and by the proposed scheme (b), (d), & (f) respectvely usng T = 0.35 and QP = 6. The performance of the Proposed-HMSL (.e., wth half moton search length) technque n terms of computatonal tme (see Fgure 11), BD-PSNR and BD-Btrate (see Table 3 and Table 4) outperforms the MAMD and VBBMD technques (wth full moton search length) by 0.35 db due to the accuracy of the PCDMV (Secton

12 III.D) and mode selecton (Secton III.A, B, and C) technques. We recommend the Proposed-HMSL technque for speedng up the entre encodng tme wth enhanced rate-dstorton performance. In the followng sectons we also see the subjectve and objectve qualty usng the Proposed-HMSL technque and other technques. Fgure 1 shows decoded frames for subjectve vewng tests by the EMS and the Proposed-HMSL scheme usng T=0.35 and QP = 6. The second frames of Tenns, Football, and Mss Amerca sequences are shown. For the Football, and Mss Amerca examples n Fgure 1, the Proposed-HMSL scheme yelds slghtly hgher PSNR. Ths s the expermental evdence of gettng slghtly better rate-dstorton performance n hgh bt rates as dscussed n Secton III.C. Although n the other example (Tenns), the proposed scheme has slghtly lost n PSNR, t subjectvely leads to better coded mage qualty (for example, the ball n Tenns sequence appears more clearly) due to ts better capablty to separate moton area and non-moton area (see Fgure 10). dfferent QPs (such as 36, 30, 4, and 0) for 10 standard vdeo sequences usng the VBBMD (t s selected for comparson snce t s smple and the best among early termnaton algorthms), MAMD (t s selected for comparson snce t s the best-performng exstng technque n terms of computatonal tme), the EMS, and the Proposed-HMSL scheme. The fgure shows that the proposed scheme outperforms the relevant exstng algorthms by up to 1.0dB. In the proposed technque, selecton of small parttons depends on the sze of the movng regons, whle n the EMS technque ths selecton s manly guded by the LM rather than the sze of the movng regons. Due to the restrcton and relaxaton of the LM, the EMS technque selects fewer smaller parttons at low bt rates and more smaller parttons at hgh bt rates rrespectve of the movng regons. Thus, the proposed technque results n lower bt rates and hgher PSNR compared to the EMS technque, as t selects small parttons for moton domnatng areas and large parttons for smooth areas under the gudance of dynamc threshold. The other algorthms ncludng the EMS could not do t properly due to the too restrcton and too relaxaton of the LM n moton estmaton as well as model selecton at the low bt rates and hgh bt rates respectvely. (a) Fgure 13: Frame level PSNR and Bts (n thousands) fluctuatons usng Hall vdeo sequence (30 frames per second, QP=4) by the Proposed- HMSL, MAMD, VBBMD, and the EMS technques. To see the frame level performance, we have plotted PSNRs (see Fgure 13 (a)) and bts (see Fgure 13 (b)) per frame for the Hall vdeo sequence wth 30 fps and frst 100 frames usng the Proposed-HMSL, MAMD, VBBMD, and the EMS technques. The fgure demonstrates that the proposed technque s also better n PSNR and bts n frame-level compared to the other technques. The other vdeo sequences also exhbt same trend. Note that to show the relatve dfference more clearly, we provde a small range of bts per frame (see Fgure 13 (b)) by gnorng bts requred by the I-frame (whch s to 3 tmes compared to the nter-frame). Fgure 14 shows rate-dstorton performance by the proposed and the MAMD methods wth dfferent GOP (.e., 4 and 15) for Hall vdeo sequence. The fgure reveals that the proposed method outperforms the MAMD method by around 1.0dB and 0.7dB at hgh bt rates wth 4 and 15 GOP sze respectvely. Both algorthms perform well at large GOP, whereas the proposed method performs relatvely better due to better mprovement n the nter-frames. Thus, n our fnal rate-dstorton performance analyss (see Fgure 15) we use GOP = 15. For comprehensve performance test, we have also provded rate-dstorton curves n Fgure 15 usng (b) Fgure 14: Rate-dstorton performance by the MAMD and the proposed methods wth two GOPs (.e., wth length of 4 and 15) for Hall sequence. The VBBMD reduces encodng tme by 19~3% aganst the EMS. It performs well at low btrates, however, suffers rate-dstorton performance degradaton at hgh btrates for most of the vdeo sequences (see Fgure 15). The MAMD reaches computatonal complexty reducton by 58~74% (as n Secton IV.A) aganst the EMS, but performs worse n terms of PSNR at hgh btrates for the most of the vdeo sequences. The LM s used to select the mode usng a cost functon comprsng bts and dstorton n the EMS. The multpler λ s derved usng expermental values from varous standard and non-standard vdeo sequences. Ths s an average value whch more or less wors for all sequences. Moble and Football sequences have more moton, and on the other hand, Mss Amerca and News sequences have less moton; the proposed-hmsl scheme performs better

13 Fgure 15: Rate-dstorton performance comparson usng the Proposed-HMSL, varable bloc-sze best moton detecton (VBBMD) [6], moton actvty-based mode decson (MAMD) [8], and the exhaustve mode selecton (EMS) based on the H.64 standard recommendaton for two SIF, four CIF, and three QCIF standard vdeo sequences. for all cases but slghtly worse for the Tenns sequence whch has moderate moton. As the proposed scheme doesn t use Lagrangan cost functon n large mode selecton, there s a chance to outperform the EMS for those cases. Wth the objectve and subjectve tests, the proposed scheme acheves smlar qualty and even outperforms the EMS n terms of ratedstorton and/or vsualzaton wth on average savng 87% of processng tme whch s close to the theoretcally derved value of 87.5% (see Secton IV.A). V. CONCLUSIONS In ths paper, we proposed a new nter-mode decson scheme where moton estmaton and compensaton modes are drectly selected usng phase correlaton nformaton nstead of exhaustve mode selecton (EMS) wth all possble large modes, based on the moton domnancy and relatve postonng. The proposed scheme successfully explots the same phase correlaton process to derve ntal moton vectors n an ntegrated fashon n order to reduce computatonal tme further. To operate n dfferent btrates, a dynamc threshold s ntroduced. The expermental results show that the proposed scheme s the fastest nter-mode selecton algorthm wthout degradng rate-dstorton performance compared to the EMS (exhaustve mode selecton) technque; n fact, t outperforms the EMS for almost all vdeo sequences n terms of rate-dstorton performance at md to hgh bt rates, whereas the other fast mode selecton algorthms degrade the performance n comparson wth the EMS. It outperforms the relevant exstng fast algorthms: () for wde range of bt rates whereas the other fast algorthms only wor for a partcular or small range of bt rates, and () for wde range of vdeo sequences (from smooth to hgh moton). The proposed technque reduces 8~9% of the computatonal tme wthout jeopardzng the codng qualty and ncreasng the btrates compared to the EMSbased vdeo codng, and therefore outperform the state-ofthe-art mode decson technques wth a bg margn, over a wde spectrum of vdeo sequences and btrates. ACKNOWLEDGEMENT

14 The authors are grateful to the Assocate Edtor J. E. Fowler and four anonymous revewers for ther encouragng and valuable advce that lead to ths mprovement verson and clearer presentaton of the techncal contents. REFERENCES [1] ITU-T Rec. H.64/ISO/IEC AVC. Jont Vdeo Team (JVT) of ISO MPEG and ITU-T VCEG, JVT-G050, 003. [] G. Sullvan, and T. Wegand, Rate-dstorton optmzaton for vdeo compresson, IEEE Sgnal Processng Magazne, pp , [3] T. Wegand, and B. Grod, Lagrange multpler selecton n hybrd vdeo coder control, IEEE Internatonal Conference on Image Processng, vol. 1, pp , 001. [4] T. Wegand, H. Schwarz, A. Joch, and F. Kossentn, Rateconstraned coder control and comparson of vdeo codng standards, IEEE Transacton on Crcuts and Systems for Vdeo Technology, vol. 13, no. 7, pp , 003. [5] A. Ahmad, N. Khan, S. Masud, and M. A. Maud, Selecton of varable bloc szes n H.64, IEEE Internatonal Conference on Acoustcs, Speech, and Sgnal Processng, 3, , 004. [6] L. Yang, K. Yu, J. L, and S. L, An effectve varable bloc-sze early termnaton algorthm for H.64 vdeo codng, IEEE Transacton on Crcuts and Systems for Vdeo Technology, vol. 15, no. 6, pp , 005. [7] D. Km and J. Jeong, A fast mode selecton algorthm n H.64 vdeo codng, IEEE Internatonal conference on Multmeda and Expo, pp , 006. [8] H. Zeng, C. Ca, and K.-K. Ma, Fast mode decson for H.64/AVC based on macrobloc moton actvty, IEEE Transacton on Crcuts and Systems for Vdeo Technology, vol. 19, no. 4, pp. 1-11, Aprl 009. [9] H. Wang, S. Kwong, and C-W. Ko, An Effcent Mode Decson Algorthm for H.64/AVC Encodng optmzaton. IEEE Transacton on Multmeda, Vol. 9, No. 4, pp , JUNE Manoranjan Paul (M 03) receved the B.Sc.Eng. (hons.) degree n computer scence and engneerng from Bangladesh Unversty of Engneerng and Technology (BUET), Dhaa, Bangladesh, n 1997 and the PhD degree from the Monash Unversty, Australa n 005. He joned the Computer Scence and Engneerng Department, Ahsanullah Unversty of Scence and Technology, Dhaa, Bangladesh as a Lecturer n 1997 and was promoted to Assstant Professor n 000. He has wored as a Research Fellow n the Unversty of New South Wales, ADFA, Canberra, Australa n 005~006 and Monash Unversty, Australa n 006~009. He s currently worng as a research Fellow at the School of Computer engneerng, Nanyang Technologcal Unversty, Sngapore. Hs major research nterests are n the felds of mage/vdeo codng, multmeda communcaton, and computer vson. He has publshed more than 40 refereed publcatons. Recently he organzed a specal sesson on New vdeo codng technologes n IEEE ISCAS 010. He s a eynote speaer on Vson frendly vdeo codng n IEEE ICCIT 010. Dr Paul s a full member of the IEEE ( 03) and ACS ( 05). Dr. Paul has served as a guest edtor of Journal of Multmeda from 008. Wes Ln (M 9 SM 98) receved the B.Sc. degree n electroncs and the M.Sc. degree n dgtal sgnal processng from Zhongshan Unversty, Guangzhou, Chna, n 198 and 1985, respectvely, and the Ph.D. degree n computer vson from Kng s College, London Unversty, London, U.K., n 199. He taught and conducted research at Zhongshan Unversty, Shantou Unversty (Chna), Bath Unversty (U.K.), the Natonal Unversty of Sngapore, the Insttute of Mcroelectroncs (Sngapore), and the Insttute for Infocomm Research (Sngapore). He has been the Project Leader of 13 successfully-delvered projects n dgtal multmeda technology development. He also serves as the Lab Head, Vsual Processng, and the Actng Department Manager, Meda Processng, for the Insttute for Infocomm Research [10] Y. H. Km, J.-W. Yoo, S.-W. Lee, J. Shn, J. Pa, and H.-K. Jung, Adaptve mode decson for H.64 encoder. Electroncs letters, vol. 40, no. 19, pp , September 004. [11] J. M. Moon, Y. H. Moon, and J. H. Km, A computaton reducton method for RDO mode decson based on an approxmaton of the dstorton, n IEEE Internatonal Conference on Image Processng, pp , 006. [1] M. Paul, M. Frater, and J. Arnold, An Effcent Mode Selecton Pror to the Actual Encodng for H.64/AVC Encoder, IEEE Transacton on Multmeda, vol. 11, no. 4, pp , June 009. [13] G. A. Thomas, Televson moton measurement for datv and other applcatons, Techncal Report, BBC Research Department, [14] To L., M. Pcerng, M. Frater, and J. Arnold, A moton confdence measure from phase nformaton, IEEE Internatonal Conference on Image Processng, pp , 004. [15] Kugln C. D. andd. C. Hnes, The phase correlaton mage algnment method, IEEE Conference on Cybernetcs and Socety, pp [16] Paul, M. and G. Sorwar, An effcent Vdeo Codng usng Phasematched Error from Phase Correlaton Informaton, IEEE Internatonal Conference on Multmeda Sgnal Processng, 008. [17] I. E. G. Rchardson, H.64 and MPEG-4 vdeo compresson, JOHN WILLEY & SONS, LTD, 003. [18] C. Chen, S. Chen, Y. Huang, T. Chen, T. Wang, and L. Chen, Analyss and Archtecture Desgn of Varable Bloc-Sze Moton Estmaton for H.64/AVC, IEEE Transactons on Crcuts and Systems-I: Regular Papers, vol.53, no. 3, pp , 006. [19] G. Bjøntegaard, Calculaton of average PSNR dfferences between RD-curves, VCEG-M33, ITU-T Q.6/SG16 VCEG, 001. [0] S. Kondo and H. Sasa, Moton-compensated vdeo codng usng slced blocs, Systems and Computers n Japan, 38 (7), 1-, 007. [1] V. Argyrou and T. Vlachos, "A study of sub-pxel moton estmaton usng phase correlaton", Internatonal conference on Brtsh Machne Vson Assocaton, 006. Currently, he s an Assocate Professor n the School of Computer Engneerng, Nanyang Technologcal Unversty, Nanyang, Sngapore. Hs areas of expertse nclude mage processng, perceptual modelng, vdeo compresson, multmeda communcaton, and computer vson. He holds ten patents, wrote fve boo chapters, and has publshed over 160 refereed papers n nternatonal journals and conferences. He beleves that good theory s practcal, so has ept a balance of academc research and ndustral deployment throughout hs worng lfe. Dr. Ln s a fellow of IET and a Chartered Engneer (U.K.). He has been nvted to gve techncal tals and tutorals n VPQM06, ICCCN07, PCM07/09, ISCAS08, ICME09, APSIPA10 and ICIP10. He currently serves on the edtoral board of Journal of Vsual Communcaton and Image Representaton, four IEEE Techncal Commttees, and Techncal Program Commttees of a number of nternatonal conferences. Chew-Tong Lau (M 84) receved the B.Eng. degree from Laehead Unversty, Canada, n 1983, and the M.A.Sc. and Ph.D. degrees n electrcal engneerng from the Unversty of Brtsh Columba, Canada, n 1985 and 1990 respectvely. He s currently an Assocate Professor n the School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore. Hs research nterests nclude wreless communcatons systems and sgnal processng. Bu-Sung Lee (M 08) receved hs B.Sc. (Hons) and PhD from the Electrcal and Electroncs Department, Loughborough Unversty of Technology, UK n 198 and 1987 respectvely. He s an Assocate Professor wth the Nanyang Technologcal Unversty. He s actve n the development of the Research and Educaton Networ n Sngapore and the Asa Pacfc Regon, eg. Trans-EuroAsa Networ. Hs research nterest covers Grd/Cloud Computng, wreless communcaton and multmeda communcaton.