Objective Video Quality Metrics

Transcription

1 Objecve Vdeo Qualy Mercs M. Vranješ,. Rmac-Drlje, D. Žagar Unversy of Osjek, Faculy of Elecrcal Engneerng Kneza Trpmra B, Osjek, HR-3000, Croaa E-mal: Absrac Ths paper dscusses mehods used n dfferen objecve vdeo qualy mercs. An expermenal comparson of dfferen objecve mehods s also conduced. Ths expermen shows he mporance of vdeo conen for a subjecve qualy evaluaon no comprsed well by he objecve mercs used. Keywords: objecve qualy mercs, vdeo qual subjecve esng. INTRODUCTION In he las decade here has been an ncreasng neres n developng objecve qualy mercs for evaluaon of dfferen ypes of dgal vdeo dsoron. In a heerogeneous communcaon envronmen dfferen compresson echnques and dfferen b raes can be used smulaneousl and dfferen ypes of errors can occur as well as dfferen demands for Qualy of ervce (Qo). Due o her smplc he mean-squared error (ME) and he peak sgnal-o-nose rao (PNR) are very wdely used qualy mercs. Usually hey canno gve an objecve qualy measure correspondng well o he qualy perceved by a human observer for a wde range of codng and ransmsson parameers. everal objecve mercs have been recenly developed showng a good correspondence wh he subjecve Mean Opnon c ore (MO) obaned from human observers, []-[7]. In hs paper we dscuss full-reference models and gve expermenal resuls for hree objecve mercs compared wh subjecve resuls.. OBJECTIVE QUALITY METRIC Bascall wo dfferen approaches are used n objecve qualy mercs. ome mercs use Human Vson ysem (HV) characerscs o exrac mporan feaures n he orgnal and a dsored sgnal and valuae dfferences beween hem accordng o characerscs of human vsual error sensv []-[4]. The second approach s based on srucural dsoron measures (lke blurrng of edges or vsbly of blocks), [5]-[7]. We presen a andard paal Observer () based model, [], and DCT based VQM model exemplfyng he frs approach, and he General Vdeo Qualy Model Ths research s suppored by he Croaan Mnsry of Educaon, cence and pors hrough he projec No (VQM G ), [6], and he rucural mlary () nde [7], as examples of he second approach... model In [], Wason and Malo propose a qualy mercs based on he sandard spaal observer model ha ncorporaes psychophyscal and physologcal vson research resuls. The basc model uses a Conras ensvy Funcon (CF) n he spaal doman o exrac feaures mporan for a human observer. The frame dfference beween he orgnal frame p( and he dsored one p'( s p( p'( () The frame dfference s furher flered by mulplcaon wh CF n he frequency doman F ( CF F ( )) () - In () F and F are he nverse Fourer ransform and he Fourer ransform, respecvely. CF s he spaal CF funcon. Vsbly of he frame dfference,, s compued by poolng over space by Mnkovsk summaon /.9.9 d ( (,, ) d x y (3) y The overall dfference for a vdeo sequence, d, s calculaed by poolng over me by Mnkovsk summaon gven by / ( d d (4) The model uses four dfferen processng of pror o spaal and emporal poolng. One s a emporal pre-fler before spaal summaon d ( F ( CF F ( )) (5) +

2 where F - and F are he nverse and he drec emporal Fourer ransform, respecvel and CF s he emporal CF funcon. The model also uses a emporal pos-fler afer spaal summaon, gven by d ( F ( CF F ( )). (6) + p A local maskng model s used n he hrd processng of, as gven by + m (7) h( * p( + c where he Gaussan kernel h( s convolued wh he frame p( o acheve localzaon of maskng. The parameer c opmzes he srengh of maskng. Fnall he based model uses an algorhm for deecon of feld duplcaon (whch s done wh some coders) and decreases overesmaon of he error observed n dsoron measures based on pxelby-pxel comparsons. The model wh hs feld replcaon algorhm s marked as +h. Auhors have esed hs model on he full se of 60 VQEG, 30 Hz sequences, []. The hghes correlaon beween model predcon d and subjecve Dfference MO (DMO) s obaned for he model +m+h wh local maskng and he feld replcaon algorhm, as well as for he followng models: +m+p+h, ++m+p+h, ++m+h, +m+p, ++m+p, ++m and +m. All of hese models have used local maskng and ha has shown ha local maskng as a sngle processng has he mos sgnfcan nfluence on predcon accuracy. In he same research, for he purpose of comparson, resuls of he ME alone and he ME wh local maskng are nroduced. I s worh menonng ha ME wh maskng (ME+m) has only slghly worse predcon resuls han he bes models... DCT- based VQM model VQM model s a DCT-based vdeo qualy merc, developed by F. Xao, [3]. Ths merc s based on smplfed human spaal-emporal conras sensvy model. Model calculaes dsoron of a compressed vdeo n four seps:. For every frame he model performs Dscree Cosne Transform (DCT) for 8 x 8 pxels blocks b ( of he orgnal vdeo frame p(, and for blocks b '( of he dsored vdeo frame p'(. DCTb ( u, DCT( b ( ) DCTb '( u, DCT( b '( ) (8). The model convers DCT coeffcens o local conras values LC (u, by usng DC componen of each block. DC DCTb ( u, 04 LC ( u, DC DC DCTb (0,0, 0.65 (9) On he same way model obans LC '(u, of he compressed vdeo. 3. The model convers LC (u, and LC '(u, o jus noceable dfference values, JND (u, and JND (u,, by usng sac and dynamc spaal conras sensvy funcon (CF). 4. The JND coeffcens of orgnal and compressed sequences are subraced o produce a dfference values Dff (. Model ncorporaes conras maskng no smple maxmum operaon and hen weghs wh he poolng mean dsoron. Fnal VQM score s obaned by: Ds Ds Mean Max VQM Ds 000 mean( mean( Dff ( )) 000 max(max( Dff ( )) Mean Ds Max (0) The VQM score decreases as qualy of compressed vdeo rses and s zero for he losless compressed vdeo..3. NTIA General Vdeo Qualy Model (VQM G ) The VQM G model s developed by he Naonal Telecommuncaons and Informaon Admnsraon (NTIA) and obaned he bes average correlaon for boh 55-lne vdeo (NTC) and 65-lne vdeo (PAL) sequences n Vdeo Qualy Expers Group (VQEG) Phase II Full Reference Televson ess, [8]. Furhermore, hs model has been sandardzed by he Amercan Naonal andards Insue (ANI), [6]. The VQM G model was desgnaed o be a generalpurpose qualy model for a wde range of vdeo sysems wh dfferen resoluon, frame raes, codng echnques and b raes. The model uses seven parameers based on dfferen qualy feaures of a vdeo sream. For all feaures he VQM G model performs bascally he same seps. In he frs sage a fler s appled o he orgnal and a dsored vdeo o enhance some propery mporan for qualy. Afer ha, feaures f ( are exraced from he spaal-emporal sub-regon, -T, usng he mean or sandard devaon of each flered -T regon. Fnall a qualy parameer q ( s obaned by comparng qualy feaures of he orgnal vdeo, f (

3 and feaures of he dsurbed vdeo, f '(. One of he followng comparson funcons s used for calculaon of qualy parameers: a) Eucldean dsance q ( ( f ( f '( ) + ( f ( f '( )) () b) The rao comparson funcon ( f ( f '( ) q ( () f ( c) The log comparson funcon f '( q ( log (3) 0 f ( paal and emporal poolng s obaned by usng some form of wors case processng (e.g. average of 0% wors-case q ( values). Tha makes he wors par of vdeo he predomnan feaure n he qualy measure. A bref descrpon of qualy parameers used n he VQM G model s gven n he remander of hs secon. For more nformaon neresed readers are referred o [6]. The frs model parameer s s_loss ha deecs loss of spaal nformaon (e.g. blurrng). Vdeo frames are flered (horzonally and vercally) wh a spaal fler, whch enhances he nformaon of edges n a vdeo frame. Parameer hv_loss deecs a shf of edges from horzonal and vercal o dagonal orenaon. Parameer hv_gan deecs a shf of edges from dagonal o horzonal and vercal orenaon (e.g. blockng). The s_gan parameer measures mprovemens of qualy caused by edge sharpenng or enhancemen. Parameer chroma_spread deecs changes n he spread of color samples dsrbuon. Parameer chroma_exreme deecs severe localzed color mparmens. Parameer c_a_gan measures percepbly of spaal mparmens n dependence of he amoun of moon as well as percepbly of emporal mparmens n dependence of he amoun of spaal daa. The VQM G measure consss of a lnear combnaon of he descrbed parameers: VQM G s _ loss hv _ loss hv _ gan chroma _ spread.346 s _ gan c _ a _ gan chroma _ exreme For no perceved mparmen he model gves he oupu value equal o zero, and for a rsng level of mparmen he oupu value rses, oo. (4) The VQM G model parcpaed n VQEG ess, whch nclude,536 subjecvely raed vdeo sequences. The Pearson lnear correlaon beween VQM G and subjecve DMO was for 55-lne es sequences, and for 65-lne sequences. In he same es, he PNR measure obaned Pearson correlaon for 55-lne es sequences, and for 65-lne es sequences..4. rucural mlary () ndex mercs uses srucural dsoron n vdeo as an esmae of perceved vsual dsoron. I s based on an assumpon ha HV s hghly adaped for exracon of srucural nformaon from he vewng feld. o, he level of perceved mparmen s proporonal o he perceved srucural nformaon loss nsead of perceved errors. For he srucural dsoron measure, uses means (µ and µ'), varances (σ and σ') and covarance (cov) of he orgnal and he dsored sequences. These values are calculaed for 8 x 8 pxels blocks b ( of he orgnal vdeo frame p(, and for blocks b '( of he dsored vdeo frame p'(. The ndex for a block b ( s calculaed as ( l ( c ( s ( 4µ ( µ '( cov ( ( µ ( + µ ' ( ) ( σ ( + σ ' ( ) (5) where l (, c ( and s ( are defned as follows l µ ( µ '( (6) µ ( + µ ' ( ( σ ( σ '( c ( σ ( + σ ' ( (7) cov( s ( σ ( σ '( (8) j Parameer l ( gves a lumnance dfference, c ( a conras dfference and s ( a srucure dfference measure beween blocks of he orgnal and he dsurbed vdeo frame. ndexes are no calculaed for he enre frame, bu only for properly seleced R blocks, hereby reducng sgnfcanly compuaonal cos whle sll provdng good expermenal resuls. The ndex ncludes srucural mparmens n Y, C b and C r color componens wh dfferen weghs. The local qualy ndex for every block s gven by Y Cb Cr ( 0.8 ( + 0. ( + 0. ( (9) j

4 Based on block, a qualy ndex s calculaed for every frame by usng weghng value w. The auhors seleced w beween 0 and, n dependence of local lumnance µ (. The frame qualy ndex Q( s gven by Q( R R w ( ( w ( (0) Fnall he overall qualy of he enre sequence s obaned as he weghed sum of frame qualy ndexes. Weghng value W( for a frame a momen depends on he moon level n ha frame. For a hgher level of moon n he frame model uses smaller W( because spaal dsoron s less vsble n a fas movng vdeo. Qualy ndex for he enre vdeo sequence s gven by W ( Q( W ( () Expermenal resuls repored n [7] show ha he qualy mercs obaned Pearson correlaon measured on es vdeo sequences from VQEG Phase I. 3. EXPERIMENTAL REULT We have made objecve and subjecve measuremens for wo CIF vdeo sequences, head and naure, wh 9.97 frames per second, coded wh an XvD coder wh 7 codng raes: 6, 50, 8, 50, 750,,50 and,500 kbs/s. One frame from each orgnal es sequence s shown n Fg.. a) b) Fg.. Frame from he es sequence: a) head ; b) naure The choce of sequences s based on her very dsnc conens. The head vdeo presens a speaker n he cenral poson of all frames. The naure vdeo s characerzed by rapd changes of conen and a hgh level of deals n every frame. Three objecve mercs are used n our expermens: PNR, VQM and. These objecve measures are obaned by usng he MU Vdeo Measuremen Tool, [9]. We have made expermenal subjecve qualy evaluaon wh non-experenced observers by usng he MU Percepual Vdeo Qualy ool, [9], and he Double mulus Imparmen cale (DI) accordng o ITU-R BT Resuls are gven as average MO for each codng rae for each sequence. Objecve and subjecve qualy evaluaon resuls for sequence head are gven n Table. and resuls for sequence naure are gven n Table. Table. Resuls for sequence head B rae PNR VQM MO (kb/s) Table. Resuls for sequence naure B rae PNR VQM MO (kb/s) In Fg.. a) MO grades versus PNR scores are gven for sequences head and naure wh dfferen b raes. MO grades versus scores are gven n Fg.. b), whereas MO grades versus VQM scores are gven n Fg.. c). Alhough he expermen s carred ou wh a small number of es sequences and a relavely small number of observers, some useful conclusons can be drawn. Resuls n Table. and Table. show ha MO grades for wo sequences are very close on a gven b rae, across whole range of b raes. Average dfference beween MO grades for wo sequences s 0.093, wh a maxmum 0.5 a 50 kbs/s. All objecve qualy mercs gve sgnfcanly dfferen resuls for head and naure sequences for a gven b rae, and, wha s more mporan, for he same (or smlar) MO grade. Ths dfference rses for hgher b raes. As can be seen n Fg.., he dfference beween he resuls for head and naure sequences are hgh for all hree objecve mercs. For he PNR mercs he average dfference s 3.6 db, whch makes 38.3% of he whole PNR range n hs expermen. For he mercs he average dfference s ,.e. 60.% of he measured range and for he VQM mercs he average dfference s 0.39,.e. 8.% of he measured VQM range. Alhough he VQM mercs gves he closes resuls for wo vdeos, he dfference beween resuls s so hgh ha one canno say wha VQM value s a hreshold value

5 o naure o naure a) acheve overall qualy grade for a vdeo sequence. Expermenal resuls of objecve and subjecve vdeo qualy measuremens for wo vdeo sequences wh dsnc conen show ha he correspondence beween he objecve and he subjecve grades depends no only on he mehods used bu also on he vdeo conen. The emporal summaon can be more crcal because of muual nfluence of spaal and emporal maskng, as well as vdeo conen mporance for human observer. Objecve qualy measures show a hgh correspondence o subjecve qualy grades (repored correlaon s close o or hgher han 0.9), and can be used for evaluaon of dfferen codng echnques or dfferen channel condons. Bu for he defnon of he Qo parameer needed for he requred qualy of he gven vdeo ransmsson, he spread of resuls s oo hgh, and conen dependence of mercs has o be mproved. REFERENCE ndcang for example he MO score 4. The naure vdeo obans MO score 4 for he VQM value close o, whle he head vdeo has MO score close o 3 for he same VQM value. Maskng properes of vdeo conen, mporance of conen for a human observer, even he poson of mparmen, nfluence subjecve experence of he dsored vdeo, as also repored n [0]. 4. CONCLUION b) c) o naure Fg.. a) MO vs. PNR; b) MO vs. ; c) MO vs. VQM Objecve mercs use spaal summaon over frame and emporal summaon over sequence o [] M. Masr.. Hemam, Y. ermade "A calable Wavele-Based Vdeo Dsoron Merc and Applcaons", IEEE Trans. on Crcus ys. Vdeo Technol., Vol. 6, No., 006, pp [] A.B. Wason, J. Malo, "Vdeo Qualy Measuremen Based on he andard paal Observer", Proc. ICIP, 00, pp. 4-8 [3] hp://se.sanford.edu/class/ee39j/projecs/ projecs/xao_repor.pdf [4] C.J.B. Lambrech e al., "Qualy Assessmen of Moon Rendon n Vdeo Codng", IEEE Trans. Crcus ys. for Vdeo Technol., Vol. 9, No. 5, 999, pp [5] E.P. Ong e al., "Vsual Dsoron Assessmen Wh Emphass on paally Transonal Regons", IEEE Trans. Crcus ys. Vdeo Technol., Vol. 4, No. 4, 004, pp [6] M.H. Pnson,. Wolf, "A New andardzed Mehod for Objecvely Measurng Vdeo Qualy", IEEE Trans. on Broadcasng, Vol. 50, No. 3, 004, pp. 3-3 [7] Z. Wang, L. Lu, A.C. Bovk, "Vdeo Qualy Assessmen Based on rucural Dsoron Measuremen", gnal Processng: Image Comm. Vol.9, 004, pp. -3 [8] Vdeo Qualy Expers Group, [9] MU Graphcs&Meda Lab, Vdeo Group, MU codecs, vdeo/ [0] M.. Moore, J.M. Fole.K. Mra, "Defec Vsbly and Conen Imporance: Effecs on Perceved Imparmen", gnal Processng: Image Communcaon 9, 004, pp