Ambient Illumination as a Context for Video Bit Rate Adaptation Decision Taking

Transcription

1 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUBER (DOUBLE-CLICK HERE TO EDIT) < 1 Ambnt Illumnton s Contxt for Vdo Bt Rt Adptton Dcson Tkng G. Nur, H. Kodkr Archch, mbr, IEEE, S. Dogn, mbr, IEEE, nd A.. Kondoz, mbr, IEEE Abstrct In ths lttr, nw bt rt dptton dcson tkng tchnqu s proposd bsd on th obsrvtons tht mbnt llumnton contxt hs n ffct on th prcvd vdo qulty. oton ctvty nd structurl complxty chrctrstcs of vdo contnt r utlzd s th contnt rltd contxts n th proposd tchnqu. Exprmntl rsults dmonstrt tht sgnfcnt mount of bt rt cn b svd by dptng th vdo contnt ccordng to dffrnt mbnt llumnton condtons usng ths tchnqu. Subjctv ssssmnt rsults show tht th proposd dptton tchnqu s cpbl of xplotng th chngs n mbnt llumnton lvl for bt rt dptton wthout scrfcng th prcvd vsul qulty. Indx Trms Adptton dcson tkng, mbnt llumnton, moton ctvty, structurl complxty. I. INTRODUCTION Unvrsl ultmd Exprnc (UE) s ky concpt tht hs vson for chvng mprovd qulty of vwng xprncs [1][]. Adptton of vdo contnt s strtgc fld undr th umbrll of ths vson [3][]. Th ovrll trgt of vdo dptton s to mxmz th usr xprnc n trms of prcvd qulty. Adptton dcson tkng, whch ms to dtrmn optmum prmtrs for dptton oprtons, cn b consdrd s th brn of vdo contnt dptton. Thrfor, ffcnt dptton dcson tkng tchnqus should b dvlopd to ccomplsh th formntond trgt of vdo dptton. Contxt-wrnss plys sgnfcnt rol n dptton dcson tkng for chvng nhncd qulty of vwr xprncs []. Contnt-rltd contxts, such s moton ctvty nd structurl complxty of vdo, s wll s xtrnl contxts, such s vwng nvronmnt, my nflunc th dptton dcsons [6]. Usng mbnt llumnton condton s contxt durng dcson tkng s n ntrstng rsrch topc, whch s yt to b thoroughly nvstgtd for contnt dptton purposs. Ambnt llumnton contxt cn b gthrd through lght snsors plcd on usr dvcs to collct nformton bout th lvl of brghtnss n th consumpton nvronmnt. nuscrpt rcvd., 010 Gokc Nur, Hmnth Kodkr Archch, Sfk Dogn, nd Ahmt. Kondoz r wth th I-Lb ultmd Communctons Rsrch, Fculty of Engnrng nd Physcl Scncs, Unvrsty of Surry, Guldford, GU 7XH, Surry, UK. (phon: +-(0) ; fx: +-(0) ; -ml: g.nur@ surry.c.uk). Th mn gol of th work prsntd n ths lttr s to dvs nw vdo bt rt dptton dcson tkng tchnqu, whch rls on drvng dcsons bsd on th mbnt llumnton. To chv ths gol, frst th prcvd vdo qulty s subjctvly ssssd undr dffrnt mbnt llumnton condtons. Explotng th knowldg gnd through th ntl subjctv xprmnts, th proposd tchnqu s dvlopd. Vdo nformton s thn dptd consdrng dffrnt mbnt llumnton condtons usng th proposd tchnqu. In ordr to ssss th rsultng qulty of th dptd vdo contnt, furthr subjctv xprmnts r conductd. Th rst of th lttr s orgnzd s follows. In Scton II, th ntl subjctv ssssmnts r prsntd. Subjctv ssssmnt rsults nd obsrvtons bout ths rsults r ntroducd n Scton III. In Scton IV, th proposd dptton dcson tkng tchnqu s dscussd. Th dptton dcson tkng rsults r provdd nd ssssd n Scton V. Fnlly, Scton VII concluds th lttr. II. INITIAL SUBJECTIVE ASSESSENTS In ordr to ssss th ffct of mbnt llumnton on th prcvd vdo qulty to dvs n dptton dcson tkng tchnqu, numbr of ntl subjctv xprmnts wr conductd. Nn tst sgmnts wr usd n th xprmnts rthr thn th ntr tst squncs to mntn homognty n th moton ctvty nd structurl complxty chrctrstcs for crtn lngth n th tst contnts. Th nm of th nn tst sgmnts r: Stfn, Footbll, Soccr, Cty, Tmpt, Flowr Grdn, Nws, Wthr Forcst, nd Slnt [7], nd thy contn th frms: , 0-10, , 0-10, 0-10, -1, 0-10, 0-10, nd 0-10 of thr rspctv squncs. Th structurl complxty nd moton ctvty r rfrrd to s C nd n th lttr, rspctvly. Th tst sgmnts wr of CIF (3x88 pls) sptl rsoluton t 30 fps. Eght dffrnt chnnl bndwdths (.., 6, 96, 18, 19, 6, 38, 1, nd 768 kbps) wr slctd s th trgt chnnl rts for tst purposs. Th sgmnts wr ncodd usng th Jont Sclbl Vdo odl (JSV) rfrnc softwr vrson [8]. A 3 Dll dsply, whch hs rsoluton of 1680x100 pls, ws usd to dsply th squncs. Th Doubl Stmulus Contnuous Qulty Scl (DSCQS) mthod [9] ws usd throughout th tsts. DSCQS ws chosn n ordr to ssss th qulty of th ncodd contnt wth rspct to th mxmum qulty tht cn b offrd to th usr (.., th orgnl vdo squnc). 16 vwrs (10 xprts nd 6 non-xprts) prtcptd n th xprmnts. Th ffct of th mbnt llumnton on

2 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUBER (DOUBLE-CLICK HERE TO EDIT) < prcptul qulty ws ssssd n four dffrnt mbnt llumnton condtons (..,, 3, 19, nd 93 lux) crtd by our lbortory fclts, n whch th xprmnts wr conductd. Ths condtons wr msurd usng Grtg cbth Ey-On Dsply dvc [10]. Ambnt llumnton s rfrrd to s I n ths lttr. Th lngths of th vdos wr rstrctd to sconds, whch s n complnc wth th Intrntonl Tlcommuncton Unon (ITU) s rcommndton [9]. Th subjctv tsts wr conductd for ch vwr to ssss ll of th tst sgmnts ndvdully, whch wr rndomly ordrd for ch nvronmnt condton to prvnt ny prjudc. Aftr th xprmnts, th n Opnon Scors (s) [9] obtnd from ll of th vwrs wr computd. III. INITIAL SUBJECTIVE ASSESSENT RESULTS AND DISCUSSION Th ntl subjctv tst rsults r dscussd n ths scton for th vdo sgmnts n vw, whch corrspond to dffrnt s rngng from hgh to low. Fg. 1 llustrts th xprmntl rsults for th tst sgmnts. As cn b obsrvd from th fgurs, th curvs rprsntng th s rcordd n th lux nvronmnt prsnt th lst fvorbl prcvd qulty rportd by th vwrs. Th 3, 19, nd 93 lux nvronmnts dmonstrt n ncrsng ordr of rtngs. Th rson s tht n th lux (.., drk) nvronmnt, th snstvty of th Humn Vsul Systm (HVS) s grtr, bcus th rs nlrgs n rspons to th lss mount of mbnt lght ntrs nto th y. As rsult, n such n nvronmnt, th comprsson rtfcts n vsul scn r mor dstngushbl to th y thn n th othr nvronmnts. As cn b sn n Fg. 1 (), th pont mrkd s A on th 3 lux curv nd th pont mrkd s B on th lux curv rsult n smlr s dspt corrspondng to dffrnt bt rts. Smlr bhvor cn lso b obsrvd for th othr ponts on th othr curvs. Th obsrvton s tht whn I ncrss (.g., from to 3 lux), th prcvd qulty of n nput vdo sgmnt s not compromsd by vwng t t lowr bt rt. In ths wy, th bt rt of th vdo sgmnt cn b rducd by sgnfcnt mount f vwng tks plc n brghtr nvronmnt. Th chng n th I condton s rfrrd to s I nd thr dffrnt Is r consdrd: -3, -19, nd -93 lux n ths Lttr. Th prncpl d hr s tht th nput vdo sgmnt cn b dptd to lowr bt rt wthout cusng ny comproms n th prcvd vdo qulty undr prtculr I. Fndng th output bt rt of th dptd vdo undr I s th mn m of th proposd tchnqu. In ordr to chv ths m, th mthmtcl functons of vry curv n th grphs hv bn dtrmnd, s llustrtd n th fgurs. Th gnrc form of functon of ny curv s gvn n (1): = K ln( B) + L (1) whr B s th bt rt, K nd L r th constnts. Tbl I shows th K nd L vlus of th functons of th dffrnt I condton curvs for th dffrnt sgmnts. As sn from th tbl, on of th fctors tht th vlus of th constnts dpnd on s I. As cn lso b notcd, th vlus chng for dffrnt s. Howvr, Nws, Slnt nd Wthr Forcst squncs rsult n vry smlr s (.., 0.096, 0.09, nd 0.09) vn f th constnt vlus vry for th sm I condtons. Thrfor, t s clr tht s not th only contnt rltd chrctrstc tht dtrmns th vlus. It hs bn drvd ftr xtnsv nlyss tht C s th othr contnt rltd contxt tht hs n ffct on th vlus. Th rson why C s lso mportnt n dtrmnng th vlus s tht HVS s typclly usd to xtrctng structurl nformton from vsul scn rthr thn th rrors n th scn [11]. Concsly, ftr th xtnsv nlyss, t hs bn found tht I,, nd C r th domnnt fctors tht dtrmn th vlus of th constnts = 0.301ln(x) +.83 = 0.36ln(x) +.16 = 0.317ln(x) +.8 = 0.3Ln(x) +.70 () = 0.96ln(x) + 8 = 0.30ln(x) + 1 = 0.316ln(x) + 1 = 0.33ln(x) +.86 = 0.97ln(x) = 0.83ln(x) +.8 = 0.318ln(x) +.69 = 0.33ln(x) () = 0.19ln(B) + 3 = 0.17ln(B) + 66 = 0.138ln(B) = 0.18ln(B) (g) = 0.81ln(x) + 6 = 0.313ln(x) + 89 = 0.3ln(x) = 0.76ln(x) + 39 (±0.0 %) 3 (±0.0 %) 19 (±0.0 %) 93 (±0.0 %) = 0.86ln(x) = 0.30ln(x) +.77 = 0.318ln(x) +.38 = 0.77ln(x) +.91 (±0.0 %) 3 (±0.0 %) 19 (±0.0 %) 93 (±0.0 %) = 0.301ln(x) (d) = 0.37ln(x) +.66 = 0.3ln(x) +.0 = 0.31ln(x) = 0.19ln(x) + 13 (f) = 0.1Ln(x) + 3 (±0.0 %) = 0.137ln(x) + 8 = 0.161ln(x) +.81 (h)

3 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUBER (DOUBLE-CLICK HERE TO EDIT) < 3.3 = 0.118ln(x) = 0.19ln(x) = 0.11ln(x) +.99 = 0.179ln(x) +.6 (±0.0 %) 3 (±0.0 %) 19 (±0.0 %) 93 (±0.0 %) () Fg. 1 scors t vrous bt rts undr dffrnt I condtons for () hgh moton: () Stfn Footbll Soccr; () mdum moton: (d) Cty () Tmpt (f) Flowr Grdn; nd () low moton: (g) Nws (h) Wthr Forcst () Slnt, rspctvly TABLE I EXPERIENTAL AND CALCULATED CONSTANTS FOR THE VIDEO SEGENTS VIEWED UNDER DIFFERENT IS Sgmnt C I Exprmntl Clcultd K L K L Stfn Footbll Soccr Cty Tmpt Flowr Grdn Nws Wthr Forcst Slnt Th optcl flow lgorthm of Lucs & Knd [1] s usd for moton ntnsty msurmnts. Th moton ntnsts of svrl sgmnts wth dffrnt spto-tmporl rsolutons wr nlyzd usng ths lgorthm. It hs bn obsrvd tht ths rsolutons should b normlzd to llow for consstncy cross dffrnt sgmnts. Th mthmtcl modl prsntd n () s dvsd for prformng th msurmnt: NoF Π( n) n= 1 F = () NoF S whr Π(n) s th moton ntnsty of th n th frm of sgmnt. NoF s th numbr of frms n th sgmnt. F nd S r th frm rt nd sptl rsolutons of th sgmnt, rspctvly. Contours, whch chrctrz th boundrs of th objcts n vdo frms, r usd to rprsnt th C of th vsul scns n ths lttr. Cnny dg dtcton lgorthm [8] ws usd to dtrmn th contours n th frms wthout supprssng th pxls tht r consdrd s dgs by sttng thm to 1 [13]. To dvlop th structurl complxty lgorthm, th numbr of pxls tht r st to 1 s countd n vry frm of vdo sgmnt [1][1][16]. Th totl vlu s thn normlzd usng th NoF nd S to provd consstncy cross dffrnt vdo sgmnts s follows: NoF =1 δ ( n ) n C = (3) NoF S whr δ(n) s th numbr of dg pxls n th n th frm. In ordr to dvs gnrc functons for K nd L n (1) whl dvlopng th proposd tchnqu, frstly, th grphs rprsntng th C vs K, vs K, I vs K, C vs L, vs L, nd I vs L r plottd, s llustrtd n Fg.. Scondly, th rltonshps btwn ll of ths prs r pproxmtd usng curv fttng functons [17], s lso llustrtd n Fg.. Th constnts n th functons r prsntd wth P, U, V, Y, G, J, D, E, H, R, nd T n th fgur. K = P c + u () 3 + ln K = V + Y IV. PROPOSED ADAPTATION DECISION TAKING TECHNIQUE Th obsrvtons from th ntl tsts hv ndctd tht n dptton dcson tkng tchnqu nds to consdr thr fctors: th I condton, th, nd th C of th nput vdo for ffcnt dptton to produc th trgt output vdo bt rt. Th of sgmnt s msurd usng ts moton ntnsty. G K = + J ln I D L = E C + 3+ln (d)

4 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUBER (DOUBLE-CLICK HERE TO EDIT) < + L = H L = R lni + T () (f) Fg. K vrsus () C I; nd L vrsus (d) C () (f) I Followng, ch of th K nd L functons r mprclly ntgrtd togthr to dtrmn th rsultng qutons tht bst corrlt wth th xprmntl K nd L. Ths hs ld to ntroducng st of nw numrcl constnts (.., 7, 0., nd 1) to th qutons whl provdng th bst corrltons. Accordngly, th gnrc functons of K nd L r dvsd s follows: C 3 + ln 0. K = + ln I 7 L = () ln I 3+ ln C Tbl 1 lso shows th K nd L vlus clcultd usng th quton prsntd n () for th sgmnts n vw. Th corrlton coffcnts computd btwn th vlus obsrvd usng th ntl subjctv xprmnts nd th ons tht r clcultd for K nd L r 0.98 nd 0.961, rspctvly. To furthr dscrb th drvton of th proposd dptton dcson tkng tchnqu, us-cs scnro, n whch usr s vwng vdo sgmnt n n nvronmnt, whr I = A lux, s consdrd. Whl contnung to wtch th sgmnt, s/h movs to brghtr nvronmnt, whr I = B lux. In ordr to utlz th shrd ntwork rsourcs bttr, th vdo sgmnt cn b dptd to lowr bt rt undr B whl mntnng th sm prcptul qulty s follows: = () whr nd rprsnt th output dptd vdo sgmnt, whch s vwd undr B, nd th nput vdo sgmnt to b dptd, whch s vwd undr A, rspctvly. From (1) nd (), th proposd dptton dcson tkng tchnqu s dvsd to solv th followng quton: K ln( B ) + L = K ln( B ) + L (6) B = K ln( B ) + L L whr K nd L r th constnts for th vdo sgmnt to b dptd, nd K nd L r th constnts for th dptd vdo sgmnt. (7) cn b xplotd to clcult th output bt rt (B ) for dptng n nput vdo sgmnt from gvn nput bt rt (B ) undr spcfd mount of I wthout compromsng th prcvd vdo qulty. K (7) V. RESULTS AND DISCUSSION In ordr to vldt th ffcncy of th proposd tchnqu, dptton dcson tkng xprmnts wr crrd out. Th xprmnts wr conductd wth th tst sgmnts usd n th ntl subjctv tsts (.., Stfn, Cty, Flowr Grdn), togthr wth thr ddtonl tst sgmnts (.., Tnns, Costgurd, Hll ontor) consdrng th us-cs scnro dscussd n Scton IV. Th Is of th sgmnts vwd n A nd B condtons r rfrrd to s I nd I, rspctvly. I ws kpt s lux to mntn common rfrnc I for ll of th dptton xprmnts whrs fv dffrnt I s (.., 7, 3, 10, 19, nd 93 lux) wr usd n th xprmnts. Th xprmnts wr crrd out for th nput sgmnts tht hv thr dffrnt nput bt rts (.., 38, 1, nd 10 kbps). Tbl II prsnts th rsultnt B s usng th nput bt rts. To vldt th rsultng B s for th dptd sgmnts, furthr subjctv xprmnts wr conductd. Th dptd sgmnts wr prsntd to th vwrs undr th formntond I condtons. Durng th subjctv vluton tsts, obsrvrs wr skd to rt th vdo squncs ccordng to th DSCQS mthod, s dscrbd n th ITU s rcommndton on subjctv qulty ssssmnts [9]. A fv-lvl qulty rtng scl rngng from 1 (bd) to (xcllnt) ws usd durng th tsts. 16 obsrvrs (10 xprts nd 6 non-xprts) prtcptd n th xprmnts. Th vlus wr clcultd for vry vdo sgmnt ftr th tsts. Fg. 3 shows th subjctv tst rsults of th dptd sgmnts. As cn b obsrvd from th fgur, th prcptul qults of th dptd vdo sgmnts thr do not chng or only slghtly vry (~0.01%) undr chngng I condtons. It cn b rgud tht th proposd tchnqu hs bn ffcnt to dpt th vdo sgmnts whl mntnng thr rspctv prcptul qults dspt th dffrnt vwng I condtons. As mntond rlr, th dptton xprmnts wr conductd consdrng th us-cs scnro dscussd n Scton IV. Th proposd tchnqu s lso cpbl of srvng th dptton nds n rvrs ordr oprton n nothr us-cs scnro, n whch usr movs from brghtr nvronmnt nto drkr on whl wtchng vdo sgmnt. In ths knd of scnro, th output bt rt clcultd by th proposd tchnqu for th dptd vdo sgmnt bcoms hghr thn tht for th nput sgmnt. Thrfor, ths dptton s only fsbl whn ncssry xtr bndwdth s vlbl. Th rson of ncrsd bt rt for th sgmnt vwd n drk nvronmnt s tht th comprsson rtfcts r mor vsbl to th y n th drkr nvronmnt thn n th brght nvronmnt. In thr of th scnros, trmnl functonlts for ncrsng or dcrsng th brghtnss of th dsply cn lso b usd for mprovng th prcvd qulty by nhncng th contnt vsblty. Howvr, t should b notd tht our focus s not to mprov th contnt vsblty, but to rdstrbut shrd ntwork rsourcs mor ffctvly by xplotng th mbnt llumnton chngs.

5 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUBER (DOUBLE-CLICK HERE TO EDIT) < SOCCER ().1. FOOTBALL SOCCER FOOTBALL SOCCER FOOTBALL Fg. 3 scors t () 38 1 nd 768 kbps undr dffrnt I condtons TABLE II RESULTING ADAPTED VIDEO BIT RATES USING THE PROPOSED TECHNIQUE Sgmnt I (lux) B (kbps) B = 38 B = 1 B = Tnns Costgurd Hll ontor Stfn Cty Flowr Grdn VI. CONCLUSION A bt rt dptton dcson tkng tchnqu hs bn proposd bsd on th knowldg gnd through subjctv vdo qulty ssssmnts prformd n dffrnt mbnt llumnton condtons. Structurl complxty nd moton chrctrstcs of vdo hv ctd s th prmry contnt rltd contxts tht nflunc th dptton dcsons n ths rsrch work. Th xprmnts dmonstrtd tht sgnfcnt mount of bt rt cn b svd undr chngng mbnt llumnton chrctrstcs usng th proposd tchnqu. Subjctv qulty ssssmnts provd th ffcncy of ths tchnqu. Th ffcts of othr contxts tht cn b usd n dptton dcson tkng wll b consdrd n futur studs. REFERENCES [1] F. Prr nd I. Burntt, Unvrsl ultmd Exprncs for Tomorrow, IEEE Sgnl Procss. g., vol. 0, no., pp , r []. Wltl, C. Tmmrr, H. Hllwgnr, A Tst-bd For Qulty of ultmd Exprnc Evluton of Snsory Effcts, Proc.s of th Frst Int. Workshop on Qulty of ultmd Exprnc(QoEX 009), Sn Dgo, USA, Jul., 009. [3] D. ukhrj, E. Dlfoss, J-G. Km, Y. Wng, Optml Adptton Dcson Tkng for Trmnl nd Ntwork Qulty of Srvc, IEEE Trns. ult., vol. 7, no. 3, pp. 6, Jun. 00. [] I. Koflr, C. Tmmrr, H. Hllwgnr, A. Huttr, nd F. Snhuj, Effcnt PEG-1 Bsd Adptton Dcson Tkng for Sclbl ultmd Contnt, Proc. 1 th Annul Elct. Imgng Conf, 007. [] W.Y. Lum nd F.C.. Lu, A Contxt-Awr Dcson Engn for Contnt Adptton, IEEE Prvsv Computng, vol. 1, no. 3, pp. 1-9, July-Spt. 00. [6] S. Shnmughm, Prcptul Vdo Qulty surmnt for Strmng Vdo ovr obl Ntworks, str s thss, Unvrsty of Knss, 006. [7] ftp://ftp.tnt.un-hnnovr.d/pub/svc/tstsquncs [8] JSV , CVS Srvr, grcon.nt.rwth-chn.d/cvs/jv [9] Intrntonl Tlcommuncton Unon (ITU) Rdo Communcton Sctor: thodology for th Subjctv Assssmnt of th Qulty of Tlvson Pcturs, ITU-R BT.00-11, 00 [10] Grtg cbth Ey-On Dsply [11] Z. Wng, A. Bovk, H. Shkh, nd E. Smoncll, Img Qulty Assssmnt: from Error Vsblty to Structurl Smlrty, IEEE Trns. Img Proc., 00, pp [1] D.J. Flt, nd Y. Wss, Optcl Flow Estmton n Prgos, Hndbook of th. odls n Comp. Vs., Sprngr, pp. 39, 006. [13]J.F. Cnny, A Computtonl Approch to Edg Dtcton, IEEE Trns. Pttrn Anlyss nd chn Intllgnc, vol. 8, pp , [1] C. Grgorscu, N. Ptkov,. A. Wstnbrg, Contour nd Boundry Dtcton Improvd by Surround Supprsson of Txtur Edgs, Img nd Vson Computng, vol., pp , 00. [1] G. Ppr, P. Cmps, N. Ptkov, nd A. Nr, A ultscl Approch to Contour Dtcton by Txtur Supprsson, SPIE Im. Proc.: Alg. nd Syst., vol. 606A, Sn Jos, CA, 006. [16] J. lk, S. Blong, T. Lung, nd J. Sh, Contour nd Txtur Anlyss for Img Sgmntton, Int. Journ. Comput. Vson, vol. 1, pp. 7 7, 001. [17] t/