Greyworld White Balancing with Low Computation Cost for On- Board Video Capturing

Transcription

1 reyword Whte aancng wth Low Computaton Cost for On- oard Vdeo Capturng Peng Wu Yuxn Zoe) Lu Hewett-Packard Laboratores Hewett-Packard Co. Pao Ato CA USA Abstract Whte baancng s a process commony used n dgta photography to estmate the umnant then compensate the coor appearance so that the whte coor appears as whte regardess of the varety n the umnaton condton when the pcture was captured. n ths paper we propose a ow computaton cost approach to reaze whte baancng for onboard dgta vdeo capturng. The reyword approach has been wdey adopted consdered as the bass for many automatc whte baancng agorthms. When the reyword approach s apped to dgta vdeos each vdeo frame n the sequence s treated ndependenty. The two modues n the approach namey the mutper estmaton the frame modfcaton are repeated on every frame wthout takng nto account the tempora correaton across adjacent frames. n fact when a segment of a vdeo cp s captured under a fxed ghtng condton f the objects n the scene do not move suffcenty rapdy the mutpers for dfferent frames often possess strong correaton. Such an observaton eads to the foowng two deas to reduce the overa computaton cost for whte baancng on-board vdeos: ) Detectng keepng track of the umnant change across frames to determne f the mutper estmaton for a new frame s necessary 2) nstead of usng the mutpers to modfy one entre frame a bock-based approach s proposed to compare bocks of the current frame wth the co-ocated ones n the prevous frame to perform coor modfcaton on seected bocks. Usng our proposed approach the computatona compexty s reduced whe the performance s mantaned. As a trade-off we need to mantan the frame buffer as we as the mutper buffer. ntroducton Whte baancng s a process commony used n dgta photography to estmate the umnant then compensate the coor appearance so that the whte coor appears as whte regardess of the varety n the umnaton condton when the pcture was captured. n ths paper we propose a ow computaton cost approach to reaze whte baancng for on-board dgta vdeo capturng. The reyword approach has been wdey adopted consdered as the bass for many automatc whte baancng agorthms. Most consumer) vdeos may endure a varety of umnatons wthn a short tme wndow. The reyword approach determnes the chromatcty of the umnant from the average of a the pxes n one mage assumng that the average coor of the scene s gray that any departure from ths average n the mage s caused by the umnant. When the reyword approach s apped to dgta vdeos each vdeo frame n the sequence s treated ndependenty. The two modues n the approach namey the mutper estmaton the frame modfcaton are repeated on every frame wthout takng nto account the tempora correaton across adjacent frames. n fact when a segment of a vdeo cp s captured under a fxed ghtng condton f the objects n the scene do not move suffcenty rapdy the mutpers for dfferent frames often possess strong correaton. Such an observaton eads to the foowng two deas to reduce the overa computaton cost for whte baancng on-board vdeos: ) Detectng keepng track of the umnant change across frames to determne f the mutper estmaton for a new frame s needed 2) nstead of usng the mutpers to modfy one entre frame a bock-based approach s proposed to compare bocks of the current frame wth those co-ocated ones n the prevous frame A subset of bocks n the current frame are then dentfed to represent the porton of the current frame that are dfferent from the prevous frame n terms of the umnant The coor modfcaton s then ony performed on those dentfed bocks. Usng our proposed approach the computatona compexty s reduced whe the performance s mantaned. Compared wth other reyword-based whte baancng approaches for vdeo capturng our proposed technque has the dstngushed advantage on the computaton cost. Through takng the tempora correaton across adjacent frames pus the consstency of umnant nto account the proposed technque can ower the amount of mutpcaton operatons nvoved n the coor modfcaton. As a trade-off we need to mantan the frame buffer as we as the mutper buffer. reyword-ased Whte aance Technque for On-board Dgta Vdeo Capture reyword Agorthm for Whte aance Varous agorthms have been deveoped to estmate adjust umnatons. Consderng the computatona compexty of the agorthm the fact that consumer) vdeo capturng may have dfferent umnatons wthn a snge vdeo cp we many nvestgate the reyword approach n ths paper. n the reyword approach the chromatcty of the umnant s determned from the average of a the pxes n one mage. t Socety for magng Scence Technoogy

2 assumes that the average coor of the scene s gray that any departure from ths average n the mage s caused by the umnant. Specfcay gven an mage n the space ettng represent the mage pxes n the channes at poston the reyword agorthm searches for a transform n the coor space α = β γ that appes to the The mutpers α β γ are obtaned ) components. α = δ / E ) 2.) β = δ / E ) 2.2) γ = δ / E ) 2.3) where E ) E ) E ) respectvey are the mean of δ = mn{ E ) E ) E ). 3) The modfed mage usng greayword s then obtaned as: x = α 4.) x = β 4.2) x γ =. 4.3) reyword-ased Whte aance for Vdeo Capture ) ) { ) +) +) { +) +2) Fgure Fundamenta procedure of reyword-based whte baance for vdeo capture n [] [2] usng the reyword-based whte baance approach for st/vdeo capture s addressed. As ustrated n Fgure n dgta vdeo capture a vdeo cp s the output of capturng a sequence of frames contnuousy wthn a certan tme perod. The whte baance scheme descrbed n [] [2] s essentay to appy the reyword approach as ntroduced n the prevous secton to each of the frame n the sequence to generate a modfed frame sequence as the fna captured vdeo. More +2) { +2) +m) +m) { +m) specfcay for the th frame ) the mutper estmaton ) bock shown n Fgure utzes Eq. 2.) 2.2) 2.3) to P = α β γ such a set of compute a set of mutpers ) { ) mutpers are used to modfy the th frame usng Eq. 4.) 4.2) 4.3) to obtan the corrected frame ). A Low Computaton Cost reyword Technque for On oard Vdeo Capture n the scheme descrbed n Fgure each frame n the sequence s treated ndependenty. The mutper estmaton frame modfcaton are repettvey conducted on each frame wthout consderng the tempora correaton across adjacent frames. n fact when a segment of vdeo cp s captured under a fxed ghtng condton the objects n the scene do not move rapdy the mutpers often possess strong correaton. Such an observaton eads to the foow two deas to reduce the computaton cost: ) Detect the umnaton change across frames to determne f the mutper estmaton s needed 2) nstead of usng the mutpers to modfy the entre frame a bock-based approach s proposed to compare bocks of the current frame wth those co-ocated ones n the prevous frame to dentfy a subset of bocks n the current frame These dentfed bocks represent the porton of the current frame that s dfferent from the prevous frame n terms of the umnaton condton. The coor modfcaton s then ony performed on the dentfed bocks. umnaton Change Detecton To detect the change of umnaton across frames the coor hstogram s wdey used as descrbed n [3]. ven a frame ) et H j) denote ts coor hstogram where j = 2 s the tota number of bns used to form the hstogram. The dstance between the hstograms of two adjacent frames ) + s obtaned as: ) SD = j= H j) H. 5) + j) f the dfference SD s arger than a gven threshod T then an umnaton change s decared. To set the threshod t usuay requres the cacuaton of the mean μ the stard devaton σ of SD over ) = 2 m. The threshod can be obtaned as: T = μ + ασ. 6) A typca vaue for α s 5 or 6. For the on-board vdeo capture t s not reastc to compute the mean μ the stard devaton σ over the entre vdeo cp to be captured for that requres the bufferng of a the captured frames. nstead an adaptve umnaton change detecton agorthm has been proposed n whch ony a sma number of hstograms need to be buffered. A smar approach has been suggested n [4] for the purpose of vdeo summarzaton. Let H denote the hstogram buffer whch keeps K hstograms. The adaptve umnaton change detecton agorthm s descrbed as foows: CSH'2008: nternatona Conference on magng Scence Hardcopy 28

3 Step : For frame ) = K obtan H et Step 2: H = K H Obtan SD = j= H j) H + j) = 0 K 2 Step 3: Obtan μ = mean SD ) σ = std SD ) TH H H = μ + ασ H Step 4: Set = K + Step 5: Obtan H from ) Step 6: Obtan SD Step 7: f SD ' > TH between ) ' = j= H K H j) H j) decare umnaton change go to Step 8 ese go to Step 8 Step 8: Set H = H+ = 0 K 2 an H = ) K H = SD+ Step 9: Set SD = 0 K 3 ' SD K 2 = SD Step 0: Obtan μ = mean SD ) σ = std SD ) TH H H = μ + ασ H Step : Set = K + 2 go to Step 5. H Adoptng the above agorthm each nput frame ) = K + can be examned to determne whether t s at the boundary of an umnaton change. To everage the tempora correaton of the mutpers under the same or smar umnaton we ony perform the mutper estmaton when the frame s determned as the boundary frame. More specfcay we mantan a mutper buffer to keep track of the mutpers used for modfyng the current frame. The vaues of mutpers w be updated through mutper estmaton ony when the ncomng frame s decared to be the boundary of umnance change. Otherwse the buffered mutpers w be used to modfy the ncomng frame. ock-ased Coor Correcton frame ) dvded to bocks) Compute mean E) E2) E3) E2) E22) E23) E3) E32) E33) Fgure 2 ustraton of bock-based computaton of means E) f frames are dentfed as beongng to the same umnance condton the same set of { P ) = α β γ ) s used throughout a these frames to perform the coor modfcaton as descrbed n Eq. 4.) 4.2) 4.3). Further we can everage the tempora correaton of consecutve frames to reduce the computaton cost. We frst dvde ) ) ) nto bocks of equa sze. Then when computng the mean of ) ) ) not ony E )) E )) E )) are computed but aso the mean of each bock of the ) ) ) as ustrated n Fgure 2. Suppose for p ) p = we dvde t nto M N bocks. The mean computaton w resut a mean of the entre p ) means of bocks E p m where m = M n = N. To perform bock-based modfcaton we examne the mean E p m wth E p m the foowng procedure s conducted: Procedure for bock-based coor modfcaton: ven p ) p ) E p m E p m for bock m of p ) { f not E m E m E m E m E m E m ) then { for a the pxes beongng to bock bock m of p ) compute bock m of p ) ) x = α x ) x = β x ) x = γ x ese { for a the pxes beongng to bock bock m of p ) compute bock m of ) x x x x = x = x =. The dea behnd ths procedure s that f the bock dfferences on channes across frames are a very sma we consder t as a strong ndcaton that the two bocks ocated n m n frame p ) p ) are very smar. Snce we use the same set of mutpers we can safey set the bock m n frame p ) to be the same as that of p ). f the bock dfferences are not a very sma we just perform the bock-based modfcaton usng the buffered mutpers. n ths way for p Socety for magng Scence Technoogy

4 bocks that have consstent pxe vaues across consecutve frames we reduce the computaton by skppng the pxe modfcaton that nvoves the mutpcaton operaton. As a trade off we need to keep the frame buffer the mutper buffer. The Compete Scheme Fgure 3 ustrates the compete scheme of appyng ths ow computaton cost reyword technque for on board vdeo capture. n genera gven an nput frame ) we frst examne whether the frame s at the boundary of a umnance change by appyng the adaptve umnance change detecton scheme. f t s we perform the mutper computaton to refresh the mutper buffer use such a mutper to modfy the entre frame for whte baance. Otherwse the nput frame s consdered to share the same umnance wth the prevous frame we use the mutper n the buffer to perform bock-based modfcaton. ) Adaptve umnant change detecton Change detected Yes No ead mutper buffer Mutper estmaton Update mutper buffer ock based modfcaton Frame based modfcaton ) Fgure 3 Proposed ow computaton cost reyword-based whte baancng approach for on-board vdeo capture a) Orgna vdeo frames b) After whte baancng usng our proposed approach Expermenta esuts Concusons Fgure 4 Test sequence Ths paper has addressed a ow computaton cost reywordbased whte baance technque for on board vdeo capture. As CSH'2008: nternatona Conference on magng Scence Hardcopy 283

5 compared wth other reyword-based whte baance technque for vdeo capture the proposed technque has the dstngush advantage on the computaton cost. y takng the tempora correaton of frames consstency of umnaton nto account the proposed technque can ower the amount of mutpcaton operatons nvoved n the coor modfcaton. eferences [] Yoon Km June-Sok Lee A.W. Moraes Sung-Jea Ko A Vdeo Camera System wth Enhanced Zoom Trackng Auto Whte aance EEE Transactons on Consumer Eectroncs vo. 48 no. 3 pg ) [2] June-Sok Lee You-Young Jung yung-soo Km Sung-Jea Ko An Advanced Vdeo Camera System wth obust AF AE AW Contro EEE Transactons on Consumer Eectroncs vo. 47 no. 3 pg ) [3] HongJang Zhang Yhong ong S.W. Smoar Shuang Yeo Tan Automatc Parsng of News Vdeo Proceedngs of the nternatona Conference on Mutmeda Computng Systems pg ) [4] Arnon Amra a Ashourb Savtha Srnvasan Automatc eneraton of Conference Vdeo Proceedngs Journa of Vsua Communcaton mage epresentaton vo. 5 No. 3 pg ) [5] Homer H. Chen Chun-Hung Shen Pe-Shan Tsa Edge-ased Automatc Whte aancng wth Lnear umnant Constrant Proceedngs of SPE nternatona Conference on Vdeo Communcatons mage Processng VCP) San Jose CA USA vo pg. 6508D- 6508D ) Author ography Peng Wu receved hs PhD n Eetrca Engneerng from Unversty of Caforna Santa arbara 200). Snce then he has worked n the magng Technoogy Department at HP Labs n Pao Ato CA. Hs work has focused mutmeda content anayss mutmeda systems. Yuxn Zoe) Lu receved her PhD n eectrca engneerng from Purdue Unversty West LafayetteN 2004). She was wth Noka esearch Center n rvng TX from Aug through Dec as a esearch Engneer was wth Sun Labs n Meno Park CA from Dec through Aug as a Staff Engneer. Snce Aug she has been wth HP Labs n Pao Ato CA as a esearch Scentst. Zoe s major research nterests have been focused on vdeo compresson meda anayss Socety for magng Scence Technoogy