AN ADAPTIVE ORDER-STATISTIC NOISE FILTER FOR GAMMA-CORRECTED IMAGE SEQUENCES

Transcription

1 AN ADAPTIVE ORDER-STATISTIC NOISE FILTER FOR GAMMA-CORRECTED IMAGE SEQUENCES Richad P. Kleihost (1), Reginald L. Lagendijk (2), and Jan Bieond (2) (1) Philips Reseach Laboatoies, Eindhoven, The Nethelands (2) Delft Univesity of Technology, Infoation Theoy Goup, Delft, The Nethelands Please diect all coespondence to: D. Reginald L. Lagendijk Delft Univesity of Technology Depatent of Electical Engineeing Infoation Theoy Goup P.O. Box GA DELFT, The Nethelands phone: fax: e-ail: ABSTRACT Oiginal video signals ae often coupted by a cetain aount of noise oiginating fo the caea electonics. As a esult of the gaa coection in caeas, the obseved noise is signal dependent. In this coespondence we pesent a spatio-tepoal ode-statistic (OS) noise filte that takes into account the gaa coection in the caea. The calculation of the filte coefficients equies highe-ode ode-statistics (HOOS) of the noise pocess. We ake use of a ange test (RT) to deteine locally fo which neighboing signal values an estiate should be foed. The noise filte that we aive at is adaptive and coputationally efficient.

2 1. INTRODUCTION Most digital video signals that we conside as being the oiginal and pefect ecodings of a natual scene, ae often coupted by a cetain aount of noise. Supisingly, the aount of noise in oiginal video signals is uch highe than one would expect fo the quantization of the luinance and choinance coponents in 8 to 12 bit. The doinant cause of the noise lies usually in theal effects in the electonic cicuity. Fo instance, the well-known CalTain video, that is often used in copession liteatue, has an estiated signal-to-noise atio (SNR) of 28 db. Reducing the aount of obseved noise is desiable to ipove the visual quality, but is also ipotant as a pepocessing stage in any signal pocessing applications of inteest, such as video copession and analysis. A coplicating facto in the filteing of caea noise is the gaa coection that is pesent in any caea to copensate fo the non-lineaity of the display's cathode ay tube (CRT). The gaa coection esults in signal-dependent obsevation noise [1]. A typical noise filte fo video signals, howeve, usually ignoes the signal dependency of the obsevation noise. We popose an ode-statistic (OS) spatio-tepoal finite ipulse esponse (FIR) estiato fo eoving signal-dependent caea noise fo iage sequences [2,3]. Vaious fos of ode statistics estiatos have been intoduced in the liteatue, anging fo siple edian, iniu and axiu filtes to adaptive estiatos using linea cobinations of odeed obsevations [4,5,6]. Due to the odeing of the obsevations pio to the actual filteing opeation, OS estiatos often outpefo linea estiatos. In this coespondence we popose a new technique to estiate the weights of an adaptive OS-estiato. The applicability of this technique is not liited to the noise eoval poble that we conside in this coespondence, but can be used fo any nonlinea signal estiation poble [7]. The esulting estiato weights ae optially adapted to the pobability density function (PDF) of the noise and to the spatio-tepoally localized popeties of the video signal. The estiato weights ae a function of Highe Ode Ode Statistics (HOOS) of the noise pocess. Futheoe we ake use of a ange test (RT) to eove outlies fo the spatio-tepoal estiation window as so to educe the estiato's vaiance. The esult is an optial and coputationally efficient non-linea filte fo caea-noise. -1-

3 2. SIGNAL MODEL The popeties of the obsevation noise depend on the caea stuctue. A siplified odel fo a colo caea is shown in Figue 1. The electical cuent geneated by the RGB (ed, geen, and blue) sensos depends diectly on the intensity of the iaged object, and takes on values between 0.0 and 1.0. Afte suitable aplification and possibly apetue coection, the RGB video signals pass a gaa-coection stage to copensate fo the CRT's non-lineaity. Finally, an RGB/YUV atix ay tansfo the RGB signals into luinance and choinance (YUV) coponents. As a postpocessing step, the luinance signal is soeties spatially enhanced (shapened). In this coespondence we assue that the shapening and apetue coection ae switched off o have a negligible effect on the video signal's statistical popeties. Futhe, we assue that the RGB/YUV atix is also switched off and that the RGB video signals ae quantized in 8 bits. The above odel can be atheatically suaized as follows. The ideal noise-fee spatio-tepoal caea signal f(i,j,k) is coupted by theal noise n(i,j,k) intoduced in the cicuity of the senso and aplifie. With i and j we denote spatial coodinates, k is the video fae nube, and f(i,j,k) efes to the ed, geen, o blue video coponent. The theal noise is odeled as a stationay zeo-ean signalindependent white Gaussian pocess [1,8]. We assue that the noise vaiance 2 n is known o has been estiated in advance. Ignoing the aplification facto fo siplicity, the obseved signal afte gaa coection is g f n (1) Noally we have 0<<1, with a typical value of =0.45, which causes the bighte pats of a video fae to be ephasized. The objective of noise filteing is to estiate the noise-fee gaa-coected signal f fo the obseved noisy signal g(i,j,k). An obvious appoach would be to fist invet the gaa coection, apply a standad filte fo additive signal-independent noise, and gaa-coect the filteed esult aftewads. Unfotunately, this obvious appoach leads to seveal pobles in pactice. In the fist place, even a sall aount of additive -2-

4 quantization noise in g(i,j,k) akes the invesion of the gaa coection ill-conditioned. Secondly, inveting the gaa coection would equie a data epesentation in a significantly lage nube of bits if the quantization popeties of f(i,j,k) need to be peseved. And finally, the non-linea invesion of the gaa coection would intoduce spatial aliasing into the signal. which hapes the subsequent noise filteing. Ou objective is theefoe to develop a filte that woks diectly on the non-linea odel fo the gaa-coected signal g(i,j,k). To bing Eq. (1) into the standad additive noise odel, we appoxiate (1) by a Taylo expansion of ode M about n(i,j,k)=0: M n f n f 1 f 1 f f n 1 n f (2) whee we have used the following shothand notation: ! (3) In (2), f is the desied signal, which is coupted by a noise te n. The Taylo expansion in (2) is aleady easonably accuate fo values of M lage than 2. In the expeiental esults section we discuss the influence of the ode M and show that a value of M lage than 4 is unnecessay. The additive noise coponent n is dependent on the signal f(i,j,k). Let us assue that f(i,j,k) is locally constant and that n(i,j,k) is Gaussian white noise with zeo-ean and vaiance n 2. Then the following expectation and vaiance of the noise n E[ n i, j, k ] f f ( ) va[ n ] f E[ n i, j, k ] can be calculated in case M=4 [1,3]: 4 2 n n (4) whee we have used the following popeties: (,, ), (,, ) E n 3 i j k 0 E n 4 i j k 3 2 n (5) -3-

5 Note that if =1 in (4), the noise eains zeo-ean and signal-independent. Since, howeve, 0<<1, the expectation and vaiance of the noise n is dependent of the signal f(i,j,k). 3. MSE OPTIMAL ORDER -STATISTIC ESTIMATOR Weighted ode-statistic (OS) estiatos have poved to be vey useful in vaious iage (sequence) pocessing pobles [4-6,10]. Any weighted OS estiato fos an estiation of a signal s value using a linea cobination of odeed obsevations taken fo a local obsevation window o windows. The vaious OS estiatos pesented in the liteatue diffeentiate in how they deteine the obsevation window(s) and how they calculate the optial estiato weights. The OS FIR noise filte that we deive in this section opeates on a spatio-tepoal window of size R. Within this sall spatio-tepoal window, the oiginal signal f(i,j,k) is assued to be constant. As a esult the calculation of the estiato weights does not equie statistical knowledge of the signal f(i,j,k). In Section 4 we will discuss pocedues fo selecting an appopiate spatio-tepoal window in which this assuption about the oiginal signal holds. The geneal expession fo the spatio-tepoal OS FIR estiato fo the gaa-coected signal has the following fo: f ( i, j, k ) w g ( i, j, k ) R 1 ( ) (6) Hee g( ) ae odeed obsevations taken fo the spatio-tepoal window of size R centeed about (i,j,k). The subscipt () denotes ank, with g g 1 fo 1<R. We will now ( ) ( ) deteine the weights w fo the MSE optial estiation of the locally constant signal f(i,j,k) iesed in signal-dependent noise n ( i, j, k ). On the basis of the Taylo expansion (2), the following expectation fo g(i,j,k) is obtained: -4-

6 E[ g] f f M 1 E[ n ] f (7) This elation still holds afte odeing of the obsevations g(i,j,k). Thus we can wite fo each ank : M E[ n ] ( ) E[ g( ) ] f f 1 R (8) f 1 The M oents of the R anks of the caea noise n( ) in (8) can be calculated in a way siila to the one descibed in [4,9] fo the fist and second ode oents of ode statistics [3,7]. Of couse, fo the calculation of these highe-ode ode-statistics (HOOS) the PDF of n(i,j,k) ust be assued known. The expectations E[ g( )] in (8) ae not available, and ae appoxiated by the actual obsevations g( ). The stochastic eo intoduced by this appoxiation is denoted by ( i, j, k ). Cobining the esulting R expessions fo the anks into a atix-vecto notation, we aive at: g g ( R) M 1 E[ n( ) ] E[ n( ) ] M 1 E[ n( R) ] E[ n( R) ] f f M f ( 1) M R (9) o in shot hand notation: g() W (10) The (RM+1) atix W is entiely known. Fo coon window sizes, we have R>M+1 which akes (10) an ove-deteined set of equations. Theefoe, (10) is solved such that ( i, j, k ) has iniu vaiance, yielding a ean-squaed eo optial estiato fo (i,j,k). Solving the ove-deteined equation (10) by fo instance singula value decoposition (SVD), the esulting estiato is given by: ( i, j, k ) W g ( i, j, k ) () (11) The top ow of W now exactly coesponds to the desied weights w of the OS-FIR estiato (6) to estiate f using the data g(i,j,k) in a spatio-tepoal window of size R. -5-

7 In an efficient ipleentation, the top ow atix enties of W ae coputed in advance. This can be done if the PDF of the noise, the Taylo expansion ode M, and the size R of the spatio-tepoal window ae known. As entioned in the beginning of this section, the calculation of the OS estiato weights does not equie statistical knowledge about the oiginal signal. Figue 2 shows the weights w of the estiato (6) fo M=2 and M=4, and fo 7R27. The Taylo appoxiation ode M can be feely chosen to ake the appoxiation eo between the odel (1) and the expansion (2) abitaily sall. This choice, howeve, does not influence the calculation coplexity of the estiato (6) o (11), but only the a pioi coputation of the atix W in (11). 4. RANGE TEST FOR REMOVING OUTLIERS The odeed obsevations g( ) used in the OS-FIR estiato (6) ae obtained fo a spatiotepoal window centeed about (i,j,k). In calculating the optial weights w, we assue that within this window f(i,j,k) has a constant value. Clealy, this assuption ay not hold in the tepoal doain if otion occus in the iage sequence, and ay not hold spatially if the window ovelays a spatial edge. Thee ae two pocedues fo adapting the window size and oientation such that the assuption of constancy of the oiginal signal within the spatio-tepoal window becoes oe eliably. The fist pocedue is egisteing successive iage faes o otion copensating the spatio-tepoal window. In this way tepoal intensity vaiations within the window can often be avoided. Howeve, this pocedue ay not always be successful [11,3], and does not avoid spatial intensity vaiations within the window. Theefoe, a second pocedue is equied to adaptively select a hoogeneous data subset fo a (otion-copensated) spatio-tepoal window. Only those values which ae not significantly diffeent fo g(i,j,k) should be used in (6). Significantly dissiila values ae consideed outlies which negatively affect the estiation pocess and should theefoe be excluded [3]. We use a ethod known as the ange test (RT) [12] to select a hoogeneous subset fo the obsevations in the spatio-tepoal -6-

8 window. This ethod uses odeed obsevations g( ), and is coputationally efficient in cobination with the OS-FIR estiato. A stict disciination citeion to decide if a paticula obsevation in the spatio-tepoal window is not significantly diffeent fo the cental value g(i,j,k) is testing fo equality. Obviously, due to the pesence of noise in the obsevations, this stict equality equieent has to be elaxed soewhat. In the RT an odeed subset g( s),, g( t ) (with t>s) of obsevations fo the spatio-tepoal window is consideed sufficiently siila if [12]: g( t ) g( s ) t s( ) (12) va[ n ] We appoxiate the vaiance of the obsevation noise afte gaa coection, i.e. n ( i, j, k ), by n Futhe, t s ( ) is the uppe -significance point fo a noalized (Studentized) ange ove t s values [13], whee a ange denotes the diffeence between two odeed values, i.e. g( ) g( ). If a paticula ange passes the test, all odeed obsevation g( ) spanned t s by this ange ae autoatically egaded to be not significantly diffeent. If a ange fails the test (12), a salle subset of the ejected ange should be econsideed. We have ipleented the RT pocedue in a ecusive way. Fist, the ange spanned by g( ) and g(1) is tested, i.e. the entie window R is tested fo hoogeneity. Upon failue, the RT is epeatedly applied on salle anges, until the lagest ange is found that passes the test and that includes the cental value g(i,j,k). 2. Afte applying the RT, Equation (11) has to be solved fo a value of R equal to the nube of obsevations that passed the RT. Although the diensions of the atix W vay depending on the esult of the RT, only R-M+1 unique cases exist. Theefoe, it is attactive to pe-copute and stoe the estiato weights w fo all possible values of R. The esulting oveall stuctue of the adaptive OS-FIR estiato is illustated in Figue

9 5. EXPERIMENTAL RESULTS The ain objective of the expeiental evaluation is twofold. Naely, in the fist place to evaluate the effect of the ode M of the Taylo expansion, and in the second place to evaluate the effect of the ange test. We futhe evaluate the effect of using a otion-copensated window additionally to the ange test. To ake nueical copaisons possible, we have consideed siulated caea noise and siulated gaa coection only. Two diffeent values fo have been used naely 0.35 and To avoid influence of the actual caea noise pesent in ou oiginal sequence, we have added noise to an aount that is uch highe than noally encounteed, naely to an SNR of 10 db. Futhe, ou siulated gaa coection opeates on spatially sapled data, which ay cause ino aliasing effects that ae not pesent in pactice. Afte noise filteing, the ipoveent in SNR is coputed on an invese gaa-coected vesion of the filteed esult. In this way ou nueical esults can be copaed against esults obtained by othe noise filtes. As oiginal we have used the CalTain iage sequence. In the fist expeient, the OS-FIR filte used a fixed spatio-tepoal cube of 3x3x3 pictue eleents, i.e. 27 obsevations. One set of esults was obtained without and a second set with using the ange test. The coluns labeled Fixed and RT in Table 1 suaize these esults, and show that (i) the SNR ipoveent satuates fo M lage than 3, and (ii) the faily siple window adaptation though the ange test geatly ipoves the esults, ainly because the assuption of constancy of f(i,j,k) within the selected window becoes oe eliably. A final conclusion that can be dawn fo this expeient is that disegading the gaa coection (M=1) is definitely not advisable, especially when the ange test is not used. It is wothwhile noting that since the noise is assued Gaussian and the signal is assued to be constant within the spatio-tepoal window, the esults listed unde M=1 ae identical to the ones obtained by a noal linea filte with weights 1/R. In the second expeient, a otion-copensated spatio-tepoal window was used. The pocess of otion estiation is uch oe coputationally intensive than the ange test. We used the ecusive block atching pocedue descibed in [14] to estiate the otion tajectoy. In the coluns labeled as -8-

10 RT+MC in Table 1 we see that the cobination of otion copensation and the ange test leads to the best esults. Howeve, the additional gain obtained by using a otion-copensated window is faily sall copaed to the gain obtained aleady by the uch siple ange test. 6. DISCUSSION In this coespondence we have poposed a new appoach to solving non-linea estiation pobles, exeplified by the poble of filteing gaa-coected noisy video signals. An adaptive OS estiato was foulated of which the weights depend only on the HOOS of the noise, and of which the data window used depends on the ange test. Essential in the deivation is that the non-linea signal odel is appoxiated by a Taylo expansion of ode M. This ode can be chosen feely and does not influence the coputational coplexity of the esulting estiato once the weights of the estiato have been deteined. The filteing esults that we obtained ae nealy independent on the value of, which eans that the poposed filteing appoach is quite successful in handling the signal-dependent noise esulting fo the non-linea odel. -9-

11 REFERENCES [1] G. Cotelazzo, G.A. Mian, and R. Paolai, Statistical chaacteization of ganula caea noise, IEEE Tans. Cicuits and Systes fo Video Technology, vol. CSVT-6, pp , [2] R.P. Kleihost, R.L. Lagendijk, and J. Bieond, An efficient spatio-tepoal filte fo gaacoected video signals, IEEE Int. Conf. on Iage Pocessing, Austin, Texas, [3] R.P. Kleihost, Noise-filteing of iage sequences, Ph.D. Thesis, TU-Delft, Infoation Theoy Goup, The Nethelands, [4] A.C. Bovik, T.S. Huang, and D.C. Munson, A genealization of edian filteing using linea cobinations of ode statistics, IEEE Tans. on Acoustics, Speech and Signal Pocessing, vol. 31, pp , [5] I. Pitas and A.N. Venetsanopoulos, Non-linea ean filtes in iage pocessing, IEEE Tans. on Acoustics, Speech and Signal Pocessing, vol. 34, pp , [6] G.R. Ace, and R.E. Foste, Detail-peseving anked-ode based filtes fo iage pocessing, IEEE Tans. on Acoustics, Speech and Signal Pocessing, vol. 37, pp.83-98, [7] R.P. Kleihost, R.L. Lagendijk, and J. Bieond, Highe-ode ode-statistic filtes fo signal dependent noise, Poc. of the 1995 IEEE Wokshop on Nonlinea Signal and Iage Pocessing, Halkidiki, Geece, pp , [8] A. van de Ziel, Noise, Pentice Hall, New Yok, [9] H.A. David, Ode Statistics, 2nd edition, New Yok, Wiley and Sons, [10] G.R. Ace, Multistage ode statistics filtes fo iage sequence pocessing, IEEE Tans. Signal Pocessing, vol. 39, pp , [11] M.K. Özkan, M.I. Sezan, and A.M. Tekalp, Adaptive otion-copensated filteing of noisy iage sequences, IEEE Tans. Cicuits and Systes fo Video Technology, vol. 3, pp , [12] D.B. Duncan, Multiple anges and ultiple f-tests, Bioetics, vol. 11, pp. 1-42, [13] A.E. Sahan and B.G. Geenbeg (eds), Contibutions to ode statistics, John Wiley and Sons, New Yok,

12 [14] G. de Haan, P.W.A.C Biezen, H. Huijgen, and A.O. Ojo, Tue-otion estiation with 3-D ecusive seach block-atche, IEEE Tans. Cicuits and Systes fo Video Technology, vol. 3, pp ,

13 senso A apetue coection ( ) R V b senso A apetue coection ( ) B RGB/ YUV atix U g senso A apetue coection ( ) G Y contou enhanceent Figue 1: Siplified colo caea odel window size R M=4 M=2 Figue 2: OS-FIR estiato weights fo M=2 and M=4, and fo 7R

14 M decision table size selection filte weights g(i,j,k) odeing stage Range Test OS-FIR filte γ fˆ (i,j,k) Figue 3: Stuctue of the adaptive OS-FIR estiato. -13-

15 Table 1: Aveage ipoveent in SNR (in db) using a fixed spatio-tepoal windows ( Fixed ), using an adaptive window based on the ange test ( RT ), and using an adaptive windows based on otion copensation and the ange test (MC+RT). =0.35 =0.45 Fixed RT RT+MC Fixed RT RT+MC M= M= M= M=