Digital Image Restoration Using Autoregressive Time Series Type Models. Casilla 110-V, Valparaíso, Chile 2 Universidad Católica de Valparaíso,

Transcription

1 Digital Image Restoation Using Autoegessive ime Seies ype Models HÉCOR ALLENDE O. JORGE GALBIAI R. 2 RONNY VALLEJOS A. 3 Univesidad écnica Fedeico Santa Maía, Casilla 0-V, Valpaaíso, Chile 2 Univesidad Católica de Valpaaíso, Casilla 4059, Valpaaíso, Chile 3 Univesidad de Valpaaíso, Casilla 5030, Valpaaíso, Chile Abstact. We conside an non-symmetic half plane autoegessive image, whee the image intensity of a point is a linea combination of the intensitites of the eight neaest points located on one quadant of the coodinate plane, plus a nomal white noise innovations pocess. wo types of contaminations ae consideed. Innovation outlies, whee a faction of innovations ae coupted with a heavy tailed outlie geneation pocess, and additive outlies, whee a faction of obsevations ae coupted. We develop a GM-estimato fo the obust estimation of paametes of a contamined autoegessive image model, based on time seies GM-estimatos intoduced by Denby & Matin (979) applied to the estoation of ada geneated images. Odinay leastsquaes estimatos ae asymptotically efficient with a non-contamined gaussian pocess, like the one consideed hee. M-estimatos behave bette when innovation outlies ae pesent, but ae vey sensitive to additive outlies. A simulation study is caied out, which shows that the GM-estimato intoduced hee has a bette pefomance with an additive outlie contamined image model than M-estimatos and odinay least squaes estimatos. Keywods: GM-estimatos, Image Restoation Robust Estimation, wo-dimensional Autoegessive Models. Intoduction Restoation of an image in the pesence of noise is one of the fundamental poblems in image pocessing. Paametic epesentations of two-dimensional pocesses suitable fo this poblem, have been well studied. Howeve, in these models, the image intensity aay is assumed to be a two-dimensional Gaussian Pocess. hee ae many image estoation methods based on the Gaussian assumption. Fo instance, Chellepa and Kashyap (982) used spatial inteaction models to epesent image intensity aays and estoed images obtained with minimum mean squae his wok was suppoted by FONDECY gant N o 96052

2 eo citeion. Howeve, when the image is contamined with outlies, the estimated paametes obtained fom the Gaussian model do not appea to be appopiate. A moe ealistic assumption fo the image model is a contaminated Gaussian noise. he impotance of the ε-contaminated models has been legitimated by numeous publications about applied woks in the aea of image pocessing and image analysis. See fo instance Kashyap and Eom (988). We develop a estoation method based on a obust image model in this wok. In the poposed method, the image intensity aay is epesented by a causal autoegessive model. A obust paamete estimation algoithm and a data cleaning pocedue is applied to estoe contaminated images. he estoation algoithm based on the obust modelling is tested with seveal simulated images. Ou contibutions ae theefold. We fist develop an algoithm fo the obust estimation of paametes of an image model in which the innovations pocess is a mixtue of a Gaussian and an outlie pocess. It is a GM-estimato. We pove the convegence and confim the convegence via simulation. Next we conside the obust estimation of the paametes of a model whee the image obeying the model is not available, the coupting innovations pocess being a mixtue of a Gaussian pocess and an outlie pocess. We develop an algoithm to ecove the paametes of the model fom a noisy image. he pocedue involves altenate paamete estimation and data cleaning. We povide intuitive easons fo the convegence of the pocedue and confim ou intution by seveal simulations. Finally, we use the above esults to estoe an image coupted by diffeent types of outlies. 2 he additive outlie in nonsymmetic half-plane autoegessive models Conside a nonsymmetic half-plane autoegessive two dimensional model with additive outlies. Assume that the image intensity of an image follows the nonsymmetic half-plane model. Let ( i, be an index fo the coodinate location, and y be the intensity at the coodinate ( i,. Let us define a nonsymmetic half-plane (NSHP) model as follows: Ω : = {( i, : ( i = 0 and j < 0) o ( i < 0 and j is abitay}. (2.) of Let u and v be indexes fo two-dimensional coodinate locations. One impotant popety Ω is that if u Ω and v Ω then ( u + v) Ω. And NSHP autoegessive model can be witten as y( u) = α v y( u + v) + µ + a( u) (2.2) v N

3 whee a (u) ae independent identically distibuted andom vaiables with a symmetic distibution G with mean zeo and scale σ a. he density of G will be denoted by g. he a' s ae called innovations. he neighbohood set N is a subset of the nonsymmetic half-plane Ω. he NSHP autoegessive model (2.2) can be witten in the linea model fom y( u) = α z( u) + a( u) (2.3) whee α is a paamete vecto and z (u) is a vecto which consists of intensities of pixels in the neighbohood set N and unity. he last element of the vecto z (u) is equied to epesent a constant gay level in the image. If N = {(0, ),(,0),(, ),(0, 2),(, 2),( 2,0),( 2, ),( 2, 2)} the NSHP autoegessive model (2.2) can be ewitten as follows: Y = α Z( + a( (2.4) whee [ Z( ] = { j ), i,, i, j ), j 2), i, j 2), i 2,, i 2, j ), i 2, j 2),} he model given in (2.4) is called an eight neighbo causal autoegessive model, and this model is used in ou simulation study. Suppose now that the NSHP autoegessive pocess cannot be pefectly obseved because a small faction ε (in pactice we usually have ε 0. ) of obsevations ae distibuted by an outlie-geneating pocess { ν V }, whee { ν } is one o zeo, with Ρ ( ν ( i, = ) = ε, Ρ( ν ( i, = 0) = ε, and the vaiables V have abitay distibution function H. hus the obsevational model is i =,..., n X = + ν V (2.5) j =,..., m heefoe with pobability ( ε) the NSHP autoegessive pocess Y itself is obseved, and with pobability ε the obsevations X ae coupted by an eo with distibution H. It is well known that the LS estimates ae asymptotically nomal and asymptotically efficient when G is Gaussian and V = 0. Howeve, when the innovations density is non-gaussian

4 (Innovative Outlies), the above estimates ae no longe efficient and heavy-tailed innovation distibutions can esult in lage losses of efficiency. he latte fact suggests that a good altenative to the LS estimate can be the M-estimate as poposed by Hube (98) fo the NSHP autoegessive case (Kashyap and Eom, 988). Howeve, the LS estimate and even the M-estimate ae extemaly sensitive to the pesence of additive outlies (AOs). his fact is epoted by Bustos and Yohai (986) fo one dimensional autoegessive pocesses. In this wok we pesent the esults of a Monte Calos simulation which shows that fo a two dimensional eigth neighbo causal autoegessive model the LS and M- estimates ae moe sensitive to AO-s than in the case of causal autoegessive with innovative outlies. 3 Genealized M-estimates Conside the paamete estimate in the NSHP autoegessive pocess. In the least squaes estimation, we need to minimize the function j 2 [ X α Z( ] (3.) with espect to α, whee Z (u) is a vecto which consists of the obsevations X in the neighbo set N. he idea of a least squaes estimation is to minimize the esiduals. Howeve, if one obsevation is an outlie, then the coesponding esidual is vey lage, and the least squaes estimato is not obust. Similaly, the class of M-estimatos poposed by Kashyap and Eom (988) fo causal autoegessive pocesses, defined by minimizing the function of a finite sample of obsevations X α Z( Q( α, σ ) = ρ + σ mn σ (3.2) j 2 is obust fo innovative outlies, when the function ρ has bounded influence. But the situation is totally diffeent when the contamination model is given by (2.5), that is, when the autoegessive model is distubed by additive outlies. his suggests intoducing a moe geneal class of obust estimatos, known as GM-estimatos, which ae an extension of the M-estimatos, obtained by assigning a weight function to the obsevations of the model. he esiduals X α Z( in a NSHP autoegessive model contamined with additive outlies may be vey lage. he way the GM-estimatos educes this effect is by intoducing smalle weights to lage esiduals. A GMestimato fo the paametes α and σ of model (2.5) is the solution to the poblem of minimizing the non-quadatic function defined by

5 Q( α, σ ) = lt mn j ρ α X Z( + β σ l σ (3.3) whee ρ is a diffeentiable function, convex, symmetic with espect to the oigin, with bounded deivative, and such that ρ ( 0) = 0. he l and t ae the weights coesponding to the espective Z. In ode to obtain consistency of the scale estimate at the nomal model, we conside β = E [χ] φ (see Hube 98). he GM-estimato is obtained by solving the equations j j X α Z( tψ Z = 0 (3.4) lσ l t (, ) (, ) X i j α Z i j χ β = 0 (3.5) lσ ρ( x) whee ψ ( x) =, χ( x) = xψ ( x) ρ( x) and ψ is a bounded and continuous function. hee x ae seveal poposals fo the choice of ψ due to the fact that the obustness of the pocedue and the ate of convegence of the pocedue depends on these functions: the Hube had-limite type, given by ψ ( x) sgn( x).min{ x, c} and uckey's edescending bisquae function given by H = ψ B x[ x / a] ( ξ ) = 0 2, x a, x > a ypical values fo the adjusting constant c in ψ H ange fom.5 and 2.0 and fo a in ψ B ange fom 4.5 and 6. he pincipal types of GM-estimatos ae: i) Mallows type, whee l = and ii) ˆ t ψ ( b / c ) = with ˆ b = p z( C Z, whee b / c C is a obust estimate of C and C is the a pioi unknown covaiance matix fo the NSHP autoegessive pocess, which may be expessed as C (α ). he constuction of C ˆ is descibed by Matin (980). Schweppe type l ψ ( b / c ) = t =. b / c

6 4 Implementation of GM-estimates Assuming that an estimate of C equied to constuct the weights t, is available, then good appoximate solutions of equations (3.4) and (3.5) can be conveniently obtained by using an iteately-weighted-least-squaes (IWLS) techniques simila to that descibed by Matin (980). It may be shown that the estimating equations (3.4) and (3.5) have a unique solution when ψ is stictly monotone. he GM-estimation of the NSHP autoegessive model unde egulaity conditions peseve the popeties of consistency and asymptotic nomality of the unidimensional autoegessive models. But they also have thei computation difficulties, because they involve the minimization of a non quadatic function of multiple paametes. o obtain the GM-estimato of α and σ we use the algoithm known as IWLS, whose convegence is established in Hube (98). IWLS algoithm Let X be the obsevations of the contamined causal autoegessive model defined in (2.5) and let α and i =,n, j =, m, stating values. σ be the initial values, ε a toleance value and weights l, t,. Set k = At the k -th iteation of α obtain the esidual = X α Z(, i =,, n, j =,, m 3. Compute the new value of σ using σˆ =.483Med{ Med } 4. Compute the weights W, fom, l, and t fo the Mallows o the Schweppe type GM-estimatos. W ( k ( t. ψ / lσˆ σˆ t / l = 0 ) if if if if if 0, l 0 = 0, l 0 = l = t = 0 0, l = t = 0, 0, l = t = 0, ψ ( t) = t ψ is bounded whee i =, n, j =, m. Define diagonal element. W as a diagonal matix with W as its [( n )( j + ) + i ] -th

7 fo 2 5. Solve [ Z τ ] W = min τ, the solution given by τˆ = [ Z W Z] Z W X α whee the ows of Z ae the 6. Compute the new value of α, α = α + λτ constant. Z defined in (2.4), and X is the vecto of obsevations. ( k + ) ( ) ˆ k with 0 < λ < 2, an abitay elaxation ( k+ ) 7. Repeat 2 to 6 until the stopping ule: α α = λτ < εσˆ 5 Applications to Image Restoation is eached. Restoation of an Image in the pesence of noise is one of the fundamental poblems in image pocessing. he image degadation pocess can be modeled by the obsevational model (2.5). We assume that the obsevation X is coupted by a contamined pocess which contains a small faction of additive outlies. hee ae many image estoation methods based on the Gaussian noise assumption. Chellapa and Kashyap (982) used a spatial inteaction model to epesent an image intensity aay and estoed images with minimum mean squae citeion. Geman and Geman (984) used the equivalence of Makov andom field and Gibbs distibution and estoed images by a stochastic elaxation method with maximum a posteioi citeion. Wu (985) used a multimensional Kalman filteing appoach and nonsymmetic half plane autoegessive model. Unfotunately, most image estoation methods based on the Gaussian assumption ae not effective to impulse noise. Image estoation is an estimation of oiginal intensity Y fom the obsevation X. Fo a small size image, the oiginal image intensity can be modeled by a causal autoegessive model. If the oiginal image intensity follows a causal autoegessive model, then the oiginal image intensity can be easily estoed by data cleaning with obust paamete estimation. he data cleaning pocedues emoves outlies at each iteation without degading the oiginal signal. he estoation method based on the obust image model has an advantage ove conventional methods such as median filte o α -timmed mean filte. he obust image model based method does not poduce blued images afte estoation. Conventional methods, such as median filte o α -timmed mean filte, eplace evey pixel by its location estimates. Because these methods ae based on the constant intensity assumption, the details of the oiginal image ae significantly blued. his pocedue is descibed in the following algoithm.

8 Restoation Algoithm Based on a Robust Model. Initially, set Y = X. Compute the initial estimate contaminated obsevation X by the least squaes algoithm. 2. Conside the k -th iteation, whee Y and ( k+ estimate ) Y ( ) fom Y ( ) by the following ecusive equation α, σ fom the α ae available. Obtain the updated = Y ˆ α Z = ψ σˆ σ whee ψ is one of the bounded and continuous functions as discussed in the GM-estimation. ( 3. Restoe the image Y k+ ) using Y ( k + ) = α Z + ˆ 4. Obtain estimatos ( k+) α fom the cleaned data ( k+) Y by minimizing the function Q( α, σ ) = mn j l t ρ Y ( k+ ) a Z l σ ( k + ) + β σ his can be computed by the IWLS Algoithm. 5. Repeat Steps 2-4 until the diffeence of estimates between iteation becomes small. Simulation Study A simulation study was conducted to obseve the behaviou of the GM-estimato and compae it with the LS and M estimatos. In each case one hunded images wee geneated using (2.4), with additive contamination geneated by (2.5), with H a lage vaiance σ 2 H nomal distibution. he paamete values wee and σ = a α he following cases wee simulated: = ( 0.2;0.37; 0.6;0.25;0.3;0.24;0.40; 0.6)

9 No contamination 5 % contamination, σ 2 = 0. and 0.5 0% contamination, σ 2 = 0. and 0.5 5%contamination, σ 2 = 0. and 0.5 Each was un thee times, estimated using LS, M and GM, espectively. he mean squae eo of the estimated α paametes is used as a measue of pefomance of the estimatos. he esults ae shown in able. H H H able - Compaison of GM-estimado, M-estimato and least squaes estimato fo NSHP autoegessive model with additive contaminaion. Numbe of uns in fach case is 00. Image size is Pecentage of outlies Outlie Standad Deviation Estimato LS M GM Mean Squae Eo Afte obseving the esults of the simulation study, we conclude that obust estimatos have a bette pefomance than the LS-estimato. he diffeence between the M and GM estimatos is less significant than expected, although the GM is slightly bette in some cases. Refeences [] Allende, H. and Heile, S. Recusive Genealized M-estimates fo Autoegesive Moving Aveage Models. Jounal of ime Seies Analysis, Vol. 3, N o [2] Boente, G.; Faiman, R. and Yoha V. J. Qualitative Robustness fo Geneal Stochastics Pocess, echnical Repot N o 26. Depatment of Statistics. Univesity of Washington. Seattle. WA. (982). [3] Bustos, H.O. Geneal R-Estimates fo contaminated ph-ode autoegesive pocesses: consistency and asymptotic nomality, Z. Wahscheinlich. Vew. Geb. 49, [4] Bustos, H. O. and Yoha V. J. Robust estimates fo ARMA models. J. Anv. Statist. Assoc. 8(393),

10 [5] Denby, J. and Matin, R.D. Robust estimation of the fistode autoegessive Paamete. J. Am. Statist. Assoc. 74(365), 979. [6] Dutte, R. Robuste Regession. Beicht N o 35, Math. Statist. Eidgenoessische echn. Hochschule. Züich 980. [7] Dutte, R. and Hube, P.J. Numeical Methods fo the Nonlinea Robust Regession Poblem, J. Statist. Comput. Simul. Vol. 3,2, [8] Hampel, F.R. A Geneal Qualitative Definition of Robustness, Ann. Math. Statist. 43, 97. [9] Hampel, F.R. Robust Estimation: a Condensed Patial Suvey. Z. Wahscheinlich. Vew Geb [0] Hampel, F.R. Robust Statistics, New Yok: Wiley. 986 [] Hube, P. J. Robust Statistics. New Yok: Wiley, 98. [2] Kashyap, R. L. IEEE ans. on Patten Analysis and Machine Intelligence, Vol PAMI [3] Kashyap, R. L. and Eom, K. B. Robust Image echniques with an Image Restoation Application IEEE ans. on Acoustics, Speech, and Signal Pocessing, Vol. 36, N o 8, pp , Aug [4] Kleine, B.; Matin, R.D. and homson, D.J. Robust Estimation of Powe Specta J. Roy. Soc., Seies B, Vol. 4, N o 3, pp , 979. [5] echnical Repot C.L. Mallows On some topics in obustness. echnical Memoandum, Bell Laboatoies, Muay Hill. NJ [6] Matin, R.D. and Zeh, J.E. Genealized M-Estimates fo Autoegesions, Incluiding Small- Sample Efficency Robustness. echn. Rep. N o 24, Depatment of Electical Engineeing, Univesity of Washington, Seattle [7] Matin, R.D. Robust estimation fo Autoegessive Models, In: Billinge D.R. and G.,C. iao (Eds.): Diections in ime Seies. Inst. Math. Statist. Publications, Haywood, C.A., 980. [8] Matin, R.D. and Yoha V. J. Robustness in time seies and estimatin ARMA models. Handbook of Statistics Vol. 5 (eds. E. J. Hanann, P.R. Kishnaiah and M.M. Rao), 985. [9] Rousseuw, P. J. Least median of Squaes egession. J.Am. Assoc. 79, 984. [20] Zeh, J.E. Efficiency Robustness of Genealized M-Estimates fo Autoegession and thei use in detemining outlies ype. Ph.D. hesis. Univesity of Washington. Seattle [2] Chellepa, R.and Kashyap, R.L. Digital Image Restoation Using Spatial Inteaction Models. IEEE ans. on Acoustics, Speech and Signal Pocessing, Vol 30 N o 3, pp , June 982. [22] Geman, S. and Geman, D. Stochastic elaxation, Gibbs distibutions, and Bayesian estoation of images. IEEE ans. Patten Anal. Machine Intell. Vol 6 pp Nov. 984.

11 [23] Wu, Z. Multidimensional state-space model Kalman filteing with application to image estoation. IEEE ans. Acoust. Speech. Signal Pocessing Vol 33 pp Oct. 985.