Some Filtering Techniques for Digital Image Processing

Transcription

1 Some Filtering Techniques for Digital Image Processing by Chi-M ing Leung l.l.sc. McMaster University, 1975 M. A. University of British Columbia, 1971 B.Sc. T h e Chinese U niversity of Hcng Kong, 1969 A Dissertation Subm itted in Partial Fulfillment of the Requirem ents for the Degree of DO CTOR OF PHILOSOPHY in. the Department of Electrical and Computer Engineering We accept this thesis as conforming to the required standard Dr. W -S T.ii Srmp.rvisnr fo ept. of Elec. & Com p. Eng.) Dr. P. AV^hoklis, D epartm ental Member (Dept, of Elec. & Comp. Eng.) Dr. R. L_KiUin. D eoartm ehtal Member (Dept, of Elec. & Comp. Eng.) Dr. A. R. Sourour, O utside M ember (D ept, of M athem atics) Dr. P. van den D riessche, Orrtsidh M ember (D ept, of M athem atics) < Dr. F.HT'T'eet, ExternaT Exam iner (Forestry Canada) C hi-m ing Leung, 1994 University of Victoria All rights reserved. Dissertation may not be reproduced in whole or in part, by photocopying or other m eans, without the permission of the author.

2 I.I Supervisor: Dr. W.-S. Lu. Abstract This thesis is primarily concerned with the theory and implen:cntation of scvna.i digital filtering methods as applied to restoration and edge detect.ion of noise contaminated images that are further degraded due to either linea.r rnotio11 of t;he obj< d. or defocusing of the camera. It is known tha.t restoring a. severely blurred but noisefree image is relatively easy, but restoring a. blurred pidure that; ha.s been con tq,t;( d by noise is not. This is especially the case when the noise is widc-ba.11d, since conventional linear filtering techniques using lowpa.ss, highpa.ss, bandpass, or their combinations will in general not work. A fundamental question that; a.rises from the above observation is is there a filtering approach to this image restora.tio11 problcrn that is significantly better than conventional techniques ment;ioued a.bovc 1 with the filtering mechanism remaining linear? This thesis presents a. systematic study of this question and gives a possible answer by proposing a class of linear, recursive regularization filters (R-filters). These R-fiJters extend the one-dimensional recursive R-filters proposed by M. Unser and his co-workers in 1991 for noise removal!;~ the two-dimensional case. The R-filters derived ar:c image independent;, therefore they can be designed off-line and stored in the computer. Furthermore, the convent;ional single-parameter regularization methods are extended to a multiplc-para.rnctcr setting, rendering a better balance of fidelity and smoothness of restored images. In

3 Ill addition, other image processing tasks such e:.s edge detection and enhancement can a.lso be performed within this 11-filter framework. Filtering techniques other than R-filters may also be useful in various image processing tasks. In particular, a modified Wiener filter for the restoration of blurred images is presented, showi::i.v; how low-order one-dimensional and two-dimegsional linear finite impulse response filters can be used for detecting edges in noisy images. Conventional smoothing filters always tend to blur the images, so for noise rernova.i tasks, the filter should have the ability to preserve features in an image while reducing the noise. Two approaches are taken here. In the first, the Lee filter is extended to two nonlinear filters utilizing local statistics of the degraded image. The nonlinear filters so designed have the advantage that they reduce noise without blurring the details of the image. The second apprnach proposed for noise removal is a two-step regularization algorithm. In the first step of the algorithm, a smoothing filter is applied to reduce the noise level of the image. Of course, the sharp features that; occur in the image are then unavoidably blurred. A critical observation ma.de at this point is that the blurring mechanism is precisely known. In the second step of the algorithm, an R-filtcr is applied to restore the image tr~ating the smoothing filter i11 the first step as a degradation operator. As is expected, this two-step approach permits smoothing without introducing undesirable blurring. Finally, a blur identification approach is proposed, that makes use of the distri-

4 i \' bution of relative minima. of the power spectra. of subimag;< s f.ha.t n.rl',)bl.a.ined fr,)111 the given degraded image. This algorithm is efficient. for the ide11t.ifica.t.io11 of t.hc linear motion blur, defocused blur, and exponential blur. As the second meu1od for blur identification, a two-phase blind restoration algorithm is proposed. Jn the first phase of the algorithm, the blur function is identified by the prccedin~ blur identification approach. The power spectra of the f-ubimagcs are then estimated. In the second phase, Wiener filtering is employed to obtain a restored iniagc using the estimated power spectra obt;i.ined in phase one

5 * Dr. W.-S. Lu, Supervisor (D ept, of Elec. & Comp. Eng.) Dr. P. A^hthoklis, D epartm ental M em ber (D ept, of Elec. & Com p. Eng.) * Dr. R. L. Kirlin, D epartm ental M ember (D ept, of Elec. & Com p. Eng.) Dr. A. R. Sourour, O utside M ember (D ept, of M athem atics) Dr. P. van den Driessche, O utside M em ber (D ept, of M athem atics) v Dr. F. G. P eet, External Exam iner (Forestry Canada)

6 C ontents A bstract ii Contents vi List of Tables xv List of Figures xxiv List o f A bbreviations xxv A cknow ledgem ents xxvii D ed ication xxviii 1 Introduction D igital Image P r o c e s s in g... 1

7 CONTENTS vii 1.2 Problems E n c o u n te r e d Image Restoration as an Ill-Posed P r o b le m Edge D e t e c t io n Identification of Image M o d e l C o n tr ib u tio n s Organization of the T h e s is One-Dim ensional and Two-Dimensional R-Filters One-Dim ensional R -F ilte r s L I Introduction The M ethod of Unser, Aldroubi, and E d e n Generalized 1-D R - F i lt e r s A Design I s s u e Recursive R egularization Filters w ith A pproxim ate Linear P hase Realization Com putational Comparison of Spatial and Frequency Dom ain Im p lem en ta tio n A pplication to the R estoration of M otion Blurred Im ages.. 36

8 CONTENTS C o n c lu sio n s Two-Dim ensional R -F ilte r s Introduction D R - F ilt e r s An Iterative Im plem entation of 2-D R-Filters An Approxim ate Factorization for the Implementation of 2-D R - f ilt e r s Experim ental R e s u lts C o n c lu s io n s... 3 Optim al D eterm ination of Regularization Parameters and the S tabilizing O perator 3.1 An L-Curve Approach to Optim al Determ ination of Regularization Param eters In tro d u ctio n More About Tikhonov R egularization Determ ination of the Regularization Parameters via the L- Curve A p proach...

9 CONTENTS ix An Adaptive Restoration M ethod for Linear Motion Blurred I m a g e s C o n c lu s io n s Other O ptim al Approaches to the D eterm ination of the Regularization P a r a m ete rs in tr o d u ctio n O ptim ization Form ulations The Constrained Least-Squares M ethod The Generalized Cross-Validation M e t h o d T he MSE and ISNR M e t h o d s Experim ental R e s u lts C o n c lu s io n s A Com m ent on the Stabilizing Operator Generated by D iscrete Laplacian O p e r a to r Sim ultaneous O ptim al Determ ination of the Stabilizing Operator and Regularization P a r a m e te r s A M ultiple-param eter Generalization of the Tikhonov Regularization M ethod In tro d u ctio n... 90

10 CONTENTS x A G eneralization of the Tikhonov Regularization Met hod Choice of M ultiple Regularization P a r a m e te r s Experim ental r e s u l t s M ultiple-parameter 2-D R-.fi..lters C o n c lu s io n s A M odified W iener Filter for the Restoration of Blurred Images In tro d u ctio n W iener Filtering and Regularization M e t h o d s A M odified W iener F ilte r Experim ental R e s u lts C o n c lu s io n s N oise Removal Introduction Conventional Sm oothing F ilte r in g Average F ilt e r in g Gaussian Filtering Noise Removal by R -filte r s

11 CONTENTS xi 5.4 Noise Removal by a Two-Step Regularization Approach Nonlinear Filters for Noise R e m o v a l Noise Removal by Lee F i l t e r s T Noise Removal by Quadratic Filters Noise Removal by Cubic F i lt e r s Simulation R e s u l t s Performance Evaluation of the Smoothing F i l t e r s Deblurring W ith a Pre-Smoothing S t e p C o n c lu s io n s D etection of Edges in N oisy Images by 1-D and 2-D Linear FIR D igital Filters In tro d u ctio n D First-Order FIR Edge Detectors D First-Order FIR Edge Detectors D Second-Order FIR Edge D e t e c t o r s C o n c lu s io n s R estoration o f Im ages w ith Unknown D egradation M echanism 161

12 CONTENTS xii 7.1 In tro d u ctio n A Posteriori Blur Identification in the Spectral D o m a in Linear M otion B l u r Defocusing B lu r Blur with Circularly f ym m etric P S F s A B lind Restoration A lg o r it h m C o n c lu s io n s C onclusions and Future Research Considerations C o n c lu s io n s Suggestions for Future R e se a r c h B ibliography 180

13 xm List o f Tables 2.1 Number of operations for obtaining fk\ through base-2 D FT algorithm. F(C) represents the D F T of C Num ber of operations for obtaining fk\ through spatial R-filtering approach. The blurring distance is L sampling units Q uantitative comparison between R-filtering and W iener filtering of the degraded im age of Fig The filter that produces smaller MSE and NM SE, or larger ISNR is considered to be a better one Q uantitative comparison between R-filtering and W iener filtering of the degraded im age Lena on Fig The filter that produces smaller M SE and NM SE, or larger ISNR is considered to be a better one Q u an titative com parison betw een the R-filter and W iener filter (related im ages are shown in Fig. 2.5)...55

14 LIST OF TABLES xiv 2.6 Q u an titative comparison betw een the R-filter and W iener filter (related images are shown in Fig. 2.6) R estoration results for exponentially blurred Airplane im age with 30 db noise Restoration results for linear m otion blurred Lena image with 20 db noise R estoration results for defocused (r = 7) Lena, im age with 28 db noise Q u an titative com parison between single-param eter and m ulti pie-param eter regularized im age restoration. The sample im age is a linear motion blur degraded Lena im age Q u an titative com parison betw een single-param eter and m ultiple-param eter regularized im age restoration. The sample im age is a defocused and noise-contam inated Lena im age Q u an titative com parison betw een single-param eter arid m ultiple-param eter regularized im age restoration. The sample image is a defocused and noise-contam inated Text im age Q uantitative comparison between the Wiener filter and modified Wiener filters. Related images are shown in Fig

15 LIST OF TABLES xv 4.2 Q uantitative comparison between the Wiener filter and m odified Wiener filters. Related images are shown in Fig Q uantitative comparison of sm oothing capacity of different filtering techniques. (The sample im age is Peppers.) Q uantitative comparison of sm oothing capacity of different filtering techniques. (Corresponding sm oothed im ages are shown in Fig. 5.8.) Q uantitative comparison of sm oothing capacity of different filtering techniques. (Corresponding sm oothed im ages are shown in Fig. 5.9.) 145

16 x v i List o f Figures 1.1 An im age formation m odel A m plitude response of S\(z) with A = 0.2 and L = R-filtered and W iener filtered im ages, (a) Original im age, (b) Linearly blurred im age w ith L 14 units and pseudo white Gaussian noise at SNR = 30 db level, (c) Restored image by Wiener filter, (d) Restored im age by 1-D R-filter with A = R-filtered and W iener filtered im ages, (a) T he original im age Lena. (b) Linearly blurred im age with L = 14 units and pseudo white Gaussian noise at SN R = 25 db level, (c) Restored image by Wiener filter. (d) R estored im age by 1-D R-filter with A = (a) A m plitude response of the discrete Laplacian operator (2.54). (b) The corresponding contour plot of (a)...45

17 LIST OF FIGURES xvii 2.5 Comparison between R-filtered and Wiener filtered images, (a) The Lena image, (b) Defocusing blurred Lena image with r = 7 units and pseudo w hite Gaussian noise at SNR = 28 db level, (c) Restored im age by W iener filter, (d) Restored im age by R-filter w ith A = Comparison between R-filtered and Wiener filtered images, (a) The im age Text, (b) Defocusing blurred Text im age with r = 5 units and pseudo white Gaussian noise at SNR = 33 db level, (c) Restored image by Wiener filter, (d) Restored image by R-filter with A = Im age restoration by R-filtering. (a) Linearly blurred Lena im age with L = 14 units and pseudo white Gaussian noise at SN R = 25 db level, (b) Restored image by R-filtering with A = Image restoration by approximate-factorization R-filtering. (a) The im age Airplane, (b) Defocused Airplane im age w ith r 3 units and pseudo w hite Gaussian noise at SN R = 30 db level, (c) Restored im age by approxim ate-factorization R-filtering w ith A = An L-curve (a) Linear m otion blurred Lena image with L = 14 units and pseudo white Gaussian noise at SN R = 30 db level, (b) Restored Lena image with A = obtained by L-curve approach, ISN R =

18 LIST OF FIGURES 3.3 (a) Defocused Lena im age with r = 7 units and pseudo white Gaussian noise at SN R = 28 dti level, (b) Restored Lena image with A = obtained by L-curve approach, ISNR = (a) T h e Tiffany im age w ith a synthetic line segm ent, (b) Linear m otion blurred Tiffany image with L = 9 units and pseudo white G aussian noise at SN R = 40 db level, (c) Restored Tiffany im age using adaptive approach, (d) Restored Tiffany im age using nonadaptive approach D eterm ination of the regularization parameter, (a) Original im age Airplane, (b) Exponential blur (r = 5 and a2 = 1 ) with noise at SN R = 30 db level, (c) Restored image by CLS m ethod (A = 0.012). The im age is overly sm oothed, (d) Restored im age by GCV method (A = ). (e) Restored image by MSE or ISN R m ethod (A = ). 3.6 D eterm ination of the regularization param eter, (a) T h e Lena im age. (b) Linear m otion blur (L = 1 1 ) with noise at SN R = 20 db level, (c) Restored image by CLS m ethod (A = 0.088). The image is oversm oothed, (d) Restored image by GCV m ethod (A = 0.02). (e) Restored im age by MSE or ISNR method (A = 0.025)...

19 LIST OF FIGURES xix 3.7 (a) Am plitude response of the discrete Laplacian operator L'. (b) The corresponding contour plot of (a) (a) Defocused (r = 7) Lena image with noise added at SNR - 28 db level. R estored im age by (b) CLS m ethod, (c) M SE m ethod and (d) ISNR m ethod M ultiple-parameter regularized im age restoration, (a) Linearly blurred Lena im age with L = 14 units and pseudo white Gaussian noise at SNR = 30 db level, (b) Restored image by m ultiple-parameter restoration M ultiple-param eter regularized im age restoration, (a) Defocused Lena im age with r = 7 and contam inated by noise w ith SNR = 28 db. (b) Restored im age by m ultiple-parameter restoration M ultiple-param eter regularized image restoration, (a) Defocused Text image with r = 5 and contam inated by noise w ith SNR = 33 db. (b) Restored im age by m ultiple-parameter restoration The m agnitude response /T(cji, oj2) of (a) linear m otion blur w ith 8 units; (b) exponential blur; (c) defocusing w ith radius o f COC 7 units. I l l

20 LIST OF FIGURES xx 4.2 Com parison of m odified W iener filtered and W iener filtered im ages. (a) T he Lena image, (b) Noise-contam inated linear motion blurred Lena im age with J, = 14 and SNR = 30 db. (c) Restored Lena image by m odified W iener filter VV3. (d) Restored Lena image by modified W iener filter W2. (e) Rest; <ed Lena image by conventional Wiener filter W \ ; 4.3 Com parison of m odified W iener filtered and W iener filtered im ages. (a) T he im age Text, (b) Noise-contaminated defocused Text image with r 3 and SNR = 15 db. (c) Restored Lena image by modified W iener filter W3. (d) Restored Lena im age by modified Wiener filter W2. (e) Restored Lena im age by conventional Wiener filter W\ Average filtering, (a) T he im age Peppers, (b) Noise-contaminated Peppers im age with SNR = 10 db. Restored image with mask of size (c) 3 x 3; (d) 5 x 5; (e) 7 x 7; (f) 9 x Frequence response of th e 1 x 3 average filter (the sam pling frequency f s is set equal to 1) Gaussian filtering. Restored im age of the noise-contaminated image Peppers (Fig. 5.1(b)) using a 7 x 7 Gaussian filter with <Jr equal to (a) 0.5 pixels; (b) 1 pixel; (c) 2 pixels and (d) 3.5 pixels

21 LIST OF FIGURES xxi 5.4 Suppression of noise by R-filtering. (a) 1-D R-filtering. (b) 2-D R- filtering Two-step regularization smoothing, (a) Restored image in which an average filter is used in the pre-sm oothing step, (b) R estored im age in which a Gaussian filter is used in the pre-sm oothing step Suppression of noise using nonlinear filters, (a) Lee filter, (b) Quadratic filter, (c) Cubic filter Suppression of noise using nonlinear filters, (a) T he original im age Robot, (b) Noise-contam inated Robot with SNR = 9 db. (c) Lee filter, (d) Quadratic filter, (e) Cubic filter Com parison of sm oothing filters (this page and the next page), (a) The Lena image, (b) Noise-contam inated Lena im age, SNR = 7 db. (c) Average filtering, (d) Gaussian filtering, (e) W iener filtering, (f) M edian filtering, (g) Lee filtering, (h) 2-step average filtering, (i) 2- step G aussian filtering, (j) 2 -step W iener filtering, (k) 1 -D R-filtering. (1) 2 -D R-filtering. (m) Q uadratic filtering, (n) Cubic filtering

22 LIST OF FIGURES xxii 5.9 Com parison of sm oothing filters (this page and the next page), (a) The im age Airplane, (b) Noise-contam inated Airplane im age, SN R = 5 db. (c) Average filtering, (d) Gaussian filtering, (e) W iener filtering. (f) M edian filtering, (g) Lee filtering, (h) 2-step average filtering. (i) 2 -step Gaussian filtering, (j) 2-step Wiener filtering, (k) 1-D R-filtering. (1) 2 -D R-filtering. (m ) Quadratic filtering, (n) Cubic filtering Com parison of deblurring w ith and w ithout a pre-sm oothing step, (a.) The im age Airplane, (b) Linearly blurred Airplane image with L 7 and S N R = 15 db. (c) R estored Airplane im age w ithout sm oothing. (d) R estored Airplane w ith a Gaussian pre-sm oothing filtering step Com parison of edge m aps obtained by Sobel operator and the proposed 1-D first-order edge detector, (a) Original image, (b) Noise- contam inated im age with pseudo white Gaussian noise at SN R = 10 db level, (c) Edge map by Sobel operator, (d) Edge map by the proposed 1-D first-order edge detector (6.10) Edge m ap by th e proposed 2-D first-order FIR edge detector (6.14). (a) Original image, (b) Edge m ap...153

23 LIST OF FIGURES xxiii 6.3 Comparison of zero-crossings obtained by LoG and the proposed 2 - D second-order edge detector designed by combined SVD and BA m ethod, (a) Original image, (b) Noise-contam inated image with pseudo white Gaussian noise at SNR = 15 db level, (c) Zero-crossings by LoG. (d) Zero-crossings by the proposed 2-D second-order edge detector (6.14) designed by combined SVD and BA m ethod Zero-crossings by a proposed 2-D second-order edge detector designed by the BFGS m ethod, (a) Original image, (b) Noise-contam inated image w ith pseudo white Gaussian noise at SNR = 12 db level, (c) Zero-crossings by the proposed 2-D second-order edge detector (6.17) designed by BFGS m ethod (a) Original im age Text, (b) Linearly blurred Text with L = 8. (c) Noisy and linearly blurred Text with L = 8 and SNR = 30 db. (d) Graph of the noise free log(vg(k, 129)). (e) Graph of the noise contam inated \og(vg(k, 129)) (a) Defocused im age Lena (r = 5) and noise added at SNR = 35 db. (b) Graph of lo g ^ ^ A :, 6 5 ) ) Restored im age by two-phase blind restoration m ethod, (a) Linearly degraded Lena, (b) Restored Lena...171

24 LIST OF FIGURES xx iv 7.4 Restored im age by two-phase blind restoration m ethod, (a.) Exponentially degraded Lena, (b) Restored Lena

25 XXV List of Abbreviations [{ fi I i,>-r Regularization filter 1-D O ne-dim ensional 2-D Tw o-dim ensional * Convolution operator (unless otherw ise stated) A 1 PSF D F T PO C S LoG AR, A R M A FIR, IIR The transpose of m atrix A Point spread function D iscrete Fourier transform Projection onto convex sets Laplacian of Gaussian Autoregressive Autoregressive m oving-average F inite im pulse response Infinite im pulse response Lz Argument of the com plex number z z ej0, (i.e., Lz 0) M SE NM SE SN R ISN R SV D Mean square error Normalized m ean square error Signal to noise ratio Im provem ent signal to noise ratio Singular-value decom position

26 B A COC MMSE LMMSE CLS BF G S G C V CCD Balanced approxim ation Circle of confusion M inim um mean square estim ate Linear m inim um mean square estim ate Constrained least-squares Broyden-Fletcher-G oldfarb-shannon G eneralized cross-validation Charged coupled device

27 XXV11 A cknow ledgem ents The author would like to thank his supervisor, Professor W.-S. Lu of th e D epartment, of Electrical and Com puter Engineering, for his encouragement, patience and advice during the course of this research, and for his help in the preparation of this dissertation. Financial assistance provided by Professor W.-S. Lu s research grants from NSERC and IRIS is also gratefully acknowledged.

28 my family

29 1 C hapter 1 Introduction 1.1 D igital Im age P rocessing By an im age we m ean a two-dim ensional (2-D) distribution of light intensity. Im age processing is the treatm ent of an image to produce a second im age for a desired purpose. This is often performed by using either graphic arts or computer programs. A digital im age can be thought of as a m atrix of light intensity represented by a finite number of bits. The elem ents of the m atrix are called pictu re elem ents or pixels, each representing the intensity of a sm all area of the corresponding coordinates in the digital im age. Images are usually digitized uniformly. It is a com m on practice that the sizes of resulting digital images are taken to be at least 256 x 256 pixels, thus a large am ount of data is involved in a digital image. Fortunately, the m atrix representation of an im age perm its the use of a digital com puter as well as existing m atrix theory to develop various algorithm s useful in processing the im age.

30 D igital im age processing can be divided roughly into two subcategories, namely su b jective digital im age processing and qu an titative digital im age processing. Subjective digital im age processing is designed to improve human visual-perception of an image. This involves a close exam ination of the human visual system, especially in the adaptation and discrimination aspects. The principal objective of subjective digital im age processing is to process a given image so that the analyst can detect the inform ation of interest in the resultant image. It is mainly a tr.ial-and-er.ror process. The m ain techniques employed in subjective digital im age processing are m odification techniques (e.g., histogram equalization), sm oothing techniques (e.g., m edian filtering and lowpass filtering) and sharpening techniques (e.g., differentiation and highpass filtering). These techniques are often called im age enhancem ent techniques, and in this regard im age enhancement is another name for subjective digital im age processing. Image enhancem ent techniques have been well-developed [1, 2, 3, 4] and can be readily implem ented on computerized image display system s. Q uantitative digital image processing techniques are based on mathematical, m odels; th e resultant im ages are produced w ithout analyst intervention. An im portant exam ple of quantitative digital im age processing is im age restoration, m inim ization o f known or unknown degradations and noise in an image. In this thesis, we are m ainly concerned with analytic techniques and m athem atical tools which are related to im age restoration, noise removal, edge detection and identification.

31 3 n {*, y) f i x,y)- * 1 J Linear blur Noise Figure 1.1: An im age form ation m odel. 1.2 P roblem s E ncountered Im age R estoration as an 111-Posed P roblem Images are produced to record or display useful information about a phenom enon of interest. Q uite often, the process ol image formation and recording is degraded. Due to the im perfection of im aging system s, recorded images are usually degraded versions of the original scenes. The basic goal of image restoration is to reduce the degradations in a recorded im ags so that the resulting im age will best approxim ate the original scene subject to som e criteria. This requires the knowledge of the im aging system used and the degradation m echanism involved. The image degradation will be modeled in this thesis by a linear blur and an additive noise process. The additive-noise assum ption in the im age formation m odel is justified by the nature of m ost im age sensors. Moreover, it is unanimous in practice to m odel the noise process as w hite. As shown in Fig. 1.1, th e degradation m odel is given by the expression OO / foo / h (x,y,a,p )f(a,l3 )d a d /3 + n(x,y) (1.1) * o o */ OO

32 where / ( a, ft) is the object (the original signal), g(x,y) is the image recorded (the degraded signal), n(x,y) is the additive noise function, and h(x,y\a,ft) is called the two dimensional (2-D) im pulse response or point spread function (PSF) of the system. A typical im age restoration problem is to find an estim ate of f(x, y) given the degraded im age g(x,y), the PSF /i(x, y\ a, ft), and in many cases the statistical properties of the additive noise process n(x,y). If the im pulse response in (1.1) is shift-invariant (i.e., stationary), then h(x,y,a, ft) = h(x a, y ft). It follows that (1.1) becomes the convolution integral OO / t OO / h(x - a, y - ft)f{a, ft)dadft + n (x,?/) (1.2) -OO J OO which m ay be regarded as a Fredholm integral equation of the first kind [1, 5, ()]. Three observations concerning equation. (1.2) im m ediately follow. First of all, a solution f{x,y) may not exist. Indeed, this will be the case when g(x,y) is too rough and h is too sm ooth or when h(x,y) and f(x,y) are orthogonal. Second, the solution (1.2) m ay not be unique. This can be demonstrated by means of the Riem ann-lebesque lem m a [7]. If h(x,y) is an integrable function, then it can be show n that [1] fb fb ^lim Jhn J J h(x a, y ft) sin (//a) sin ( ft)dadft = 0 (1.3) It follows that a sinusoid of high frequency can be added to the object f{x,y) and the resulting sum is identical to the degraded image </(x, y). T he last concern, which is always a serious problem in many practical applications, is that a solution / ( x, y) does not depend continuously upon the recorded data g(x,y). This means that sm all perturbations in g(x,y) might lead to large changes in f(x,y). This can also be seen in (1.3). B ecause of these reasons, the problem o f solving equation (1.2) is

33 5 called an ill-posed problem [8, pp. 9, 39, 49] a common usage in m any diverse fields. Throughout the thesis, it will be assumed that the images / ( x, y), g(x, y), and the PSF h(x, y) are of finite extent, and that they all have support in the first-quadrant. Sam pling the equation (1.2) for a discrete approxim ation, we have <?(M) = + (1-4) * i I We assum e that the size of the original image m atrix [f(k, /)] is P ix Q, and the size of the PSF m atrix / / p s f = [^(^, 01 J x ^ Thus the degraded im age m atrix [g(k, /)] and the noise m atrix [n(fc, /)] are of size (P + J 1) x (Q + K 1). Following H unt s approach [9], we express the convolution format of the discrete im age m odel (1.4) in a vector-m atrix form by the following steps. Step 1: Choose M > P T J 1 and N > Q + K 1. Form new extended m atrices [/e], [/*c], [(Je\ and [ne] of size M x N as follows: /«(*,/) = f(k,0 0 0<k<P-l, 0<1<Q-1 otherw ise (1.5) he(k,l) = h(k, I) 0 0 < k < J - l,0 < l< K - l otherw ise (1.6) 9e(k,l) g(k,i) 0 0<k<P + J-l,0<l<Q + K - l otherw ise (1.7) n (k,l) fc = n(k,l) 0<k<P + J-l,0<l<Q + K - l (1.8) 0 otherw ise

34 (> Step 2: Construct column vectors fe, g,;, n,., of length M N by lexicographically ordering the m atrices [ /e], [ge], and [rac], respectively. In this construction, the first row of a m atrix becom es'the first segm ent of I,he corresponding vector, the second row th e second segm ent, etc. For exam ple, M 0, 0 ) /e(0, 1) /e(0,/v- 1) (1.9) /e (l,0 ) h (M - 1, N - 1) The vectors ge and ne are constructed in a similar manner. Step 3: Construct the M N x M N m atrix H as Ho H m -I Hm -2 //, n \ Ho I lx i / / = h 2 H i Ho t h ( ) H m - 1 H m 2 H m - 3 //o

35 7 where each //, is formed from the *th row of the extended PSF m atrix [he], i.e., he(i, 0) he( i,n - 1) he( i,n - 2) he(i,l) he(i, 1) he(i, 0) he( i, N - 1) he(r,2) II, = he(i, 2) ^e(i, 1) /»e(*,0) he(i,3) he( i, N - 1) he(i,n -2 ) he(i,n - 3) Ae(*,0) Step 4: Finally, write (1.4) in the vector-m atrix form ge = Hie + ne ( 1.12) Throughout this thesis, M and N will be chosen such that M J and N K, and for convenience, they are chosen to be equal. Thus, with uniform sam pling on an N x TV 2-D lattice, equation (1.2) can be reduced to a discrete approxim ation of a convolution form sim ilar to (1.4), and then expressed in th e m atrix form g = H f + n (1.13) where /, g and n are lexicographically ordered vectors of size N 2 x 1, and H is the linear blurring (distortion) operator of size N2 x N2. N ote that m atrix H in (1.10) is block circulant with circulant blocks. For sim plicity, block circulant m a trix will be used instead of block circulant m atrix with circulant blocks in the rest of this thesis. For a typical im age restoration task, N is a large integer ranging from 256 to 1,024. C onsequently, equation (1.13) represents a linear system of equations of

36 8 very large size, and careful treatm ent must be given in solving such a system for /. Hunt [9] introduced m ethods for handling such large matrices. In this thesis, the term image form ation m odel always refers to (1.13) as we only deal with digital im ages. Tikhonov [10, 11], Tikhonov and Arsenin [12] proposed an approximate solution to (1.13) by m inim izing th e functional Q(f) = \\U f-9\\2 + M\Cf\\2 (1.14) where A > 0, C is the m atrix which is constructed in the sam e way as matrix M from a given operator C s designed to damp out the high frequencies of /, and denotes the 2-norm. T he term C,/ 2 can be obtained by convolving operator C's with the m atrix [f(k, /)] and sum m ing up the squares of the results at each point (fc, /). The parameter A, in a certain sense, is a measure of tradeoff between sm oothness (for large A) and fidelity (for small A) of the approximate solution /. The approxim ate solution / thus obtained is always stable in the sense that small changes in g will only cause sm all changes in /. M atrix C is therefore called the stabilizing operator (operator C s is also called the stabilizing operator by many researchers). In order to avoid the am biguity, we will call C s the generating stabilizing operator in this thesis. For a given A, the fx that provides the m inimum of the functional (1.14) yields fx = (Ht H + \C TC)~lH Tg (1.15) This fx is considered as an approximate solution of (1.13). The above method and other similar m ethods have been studied extensively in the last three decades and are often referred to as regularization m ethods: m ethods for finding a stable approxim ate solu tion of an ill-posed problem. T he param eter A is therefore called

37 9 the regularization param eter. Regularization theory provides a formal basis for the developm ent of acceptable solutions of ill-posed problems. Phillips [13], Tikhonov [10, 11] and M iller [14] introduced the general concept of a regularization operator in determ ining regularized solutions for ill-posed problems in the early 1960 s. Hunt [9] was one of the first to apply the regularization theory to image restoration. In [9], he also developed a com putationally efficient technique for image restoration based on the 2-D discrete Fourier transform (D F T ), a m ethod which is still used today by many researchers in this field. Note that there is an inconsistent usage of the term s of regularization and constrained least-squares (CLS) m ethods. Regularization m ethods provide estim ates of the im age problem (1.13) by im posing a set of constraints on the problem such as the functional (1.14). A good estim ate of (1.13) depends on an appropriate choice of the corresponding regularization parameter A. There are several m ethods of finding A, Some of these are considered in Chapter 3. One m ethod of finding A is known as the CLS m ethod [9]. However, the term CLS m ethod has been used to describe a synonym of regularization as a whole, i.e., finding the solution and finding an optim al A through the Lagrange m ultiplier m ethod. / For exam ple, the CLS m ethod proposed by Hunt is in fact a special application of the idea of the Tikhonov regularization. Another exam ple is K atsaggelos paper [19] in which the Tikhonov m ethod and the CLS m ethod are equated. Early developm ents in im age restoration are discussed in detail in references [l]-[4]. A fairly com plete treatm ent of regularization theory as applied to im age restoration can be found in [15, 16, 17]. During the last decade iterative restoration algorithm s including basic iterative algorithms, m ethod of projection onto convex sets (PO C S), and regularized constrained iterative algorithms have been studied exten sively [18]-[23], Unser et. al. [24] presented a frequency-dom ain version o f th e

38 10 regularization theory and applied a class of one-dim ensional (1-D) recursive regularization filters (R-filters) to som e noise removal problems. Recently this approach has been extended to a broader class of 1-D and 2-D R-filters that can be used for im age restoration and other im age processing tasks [25, 26] E dge D etectio n Edges in an im age correspond to intensity changes in the physical properties of surfaces such as illum ination, reflectance and geometry. Edge detection is mainly a process that detects and localizes light-intensity changes. It is an important topic not only in im age processing, but also in pattern recognition, computer vision, and robotics. A variety of edge detection schemes have been proposed in the past two decades, and they work w ith various degrees of success for different images [27, 28, 29, 30, 31]. T he edges in an im age can be roughly modeled by step functions, ramp functions or roof functions. Numerical differentiation is an obvious tool in detecting edges. Specifically, edges can be detected by taking the gradient of the image and applying an appropriate threshold to the resultant 2-D signal; or by taking the Laplacian of the im age and then identifying its zero crossing points. Com m only used gradient operators include Prew itt, Sobel, Roberts and isotropic operators [4]. Most of these operators perform reasonably well on noise-free images whereas they do not work well for noise-contam inated im ages. This is because derivative operators enhance and em phasize high frequency noise. One way to overcome this difficulty is to sm ooth the data before differentiation is applied. An exam ple that realizes this idea is the Laplacian of G aussian (LoG) operator proposed by Marr arid Hildreth

39 11 [27], which suppresses noise by first sm oothing the im age by convolving it with a Gaussian function, and then applying a Laplacian to the resultant image. The edges of the im age at hand are finally identified by zero crossings of the output from LoG. It should be pointed out that the above pre-sm oothing step tends to blur edges, and consequently the LoG m ethod degrades the localization of edges. Thus, an edge detector which can provide accurate locations of edges for noisy images is desirable. Indeed, good localization of the edge is one of the three criteria proposed by Canny [29] in obtaining an optim al edge detector. Canny s other two criteria are good detection ability and single response per edge. It is noted that Canny s detector is optim al only for a small fam ily of edge detectors (e.g., LoG) in detecting stcp-function edges. Nevertheless, Canny s approach can be applied to different types of edges, and the approach leads to a new direction of designing optim al edge detectors. Following the work [29] m any different optim al edge detectors have been designed which m odify and improve Canny s detector by adding more criteria, or by considering ram p-function and roof-function edges, rather than step edges. These optim al edge detectors include the optim al edge detector for ramp edges by Petrou and K ittler [32] and the optim al infinite im pulse response edge detection filter by Sarkar and Boyer [33]. * Identification o f Im age M odel The objective of image restoration is to generate an estim ate of the original scene that, is as good as possible based on the available (degraded) image. There are three m ajor types of degradations, namely, point degradations, spatial blurs, and noise. Point degradations are referred to as those that affect th e gray levels of individual

40 12 pixels, without introducing spatial blur, and they are in general nonlinear. A typical exam ple of point degradation is film nonlinearity caused by film s sensitivity to light intensity. The density of silver grains on developed film varies approximately logarithm ically w ith the incident light intensity with saturation in both black and w hite regions [34]. Potential spatial blurs include relative linear motion between the object and the im age, out-of-focus in an optical system, atmospheric turbulence effects, geom etric distortions, and sensor nonlinearities. Noise disturbances may be introduced by electronic im aging sensors, transmission devices, recording devices, m easurem ent errors and quantization errors. The objective of im age restoration is to generate an estim ate of the original scene as good as possible based on the available (degraded) image. If the imaging system is modeled by taking all the above m entioned degradations into cons' nation, the restoration problem would be too involved and there would be no general solution for the problem. In this thesis, it is assum ed the im aging system is of the form (1.13) where the spatial blur is shift-invariant, the noise is additive, and point degradation can be neglected. Under these assum ptions, the corresponding restoration problem becom es m athem atically m ore tractable. A nother im portant issue w ith which we have to deal is that in practical, applications blur characteristics are often unknown, hence the blur operator H (or: equivalently th e blur PSF k(k,l)) m ust be identified from the degraded im age g itself. In other words a practical im age restoration problem is to find an estim ate of / given th e degraded im age g only, w ithout priori knowledge of the blur degradation and noise degradation. In this case, the blur PSF and noise characteristics have to be first estim ated from the image to be restored. The associated image restoration problem is therefore called a blin d restoration problem. There are three

41 13 approaches to a blind restoration problem. One approach is to locate and exam ine sharp points and sharp lines in the image to be restored. B y definition, the blur impulse response is the im age of a point-source object. Therefore, a point source in the degraded image yields a direct indication of the blur PSF. This would be the case in an astronomical image, where the image of a faint star could be used as an estim ate of the blur PSF. The image of a sharp line can also be utilized to determ ine the blur PSF and details can be found in [35]. The second approach is m ainly to identify those blur PSFs whose spectra show a regular pattern of zero crossings. Since these zeros can also be located using the spectrum of the blurred image, spectral or cepstral techniques [20] can be used to estim ate the PSF from the distance betw een spectral zeros o f tne blurred im age. T hese techniques are especially effective for dealing with linear m otion blur and out-of-focus blur. D etails of this approach can be found in [2, 20, 36, 37]. In the third approach, the original im age (the object) is first m odeled as a 2-D autoregressive (A R ) process and then the identification problem is formulated as a m axim um likelihood problem, which turns out to be equivalent to a 2-D autoregressive moving-average (ARM A ) m odel identification problem. The earliest work in this area was described by Tekalp et. al. [38, 39], in which m ethods for the identification of various blur PSFs are proposed. Recently, following the work of [38, 39] Lagendijk et. al. [20, 40] and Biem ond et. al. [41] proposed an expectation-m axim ization im age identification algorithm which can be applied to blurred im ages w ith relatively low signal to noise ratio (SN R ).

42 C ontributions This thesis is intended to be a self-contained description of som e filtering techniques for restoration, noise removal, edge detection and blur identification of degraded im ages. For im age restoration, we have developed a frequency-domain recursive regularization filter (R-filter) theory of the well-known regularization techniques to efficiently restore noisy images whose degradations are due, for exam ple, to camera defocusing, linear m otion, or exponential blurring. The R-filters proposed here are derived by generalizing the 1-D recursive R-filters proposed by M. TJnser et. al. [24] in 1991 for noise removal purposes to various im age blur mechanism s and to extend it further to the 2-D case. Unser s spatial domain approach was transferred to frequency dom ain through the use of Parseval formula. The filter can either be applied in the frequency dom ain or transferred back to the spatial dom ain. In the frequency dom ain, the number of operations carried out is sam e as the D F T im plem entation proposed by Hunt [9]. In the spatial dom ain, the tim e required might be longer but one has th e advantage of adaptive filtering and region of interest filtering. For im age noise removal, several new nonlinear 2-D filters that make use of local sta tistics of th e im age have been designed. Moreover, a tw o-step regularization approach is proposed for noise removal. For im age edge detection, we have developed several design techniques for detecting edges of noisy images by using linear, low-order, 2-D finite im pulse response (FIR) filters. The design techniques are based on sim ultaneous consideration of the conventional edge detectin g procedures the pre-sm oothing step and the differentiation step. It is also dem onstrated that both noise removal and edge detection tasks can also be performed within the unified R-filter fram ework. Finally a blur identification algorithm based upon the

43 15 relative minimum pattern of spectra of degraded images is proposed. The algorithm proposed is found to be successful when identifying linear m otion blurs, defocusing blurs, and blurs with circularly sym m etric PSF. Making use of the proposed blur identification algorithm, a two-phase blind restoration algorithm is presented. In the first phase of the algorithm, estim ated power spectra of subimages of the original im age are determ ined from the degraded image. In the second phase, a blind restoration algorithm is em ployed to obtain a restored im age usinb estim ated power spectra obtained in phase one. 1.4 O rganization o f th e T hesis The principal objectives of this thesis are to present the unified R-filter approach to im age restoration, noise removal and edge detection; nonlinear filtering for noise removal; low-order FIR filters for edge detection; and a two-phase blind restoration algorithm. T he remainder of this thesis is organized as follows. In Chapter 2 we provide a detailed description of the 1-D R-filter theory. The 1-D R-filter theory on noise removal by Unser et. al. [24] is briefly reviewed. Then it is extended to include a nontrivial 1-D deblurring mechanism. As a result, the generalized 1-D R-filters can be used to restore noise contam inated im ages that are further distorted by a linear m otion blur. It is shown that 1-D R-filters can be decom posed into two com p lem entary causal and anti-causal, recursive, stable filters with identical coefficients. Furtherm ore, th e 1-D R-filter theory is extended to yield a 2-D R-filter theory. B e ginning with the im age restoration m odel (1.13), the problem at hand is form ulated as a least-squares optim ization problem. Tw o techniques for the im plem entation

44 16 of recursive 2-D R-filters are presented. The first is iterative in nature, while the second is accom plished by approximating the square root of m agnitude response of the R-filter by a stable causal filter. Applications of the proposed restoration algorithm s are illustrated using a number of sample images. Chapter 3 presents m ethods for determ ining optim al regularization parameters and/or optim al stabilizing operators for image restoration. The first part of the chapter is devoted to a study on optim al determination of the regularization parameter A by an L-curve approach. Definition of the L-curve is given and the Tikhonov regularization is discussed in some detail. Several m ethods for choosing suitable regularization parameters are reviewed. It is then shown that the regularization parameter corresponding to the largest curvature of the L-curve gives a nearly optim al regularized solution for a given image restoration problem. Dealing with im ages blurred by linear m otion, an adaptive restoration m ethod is proposed where the regularization parameter is determined by the proposed L-curve m ethod. The second part of the chapter presents several optim ization techniques for determ ining th e regularization param eter and stabilizing operator. T he generality of th e proposed techniques im plies that they can be applied to other 1-D and 2-D ill-posed problem s as w ell. Several sam ple im ages are used to illustrate the techniques proposed. Finally, the conventional Tikhonov regularization m ethod where a single regularization param eter is em ployed is extended to a regularization schem e where m ultiple regularization parameters can be incorporated. It is shown that when the proposed m ultiple-param eter regularization m ethod is applied to an im age restoration problem, the solution accuracy and smoothness is better balanced. It is also shown that th e im plem entation of th e proposed algorithm requires 2-D D F T operations only. Finally, th e R-fiiters for the m ultiple-param eter im age restoration

45 17 problem are derived. Chapter 4 describes an improved Wiener filtering technique and its application to im age restoration. It is demonstrated that the conventional Wiener filters can be improved by taking the information contained in the spectra of the blurring operator, the noise and the image into consideration. A modified W iener filter is derived based on this idea together with the regularization concepts. It is dem onstrated that better quan titative results can indeed be achieved by using the m odified W iener filter. Chapter 5 is devoted to several filtering techniques for noise removal. In the first part of the chapter, average filtering and Gaussian filtering are briefly reviewed, and a two-step regularization approach is introduced. Application of the R-filtering techniques to noise removal is presented. The second part of the chapter provides a treatm ent of the noise removal problem using quadratic and cubic filters which utilize local statistics of th e im age. In Chapter 6, num erical optim ization techniques are used to design edge d etectors. Here the idea is to integrate the differentiation step w ith the pre-sm oothing step to obtain a design specification for a band-lim ited differentiator. This leads to several im age detectors that are iow-order 1-D and 2-D FIR filters. These filters are linear and shift-invariant, and can be im plem ented efficiently by discrete convolution. Experim ental results are presented to show the performance of the design edge-detector filters. Chapter 7 proposes a two-phase blind restoration algorithm. We first present a practical identification m ethod by exam ining the m inim um -pattern of the spectra of blurred im ages. T he m ethod is shown to be successful in identifying linear m otion blur, defocusing blur, and those with circularly sym m etry PSFs. The identification procedure is th en incorporated into th e proposed tw o-phase restoration m eth od

46 18 which is described in the second part of the chapter. Chapter 8 sum marizes the major contributions of the thesis and discusses some- future research considerations in the field.

47 19 C hapter 2 O ne-d im ensional and T w o-d im ensional R -F ilters 2.1 O ne-d im ensional R -F ilters Introduction A problem is well-posed in the sense defined by Hadamard [8] when its solution exists, is unique, and depends continuously on the initial data. Ill-posed problems fail to satisfy one or more of these criteria. It has been known that im age restoration problems are ill-posed problems [1, 10, 11, 12]. This can be seen by considering an im aging system m odeled as g = H f + n (2.1) Typically, im age restoration is a procedure that, for given degradation m echanism H and degraded im age g, one seeks a good approximation to the original im age /. In the discrete version the large size m atrix H as an analytical representation of the degradation m echanism is often ill-conditioned. As a result the widely acceptable least-squares solution / = H^g H^n, where H* = (HTH)~lHT is the pseudo

48 20 inverse or th e M oore-penrose generalized inverse [42] of H, is not of use as it is too sen sitive to th e system noise n. Regularization theory provides a m ethod to solve ill-posed problems and to. com pute solutions that satisfy a priori smoothness constraints. In this section, the recent work of Unser, Aldroubi, and Eden [24] on recursive regularization filters (R-filters) will be extended to a more general setting to include nontrivial 1-D deblurring dynam ics in the regularization filters. As a result, the generalized R-filters derived here can be utilized to restore noise contam inated images that are further degraded by a 1-D m otion blur. As a key in our developm ent, a decom position theorem for the 2 -transform1 of generalized R-filters is presented, which shows that R-filters are sym m etric stable lowpass filters with an adjustable regularization parameter and can be decom posed into two complementary linear shift-invariant2, 1The z-transform X(z) of a sequence far(ar)} is defined as OO * 0 0 = *(*)* * k QQ where z is a complex variable. The z-transform of the impulse response {/>(&)} of a linear filter is often referred to as the system function or transfer function of the filter. 2The class of shift-invariant filters is characterized by the property that if {y(k)} is the response to {ar(fc)}, then {y(k /)} is the response to {x(k /)} where / is a positive or a negative integer. It can be shown that any linear shift-invariant filter is completely characterized by its impulse response {h(k)} [43].

49 21 causal3 and anti-causal recursive4 stable5 filters with identical coefficients. Related design and im plem entation issues will also be addressed, and experim ental results will be included to illustrate the proposed theory The M ethod of Unser, Aldroubi, and Eden In [24], the imaging system m odel is given by g (k ) = f ( k ) + n (k ) (2.2) where {</(&)} and {/(& )} represent the lexicographically ordered recorded and original im ages, respectively; and (n(a:)} is the additive noise. T h e problem to be solved here is to find a good approxim ation to th e original im age / from the noise-contam inated image g. In [24], Unser, Aldroubi and Eden proposed a recursive regularization filter to accom plish this noise removal task. Their m ethod is described as follows. For a given A > 0, one seeks a regularized 3 A causal filter is one for which the output for any k = ko depends on the input for k < ko only. It can be shown that a linear shift-invariant filter is causal if and only if the impulse response h(k) is zero for k < 0. 'A linear shift-invariant filter is recursive if there are infinitely many nonzero impulse response terms in h(k). 5A stable filter is a filter for which any bounded input produces a bounded output. A linear shift-invariant causal filter with system function H(z) is stable if all the poles of H{z) lie inside the unit circle.

50 22 approximation to signal {/(& )}, denoted by {/,\(A:)}, that minimizes CO?»(/) = I/W - 9(fc)]2 + A [/(*). c(fc)l2 (2-3) fc = oo k = oo where A is called the regularization parameter, {c(ar)} satisfies J2T=-oo c(^:) = 0 <nid defines a stabilizing operator that in conjunction with an appropriate regularization param eter controls th e degree of regularization for th e optim ization problem defined by (2.2) and (2.3). It can be shown that the optim al {/a(a")} satisfies the recursive relations [24] OO f\{k) ~ g(k) + A (j>c(l)fx(k - /) = 0, - o o < A: < 0 0 (2.4) / = n where (j>c{k) = c(k ) * c( k) (the convolution of c(a;) with c(~k)), and {c(a:)j is an nth order FIR type stabilizing kernel. It follows that the regularized solution f\(k) can be derived from g(k) by linear filtering fx(k) = wx(k) * g{k) (2.5) where {iwa(a;)} is the R-filter in the spatial domain. Application of the 2 -translorm to (2.4) then leads to Wx^ = G(z) = 1 + A C(z)C{z~') ^ where Fx, G and C are the 2 -transform of /a, </, and c, respectively. Furthermore, th e standard sp ectrum factorization technique [44, Ch. 4] gives Wx{z) = V{z)V{z~') (2.7) where V{z) represents a linear shift-invariant, stable, recursive, causal 1-D digital filter which is of the form V(z) = ^ (2.8) v ' 1 -I- 0 i \- anz~ n v

51 23 where n + 1 is the length of the stabilizing functional c (k ) and bo, ax,, a n are real coefficients that can be obtained directly from (2.6). Note from (2.2) that the m ethod of [24] treats restoration problems where the degradation effect from the imaging system has been neglected. In the next section, the concept of recursive R-filters will be extended to include nontrivial 1-D distortion dynam ics G eneralized 1-D R -F ilters Consider now an imaging system m odeled by the 1-D linear shift-invariant discrete system OO g(k)= h(k-l)f(l) + n(k) (2.9) / = OO where {g(k)} and {/(& )} represent the lexicographically ordered recorded and original images, respectively; {«(&)} is the additive noise; and {h(k)} characterizes a 1-D im age blurring process and is assumed to possess finite length. Denoting the convolution of h with f by h * f, (2.9) can be written as g(k) = h(k) * f(k) + n(k) (2.10) The restoration problem now is to find a regularized approxim ation to {f(k)}, denoted also by {f\(k)}, that m inim izes OO OO W/)= E «*)»/(*)-*(*}] +AE M*) */(*)] (2.11) fc= oo fe= oo Let F(u), G{u>), H(u>), and C{uj) be the z-transform of /, g, h, and c evaluated on the unit circle z =. By Parseval s formula, (2.11) can be written as (\{f) = ^ e\(f, w)du> (2.12)

52 24 where ea( y» = IH(u)F(u) - G(u) 2 + A C (w )^(w ) 2 (2.13) It can readily be shown that, regarding u as a real parameter, the minimum of ea( /, <x>) (and therefore the m inim um of \ ( / ) ) is achieved by A(W) tf(w ) 2 + A C(w) * In z-d om ain, (2.14) defines an R-filter with transfer function R(z) = F& ) H i* " ') n G(z) H(z)H(z-') + \C{z)C(z-1) = H(z~1)S\(z) (2.15a) where = H{z)H{z~') + A C{z)C{z~l) ^2,15b^ In w hat follow s the non-causal transfer function will be referred as a 1-D recursive R-filter. Som e elem entary properties of S\(z) are summarized in the following remark. Rem ark 2.1 (1) The R-filter W\(z) obtained in [24] is a special case of R\(z) with Ti(z) = 1. (2) 5a(z) is sym m etric: S \(z ) = 5a(z-1). (3) S\(z) is stable if H(z)H(z~l)+\C(z)C(z~l) ^ 0 on the unit circle, a condition is assum ed in the rest of this thesis which is the case in real world situations.

53 25 (4) (i) lim,s'a( / ) = jjrfjw ', w h ere S\(f) = S A(*) 3=ei**/ a" o AW ' ~ lw(/)l2 (ii) lim S A( / ) = A ^oo tf(/)l2 ^ = 0 0 otherwise In th e rest of this section, the blurring process {/i(&)} and th e stabilizing functional {c(ifc)} are assumed to process finite lengths (m-fl) and (n + 1), respectively. We will show that S\(z) can be efficiently implemented by decom posing it into a causal and an anticausal filters. Let us first state th e following lem m a. Lemma 2.1 Let A(z) = E L-oo a(k)z~k. Then A{z)A(z~L) = M k)z~k where <j>a(k) = a(k)*a( k) is the autocorrelation function of a(k). Furthermore if a(k) is of length m + 1, then <j>a(k) is of length 2m + 1 and A{z)A{z~l) = L -m <f>a(k)z~k. Let M m ax(m, n) and make use of Lemma 2.1, we obtain Sx(z) = 1 ZZL-MlMk) + \ M k ) \ z ~ k+m P-2Xf(z) (2.16) Since S'a(z) = S^z-1), it follows that 1 PM z~l) = P2m (z) (2.17) Therefore if z,- / 0 is a root of P2m(z), s o is zi 1. This m eans that the roots of the polynom ial P2m{z) appear in reciprocal pairs. Furthermore, S A(z) is stable (see

54 26 Rem ark 2.1). T his im plies that th e roots cannot be on the unit circle. C onsequently, there m ust be M roots with modulus smaller than one, which we denote by { z i,i = 1,2,...,M }. N ote that w hich leads (2.16) to M P*m{z) = [M M ) + XMM)) Y[{z - zi){z - z~[) (2.18) i=l where the constant a is positive and is given by If we define th e stable rational function a = (2.2 0 ) M M ) + A <f>c(m) ( ) then we can express S\(z) in the sim ple form Sx(z) = U(z)U{z~x) (2.22) Hence we can state T heorem 2.1 (D ecom position Theorem) Suppose h(k) and c(k) are of finite lengths. Then S\(z) can be decomposed as (2.22), where th e constant a is given by (2.20) and U(z) given in (2.21) is a stable filter. It im m ed iately follows from Theorem 2.1 that «(z ) = H(z-l )U(z)U(z-') (2.23)

55 27 Equation (2.23) suggests that the R-filter R\(z) can be im plem ented by filtering the input im age w ith U(z -1 ), and then filtering its outcom e image by U(z), and finally filtering the resulting im age by H(z~1). M otion blur occurs when there is relative m otion between the object and the camera during exposure. Let f(x, y) be the original im age of an object being imaged. Suppose it m oves in the a:-direction at a constant velocity V during the exposure interval [0, T], the recorded image (observed imaged) can be expressed as [4, 34] 9{x,y) = ^ jf f{x-v t,y)dt 1 fvt = VT Jo f(x - P VVP = V t C l e c t { v r ~ 5)/(x p > ) d p 1 f+ o o r+ oo n ^ = VT J-oo J - oo r e c t ^ V T y ~ ^ d p d q ( ) where S(-) is the Dirac delta function and f 1 1*1 < 5 rect(x) = < I 0 otherw ise This shows that the linear m otion PSF blur in the x-direction is v) = v ^ rect( ^ ) % ) (2-25) The discrete equivalent PSF is { t t t fc = 0,l,2,...,L, 1 = 0 + (2.26) 0 otherwise and th e corresponding frequency response is given by h ( ^ ) - i L + 1 sm (a;i/2 ) (2.27)

56 28 where L is the blurring distance which is defined as the number of sampling intervals equal to the distance traveled per unit of tim e by a particle that moves at the sam e speed as the linear m otion. If th e second-order difference operator, nam ely the Laplacian filter, is used as C (*), C{z) = z~1-2 + z (2.28) then at u = 0, C (ej0) = 0, and hence from (2.15(b)) S\(e^ ) = l/7 :/2 (e-, ), which is equal to unity if H(z) = 1 or H(z) = j^ j-(l + z_ (- z~l) (the system function of a 1-D blurring operator with distance L). Thus in low frequency range the behavior of the R-filter has in general little to do with the regularization parameter A. On the other hand, at ui = 7r /2, ( ^ (e ^ 2) = 2, thus from (2.15(b)) S\(fJn^2) 1 / ( ( z 7+ t ) 2 + 4A) for a linear blurring operator with distance L. This suggests that a reasonable initial choice of A should be A = 0.25(1 ( 7^-j-)2) in order to preserve useful im age inform ation over high-frequency range. Experim entally it is found that optim al values of A corresponding to a variety of R-filters for different images with different noise levels are always close to the suggested initial choice. For exam ple, with L = 14 the optim al value of A is The am plitude response of the corresponding filter S\(z) is shown in Fig. 2.1 with A = 0.2. Note that the am plitude at low frequencies (u; < 0.25) and high frequencies (u> > 1.6) are approximately the same. It is found that in restoring the degraded images (with L = 14) in Figs. 2.2 and 2.3 (pages 38, 39), th e op tim al A s are 0.2 and 0.3, respectively.

57 Figure 2.1: Am plitude response of S\(z) with A = 0.2 and L = 14.

58 A D esign Issue It is observed from (2.15b) and (2.22) that the am plitude response of the transfer function U(z) is determ ined by = tf(e ^ ) 2 + A C(eJW) 2 ^ 2<^ This suggests an alternative approach to obtain an approximation U(z) to U\z) as follows. Given H(e^tJJ) and C'(eJU'), a stable, recursive 1-D digital filter U(z) can readily be designed w hose am plitude response, D (ejw)j, approxim ates 1 [ fl'(e ^ )!2 + A C ( e ^ ) 2]1/2 V2 J 0 ) by using one of the well established design m ethods [45]. Assume that sampled am plitude response S\{u>k) is available where = ( ^ -^ )tt, k = I,,]{ and K is a pre-chosen number of sample points. Let the transfer function of U{z) be in the form of U(z) = (2.31) 1 + a\z~x anz~n The coefficients b0,ai,...,a n of U(z) can be determined numerically by m inim izing the L 2P error function EiP = ( \ U M - J s l M v*)'1* (2.32) k=l with th e stability constraint; that is, each pole of U(z) lies within the unit circle. A lthough th e 1-D R-filters can be designed precisely using the approach of Sec , the observation m ade here has been found instrumental in extending the concept of 1-D R-filters to the 2-D case. Details of a 2-D R-filter theory will be discussed in Sec. 2.2.

59 Recursive Regularization Filters w ith Approxim ate Linear P hase Filters U(z) and U(z) designed in preceding sections are infinite im pulse response (H R) filters (recursive filters) and therefore phase distortions are inevitably introduced in restored signals. A possible approach for significantly reducing the distortion is to cascade U(z) w ith a stable allpass filter IIan(z). An allpass filter has a unit m agnitude response for all frequencies. The transfer function of the nth-order allpass filter is of the form [46] = + +«. (2 JB ) where a j,, an are real coefficients. It is easy to see that the z-transform of a stab le 3rd-order allpass filter can be expressed as U (~\ - (r + g~1)(rl2 + 2r^COS 9z~1 + Z~2) (0 VA\ W (1 + r0 2-1 ) ( l + 2 n cos 6z- 1 + r\z-2) { w ith constraints r,- < 1, * = 0,1, and 0 < 7r. In general th e z-transform of a stable nth order allpass filter is of the form Hall{z) = i- rn /2 (r?+ 2r,-cosfl,z 1+ z 2 ) n,= i (1; 2ric o B ^ - ^ - 2) «is even 3 5 ( r p + z - 1) - r ( n - l ) / 2 ( r ^ r. c o s f l. z ^ - j - z - 2).,, (1 + ro z - 1 ) I b = l ( l+ 2 r j cos 0,z 1 + r 2z 2) Now, for a fixed n, we try to find an allpass stable filter Haii(z) such that the group delay [43] G(u) = ~ L V ( u ) H M(u) (2.36) of U(z)Haii(z) is nearly constant. Hence a possible approach to designing a stable Hau to achieve approxim ately linear phase for U{z)Haii is to form ulate the design

60 32 problem as a constrained optim ization problem as.folic tvs: m in G(u;,) - G'(wt-, ) '2 (2.37) r,,#i. subject to r,j < 1, l^.l < 7T The resulting filter U(z) = U(z)Hau(z) is stable since both U(z) and H u(z) are stable R ealization From (2.2 1 ) it follows that U(z) is an all-pole transfer function of the form U(z) = (2.38) 1 -f- a\z + -!- anz n For a linear 1-D blurring process, H(z 1) = hr, -f- h\z + -(- hnz11 (2.39) where n = L = the blurring distance in one direction in sampling units and h0 h\ = = hn = 1 /{L + 1). N ote that the order of H(z~l) has been assumed to be the sam e as that of U(z). This is the case whenever one deals with blurred im ages with L larger than 2, where C(z) represents the Laplacian filter whose order is 2. T he realization of R-filter R\(z) can be accomplished in two steps. First an interm ediate im age, denoted by is generated by filter S\(z) as follows ft(k) = g ( k ) - a if f ( k - 1 ) anf f ( k - n ) f \ { k) = f t ik) - o i/a ik + 1 ) /a {k + n) (2.40b) A (*) = %fx(k) (2.40c)

61 33 This js a step identical to the one proposed in [24], although the parameters a[s and b0 are obviously different from that in [24] due to the presence of the blurring process. Next the FIR filter (2.39) is implemented to generate f\(k) as / a (*0 = X > / a ( * + O (2.41) l=o As pointed out in [24], disturbing artifacts due to the utilization o f boundary conditions can be significantly reduced by im posing the following boundary conditions f t (k) = ^ for k = 1,, n (2.42a) Jxy ' ( l + ax + + <* ) V ' fi W J. ' for k = K - n - H, -, K (2.42b) + + an) where K is the number of image pixels in one line of the image. Similarly, reasonable boundary conditions for (2.41) can be set as / a( * ) = ( I > ) A ( * ) iovk = K - n + l,---,k (2.43) 1= C om putational Comparison of Spatial and Frequency Dom ain Im plem entation The 1-D R-filtering approach provides a spatial dom ain im plem entation of the restoration of a linear m otion blurred im age in comparison to the conventional D F T im plem entation. Since th e im age is linear m otion blurred, th e im age can be processed lin e by line. Suppose th e linear m otion blur is in the horizontal direction, then the 2-D im age m odel (1.13) can be reduced to 9k = H fk + nk (2.44)

62 where and fk represent the kth row of the recorded and the original im ages, respectively; H is a circulant m atrix representing the linear motion blur operator. For a given A, the Tikhonov solution of (2.44) is given by (see (L 15)) fkx = (Ht H + A CTC)~lHTgk (2.45) which can be obtained efficiently using the 1-D DFT [9]. B y the proposed spatial 11- filtering approach, the estim ate fk\ of fk can also be obtained by a cascade filtering (see(2.23)) w ith filters H(z~l), U(z) and U(z~l). In the rest of this section, the com putations required in each of these two different im plem entations is com pared. Let N be the size of gk. In practice, N = 27 for some integer 7, and is taken to be 8 in this thesis. It can be shown [47, 48] that the numbers of m ultiplications and additions required to evaluate a D F T of a set of N data base on a base-2 D FT algorithm are given by ( 2 7 4)iV + 4 and ( 3 7 2)iV-j-2, respectively. Table 2.1 gives the required number of operations for obtaining fk\ through base-2 D F T algorithm. For a 1-D linear m otion blurring process with L units, the orders of H(z~ [), U(z) and U(z~1) are all equal to L + 1. The number of operations for obtaining fkx through the R-filtering approach is given in Table 2.2 (see (2.38) and (2.39)). Custom arily L is taken to be less than 1 0. In this thesis we have taken L to be 14. Even in this case it is seen that the R-filter approach in the spatial domain is approxim ately twice as fast as in the frequency domain.

63 r Im plem entation M ultiplication A ddition T{C) (27-4)JV + 4 (3 7-2 ) N + 2 H H ) (2 7-4)N + 4 ( )N + 2 F{C t)t{c) N 0 A T{C t)t{c) N 0 T {tir)t{h) N 0 T{H r )F{H) + A T{C t)t{c) 0 N i Jr(HT)Jr(H)+\T(C',')Jr(C) N 0 H i) ( 2 7-4)iV + 4 (37-2 ) i V + 2 N 0 r(ht)ha) r(ht)f(h)+\t(ct)r(c) N 0 ^ (r(ht)f(h)+\f{ct)t(c)) (27 - i)n + 4 (37 2)N + 2 Total 2(47-5)N ( 3 7-2)N + 8 Table 2.1: Num ber of operations for obtaining fk\ through base-2 D F T algorithm. J-(C) represents the D F T of C. \

64 Im plem entation M ultiplication Addition H(z- 1) N LN U(z) (L + 1 )N LN U{z~l) (L +1)N LN Total ( 2 L + 3 )N U N Table 2.2: Number of operations for obtaining fk> through spatial R-filtering approach. The blurring distance is L sampling units A pplication to the Restoration of M otion Blurred Im ages In im age processing, it is useful to have some measures of the difference between a pair of similar im ages. The m ost common difference measure is the mean square error (M SE). T he M SE measure is popular because it correlates reasonably with subjective visual quality tests and it is m athem atically tractable. Consider a discrete im age {f(k, /)} w ith size K x L which is regarded as a reference image, and consider a second im age {/(&, /}) that is to be compared to the reference image. Under the assum ption that {f(k, I)} and {/(&,/)} represent samples of a stochastic process, the M SE between the im age pair is defined as [2] M SE = E(\f(k, I) f(k, /) 2) (2.46) where E(-) is the expectation operator and the normalized mean square error (NM SE) is defined as NMSP. ( I / ( M ) - / ( M ) P ) - ( l m m ( )

65 37 In practice, it is com m on to assume that /(&, /) and f(k, /) are samples of ergodic processes. An ergodic process is defined as a random process w ith equal space averages and ensem ble averages. Under the assumption of ergodicity, MSE and NM SE are com puted as follows [2]: m s e = t ^ x ; i / ( m ) - / ( m ) i 2 (2.4 8 ) K L k=i i=i D / ( M ) - / ( M ) I 2 NM SE = 100t=i l=);. L % (2.49) D / ( M ) I 2 k= 1 /=1 In this section, the 1-D R-filters proposed in Sec are used to restore two digital images that are linearly blurred with blurring distance L 14 and are contaminated by pseudo white noise. All images have 256 x 256 pixels and 8 bits of gray levels. (In the rest of this thesis, all images have 256 x 256 pixels and 8 bits of gray levels unless otherwise stated.) The original im age in Fig. 2.2(a) was linearly blurred and pseudo white Gaussian noise was added w ith signal to noise ratio (SNR) = 30 db (Fig. 2.2(b)). Here the SNR is defined as [20] ~, variance of the blurred im age SN R = 10 log1 0 : : ) variance ol the noise For comparison purposes, a W iener filter [4] was used to restore the image. The W iener filter used is given by W (w i,w j) + (2-51) where Pj(l)i,u>2 ) and Pn(u\,u>2) are the power spectra of the original im age and the noise, respectively. T h e restored im age by the W iener filter is shown in Fig. 2.2(c).

66 (a) (b),.vavajavai.v/^av.v.,.v.v/.v/a (c) (d) Figure 2.2: R-filtered and W iener filtered images, (a) Original image, (b) Linearly blurred im age w ith L = 14 units and pseudo white Gaussian noise at SN R = 30 db level, (c) Restored im age by W iener filter, (d) Restored image by 1-D R-filter with A = 0.2.

67 39 (c) (d) Figure 2.3: R-filtered and W iener filtered images, (a) The original im age Lena, (b) Linearly blurred im age with L = 14 units and pseudo w hite Gaussian noise at SN R = 25 db level, (c) Restored im age by W iener filter, (d) Restored im age by 1-D R-filter with A = 9.3.

68 40 A lgorithm M SE NM SE (%) ISN R (db) W iener filtering S R-filtering Table 2.3: Q uantitative comparison between R-filtering and Wiener filtering of the degraded im age of Fig The filter that produces smaller MSB and NM SE, or larger ISNR is considered to be a better one. A lgorithm M SE NM SE (%) ISN R (db) W iener filtering R-filtering Table 2.4: Q uantitative comparison between R-filtering and Wiener filtering of the degraded im age Lena on Fig The filter that produces smaller MSE and NM SE, or larger ISN R is considered to be a better one.

69 41 W ith L 14, A = 0.2, and C(z) = z~x 2 z (the Laplacian), the degraded im age was restored by the proposed m ethod, and the resulting im age is shown in Fig. 2.2(d). Table 2.3 summarizes the quantitative restoration results of these two filters, where MSE and NM SE are given in (2.48) and (2.49), respectively, while the improvement SNR (ISNR) is defined as [34] IS N R = 101ogi0 s s a ^ o - A M ) - (2JH) El5 E?=1 (/(M) - /(M ))2 with /, g and / the original, the noisy and blurred, and the restored im ages, respectively. The filter that produces smaller values of M SE and NM SE, or larger value of ISN R is considered as a better one. T he second image (Fig. 2.3(a)) in our experim ent is the Lena image. Fig. 2.3(b) shows a noisy and blurred Lena im age with L = 14 and SNR = 25 db. It was restored by a W iener filter and a R-filter with the restoration parameter A = 0.3. The resulting images by the W iener filter and the R-filter are shown in Fig. 2.3(c) and (d), respectively. Table 2.4 summarizes the restoration results of these two filters. It is observed that in both exam ples the proposed m ethod provides better restoration in terms of each of the MSE, NM SE and ISNR criteria C onclusions The present section extends the design of recursive 1-D R-filters to signals which are distorted by linear shift invariant operator as well as degraded by additive noise. This is presented in Sec The derivation is carried out in the continuous dom ain with the use of Parseval s theorem. No restrictions on both the distorted operator {/*(&)} and th e stabilizing functional e(fc)} are im posed. W hen both

70 {h(k)} and {c(a:)} are of finite lengths, a decomposition similar to equations (2.22) and (2.23) is obtained. T h e analytic decom position o f recursive 1-D R-filters no longer exists when either {h(k)} or {c(fc)} is of infinite length. Even when both {/»(&)} and {c(lfc)} are of finite lengths, com putation involved in im plem entation of the corresponding recursive 1-D R-filter increases as the lengths of {h(k)} and {c(fc)} increase. In general, an approxim ation decom position can always be designed by using constrained op tim ization techniques. T his is presented in Sec In Sec , design of linear phase recursive 1-D R-filters is studied. There are at least two types of distortion that can be introduced by a linear filter, namely am plitude distortion and phase distortion. Linear filters always introduce am plitude distortion unless their am plitude response is constant for all frequencies. Therefore there is not m uch we can do to improve the am plitude distortion. Phase distortion is found in m any restoration algorithms, including those proposed in Secs and Those vague long, thin, irregular lines or bands on restored images are usually due to phase distortion. In order to diminish the phase distortion effect, a low order allpass Hau(z) is designed using optim ization technique such that U{z)Hau{z) is of quasi linear phase. 2.2 T w o-d im ensional R -F ilters Introduction This section concerns two im age restoration algorithms which extend the work of U nser, A ldroubi, and Eden [24] to 2 -D regularization filtering. As before, the im age

71 43 restoration model is formulated as a least-squares optim ization problem which is regularized with a highpass stabilizing operator. Unlike the approach of Unser, the transfer function of the 2-D R-filter is derived in the frequency domain. Furthermore, two approaches are proposed for the im plem entation of the 2-D R-filter derived. The first approach is an application of the iterative technique initiated by D. E. Dudgeon [49] to the 2-D R-filter, while the second approach is to approximate the square root of the m agnitude response of the 2-D R-filter by a stable and causal FIR or HR 2 -D filter. Experim ental results are presented in Sec to illustrate the restoration technique proposed. T he goal of im age restoration is basically to reduce the degradations in a recorded im age so that the resulting im age will best approximate the original scene subject to som e criteria. The standard m ethod for the treatm ent of this typical ill-posed problem of solving (2.1 ) for / for given g is the regularization technique m ainly due to Tikhonov [10, 11] where an approximate solution to (2.1) is obtained by m inim izing th e quadratic functional \ \ H f - g f + \\\C ff (2.53) where is the 2 -norm, m atrix C is a stabilizing operator that usually corresponds to a discrete Laplacian operator, and A is th e regularization param eter. T he discrete Laplacian operator having the filter mask L = (2.54) is m ost com m only used. For the purpose of comparing results obtained by our proposed algorithm s to conventional algorithms, the stabilizing operator C is chosen

72 as th e discrete Laplacian operator (2.54) in this thesis unless otherwise stated. The frequency response o f the Laplacian operator (2.54) is [4] L(u>i, ^ 2) = 2 cos u, i + 2 cos ijj2 4 = - ( u j + a j ) + M + M ) a _ + 0(M + ^ ) 3) (2.55) and is shown in Fig It follows from (2.55), operator (2.54) is only circularly sym m etric about the origin for low frequencies (see also Fig. 2.4). For high frequencies, the frequency response has a considerable deviation from an ideal Laplacian response (wj T u2). This m eans that operator (2.54) as the stabilizing operator in the regularized restoration is not optim al. We will not go further in this direction for the m om ent. An optim al choice of a stabilizing operator in regularized restoration will be discussed in Chapter 4. R ecently U nser et. al. [24] proposed a class of 1-D R-filters for noise removal where the signal m odel is given by (see Sec. 2.1) g(k) = f(k) + n(k) D etails of 1-D R-filters were discussed in Sec. 2.1, in which the method of [24] was generalized to design a broader class of 1-D R-filters for the restoration of signals w ith degradation m odel g(k) = h(k) * f(k) + n(k) T h e generalized 1-D R-filters were then used to restore linear m otion blur im ages. Moreover, in Unser et. al. [24] separability conditions are imposed on the 2-D R-filters. T h is characterization allows a direct use of all 1-D results to 2-D separable R-filters. However, this separable condition can rarely be satisfied in im age

73 45 o o (a) >) Figure 2.4: (a) Am plitude response of the discrete Laplacian operator (2.54). (b) The corresponding contour plot of (a).

74 restoration and its application is therefore lim ited. In this section, a generalized 2-D R-filter approach for the im age m odel (2.1) will be proposed D R -F ilters In order to m ake use of the idea developed in Sec. 2.1 and [25], the discretized im aging system (2.1) is rewritten as g(h,k2) = h{k\,k2) * f(k1,k2) + n(ki,k2) - oo <, k2 < oo (2.56) The regularization of m odel (2.56) to find {/(& i, k2)} from the recorded image {g{ki, fc2)} requires a stabilizing kernel {c(fci, k2)} satisfying YlTt,^=-<x, c(kh ^2) = 0. The cost functional in the least-squares optim ization problem associated with model (2.56) can be expressed as OO & ( / ) = [h(kh *2) * f(k 1, k2) - g(k1,k2)}2 + A K fci, h) * f{k 1, h)]2 k\,fc-2= 00 k\,k2= oo (2.57) OO where A is th e regularization param eter. In the rest of this chapter, we shall use X(zi,z2) to denote the 2-D 2 -transform of a given signal {a:(&i, k2)}. That is, By Parseval s Theorem, we have 00 OO X ( z J 4 ) = x ( h M K i'n' (2.58) k\ 00 ^2 = 00 & ( / ) = 4 ^ 2 J_^ (2.59) \{f,w\,u}2) = \H(u}i,u!2)F(u>i,u)2) G(ui,w2)\2 + \\G{wi,u)2)F(u'\,u}2)^ (2.60) It follows that \(f) is m inim ized if t\(f,ux,u2) is m inim ized. Assum ing all 2-D sequences involved are real-valued sequences, then X{u)i,u2) = X'( u)\, u>2) and

75 47 (2.60) can be w ritten as ex(f,u>) = \G{uj)\2 + [\H{w)\2 + A C (a;) 2][F ^ (a;) -f Fj{u)] + 2 [Gn(u)Fj(u) Gj(uj)Fii(ii})]Hz(U)) (2.61) where subscripts % and 1 denote the real and imaginary parts of the function, respectively, and u (uq,u;2). If can now readily be shown that ( 15 2) ^ ( u ;1,u ;2) 2 + A C (a;1,u ;2) 2 (2'6 } achieves the m inimum of e\. In z-transform representation, (2.62) represents the frequency response of the transfer function l?(~ \ _ H(zl iz2 )G{.zh z2),cy 2) ff(z1,z2)h (z;\z;1) + \C(z1,z2)C (z?,z?) [ and the transfer function of the required R-filter is in turn given by R\{zi, z2) = = H(zi \ z^)wx{zx, z 2) (2.64),Z2) Wx(Zi,z2) = J[ UZ2)H(Z- ^ Z-1) + x c iz ^ z jc iz ; 1^ ; 1) (2,65) N ote that the R-filter obtained in [24] m ay be regarded as a special case of (2.64) with H ( z i, z 2) = 1, and the properties for 5,\(^) stated in Remark 2.1 are also valid for W\(zi,z2) An Iterative Im plem entation of 2-D R -Filters Iterative algorithm s are used extensively in signal restoration because they allow incorporation o f prior knowledge about th e solutions into th e restoration process

76 48 [49]. Here we present a frequency-domain iterative algorithm for the im plem entation of the 2 -D R-filters which were proposed in the preceding section (Sec ). We write where (2-66) A(u>l,UJ2) H( U>1, U>2) B(l)i,U)2) = \H (wi,w2 ) 2 + A C(uq,U>2 ) 2 and B(uji,(jj2) in (2.66) is normalized so that its constant term is 1. As B(u>\,u2) > 0, a number fi can be found such that Let 0 < i i < ^ r (2.67) m ax B(u}i,uj2) (wi,w2) C{uj\,uj2) = 1 fib(u>i,u2) (2.68) then It follow s from (2.62), (2.66) and (2.69) that R ^ f2-^ F(uji,u!2) = ha(u}i,lo2)g(uji,uj2) + C(uji,u)2)F(u)i,uj2) (2.70) A pplying th e m eth od of successive approxim ation to (2.70) yields [20, 49] F o (u > i,u j2) = h A ( u > i, li2)g (u }i,u }2) (2.71a) Fm(wi,u>2) = fia(uji,u)2)g(u>i,ijj2) + C(u>\,u)2)Fm-\(uj\,u>2) rn, = 1,2, (2.71b)

77 49 By assuming F-m{u\, w2) = 0 for m = 1,2,, it is easy to see that Fm{ux,u2) = fia(ui,u}2) (2.72) By (2.67) and (2.68), \C(uji,uj2)\ < 1, so limn_oo Fm(wx,w2) = F(u>i,u;2). This shows that { / m(fc], fc2)} converges to {/(& i, k2)} as m >00 with /i satisfying (2.67). Obviously, the convergence rate of {fm(ki, k2)} is controlled by [i A n Approxim ate Factorization for the Im plem entation o f 2-D R-filters The implementation of R\(zi,z2) is accomplished in two steps. First, an intermediate image is generated with its 2-D 2-transform given by G(zt,z2) = H{zl 1,z2l)G(z1,z2) (2.73) Then the restored image is obtained by filtering G through Wa(zi, 22), i.e. G(zi, z2) = W\(zi,z2)G(zi,z2) (2.74) As is noted in [24] and Sec. 2.1, it is in general impossible to find a 2-D spectral factorization of W\{z\, z2). The main reason is the fundamental differences between the mathematics of 1-D and 2-D 2-transforms and polynomials. An alternative approach for the implementation of Wa(2i,z2) is to find a stable causal 2-D filter T(zj, z2) whose frequency response approximates 1 [ H(eM, e^)\2 + A C(e^, e ^ )! 2]1/ 2 (2.75) The am plitude response (2.75) is quadrantally symmetric with respect to the origin in the (o>i, u>2) plane, and filter T(zi, z2) can be designed using the combined singular- value decomposition (SVD) and balanced approximation (B A) m ethod [50, 51]. The

78 50 frequency response of T(z\, z2)t(zil, z2*) then approximates that of Ws{zx,z2) and, therefore, the restored im age {/,\(& i, k2)} can be obtained by filtering {fl(ki, k:2)}, whose 0 -transform G{z\, z2) is given by (2.73), through T(sl, z2)t(z^1, z2l). T h e advantage o f this design of 2-D R-filters is that the im plem entation is recursive. The filters T(zr, z2) are usually of low order. The number of operations per im age point is m2 + 2n 2, where m and n are respectively the orders of H{z\, z2) and T{z\, z2), which is usually a small number. Note also that the parameters involved in the 2-D R-filter are determ ined only by the blur mechanism and the regularization param eter, which are independent of the im ages to be restored Experim ental R esults A n Iterative Im plem entation Technique Three sam pled im ages which are degraded by a linear m otion blur and a defocusing blur are used to dem onstrate the performance of the proposed 2 -D R-filter and the results are compared with the classic Wiener filters based on MSE, NM SE and ISN R criteria which are defined in (2.48), (2.49) and (2.52), respectively. To be specific, two defocused and one linearly blurred digital images are used. The Lena im age in Fig. 2.5(a) was defocused with the radius of the circle of confusion (COC) [20, 34] r = 7 and was contam inated by pseudo white Gaussian noise with SN R = 28 db (Fig. 2.5(b)). T he im age processed by a Wiener filter is shown in Fig. 2.5(c) while the im age processed by a 2-D R-filter with A = is shown in Fig. 2.5(d). Fig. 2.6 shows the filtering results by Wiener and R-filter for noisy defocused Text im age with radius of the COC r = 5 and SNR 33 db. In the R-filter restoration, A = 6 x 10-5 is used. Q uantitative restoration results of these two exam ples are sum m arized in Tables 2.5 and 2.6, respectively. Finally, an R-filter with A = 0.006

79 51 was used to restore the noise contam inated (SNR = 25 db) and linearly blurred (w ith blurring distance = 14) Lena image as is shown in Fig. 2.7(a), and the restored Lena image is shown in Fig. 2.7(b) which gives MSE = , NM SE = 0.79% and ISNR = 9.7 db An A pproxim ate Factorization Technique The approach of approxim ation factorization using the combined SVD and BA m ethod for the Im plem entation of 2 -D R-filter has been tested using different lightly blurred test images. Fig. 2.8 shows one of them. The im age Airplane in Fig. 2.8(a) was defocused with radius of the COC r = 3 and was contam inated by white Gaussian noise with SNR = 30 db, and the degraded im age is shown in Fig. 2.8(b). The restored Airplane im age (w ith A = ) is shown in Fig. 2.8(c) which gives MSE = 99.54, NM SE = 5.39% and ISNR = 9.52 db. The iterative-im plem entation m ethod has been also used to restore the degraded Airplane image. The result is not presented here since the difference between the images restored by the two proposed m ethods is so sm all that it is difficult to observe visually. The sm all difference obtained with the two m ethods suggests a practical applicability of low order recursive filters in image restoration instead of using the conventional D F T technique when im ages are not severely blurred C onclusions Our developm ent has been concentrated on designing 2-D R-filters for restoring degraded im ages. T he 2-D R-filters are derived in the frequency dom ain w ith an adju stab le regularization param eter controlling the balance of sm oothness and fidelity

80 52 of the restored image. Two techniques, namely the iterative and approximate- factorization techniques, are used in im plementing the R-filters. The iterative technique is standard and can be im plemented using the DFT. The approximate- factorization technique provides two low-order recursive filters with identical coefficients which can be im plem ented with a small number of operations per pixel.

81 53 (c) (d) Figure 2.5: Comparison between R-filtered and W iener filtered im ages, (a) The Lena im age, (b) Defocusing blurred Lena im age with r = 7 units and pseudo w hite Gaussian noise at SNR = 28 db level, (c) Restored image by W iener filter, (d) Restored im age by R-filter with A =

82 54 Irnag R e sto r a tio n s^, b y R e g u la r iz a tio n ^ ^ %^ Method ** (a) (b) taupff juihuli Image R esto r a tio n Regular izafc ion Method (c) (d) Figure 2.6: Com parison between R-filtered and Wiener filtered images, (a) The im age Text, (b) Defocusing blurred Text im age with r = 5 units and pseudo white Gaussian noise a,t SN R = 33 db level, (c) Restored image by Wiener filter, (d) R estored im age by R-filter with A =

83 55 A lgorithm M SE NM SE (%) ISN R (db) W iener filtering R-filtering Table 2.5: Q uantitative comparison between the R-filter and W iener filter (related im ages are shown in Fig. 2.5). A lgorithm M SE NM SE (%) ISN R (db) W iener filtering R-filtering Table 2.6: Q uantitative comparison between the R-filter and W iener filter (related images are shown in Fig. 2.6).

84 56 Figure 2.7: Im age restoration by R-filtering. (a) Linearly blurred Lena im age with L = 14 units and pseudo w hite Gaussian noise at SNR = 25 db level, (b) Restored im age by R-filtering w ith A =

85 57 (c) Figure 2.8: Im age restoration by approximate-factorization R-filtering. (a) The im age Airplane, (b) Defocused Airplane image w ith r = 3 units and pseudo w hite G aussian noise at SN R = 30 db level, (c) Restored im age by approxim ate-factorization R-filtering w ith A =

86 58 C hapter 3 O ptim al D eterm in ation o f R egularization P aram eters and th e Stabilizing O perator 3.1 A n L-C urve A pproach to O ptim al D eterm i n ation o f R egularization P aram eters Introduction As was m entioned in Sec , the m ethod of regularization provides stable solutitflfis to im age restoration problem s w ith a tradeoff between accuracy and sm ooth ness of the solutions. The tradeoff is determined by a regularization parameter. In this section, an L-curve approach to determining this tradeoff is proposed. It is dem onstrated that a regularization parameter corresponding to the largest curvature of th e L-curve gives an optim al regularized solution of a given image restoration problem. Consider as usual the typical im age m odel (see Sec. 1.2) J s H f + n (3.1) Let us denote a regularized solution of (3.1) by f\ = R,\g where R\ is a m atrix

87 59 determined by the regularization scheme lised and A is the regularization parameter. For exam ple, if the Tikhonov regularization scheme [5, 10, 11, 12, 20] is used, then R,\ is given by Rx = {Ht H + \C tc)~1h t (3.2) and f\ = R\g is the solution of the least squares problem m i n ^ / A (7/ 2 (3.3) with C a stabilizing operator. The term \\Hfx g\\ \\H R \g <7 represents the approxim ation error between R\ and H~x which can serve as a m easure of accuracy of the solution / a, and the term C / a is a measure of the smoothness of the solution / a - The stabilizing operator C should act like a highpass filter, which is often chosen to be the discrete Laplacian operator (2.54). The regularization scheme requires a strategy of choosing a A such that both quantities \\Hfx g\\ and C '/\ are small. To achieve a sm all approximation error \\Hfx g\\, A has to be small. However to obtain an f\ w ith sm all \\C/a requires a large A. Therefore a compromise between solution accuracy (the term \\Jffx 511) and smoothness (the term C / a ) m ust be made. Concerning the choice of the regularization parameter A, several m ethods have been proposed, see for exam p le reference [52] and the references cited there. T h e aim of this section is to propose an L-curve approach for the determ ination of A in order to keep both quantities H f g 2 and C '/ 2 as small as possible. The L-curve approach was first proposed in [53, pp ] to solve a class of least-squares problems and was further explored recently in [54, 55] to analyze discrete ill-posed problems. T he rest of this section is an attem pt to apply the L-curve approach to a class of im age restoration problem s.

88 60 T he L-curve in [53, pp ] is defined as the curve ( / / / </, C / ) for 0 < A < oo. For m ost image problems, it is observed that the corresponding L- curves are in general arc-shaped with slowing varying curvature which makes, the num erical determ ination of a point of m axim um curvature difficult. Thus it is difficult to determ ine a corner point of the curve and the corresponding regularization parameter A. However, if norm square is used, the graph ( / / / </ 2, C / 2) for 0 < A < oo, always displays an L-shaped curve with a sharp corner point with large curvature which is identifiable numerically. In fact, this phenomenon can be justified m athem atically by m aking use the block circulant property of matrices II and C, and the proof will be presented in Sec M ore A bout Tikhonov R egularization To solve the im age restoration problem (3.1), the following two formulations might be considered : m in 110/11 (3.4a) J subject to \\Hf g\\ < n (3.4b) and m in j t f / - s (3.5a) subject to \\Cf\\ < B (3.5b) where B is a given bound on the smoothness requirement. The theory of Lagrangian m ultipliers leads us to the following compromise [14] between methods (3.4) and (3.5): min / / / - S + M! C 7! (.1.6)

89 61 The solution of problem (3.6) is given by / = (.Ht H + ^ C TC)-xHTg (3.7) } which is called the Tikhonov-M iller solution of (3.6) in the literature. The Tikhonov- M iller solution suggests that an optim al choice of A is n j/j3. According to Hunt [9], an optim al choice of A is the one which satisfies \\Hfx-g\\ = M (3.8) In H unt s original setting, the problem of solving (3.1) is replaced by the m inim ization problem n \\cf\\2 (3-9a) subject to \\Hf g\\2 = n 2 (3.9b) which is the well known constrained least-squares (CLS) m ethod. Since \\C/ < B and H f </ = jrc, we obtain 11^/ - sll2 + ^ I C / I I 2 < INI2 + INI2 = 2INI2 (3.10) This im plies that at most a factor of y/2 is lost in an approximate solution of (3.1) if the form ulation (3.6) is used instead of (3.4) or m ethod (3.5) [14]. As B is unknown in m any cases, a regularized solu tion of (3.1) is typically form ulated as the solu tion of th e generalized Tikhonov problem (3.3). T h e standard approach is again th e use of th e Lagrangian m ethod, and the solution is /a = (Ht H + XCTC)~lHTg (3.11)

90 As was pointed out in Sec. 1.2, the matrices H and C are block circulant matrix w ith circulant blocks. An im portant property of block circulant m atrices is that all block circulant m atrices (w ith the same size) have the same set of eigenvectors. This leads to an efficient com putation of f\ in (3.11) using the D FT. To be specific, there exists an unitary m atrix U such that any block circulant m atrix A can be expressed as A = UAU* (3.12) where A is a diagonal m atrix determined by the eigenvalues of A and * denotes the operation of com plex conjugate transposition. It can be shown [9] that this U acts like a D F T and consequently solution (3.11) can be obtained using the fast D F T D eterm ination o f the Regularization Param eters via th e L-Curve Approach It follows from (3.12) that H = U \ hu \ C = UACU* (3.13) where A # = diag{ } and Ac = diag{c,}. In other words, {/».,} and {c,} are the eigenvalues of H and C. Hence solution (3.11) can be expressed as / A = U(A*hAh + XA*cAc)~1A*hU*(j (3,14) B y using (3.14), it is found that l i C ' M I 2 = E l A. 2 j A c, f ^ i -y)*2 (, U 5 ) \ r 2 where u, is the ith cotllmn of U. Eqns. (3.15) and (3.16) imply that

91 63 xlo \\Cfx\\ Hfx -.9II2 Figure 3.1: An L-curve. and Tx? «T a m K 91 (3'18) Hence HC/aII2 m onotonically decreases with respect to A while \\Hfx <7 2 m onoton ically increases with respect to A. As a result, if we view A as a parameter varying from 0 to oo, the mapping H f \ g\\2 > C / a 2 defines a curve in the ( \ \ H f x 9 2, C / a 2 ) plane. We call this curve the L-curve associated with the optim ization problem (3.3). Further notice from eqns. (3.15)-(3.18) that for small values of A, both ///,\ </ 2 and its derivative are small while both \\Cf\ 2 and its derivative are relatively large,

92 this im plies that dwchu* d\\chn<t\ d\\uh-s\\2 i\\h h-srids ' 1 is always negative on an interval 0 < \\Hh - g f < 6, where &i is a certain small number, and hence the L-curve has a sharp decline there. In other words, the left; portion of the L-curve is nearly a vertical line and hence its curvature1 is small. Obviously, any point on this portion of the L-curve corresponds to a /,\ with good solution accuracy (i.e. small \\Hf\ g\\2) but poor sm oothness (i.e. large C '/,\ 2). Furthermore, for the large values of A, eqns. (3.15)-(3.18) indicate that both <7/a 2 and its derivative are sm all while Hf\ g\\2 tends to be independent of A. Fqn. (3.19) now im plies that the right portion of the L-curve corresponding to large A (and thus large \\Hf\ g 2) is nearly a horizontal line and hence its curvature is also sm all. Obviously, any point on this portion of the L-curve corresponds to a sm ooth f\ w ith poor solution accuracy since Hf\ g 2 is large. Moreover, from (3.15)-(3.18) it can readily be shown that the second-order derivative of \\C/,\ 2 with respect to \\Hf\ g 2 is always positive meaning that the L-curve is globally convex. Based on the observations m ade above, we conclude that the shape of the L-curve w ill look like a letter L as shown in Fig. 3.1 with the optim al A corresponding to the corner point c on the curve. Since the maximum curvature will occur at the corner c, we propose an easy-to-im plem ent approach to find the numerical value of this A as follows. First, a number of points ( H f\ </ 2, C'/a 2) with A varying over a wide range are calculated. Then an interpolation method such as the least xthe curvature of a curve y f{x) at the point (x,y) is defined as the number

93 65 squares m ethod is used to obtain a rational function that approximates the L-curve. N ext the curvature of this rational function is computed to determ ine the best value o f A that corresponds to the L-curve s m axim um curvature. To dem onstrate the proposed approach, we apply it to two sampled images which are degraded versions of im age Lena by a linear m otion blur and a defocusing, respectively. The image Lena (Fig. 2.3(a)) was linearly blurred with 14 blurring distance units and was contam inated by pseudo-white Gaussian noise with SN R = 30 db, and the blurred im age is shown in Fig. 3.2(a). The corresponding L-curve obtained using an interpolation with 24 (\\Hfx g 2, C /a 2) points is depicted in Fig It-w as found that A = 0.01 corresponds to the m axim um value of the curvature. Using this value of A in (3.11), the restored Lena im age is shown in Fig. 3.2(b) and it has a fairly high improvement signal-to-noise ratio (ISNR) value (ISNR = 10.5 db). Here the ISNR is defined by (2.52). Fig. 3.3(a) shows a defocused Lena im age with radius of the COC r = 7 which was contam inated by pseudo v " ite Gaussian noise with SNR = 28 db. By using the proposed L-curve approach, A = was obtained. Using this value of A in (3.11), the restored Lena im age is shown in Fig. 3.3(b) and it has an ISNR value equal to 11.0 db A n Adaptive R estoration M ethod for Linear M otion Blurred Im ages In Tikhonov regularization, the A is chosen as a compromise between accuracy and sm oothness and the same A is used for the whole input image. However, the value of A so chosen tends to be larger than necessary for background regions. This is because a sm all value of A used in the restoration process leads to a noisy restored im age in general. As a result, isolated edges lying in a uniform zone of the im age may

94 66 (a) (b) Figure 3.2: (a) Linear m otion blurred Lena image with L = 14 units and pseudo w hite Gaussian noise at SNR = 30 db level, (b) Restored Lena im age with A = 0.01 obtain ed by L-curve approach, ISN R = Figure 3.3: (a) Defocused Lena image with r = 7 units and pseudo white Gaussian noise at SNR = 28 db level, (b) Restored Lena image with A = obtained by L-curve approach, IS N R = 11.0.

95 67 be sm oothed out. A possible remedy for this problem is to localize the rest,oration problem so that A becomes a region-dependent local parameter. For exam ple, if the linear blur is 1-D, say in the vertical direction, then the 2-D image model (3.1) can be reduced to 9j = Hfj + nj (3.21) where gj and fj represent the j th column of the recorded and the original images, respectively; H is a circulant (not block circulant) m atrix corresponding to the I - 1) invariant linear blur, assumed to be known; and rij is the additive noise. For a given Aj, the Tikhonov solution of (3.21), i.e. = (Ht H + A,CTC y lu Tm (3.22) can be obtained efficiently using 1-D DFT. For different columns, the optim al Aj s chosen by the proposed L-curve approach may be different, e.g., for the column with high SNR, the corresponding A j should be small and for the column with small SNR, the corresponding Aj should be large. Therefore the restored im age can be recovered colum n by colum n adaptively with respect to A j. It is also noted that the m odel (3.21) is com putationally more efficient in implementation than its 2-D counterpart (3.1). Fig. 3.4(a) shows the image Tiffany with a synthetic horizontal line segment on the top left corner. It is used to exam ine the performance of the proposed 1-D adaptive restoration algorithm in which the L-curve approach described in Sec is incorporated. The Tiffany ima,ge was linearly blurred in the vertical direction with, 9 shifting units and was further contam inated by pseudo-white noise with SNR = 40 db. T he resulted noisy blurred Tiffany image is shown in Fig. 3.4(b). For each colum n o f g, a Aj was determ ined using the L-curve approach. It was found that

96 6 8 A j 0.01 for j = 1 to 30 and the other Aj s vary over the range [0.2, 0.4]. Four Aj values, namely 0.1,0.25,0.3 and 0.35 were employed in the 1-D adaptive Tikhonov regularization restoration. The adaptive threshold was set as follows. Let SNR^ be the SN R of the whole im age g and let SNRj be the SNR of the j th column of g. The Aj in the jth column restoration was chosen as follows 0.01 j = l,---,30 Aj = < 0.25 SNRj < (1 A;)SNRtu 0.3 (1 - k)snr < SNRj < (1 + Ar)SNRu, 0.35 SNRj > (1 + fc)snru, where k = 0.1 representing 10% standard deviation of SNRW. The restored im age is shown in Fig. 3.4(c). For comparison purpose, the 1-D conventional Tikhonov regularization w ith A = 0.3 is also applied to the sam e im age and the restored im age is shown in Fig. 3.4(d). It is clear from Fig. 3.4(c) and (d) that the synthetic line segm ent is better restored by the adaptive restoration m ethod C onclusions We have introduced an L-curve approach to obtain an op tim al choice of the regularization parameter A in the Tikhonov regularization. The L-curve is defined as the graph (( Hf\ g 2, C '/a!) the norm square of the residue versus the norm square of the sm oothness of the solution. The optim al A is the one corresponding to the largest curvature of the L-curve. The display of the L-curve not only leads to a determ ination o f an o p tim al A, it also helps to understand the Tikhonov regularization theory and its numerical treatm ent of discrete ill-posed image restoration problem s.

97 69 m& (c) (d) Figure 3.4: (a) T h e Tiffany im age with a synthetic line segm ent, (b) Linear m o tion blurred Tiffany im age with L = 9 units and pseudo white Gaussian noise at SN R = 40 db level, (c) Restored Tiffany image using adaptive approach, (d) R estored Tiffany im age using nonadaptive approach.

98 70 In addition, an adaptive restoration algorithm using a 1-D R-fiitering technique with the L-curve approach in determ ining the adaptive regularization parameter A is proposed for restoration of linear-motion-blurred images. Experim ental results depicted in Fig. 3.4 have shown that the proposed adaptive restoration algorithm leads to a significant im provem ent over the non-adaptive (conventional) algorithm. 3.2 O ther O ptim al A pproaches to th e D eterm i nation o f th e R egularization P aram eters Introduction Again consider the discrete m odel of the imaging system (3.1) together with the regularization-related m inim ization problem (3.3) which is rew ritten here nun ^ / - ^ 2 + A C'/ 2 (3.23) for convenience. The operator C is chosen such that the m inim ization incorporates som e a priori information about the original image into the regularization problem. Conventionally, C is always chosen as the discrete Laplacian operator (2.54) m eaning that sm oothness is im posed on the estim ated images. We shall develop two optim ization strategies, according to the M SE and ISNR criteria respectively, to determ ine an o p tim a l A for the Tikhonov regularization m ethod. Each of the strategies optim izes a criterion J{f\) subject to constraints on the solution f\. Although the optim ization strategies of choosing a regularization parameter are developed for the Tikhonov regularization m ethod in this section, they can be applied to any other regularization m ethods as well. We firstly form ulate th e problem of selecting an optim al param eter A as an

99 7.1 optim ization problem min J ( A) (3.24) Then we dem onstrate how the most commonly used constrained least-squares (C.LS) m ethod can be formulated as the optim ization problem (3.24). Formulation of the generalized cross-validation (GCV) m ethod as an optim ization problem will also be discussed. Finally, we propose two new J(A) defined by the MSE and 1SNR respectively O p tim ization Form ulations A natural choice of J(A) in (3.24) is th e Tikhonov regularization quadratic functional J(\) = \\Hfx - g f + \\\C h f (3.25) Sub stitu tin g (3.1) into (3.11), we have fx = (Ht H + \C TC)~lHT(Hf + n) (3.26) which indicates that the restoration error / A / depends on the noise level n. Note that \\h ~ f\\ = ^ 5 - / < \\Rxg R\Hf\\ + \\R\Hf f\\ < \\Rx\\\\g -E f\\ + \\RxH f - f \ \ < p A n + ^ / / / - / (3.27) So that the restoration error / A / possesses an upper bound that depends on n. From (3.27), we see that if n tends to zero then fx converges to / as A > 0.

100 72 This suggests that a suboptim al A can be identified by finding the best compromise between the term s./7,\ n and R\H f /. Since l i m p A/ f / - / = 0 (3.28) and lim E A n = o (3.29) we see that A plays the role of a tradeoff parameter between ( R \H f / ) which represents the solution accuracy and (H-RaMMI) which gives a m easure of solution stability. This m eans that we should choose a parameter A which m inim izes the functional J ( A ) = f i j / f / - / + A fla n (3.30) or sim ilarly the quadratic functional J(X) = \\RxHf - f f + A llf t lp lln f (3.31) which is a functional easier to handle than (3.30). N ote that functionals (3.30) and (3.31) can only be evaluated when / is known. In practice, it is difficult to determ ine A via (3.30) and (3.31). However, they suggest reasonable form ulations of the functional J. R em arks (1) N ote that the form ulations of (3.25) and (3.31) are similar but the m eaning of solution accuracy in each expression is somewhat different. The solution accuracy in (3.25) is referred to the accuracy of Hf\ as an estim ate of </, whereas the accuracy in (3.31) is referred to the accuracy of f\ as an estim ate of /.

101 73 (2) In practice, the Tikhonov regularization quadratic functional (3.25) can be used directly to obtain an optim al A, since H, C and g in the formula are all known. (3) T h e functional (3.31) can only be used to obtain a suboptim al A by the follow ing steps: Step 1. Obtain a reasonably good estim ate / of /. This can always be achieved using one of the regularization m ethods described in Chapter 2, e.g., the CLS m ethod, W iener filtering or modified Wiener filtering. Step 2. E stim ate the variance a2 of the noise process n. Form g = II f + n where h is w hite noise with variance a2. Step 3. Obtain an optim al A* by m inim izing Step 4. The A* obtained in Step 3 can then be taken as a suboptimal A in the Tikhonov regularization of th e original im age problem (3.1) T he C onstrained Least-Squares M ethod For a Tikhonov estim ate f\ of the solution of problem (3.1), the residual sum of squares is defined as R S S (/a) = \ \ H h - n f The CLS m ethod finds a A which satisfies ussy*) = IMI2

102 74 where ra 2 has to be estim ated at first. In many applications, the noise process n in (3.1) is assumed to be a zero-mean white Gaussian process with variance cr2, in which case n is multivariate normally distributed w ith zero m ean and covariance m atrix a21. In statistical notation we denote the noise process n by n ~ A'(0, a21) (3.36) It then follows that [56] ~ X2(JV2) (3.37) in which X2(/V2) is the chi-square distribution with TV2 degrees of freedom, where TV x TV is the size of the im age g. Then we have [ M 2] = 2N 2 (3.38) where E denotes the m athem atical expectation. As a2 can be estim ated in uniform zones o f g, (3.38) provides a m ethod of estim ating?i 2. Finally, we note that the CLS m eth od can be form ulated as the optim ization problem (3.24) w ith J(A) = (R SS(/a) - n 2)2 (3.39) T he G eneralized Cross-V alidation M eth od The generalized cross-validation (GCV) m ethod was first proposed by Golub et. al. [57] in obtaining a satisfactory solution for a linear regression m odel. If the noise process n in (3.1) is a zero-mean white Gaussian process, then we m ay consider the im age m odel (3.1) as a linear regression m odel as defined in [57]. S tatistical m otivation for using the GCV m ethod can also be found in the sam e paper and th e references cited there. A pplication o f the G CV m ethod in regularized im age

103 75 restoration has been initiated in [52, 58]. The GCV m ethod for determ ining an optim al value of A is to com pute the minimum of the GCV functional which, up to a constant factor, is defined as GCV(A) = \\H h-g\\2 [trace { / /v,\}]2 R S S (/a) [trace {I /C \}]2 (3.40) where K x = H(Hr H + A CtC)~xH t (3.41) Im plem entation of GCV(A) necessitates the calculation of the trace of the matrix I K \ which can not be obtained directly as the matrices involved are of large dimension. Let {/*.,} and {c,} be the eigenvalues of H and C, respectively. It can be shown that [52] (tra ce)/ ~ ^ <* «> Su b stitu tin g (3.42) into (3.40), we obtain R S S(A ) G C V (^) - 7 jy r ( ik-p. 3na ) Under the assum ption that the matrices H and C are block circulant, {/;,,} and {c,} can be obtained by the D F T s of the fundamental matrices [hc] and [ce], respectively. T his gives an efficient im plem entation of the function GCV(A). Finally, by Jetting J(A) = GCV(A), the GCV problem becomes a special case of (3.24).

104 The MSE and ISN R M ethods Restored images are usually evaluated by measures such as MSE or ISN R which are defined in Chapter 2. It is worthwhile to note that the commonly used W iener filters, Tikhonov filters, and their variations are related to the MSE m easure in a natural way. As a m atter of fact, T he W iener filter is the restoration filter that m inim izes the expectation of the MSE between the original / and its estim ate /. T he Tikhonov filter is developed from least-squares Lagrangian methods: find an / such that the square error \\g Hf\\2 is m inim ized subject to som e sm ooth constraints. N ote that l l s - i f / l l = \\Hf + n - Hf\\ < \W - Hf'\ + n < W i l l / - / I I + I N I < ] ff \/M S E + n (3.44) and we m ay consider the least-squares measure as an equivalent m easure of the MSE in this regard. Similarly, modified W iener filters (see Chapter 4), inverse filter, pseudo inverse filter (setting C = / ), CLS filter, and R-filters (see Chapter 2) can also be related to the MSE measure explicitly or im plicitly. It is for this reason that J{\) = M SE = / - jtf (3.45) is an appropriate m easure.

105 77 Another possible m easure is related to the ISNR which is defined, by (2.52) as T C T V T D m i 0 / ( k ) 0 ) 2 ISN E = 10 lo g jo (0,(6) where K x L is the size of the image. Note that ISNR is inversely related to MSE, so one m ay take J M = i s k <W7> as an appropriate m easure. Obviously, these two perform ance m easures can only be adopted w hen / is known, but th e difficulty can be resolved by applying the technique described in the remarks of Sec Experim ental R esults In our experim ental sim ulation, three sam ple images (Lena, Text and Airplane), three different kinds of blurs (defocusing, linear m otion and exponential) with, different blur parameters and noise levels have been used. In what follows, we only provide two exam ples to explain the performance of the various functionals J(A) chosen to obtain an optim al A. The first exam ple is the Airplane im age (Fig.3.5(a)) which was exponentially blurred with r = 5 and a2 1, and was further contam inated by w hite noise with SN R = 30 db (Fig. 3.5(b)). The discrete Laplacian operator (2.54) was used as the stabilizing operator. The values of A determined, by CLS, GCV, M SE and ISNR m ethods are 0.012, , and , respectively. N ote that the M SE and ISN R m ethods provide approximately the same value of A. The restored im ages are shown in Fig. 3.5 (c), (d) and (e). The second exam ple is the Lena im age. Linear m otion blur in the vertical direction was sim ulated with L 11 units. W hite noise was added to a level of SNR = 20 db. Similarly, the

106 78 discrete Laplacian operator was used as the generating stabilizing operator in this exam ple. The Lena, the degraded Lena and the restored Lena images are shown in Fig T he quantitative restoration results of these exam ples using ISN R and MSE measures are summarized in Tables 3.1 and 3.2. Of course, the ISNR and MSE m ethods provide the optim al A with respect to the performance measures ISNR and MSE, respectively. N ote that the CLS m ethod often overestim ates the value of A. As a result, the respective restored image is oversmooth and lacks fidelity. The exam ples also dem onstrate that the GCV m ethod leads to good (nearly optim al) ISNR and MSE values Conclusions Various m ethods of choosing a good regularization parameter A are studied, of which two approaches are derived directly from the basic idea of regularization, nam ely minim ization of a solution-accuracy-and-sm oothness measure and m inim ization of a solution-accuracy-and-stability measure. The well-known CLS and GCV m ethods are reviewed and used for comparison purposes. In addition, we have proposed the M SE and ISN R m easures. T hese six measures are com pared in an exten sive sim ulation study using the Lena, Airplane and Text images with different noise levels as testin g im ages. R egularization performances in term s of M SE and ISN R m easures produce the best results. Although the GCV m ethod performs well usually, it som etim es produces a negative or an underestim ated value of the regularization parameter. The CLS m ethod tends to overestim ate the value of the regularization param eter and consequently produces sm ooth but blurred im ages. In order to con

107 79 trol this section to a m anageable size, only partial results of the last four m ethods based on the im ages Lena and Airplane are reported. 3.3 A C om m ent on th e Stabilizing O perator G enerated by D iscrete Laplacian O perator The outcom e of the regularized restoration process depends not only on the choice of the regularization parameter A. As the smoothness C/,\I of the solution f,\ is m easured by the stabilizing operator, the role of the stabilizing operator should not; be overlooked. In this section, an attem pt is made to improve the performance of a regularization procedure by choosing a more suitable stabilizing operator C along w ith a suitable regularization param eter A. T he idea of including the term A C / 2 in (3.23) to control the sm oothness of the solution / can be explained analytically as follows. First observe that a solution / of (3.23) m ust satisfy the normal equations (Ht H + A CTC)f = H'r g (3.48) Let r(a ) = A A -1 (3.49) be the condition number of A [42]. If F(A) is large, then A ~ v is sensitive to small perturbations in A, and the problem of inverting A is ill conditioned. For a given A > 0, the idea of regularization of the ill conditioned problem (3.1) is to choose a m atrix C such that the linear system (3.48) is no longer ill-conditioned. This will be the case if T(Ht H + ACTC) < F (H) (3.50)

108 80 (d) (e) Figure 3.5: D eterm ination of th e regularization param eter, (a) O riginal im age Airplane. (b) Exponential blur (r = 5 and a1 1) with noise at SN R = 30 db level. (c) R estored im age by CLS m ethod (A = 0.012). T he im age is overly sm oothed. (d) Restored im age by GCV m ethod (A = ). (e) Restored im age by M SE or ISNR m ethod (A ).

109 81 (d) (e) Figure 3.6: D eterm ination of the regularization parameter, (a) The Lena im age, (b) Linear m otion blur (L = 11) with noise at SNR = 20 db level, (c) Restored image by CLS m ethod (A = 0.088). The image is oversmoothed, (d) Restored image by GCV m ethod (A = 0.02). (e) Restored image by MSE or ISNR m ethod (A = 0.025).

110 8 2 M ethod A MSE ISNR (db) CLS G CV MSE ISN R Table 3.1: Restoration results for exponentially blurrcd Airplane im age with 30 db noise. M ethod A MSE ISN R (db) CLS GCV M SE ISN R Table 3.2: Restoration results for linear m otion blurred Lena im age w ith 20 db noise.

111 Here both m atrices H and C are block circulant, and hence they have the sam e set of eigenvectors. It can be shown that [19] max(/i- - - Ac?) I ll1 II \ mm (hf + A cf) (3.51) K where {ft,} and {c,} are the eigenvalues of II and C, respectively (see Sec ). In order to satisfy (3.50), (3.51) suggests that C should be chosen such that if hf is large, hf + Ac? is approximately the sam e as hf, and if hf small, hf + Ac? is significantly different from zero. If the sequence hf is decreasing, the cf should be chosen as an increasing sequence. The {/*,} and {c,} are the D F T s of the fundamental matrices [he] and [ce], respectively [9], and that II represents in general a lowpass filter, as a result C should be chosen as a highpass filter. This explains why the discrete Laplacian operator (2.54), which is a highpass filter of order (2, 2), has been used in regularization im age restoration. However, it does not, necessarily mean that, it is an optim al choice, as can be seen from an examination on the isotropic properly (will be defined soon) of the continuous Laplacian operator (d2/dx 2 -f- d2/d y 2). In im age processing, we always want to sharpen blurred features such as edges and lines that m ay occur in any direction. Therefore we want our differential opera,- tors to be isotropic, i.e., rotation-invariant (in the sense that the rotation and the differentiation are interchangeable). It is easy to see that the Laplacian operator (d2/d x 2 + d2/d y 2) is isotropic. Moreover, it can be shown that an isotropic linear differential operator involves only even-order derivatives. The Laplacian operator is the sim plest possible isotropic operator consisting of even-order derivatives. Recall that the discrete Laplacian operator (2.54) is only sym m etric for low frequencies (see Sec ), and hence it is not a good approximation to the Laplacian operator. There are other ways to construct a digital Laplacian, a discrete approxim ation to

112 8 4 the Laplacian operator. For exam ple, one can use different sizes of neighborhood (e.g., the 3 x 3 neighborhood of consisting point (k, I) and its 8 horizontal, vertical, and diagonal neighboring points), or use a weighted average over a specific neighborhood. Another exam ple is the m th order binomial filter [59]. W ith m = 2, the 2nd-order binom ial filter is described by its im pulse response (3.52) Its frequency response is given by rl/ \, C O S ^ -td j) COs(wi+W 2) X, (u>j, U>2 ) = cos 0*1 -I- COS u> ( u? 4-LJ2'i 4- ^ _*"a ; 2 ) 2, 1 ^ 2 I nil.. 2 I,, 2 \ 3 + ^ + 0 ( ^ 1 + ^ ) (3.53) 24 (W l r 2 / + and the am plitude response of V is shown in Fig 3.7. At high frequencies, the am plitude response of V as well as L (2.54) exhibits considerable deviation from the ideal Laplacian. But by comparing the contours of L in (2.54) w ith that of L', we conclude that L' is slightly more sym m etric than L. A good approximation to the ideal Laplacian can be obtained when the order of the binom ial filter is sufficiently large. T heoretically, th e larger the m ask used in designing the digital Laplacian operator, the better the resulting approximation. Since our im age system is assumed linear shift-invariant and both H and C are block circulant, the restored im age can be com puted through D F T im plem entation. This im plies that the size of the generating stabilizing operator Cs can be as large as N x N, the size of the recorded

113 0 A 40 (a) 60 SO SO Figure 3.7: (a) A m plitude response of the discrete Laplacian operator V. (b) T he corresponding contour plot of (a).

114 86 im age </, without any additional com putation cost. It follows im m ediately that the N X TV discrete Laplacian LN defined im plicitly as D F T ( n ) = (( N - 1 T (( Ar - 1 )lr) ( «) (3.54) which is the TV xvv uniform sampling of the continuous Laplacian am plitude response should be the m ost suitable discrete Laplacian operator as a generating stabilizing operator. N ote that since restored images are obtained Uirough the D FT, an explicit LN is unnecessary. An extensive exam ination was performed on the restoration performance using different test im ages with different blurs and noise levels has been studied. It is found that restoration performance measures are practically independent of choices of digital Laplacians as generating stabilizing operator. T his provides strong support for using the sim plest discrete Laplacian operator (2.54) in regularized im age restoration. 3.4 Sim ultaneous O ptim al D eterm in ation o f th e Stabilizin g O perator and R egularization P a ram eters In regularized im age restoration, th e stabilizing operator has already been determined before the regularization parameter is chosen, and the determ ination of the stabilizing operator is usually m ade on the basis of a priori knowledge of the im age, the blur and the noise process. As was discussed in the previous section, discrete Laplacian operators are often suitable choices for stabilizing operators. In this section, the regularization parameter A is treated as a positive variable and the stabilizing operator C a variable m atrix. For a given C, the Tikhonov regularization

115 S7 algorithm gives reasonable approximated solutions {/(A, C) : A > 0} where /(A, C) = (HTH + A CTC)-lB Tg (3.55) W hen C is the discrete Laplacian, f(\,c) is the same as f\, the notation we used to denote the Tikhonov regularization solution previously. For the case for which C is fixed, m ethods have been proposed in Secs. 3.1 and 3.2 for choosing optim al regularization param eters. In what follow s, an op tim ization m ethod is presented for obtaining optimal, estim ates for both the fegularization parameter A and the stabilizing operator G simultaneously. Let 0 be the param eter vector that specifies the generating stabilizing operator Cs of C. For exam ple, when Cs is a 3 x 3 mask, then 0 = [0 \ 02 0<>] is the lexicographically ordered vector of $3 C = Thus, the stabilizing operator C is described in a parametric form as C(0). The objective function J(A) used before to identify an optim al A needs to be written as J(\,C(0)) to explicitly indicate our intention of seeking a vector [A* 0*] that m inim izes «7(A,C(0)). T he objective functions J(\,C(0)) corresponding to various perform ance m easures are listed as follows: (1) A ccuracy and Sm oothness M easure J(KC(6)) = ff/(a,c («)) - g f + A C («)/(A,C (»)) i'

116 8 8 (2) Accuracy and Stability Measure J(A,C(«)) = P(A,C (0))tf/(A,C (0)) -/(A,C (0 )) 2 + A p(a,c (0)) J M J (3.58) (3) CLS M easure J(A,C(«)) = (RSS(/(A,C(«))) - H 2)2 (3.59) (4) GCV M easure J(A,G (0)) = R S S (/(A,C (0 ))) tra c e {/ I<(\,C{0))} (3.60) (5) M SE M easure /(A, C (0)) = M SE(A,C (0)) (3.61) (6) ISN R M easure ISNR(A, C{6)) (3.62) Since J(A, C(0)) is a nonlinear function of [A 0] with m inim a that cannot b e deter- mined analytically in general, numerical optim ization techniques m ust be used for determ ining [A* 0*]. The test im age is the im age Lena. Distortion due to defocusing was sim ulated with radius of COC, r 7 units. Noise was added, resulting in SN R = 28 db. The blurred Lena im age is shown in Fig 3.8(a). Three different J(\,C(6 )), nam ely CLS, M SE and ISNR were applied to obtain the optim al values of A and C. The Broyden-Fletcher-Goldfarb-Shannon (BFGS) optim ization algorithm [60] was used in each case. T he generating stabilizing operator Cs was assumed to be size 3x3,

117 89 and of the following form a b a Cs = b o b a b a with constraint 4a -f 46 -f c = 0. The c rimal values (A*,Cs) corresponding to the CLS, M SE and ISN R m easures are given below. CLS Measure: A = Cs (= Lt) (3.64) M SE Measure: A = , Cs ( = L2) = (3.65) ISN R Measure: A = , Cs ( = L3) = (3.66) The restored im ages are shown in Fig. 3.8(b), (c) and (d), respectively, arid the num erical results are sum m arized in Table 3.3.

118 9 0 M ethod A Cs MSE ISNR (db) CLS Lx M SE l ISN R l CLS with Laplacian (2.54) G C V with Laplacian (2.54) Table 3.3: R estoration results for defocused (r = 7) Lena im age w ith 28 db noise. We have extended all six m ethods of choosing regularization parameters to m ethods of choosing regularization parameters and stabilizing operators sim ultaneously. Sim ulation experim ents indicate that determ ination of a suitable regularization parameter is most important in obtaining a good restoration result when digital Laplacians are used as generating stabilizing operators. 3.5 A M ultip le-p aram eter G eneralization o f th e T ikhonov R egularization M eth od Introduction The Tikhonov regularization approach has long been utilized for restoring im ages that are contam inated by noise and are blurred due for exam ple to camera defocusing or linear m otion. It is posed as a least-squares approximation problem in the I? space that provides a parameterized tradeoff between accuracy and sm oothness of the restored image, with the tradeoff being controlled by a scalar regularization

119 Figure 3.8: (a) Defocused (r = 7) Lena image with noise added at SNR = 28 db level. Restored im age by (b) CLS m ethod, (c) MSE m ethod and (d) ISNR m ethod. 91

120 9 2 parameter. In this section the Tikhonov m ethod is generalized by incorporating m ultiple regularization param eters into the regularization process. As th e regularization parameters can be chosen based on a frequency- dom ain criterion to better balance the approxim ation accuracy and solution smoothness, the proposed m ethod leads to improved restoration results as compared to the conventional regularization methods. As usual, the degradation m odel is described by (3.1). It is also assumed that H is constructed as a block circulant m atrix (see Sec. 1.2). An important property of a block circulant m atrix is that it can be diagonalized by the 2-D D FT [9]. Conventional m ethods for determ ining solutions of (3.1) can be found in [1, 2, 3, 4, 20]. M ethods developed recently by the author can be found in [25, 26, 61, 62]. T h e standard regularization is to m inim ize the ob jective function A(f,g) + \S(J) (3.67) where A(f,g) is a measure of accuracy of /, S(f) is a measure of sm oothness of /, and the scalar A > 0 is a regularization or sm oothing parameter. The role of A is to compromise between A(f,g) and S(f). From (3.1), the Tikhonov regularization approach specifies (3.67) as the quadratic functional \\Hf-g\\> + \\\Cf\\2 (3.68) where C is a stabilizing operator which should act like a highpass filter and is assumed to be a block circulant. The stabilizing operator C is usually chosen to be the Laplacian operator (2.54). For each A > 0, the necessary condition for (3.68) to have a m inim um is that the gradient of (3.68) with respect to / is zero. This leads to the solution o f (3.1) f\ = (H t H + A CTC)~xH Tg (3.69)

121 M ethods concerning an optim al choice of the regularization parameter A have been introduced in Secs. 3.1 and 3.2. In a regularization process, A is chosen as a compromise between accuracy and sm oothness and the sam e A is used for the whole input image. Tire value of A so chosen tends to be larger than necessary for background regions. As a result, significant features lying in a uniform zone of the im age may be smoothed out. A possible rem edy for this problem is to generalize the regularization m ethod by introducing m ultiple regularization parameters into the regularization process. As will be dem onstrated in this section, by using a multiple-parameter-regularization scheme, a better balance between the approximation accuracy and solution sm oothness can be obtained locally A G eneralization of the Tikhonov Regularization M ethod In th is section, th e objective function (3.68) is generalized to the form IIH f - 9 f + \\LCf\\2 (3.70) where L is a block circulant m atrix of regularization parameters. Note that when L = A2 / w ith I denoting the identity m atrix, (3.70) is reduced to (3.68), and the operator L in (3.70) acts like the scalar parameter A in (3.68). More importantly, it will be shown shortly that an appropriate assignment of a nontrivial L leads to a solution of (3.70) with improved restoration quality. Such a parameter matrix L is characterized by the squares of its eigenvalues, that gives a feasible way of choosing an optim al L in the sense that the accuracy and sm oothness of the restored image are well balanced. Let fi be an estim ate of the solution to (3.1) obtained by m inim izing (3.70),

122 9 4 then it is straightforward to verify that fa = (Ht H + CTLTLC)-1HTg (3.71) For each L, define an operator Rl as Rl = (Ht H + CTLTLC)~lHT (3.72) It can be shown that [5] lim RLH f = f (3.73) where j X /r denotes the Frobenius norm of L, i.e. therefore Jl is a regularized solution of (3.1). I f = ( $ ) * (3-74) «i N ote that the fast 2-D DFT can be em ployed to generate fa. It is shown [9] that there exists an unitary m atrix U such that any block circulant m atrix A can be expressed as (see (3.12)) A = UAU* (3.75) where A is a diagonal m atrix determ ined by the eigenvalues of A operation of com plex conjugate transposition. Since m atrices H, and * denotes the C and L are block circulant, it follows from (3.75) that H = U \ hu \ C UACU*, L = UAlU* (3.76) where Ah = d iag{/i,}, i\c = diag{c,} and A/, = d iag{/,} are the diagonal m atrices with eigenvalues of i f, C and L placed along the diagonal, respectively. Let [fte], [ce] and [/e] be the fundam ental blocks (see [9] and Sec. 1.2) of m atrices H, C and L, respectively, then Ah, Ac and Al can be obtained efficiently by taking the 2-D

123 95 D FT of [/ie], [ce\ and [/e], respectively. Making use of (3.76), (3.71) can be written as fl = C7(A*h A + (3.77) i.e. U'fL = + \'c A}AL\ G)-l\*HU*g (3.7S) It can be shown [9] that U*fi and U*g in (3.78) are the 2-D DFT of //, and g, respectively. H ence, in th e frequency dom ain (3.78) im plies that = i ^ { W } i 2 + i ^ { [ y ) r v { [ c. j } i 2 n, ' ) (3,79) where F denotes the operation of 2-D DFT, and (3.79) gives an efficient; im plem entation of (3.71). B y (3.77), we have ^ =? f c i 2 + /< 2 ci 2^ U< ^ ' 80^ where u, is the zth colum n of U. As a remark, if we let /, 2 = constant = A for all if then Jl is reduced to the conventional Tikhonov solution (3.69). N ote that {/»,}, {c,} and {/,} are actually the frequency responses of [he], [ce] and [/e], respectively. As the blur operator [he] usually represents a lowpass filter while [ce] represents a highpass filter, /i, 2 + c, 2 > 0 always holds in practice, i.e., and c, can not be zero sim ultaneously. B y expressing (3.80) in the following form f l =? N 2 + l l l 2M 2 ' h i ^ Ui the term /, 2 c, 2 can be regarded as a term that sm ooths the inverse filter l//t,\ It is now clear that the sm aller the value j/t, 2, the larger the value /, 2 c, 2 should be used to regularize the inverse filter.

124 Choice of M ultiple Regularization Param eters A m ethod of determ ining an optim al frequency domain regularization m atrix A^A/, is proposed as follows. First, one obtains a regularization param eter A, that is op timal in a certain sense for the conventional Tikhonov regularization. The constrained least squares m ethod [9] or the L-curve m ethod [54] [55] (also see Secs. 3.1 and 3.2), for exam ple, might be used for this purpose. N ext, let 0 < T\ < T2 < < Tk = max hi = 1. be a set of thresholds and let {/;} be defined as \ h \2 = A, 0 < ht-1 < Ti / A Tk < \h i\< T k+i, * = 1, (3.82) where 0 < p < 1, and the numbers K and p can be determ ined by trial and error. T h e following m ethods of choosing A^A/, are proposed. (1) M = ( A!,il" ' [ o /. > p (2) where 0.8 < p < 1 determ ined by experim entation.,2 = A>(1 ~ fe, 2) 1,1 1 + h, 2 (3) IUY = A. 1 + p\hi\2 where 10 <: p < 102. The theoretical argument for making these choices can be found in [5].

125 97 (a) (b) Figure 3.9: M ultiple-param eter regularized image restoration, (a) Linearly blurred Lena im age w ith L = 14 units and pseudo white Gaussian noise at SN R = 30 db level, (b) R estored im age by m ultiple-param eter restoration. (a) (b) Figure 3.10: M ultiple-param eter regularized image restoration, (a) Defocused Lena image w ith r = 7 and contam inated by noise with SNR = 28 db. (b) Restored image by m ultiple-param eter restoration.

126 98 Algorithm M SE NM SE (%) ISN R (db) Single-param eter M ultiple-param eter Table 3.4: Q u antitative com parison betw een single-param eter and m u ltiple-param eter regularized im age restoration. The sample im age is a linear m otion blur degraded Lena im age. A lgorithm MSE NM SE (%) ISN R (db) Single-param eter M ultiple-param eter Table 3.5: Q u an titative com parison betw een single-param eter and m u ltiple-parameter regularized im age restoration. The sample im age is a defocused and noise-contam inated Lena image.

127 E xperim ental results To dem onstrate the performance of the proposed approach to image restoration problems, we apply it to three sample im ages, of which two are degraded versions of im age Lena by a linear m otion blur and one is by a defocusing blur. The im age Lena (Fig. 2.3(a)) was linearly blurred with 14 blurring distance units and pseudo white Gaussian noise was added with SNR = 30 db, and the degraded image is shown in Fig. 3.9(a). B y using the constrained least-squares (CLS) m ethod [9], it was found that A* = The restored image using the proposed m ethod (with T\ = 0.01 and T2 = 1, p = 0.05) is shown in Fig. 3.9(b). Fig. 3.10(a) shows a defocused Lena im age with radius of the COC r = 7 which was contaminated by pseudo white Gaussian noise with SNR = 28 db. B y using the constrained least squares m ethod, A* = was obtained. The restored im age using the proposed m ethod (with Ti = and T2 = 1, p = 0.6) is shown in Fig. 3.10(b). Q uantitative measures of these two exam ples are sum marized in Tables 3.4 and 3.5, respectively. The third sam ple im age Text (Fig. 2.6(a)) was defocused with radius of the COG r = 5 and was further contam inated with noise at SNR = 33 db (Fig. 3.11(a)). The restored im age (w ith T\ = 0.001, T2 = 0.001, A» = and p 0.6) is shown in Fig. 3.11(b). Corresponding quan titative results are given in Table M u ltip le-p aram eter 2-D R-filters In order to derive the transfer function of the multiple-parameter 2-D R-filter of the im age m odel (3.1), first o f all the associated quadratic cost functional is expressed

128 100 as U f ) = [h(hlth2)*f(hllk2) - g ( k u h2)]2 ki,fc2 ~ co co +A 2 [/(A-i, fc2) * c(^i, fc2) * /(An, Ar2)]2 (3.83) k],a:2 = oo where { /(A:i, At2)} represents the set of regularization parameters. B y Parseval s Theorem, we have a (/) = ^ ^ J_ ex(f,ui,u2)du1du2 (3.84) and e,\( /,W l,u ;2) = \H (u i,u}2)f(& i,u)2) G(uJl,L02)\2 + \L(uji, u2)c(ui, uj2)f(uji, u;2) 2 (3.85) It follows that \ ( / ) is m inimized if e\(j ui,u>2) is m inim ized. Then following the sam e idea developed in Chapter 2, it can be shown that EV,,,, \ A 15 2 \H(uv,u;2)\2+ \L(u)i,u2)\2\C(ui,w2)\2 achieves the m inim um of e\. In th e 2-transform representation, (3.86) represents the frequency response of the transfer function,zn )G(ZU Z2) r 1*1, z2) -, Mr/ -1, IV- _ U /. - l H (2l? 22) /f ( 2 a \ 22 J) + 1 (2!, 22)T (2x 1, 22 1)C («i, 2:2)17(2! \ 22 *) (3.87) and the transfer function of the required multiple-parameter 2-D R-filter is in turn given by R\{zi,z2) = fj(zl,z2\ = H (zil,z2 l)wx(z1,z2) (3.88) C r f2l, 22 j

129 For the im plem entation of multiple-parameter 2-D 11-filters, the iterative and the approxim ate factorization m ethods introduced in Chapter 2 can be similarly applied. The only m odification in the iteration m ethod is to replace the regularization parameter A in eqns. (2.66) to (2.72) by T(a;i, u;2) 2, and the only modification in the approxim ate factorization m ethod is to replace (2.75) by the am plitude response I ( [ i f ( e ^, e ^ ) 2 + \L(e^, ej^)c{e^, ) 2]11' C onclusions In order to have more degrees of freedom to control the fidelity of the restored image, the Tikhonov regularization m ethod is generalized based on the sam e im age model but w ith an extension of one regularization parameter to m ultiple regularization parameters. T he restored image solution is then expressed in a closed form similar to the one-parameter case and can be obtained through the 2-D D F T as before. M ethods of choosing m ultiple regularization parameters have also been presented. Experim ental results indicate the proposed multiple-parameter restoration method does lead to better results than its one-parameter counterpart. It is worth pointing out that the proposed m ultiple-parameter method requires only an extra m ultiplication of two D F T s, in addition to the com putation needed for the single-parameter regularization. Finally, 2-D R-filters for m ultiple parameters are developed based on the sam e idea as introduced in Chapter 2.

130 102 m * * * * * * * - * * * ililll!lillfciilplllliliill Image R e sto r a tio n R e g u la r is a tio n Method (a) (b) Figure 3.11: M ultiple-param eter regularized image restoration, (a) Defocused Text image w ith r = 5 and contam inated by noise with SN R = 33 db. (b) Restored im age by m ultiple-param eter restoration. Algorithm MSE NM SE (%) ISNR (db) Single-param eter M ultiple-param eter Table 3.6: Q u an titative com parison between single-param eter and m u ltiple-param eter regularized im age restoration. The sam ple im age is a defocused and noise-contam inated Text im age.

131 103 C hapter 4 A M odified W iener F ilter for th e R estoration o f Blurred Im ages 4.1 In trod u ction W iener filters give the linear least mean square estim ate of the object im age from the observations and have been used extensively for the restoration of noisy and blurred im ages. The underlying idea here is to make use of the information contained in the im age at hand as well as in the imaging system involved. In this chapter, it will be shown that conventional W iener filters can be improved by taking the characteristics exhibited in the Fourier transform of the blurring operator into account. A*modified W iener filter based on the idea will be derived. Experimental results are included in Sec. 4.4 to illustrate th e proposed im age restoration algorithm. Im age restoration is a process that attem pts to recover an im age that has been contam inated by noise and blurred by the im age system involved. In this chapter, the im age system is assumed to be linear, and is described as usual by (see Secs. 1.2 and 2.1) g = H f + n (4.1)

132 104 where g and / represent, respectively, the lexicographically ordered recorded and original images; m atrix H represents the spatially invariant linear blurring operator; and n represents the additive white noise. W ith different operators H, this m odel covers m any practical degradation mechanisms such as linear m otion blur, exponential blur and defocusing, with satisfactory m odeling accuracy. T he noise process n in (4.1) is not known exactly, although its statistical propertiec are assum ed to be known. Hence for a given g, it is im possible to recover / with perfect precision. The image restoration task here is to find an estim ate f of /, such that the MSE e2 = E[(f - ff] (4.2) is m inim ized, where E(-) denotes the m athem atical expectation. This / is usually called the m inim um m ean square estim ate (MMSE) of / w ith a given g. It is well known that value e2 is m inim ized when f is equal to the conditional expectation of f(k, I) given g(k, /) [4, 35]. However, this solution is in general a nonlinear function of g(k, /) and for its evaluation one needs the knowledge of the local statistics of / and g. This solution is therefore of little use from an analytical as well as a practical point of view. The problem becom es feasible if we m inim ize e2 by restricting to linear functions of g(k, /). Such an estim ate is called a linear m inim um m ean square estim ate (LM M SE). It is noted that the LMMSE does not absolutely m inim ize (4.2), but am ong the linear estim ates it yields the sm allest value for e2. Also the LMMSE is the sam e as the optim um nonlinear estim ate if /, g, and n are stationary and jointly G aussian, a condition which is hardly valid for m ost im ages [35]. A If the estim ate f(k, /) is a linear function of g(k, I), it can be expressed as f(k>l) = '%2Y,w(k - P l - <l)9(p,<l) (4-3) P 9

133 105 or in short, / = Wg in operator format, where W is determined such that (4.2) is m inim ized. It can be shown [4] that any / satisfying the orthogonality condition E[{f(k, /) - m l)h(k\ I')] = o for all (*, /), (k\ /') (4.4) A m inim izes e2 defined by (4.2). Substituting / = Wg into (4.4), we obtain W E[g(k, l)g(k',;')] = E[f(k, l)g(k', /')] (4.5) Taking the Fourier transform on both sides of (4.5) leads to where Pfg(us 1,0*2) is the cross power spectrum of the undegraded and the degraded im ages, Pg(u>i,u>2) is the power spectrum of the degraded image and W(uii,u>2) is the Fourier transform of w(k,l). If in addition f(k,l) and n(kj) are uncorrelated and either f(k, I) or n(k, I) has zero m ean, so that E[f{k, l)n(k\ /')] = E[f(k, l)]e[n(k', /')] = 0 (4.7) then it can be shown that [4, 35] Pfg(uj i,u>2) = H*(u1,u2)P}(<jj1,ijj2) (4.8) and Pg(u}i,u>2) = \H(u>i,u>2)\2Pj(u}i,bj2) + Pn(uji, u>2) (4.9) where u>2) is the frequency response of the linear blurring operator H. When (4.7) holds, by substituting (4.8) and (4.9) into (4.6) we obtain for the linear restoration filter un x, u>2)pf{u\, u>2).. ( 1, W 2 ) ~ ( 0 a )

134 106 H*(u i m <+,, pn{u>i,o,2) (4.10b) \H(U1,U2)\2 Pf(u 1.W2) U)2)\2 1 T B ' ( c ) which is often referred to as the Wiener fillter in the literature. 4.2 W iener F ilterin g and R egularization M eth od s The W iener filter (4.10) provides a numerical solution to the im age restoration problem that is form ulated as a m inim um m ean square error problem (4.2). In what follows we dem onstrate that the W iener filter is in fact a regularization m ethod. At the sam e tim e, we also illustrate the ill-posedness of the im age restoration problem (4.1) in the frequency domain. For sim plicity, consider the noise-free case and write the m odel of such an im age system as g(k,l) = h(k,i)*f(k,l) (4.11) If we take the Fourier transform of (4.11), we obtain G{u>i,U2) = H(Ui,L02)F{u\,U)2) (4-12) To restore / in th e spatial dom ain, we need to deconvolve (4.11); this is equivalent to dividing G(uj\,W2) by / / ( u q,^ ) in frequency dom ain and then applying the inverse Fourier transform. T he blur function H acts like a lowpass filter. This m eans 0 rapidly as y u f + t. As a result, high frequencies in G(u j get amplified considerably. Even when the original analog g is sm ooth, the digitization itself m ay introduce non-sm ooth roundoff errors, a kind of high frequency noise w ith sm all am plitude. So the deconvolution procedure will lead

135 107 to deterioration in the restored im age especially when the recorded image is also degraded w ith noise. It follows from (4.10) that the Wiener filter can be implemented in the frequency dom ain. In the absence of noise Pn{wi, W2) = 0, the Wiener filter becomes the inverse filter. In th e presence of noise, because of the term, the denom inator of (4.10b) will never be zero. In the lim iting case Pn(u>i,uj-2) 0, we have,.. 2 )^ 0 W{u1,u2) = \ (4.14) ) = 0 The term in (4.10c) can be regarded as a term that sm ooths the inverse filter m b * ) - To im plem ent the W iener filter, we have to know Pj(u q,^ ) and Pn(u>i,uq)- In m odel (4.1), the noise process is assumed to be white, that is Pw(u>i.,u>2) is a constant. This constant can be regarded as P (0,0) which is equal to a*, the variance of n. It then follows from (4.9) that 2) (4 J 4 ) provided that H(uii,uj2) ^ 0. Estim ates of can be obtained with satisfactory accuracy either using the average of local variances in uniform zones of the degraded im age g, or using the variance of the im age that contains noise only, that is, by putting / = 0 in (4.1). However, circumstances exist where //(u q ju q ) W 0 or Pg(u!hU>2) < m aking it difficult to obtain Pj(u> 1,^2) from (4.9). In the next section th e conventional W iener filters are m odified to deal with this difficulty.

136 A M odified W iener F ilter The optim al parameters in the Wiener filter for image m odel (4.1) are determ ined uniquely by the power spectra Pj and Pn, and the Fourier transform H(u) \, u>2) of H. In order to im plem ent (4.10a), we need to know H(ui, ^ ), Pj and Pn. As H is assumed to be given, the W iener filter can be implem ented upon the availability of Pf and Pn. Estim ating the noise power spectrum is relatively sim ple, but estim ating the im age power spectrum Pj is not. A number of variations of the W iener filter have been proposed to im prove its restoration perform ance [1, 2, 3]. O ne of th ese m ethods is to put for all (u>i,u2) in (4.10c), where SNR, the signalto-noise ratio, is defined by (2.50), i.e.,, variance of H f SNR = 10 log : r1 - variance of n This often gives a satisfactory result, yet there is room for im provem ent. For com parison purposes, the filter so form ed is denoted by r i r / N _ t f ( - l, U, 2) Wl,W2 \H{U}U U}2)\2 + H { u i,u2) In practice, it is found that better results can be achieved if in (4.15) is modified to with a determ ined by a trial-and-error m ethod. T he resulting filter is denoted by H \U ) \, U)2) + SNR H (u ii,u!2) Note that blurring operators H such as linear m otion blur, defocusing and exponential blurs are basically lowpass filters. More precisely, at low spatial frequencies (a?!,^ ), it is usually true that ^T(o^x,^ 2) ^ 0; that is, there is always a neighborhood D of the origin (u>i,u2) = (0,0 ) such that \H(u}i,uj2)\ 0, (wi,uj2) D

137 109 In addition, for m ost images Pg (wx, lo2) at low frequencies is relatively larger than those at high frequencies. The noise process n is assumed to be white and P (a i, w2) is approxim ately equal to cr^, the variance of n. Thus Pa{ui,w2) - P (u>i,w2) = Pg(ujx,u2) ~<rl> 0 holds for low frequencies (a>x,u>2), so in a neighborhood D of the origin (4.14) is satisfied, i.e., = K < * ) e D ( 4.n ) Equation (4.17) indicates that Pj{ux,u>2) can be more accurately estim ated at low spatial frequencies. Substituting (4.17) into (4.10c) leads to an improved estim ation of W{ujx,ijJ2) at low frequencies: VE(wx, ci;2) IH(l ^ ) 2 1 Pg(w i,w 2) - a l 1 d / \ Pg(Ul,U2) hy H{u i,u2) ( ^ i? ^ ) D (4.18) For frequencies at which P/(a;1,a;2) > Pn(u>i,uj2), the Wiener estim ate W(lox,u>2) is nearly equal to m eaning that the W iener filter acts like an. inverse filter at these frequencies. At high frequencies, Pf(uix,Li2) is often much larger than Pn(u>x,u>2), thus condition Pf(u>x,u2) > Pn(u>i,w2) holds in D c where D c denotes the com plem ent o f D w ith respect to the whole frequency dom ain. For linear m o tion blur, exponential blur, and defocusing, \H(u}X,uj2)\ at high frequencies is relatively small. Especially when (ujx,u>2) approaches a region with large values of \J uj\ -(- ui2, \H(ujX:uj2)\ is nearly zero. Therefore when Pj(uix,u>2) > Pn(uii,u)2) and FT(u;x,u;2) ps 0, the W iener estim ate W(u;i,u2) = is ill-conditioned. It is for the same reason that the estim ate of Pj(u\,u>2) by equation (4.14) is also ill- conditioned. Consequently, this estim ate will lead to a poor W(tx>i,w2). In order

138 110 to obtain a reasonable VF(wi,u;2) in Dc, regularization techniques together with the rough substitution = g^n f r all (<*>!,u>2) 6 D c are used sim ultaneously to enhance the restoration capability of the W iener filter. This leads to the following m odification of VF(u>i, u>2) in D c ( ' 2 ) D e ( 4 -i 9 ) where a is a regularization parameter. Combining equations (4.18) and (4.19), we obtain the following m odified W iener filter: ( ^ 1, u > 2 ) 5if e f e 4 2T JL' h ( ^ 2) ^ 2) e D ^ ^ ff(q > l,q > 2 ) 2 1 (,,,, \ c n c tf(u>i,u>2) 2+ a /S N R ' / / ( Wl,w2) u The region D and the regularization parameter a in the m odified W iener filter W 3 are determ ined experim entally in order to obtain better restoration results. Fig. 4.1 shows the m agnitude U(u;i,u;2) of three different blurring operators, nam ely linear m otion blur, exp onential blur and defocusing. These figures suggest that a rectangular D should be chosen if H represents a linear m otion blur, whereas a circular D is preferred if H is either an exponential blur or a defocusing. For m ost im ages, P/(o;i,o;2) /P n(a;i,a;2) at high spatial frequencies is larger than SNR,. This suggests that an a < 1 should be used. The m odified W iener filter W 3 in this case acts more or less like a highpass filter. 4.4 E xp erim en tal R esu lts To evaluate the performance of filters VFj, VF2 and W 3 quantitatively, we use two criteria, nam ely th e N M SE and th e ISN R, which are defined in (2.49) and (2.52),

139 I l l 5 (a) 0>) l i i ; ( c ) Figure 4.1: T he m agnitude response \H (u>i, u 2)\ of (a) linear m otion blur w ith 8 units; (b) exponential blur; (c) defocusing with radius of COC 7 uiuts.

140 112 respectively. These performance measures can only be evaluated for the experim ents in which the original image is known. It is easy to see that an algorithm that produces smaller NM SE or larger ISNR is considered as a better one. In order to evaluate the performance of the proposed linear filter W 3, two images degraded by a linear m otion blur and a defocusing, respectively, are used. The Lena im age in F.ig. 4.2(a) was linearly blurred with 14 units and was further contam inated by pseudo white Gaussian noise with SN R = 30 db (Fig. 4.2(b)). T he im age processed by linear filters W 3 (a = 0.25, D is a rectangular neighborhood with dimensions 15 x 120), W'i (a = 0.25) and the classic W iener filter W \ are shown in Figs. 4.2(c), (d) and (e), respectively. Table 4.1 sum marizes the performance assessm ent of these three filters. The second image Text (Fig. 4.3(a)) was defocused with radius of the COC r = 3 and SN R = 15 db (Fig. 4.3(b)). The restored images by linear filters W 3 (a = 0.05, D is a circular neighborhood w ith radius 30 units), W 2 (a = 0.05) and Wi are shown in Figs. 4.3(c), (d) and (e), respectively. The performance evaluation is summarized in Table C onclusions A popular approach to obtain an estim ate / is to use the LMMSE criterion in the restoration process, where the statistics of g and n are taken into consideration. This yields the well-known W iener filter in which the information lying in the distortion operator H is never used explicitly. It is observed in im age restoration that the m agnitude response /f(u;i.,u;2) of m ost blurs drops rapidly with distance from the origin in the (u^u^-pl& ne. The m agnitude iv(u;i,u;2) of th e noise, on th e other hand, is custom arily a constant.

141 H ence for frequencies (u ^ u ^ ) far away from the origin, the quotient becomes very large. In order to avoid the amplification of noise, the modified W iener filter places a very large weight in a well-chosen neighborhood D of the origin. As the SN R of th e degraded image is likely to be very low in the vicinity of zeros of i/(u;i,u>2), D is chosen so as to exclude the zeros of \H (u\,u)2)\. This choice also avoids th e ill-cond ition incurred by In D c, th e m odified W iener filter is designed using the regularization approach to enhance the fidelity of the restored image. Experim ental results have demonstrated that the proposed approach (ITa) yields better results than those obtained by the classic W iener filter (H i) or its existin g m odifications (such as H i).

142 114 (c) (d) (e) Figure 4.2: Comparison of modified W iener filtered and W iener filtered im ages, (a) The Lena image, (b) Noise-contam inated linear m otion blurred Lena im age with L = 14 and SN R = 30 db. (c) Restored Lena image by modified W iener filter W3. (d) Restored Lena im age by modified Wiener filter W i. (e) Restored Lena im age by conventional W iener filter W \.

143 Image R e s to ra tio n by R e g u la riz a tio n Method ::-:.v _ ' ;-v. (a) (b) Image Restoration by Regularization. Method Imags Restoration by Rsgularioatiofj Method (c) (d) (e) Figure 4.3: Comparison of modified Wiener filtered and Wiener filtered images, (a) The im age Text, (b) Noise-contam inated defocused Text im age with r = 3 and SNR = 15 db. (c) Restored Lena im age by modified W iener filter W:). (d) Restored Lena im age by modified W iener filter W i. (e) Restored Lena im age by conventional W iener filter W i.

144 116 Filter NM SE (%) ISN R (db) W W Table 4.1: Q uantitative comparison between the Wiener filter and modified W iener filters. R elated im ages are shown in Fig Filter NM SE (%) ISNR (db) U' w Wy Table 4.2: Q uantitative comparison between the W iener filter and modified W iener filters. R elated im ages are shown in Fig. 4.3.

145 C h ap ter 5 N o ise R em oval 5.1 In trod u ction Noise is a random process in general and special filters have to be designed to reduce the noise in im ages. In this chapter, it is assumed that the noise in an im age is an \ additive random process and the corresponding im age formation model is sim ply given by g(k,l) = f(k,l) + n(k,l) (5.1) where f(k, /) and g(k, I) are the im age intensity (gray level) of the original image arid the noisy im age at point ( k, /), respectively, and n(k, I) is the additive noise at (k, I). The noise removal task is to provide a reasonable smooth estim ate, f(k, /), of f(k, I) given the noisy im age g(k, I). Lowpass filtering nets been one of the conventional tools for sm oothing noisy im ages. Generally, the lowpass filters are characterized by attenuating high-frequency structures in the images. Although lowpass filters reduce the noise, they could degrade sharp details such as edges and lines. This causes an artifact of blurring the image. Average filters and Gaussian filters are com m only used lowpass filters in noise removal. A review of these two types of

146 118 filters is presented in Sec Application of R-filters to noise removal is presented in Sec Recall that associated with an R-filter there is a regularization parameter which indicates a tradeoff between the accuracy and sm oothness of the estim ated im age, and a suitable choice of the regularization parameter can reduce the blurring effect of a sm oothing R-filter. In Sec. 5.4, a two-step regularization algorithm is proposed for im age noise removal. In the first step of the algorithm a sm oothing filter such as a lowpass or a W iener filter is applied to reduce the noise level of the im age, and a new im age m odel is formulated where the sm oothing filter is treated as a degradation operator. In the second step of the algorithm a generalized regularization filter is applied to deblur the image. The filter proposed by J. S. Lee [63] in 1980 for im age enhancem ent and noise filtering is well known as an effective technique for noise removal. Since the Lee filter is characterized by the local statistics of the image, it is often considered as an adaptive linear filter. Two nonlinear extensions of the Lee filter for noise filtering are presented in Sec The proposed nonlinear filters utilize higher m om ents of the available im age and include the Lee filters as a subclass of (linear) filters. Comparisons of various conventional filters such as average, Gaussian, m edian, W iener and Lee filters w ith the proposed algorithm s, nam ely, th e tw o-step regularization filters, R-filters, and nonlinear filters are presented in Sec Finally, by making use of the idea of the two-step regularization, an im age restoration algorithm w ith a pre-sm oothing step is proposed in Sec. 5.7.

147 C onventional Sm oothing F iltering Conventional sm oothing filters, such as average filters and Gaussian filters, can reduce the noise considerably but only at the price of blurred details. This is because every sm oothing filter attenuates structures w ith high frequencies A verage F iltering Average filters average pixels w ithin a small neighborhood. The sim plest 2~D average filter is the 3 x 3 average filter which gives the m ean of the pixels within the filter mask. Note that in order to preserve the gray level in uniform zones, the sum of all coefficients of a sm oothing filter m ust be one. Average filters with sizes 3 x 3, 5 x 5 and 7 x 7 are com m only used, but they are not good filters for noise removal as can be seen from Fig In the figure, the original im age Peppers (Fig. 5.1(a)) was contaminated with w hite Gaussian noise with SN R = 10 db and the noisy im age is shown in Fig. 5.1(b). Figs. 5.1(c) (d ), (e) and (f) show the results of average filters whose sizes are 3 x 3, 5 x 5, 7 x 7 and 9 x 9 respectively. It is observed that image edges get blurred in all sm oothing im ages especially when a larger mask is used. T he inadequacy of average filters as noise removal filters can also be seen from th e follow ing dem onstration:

148 Figure 5.1: Average filtering, (a) The im age Peppers, (b) Noise-contam inated Peppers im age with SNR = 10 db. Restored image with mask of size (c) 3x3; (d) 5 x 5; (e) 7 x 7; (f) 9x9.