The Family of Regularized Parametric Projection Filters for Digital Image Restoration

Size: px

Start display at page:

Download "The Family of Regularized Parametric Projection Filters for Digital Image Restoration"

Chester Dwight May
5 years ago
Views:

1 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH PAPER The Family of Regularized Parametric Projection Filters for Digital Image Restoration Hideyuki IMAI, Akira TANAKA, and Masaaki MIYAKOSHI, Members SUMMARY Optimum filters for an image restoration are formed by a degradation operator, a covariance operator of original images, and one of noise. However, in a practical image restoration problem, the degradation operator and the covariance operators are estimated on the basis of empirical knowledge. Thus, it appears that they differ from the true ones. When we restore a degraded image by an optimum filter belonging to the family of Projection Filters and Parametric Projection Filters, it is shown that small deviations in the degradation operator and the covariance matrix can cause a large deviation in a restored image. In this paper, we propose new optimum filters based on the regularization method called the family of Regularized Projection Filters, and show that they are stable to deviations in operators. Moreover, some numerical examples follow to confirm that our description is valid. key words: the family of projection filters, the family of parametric projection filters, regularization theory 1. Introduction Functional analysis is an effective approach to an image restoration problem. In this framework, degradation and restoration of an image are modeled as follows: { g =Af0 + ɛ, f 0 H 1,g,n H 2,A B(H 1, H 2 ), f =Bg, f H 1,B B(H 2, H 1 ), where H 1 and H 2 denote two separable Hilbert spaces called the space of original images and that of observed images, respectively, and B(H 1, H 2 )denotes the set of linear operators on H 1 into H 2. In addition, f 0 H 1, g H 2, ɛ H 2 and f H 1 are called an original image, a degraded image, an additive noise, and a restored image, respectively, and linear operators A and B are called a degradation and a restoration operators, respectively. An aim of an image restoration problem is to obtain the restored image closest to the unknow original image. A restoration operator which gives the optimum restored image is called an optimum filter. According to the measure of closeness between two images, various optimum filters have been proposed. For example, Generalized Inverse Filter [6], the family of Projection Filters [3], [7], the family of Parametric Projection Filters [1], are some of optimum filters. Furthermore, their individual properties and mutual relations have been also studied [3], [5], [10] [12]. Manuscript received April 13, Manuscript revised October 14, The authors are with the Division of Systems and Information Engineering, Hokkaido University, Sapporo-shi, Japan. These optimum filters are formed by a degradation operator, a covariance operator of noise, and one of original images. In a practical image restoration problem, these operators are estimated on the basis of empirical knowledge. Thus, it appears that such operators differ from the true ones. In [1], [2], it is shown that small deviations in the operators can cause a large deviation in the restored image. In this paper, we propose a new class of restoration operators named the family of Regularized Parametric Projection Filters. They are shown to be stable to deviations in operators, that is, small deviations in operators cause a relatively small deviation in the restored image. In our work, the regularization theory [8] plays an important role. Although the theory have been developed in the field of the functional analysis, it becomes widely spread in many areas [9]. 2. Preliminaries Let F be a two dimensional original image of size n 1 n 2, and let vec(f )denote the vec operator which transforms an n 1 n 2 matrix to an (n 1 n 2 )vector by stacking the columns of the matrix one underneath the other [4]. In this paper, we assume that the image degradation model becomes vec(g) =Avec(F )+ɛ, where G denotes a two dimensional degraded image of size m 1 m 2, A denotes an (m 1 m 2 ) (n 1 n 2 )matrix, and ɛ denotes a noise vector of dimension m 1 m 2 whose mean vector and covariance matrix are 0 and Q, respectively. We assume that a noise vector is uncorrelated to an original image. Hereafter, vectors f = vec(f )of dimension n(= n 1 n 2 )and g = vec(g) of dimension m(= m 1 m 2 )are simply called an original image and a degraded image, respectively. Thus, an image degradation and an image restoration are modeled as { g = Af 0 + ɛ, f 0 R n, g, ɛ R m f = Bg, f R n, where A and B are an m n and an n m matrices called a degradation matrix and an optimum filter, respectively.

2 528 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH The Family of Projection Filters The family of Projection Filters (abbreviated to PFs) includes Projection Filter, Partial Projection Filter, and Averaged Projection Filter. Their common properties, mutual relation, and the unified theory of the family are found in [2], [3], [5], [10] [12]. The general form of the family becomes B PFs = BPFs 0 + W (I m UU + ), BPFs 0 = R F V + R F A U +, U = ARF 2 A + Q, V = R F A U + AR F, (1) where R F is the specific Hermitian matrix, W is an arbitrary matrix with size n m, A denotes the conjugate matrix of A, A + denotes the Moore-Penrose inverse matrix of A, and I m denotes the m m identity matrix. We obtain an optimum filter belonging to the family of PFs with I n, Projection Filter, R F = P S, Partial Projection Filter, R 1/2, Averaged Projection Filter, where P S is the orthogonal projector on S to which an original image belongs, and R denotes the covariance matrix of original images. 2.2 The Family of Parametric Projection Filters The family of Parametric Projection Filters (abbreviated to PPFs)includes Parametric Projection Filter, Parametric Partial Projection Filter, and Parametric Wiener Filter. They are proposed in order to suppress a noise component. The general form of the family becomes as follows [1]: B PPFs (γ) =BPPFs(γ)+W 0 (I m U(γ)U(γ) + ), BPPFs 0 (γ) =R2 F A U(γ) +, U(γ) =ARF 2 A + γq, (2) where γ is a positive number, W is an arbitrary matrix with size n m. We obtain an optimum filter belonging to the family of PPFs with I n, Parametric Projection Filter, R F = P S, Parametric Partial Projection Filter, R 1/2, Parametric Wiener Filter. From Eqs. (1)and (2), we see that every optimum filter belonging to the families of PFs and PPFs does not uniquely determined in general. However, a restored image is unique for any W if a degraded image belongs to R(ARF 2 A + Q), where R(A)denotes the range of a matrix A. Thus, BPF 0 or B0 PPFs (γ)is used in a practical image restoration. 2.3 Properties of the Families of PFs and PPFs A lot of properties about the families have been investigated (e.g. [1] [3], [10] [12]). At first, we clarify the definition of the norm used in this paper because we discuss convergence of an optimum filter. Let A be a matrix with size m n. The norm of A is denoted by A, and is defined as A = sup Ax x. x R n x 0 It is well-known that AB A B holds for matrices A and B when the matrix product AB is defined. By the definition, the norm of a matrix is equal to its largest singular value. It is to be noted that convergence of a matrix is independent of the choice of a norm if the space of original images and that of observed images are both finite dimensional. The following lemma shows that BPPFs 0 (γ)approaches B0 PFs as γ approaches 0. Lemma 1 ([12]): BPPFs 0 (γ) B0 PFs γ BPFs Q (AR 0 F 2 A ) + (1 + Q U + ) In practical image restoration problem, the degradation matrix and the covariance matrix are estimated on the basis of empirical knowledge. Thus, they may differ from the true ones. Let Ã, Q and R F be estimated matrices, and let B PFs and B PPFs (γ)be optimum filters belonging to the family of PFs and PPFs based on Ã, Q and RF, that is, B PFs 0 = R F Ṽ + RF Ã Ũ +, B PPFs(γ) 0 = R F 2 Ã Ũ(γ) +, where Ũ = Ã R 2 F Ã + Q, Ṽ = R F Ã Ũ + Ã R F, Ũ(γ) =Ã R 2 F Ã + γ Q. In [1], [2], it is shown that differences between the true matrices (A, Q, and R F )and the estimated matrices (Ã, Q, and R F )cause a significant influence to the optimum filters BPFs 0 and B0 PPFs. In other words, even if max( Ã A, Q Q, R F R F )approaches 0, B PFs 0 and B PPFs 0 do not converge to the true optimum filters BPFs 0 and B0 PPFs, respectively. 3. The Family of Regularized Parametric Projection Filters As we stated in the preceding section, obtaining an optimum filter which belongs to the family of PFs

3 IMAI et al: THE FAMILY OF REGULARIZED PARAMETRIC PROJECTION FILTERS 529 and PPFs is ill-posed. To solve ill-posed problems, the regularization technique is widely used [8]. In this section, we propose the family of Regularized Parametric Projection Filters (abbreviated to RPPFs). By using the family, we obtain an optimum filter that converges to B PFs and B PPFs (γ)as max( Ã A, Q Q, R F R F )approaches 0. Definition 1: The general form of RPPFs is defined as follows: B RP P F s (γ,δ) =BRP 0 P F s (γ,δ) + W (I m U(γ)T (γ,δ)), BRP 0 P F s (γ,δ) =R2 F A T (γ,δ), U(γ) =ARF 2 A + γq, T (γ,δ)={u 2 (γ)+δi m } 1 U(γ), where γ and δ are positive numbers, W is an arbitrary matrix with size n m. We obtain an optimum filter belonging to the family of RPPFs with I n, Regularized Parametric Projection Filter, R F = P S, Regularized Parametric Partial Projection Filter, R 1/2, Regularized Parametric Wiener Filter. We see that the general form of the family of RPPFs is obtained by replacing U + (γ)with T (γ,δ)in the Eq. (2). It is well-known that lim T δ 0 (γ,δ)=u(γ)+, holds, and T (γ,δ)is called the Tikhonov approximation of U + (γ)[8]. Let B RP P F s (γ,δ)be an optimum filter belonging to the family of RPPFs based on the estimated matrices (Ã, Q, and R F ), that is, B RP P F s (γ,δ) = B RP 0 P F s (γ,δ) + W (I m Ũ(γ) T (γ,δ)), B RP 0 P F s(γ,δ) = R F 2 Ã T (γ,δ), Ũ(γ) =Ã R F 2 Ã + γ Q, T (γ,δ)={ũ 2 (γ)+δi m } 1 Ũ(γ). At first, we show the following lemmas. Proofs of the following lemmas and theorems are found in Appendix. Lemma 2: Let A be a matrix with size m n, then (A A + δi n ) 1 A A + 2δ A + (AA ) +, holds for δ>0. Lemma 3: Let A and Ã be matrices with size m n such that Ã A <εholds. If 2 A ε + ε2 <δ, then (Ã Ã + δi n ) 1 Ã (A A + δi n ) 1 A ( ε ε δ +2 A 2 δ 2 )(1 + O( ε δ )) as ε δ 0, holds for δ>0. Therefore, we obtain the following: Lemma 4: If (2 R F A +1)ε + ε 2 <δ, holds, where then ε = max( R F Ã R F A, Q Q ), T (γ,δ) U(γ) + δ M 1 γ ( 3 ε + M 2 δ + M ε )( ( ε )) 3 δ 2 1+O as ε δ δ 0, holds for 0 <γ 1 and δ>0, where M 1 =2 U + 3, M 2 =2 R F A +1, M 3 =4 U 2 R F A +2 U 2. Lemma 4 shows that U(γ) + is well approximated by T (γ,δ)if parameters γ and δ satisfy suitable conditions. Applying Lemma 4, we obtain the following theorem by which we can evaluate the difference between B RP P F s (γ,δ)and B PPFs (γ). Theorem 1: Let then ε = max( R F Ã R F A, Q Q, R F R F ), B PPFs (γ) B RP P F s (γ,δ(ε)) 0asε 0, holds if the function δ(ε)satisfies 1. δ(ε) 0, ε 2. δ 2 (ε) 0, as ε 0. Applying Lemma 1 and Theorem 1, we obtain the following theorem by which we can evaluate the difference between B RP P F s (γ,δ)and B PFs. Theorem 2: Let then ε = max( R F Ã R F A, Q Q, R F R F ), B PFs B RP P F s (γ(ε),δ(ε)) 0asε 0, holds if the functions γ(ε)and δ(ε)satisfy 1. γ(ε) 0, 2. δ(ε) 0, ε 3. δ 2 (ε) 0, δ(ε) 4. γ 3 (ε) 0, as ε 0.

4 530 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH 1999 As can be seen from Theorem 1 and Theorem 2, the restored image B RP P F s (γ,δ)g converges to the true restored images B PFs g and B PPFs (γ)g if suitable real numbers γ and δ are chosen. For example, γ(ε) =ε 1/10 and δ(ε) =ε 1/3 satisfy the conditions in Theorem 2. Nevertheless, we can not obtain the value ε. Thus, to determine the parameters γ and δ is an important problem that is to be investigated. 4. Numerical Examples In this section, we show some numerical results. As optimum filters to be compared, we use Projection Filter, Parametric Projection Filter, and Regularized Parametric Projection Filter, that is, where BPF 0 = V + A U +, BPPF 0 (γ) =A U(γ) +, BRP 0 P F (γ,δ)=a T (γ,δ), V = A U + A, U = AA + Q, U(γ) =AA + γq, T (γ,δ)=(u(γ) 2 + δi) 1 U(γ). In following two examples, the true degradation matrix A and the true covariance matrix of noise Q are C 4 and C 2 C2, respectively, where C n is a matrix with size defined as c 1 c 2 c 16 c 16 c 1 c 15 C n = , c 2 c 3 c 1 with c i = { 1 n, i =1,...,n, 0, otherwise. We consider the case when a degradation matrix and covariance matrix of noise are as follows: { Ã = C4 + x(c 3 C 4 ),x R, Q = C 2 C2 ( = the true covariace matrix). In this case, B0 PF does not converge to BPF 0 as x 0, nor does B PPF 0 (γ)converge to B0 PPF (γ)[1], [2]. Example 1 In this example, we confirm that B RP 0 P F (γ,δ)converges to BPF 0 and to B0 PPF (γ)under the conditions stated in Theorem 2. In Fig. 1, the abscissa and the ordinate represent the values of δ and BPF 0 B0 RP P F (δ1/4,δ), respectively. Figure 1 shows that BPF 0 B0 RP P F (δ1/4,δ) Fig. 1 Difference between B 0 PF and B 0 RP P F. Fig. 2 Difference between B 0 PPF and B 0 RP P F. decreases as δ approaches 0 when x = 0.001, and BPF 0 B0 RP P F (δ1/4,δ) does not decrease in the neighborhood of the origin when x =0.1 and 0.3. The ε reason is that either δ or ε 2 γ is not negligible in the latter cases with ε = x C 3 C 4. In Fig. 2, the abscissa and the ordinate represent the values of δ and BPPF 0 (0.01) B0 RP P F (0.01,δ), respectively. In the case of Parametric Projection Filter, we also find that BPPF 0 (0.01) B0 RP P F (0.01,δ) decreases in the neighborhood of the origin for small ε. Example 2 In this example, we investigate behaviors of the three optimum filters when x approaches 0. At first, we show the behaviors of Projection Filter and Regularized Parametric Projection Filter. In Fig. 3, the abscissa represents the value of x, and the ordinate does the values of B PF 0 B0 PF and B RP 0 P F (0.01, ) B0 PF. Secondly, we show the behaviors of Parametric Projection Filter and Regularized Parametric Projection Filter. In Fig. 4, the abscissa represents the value of x, and the ordi-

IMAI et al: THE FAMILY OF REGULARIZED PARAMETRIC PROJECTION FILTERS 531 Fig. 5 The original image. Fig. 3 Behaviors of projection filter and regularized parametric projection filter. Fig. 6 The degraded image.

1, 0.00001)approaches the true Projection Filter and Parametric Projection Filter as x approaches 0, though B PF 0 does not converge to the true Projection Filter nor does B PPF 0 (0.

5 IMAI et al: THE FAMILY OF REGULARIZED PARAMETRIC PROJECTION FILTERS 531 Fig. 5 The original image. Fig. 3 Behaviors of projection filter and regularized parametric projection filter. Fig. 6 The degraded image. Fig. 4 Behaviors of parametric projection filter and regularized parametric projection filter. nate does the values of B PPF 0 (0.1) B0 PPF (0.1) and B RP 0 P F (0.1, ) B0 PPF (0.1). From these figures, we see that B RP 0 P F (0.1, )approaches the true Projection Filter and Parametric Projection Filter as x approaches 0, though B PF 0 does not converge to the true Projection Filter nor does B PPF 0 (0.1)converge to the true Parametric Projection Filter. Fig. 7 The restored image by B 0 PF based on A and Q. Example 3 In this example, we show an efficacy of Regularized Parametric Projection Filter for an actual image restoration. We use Projection Filter as an optimum restoration filter. Figure 5 is the original image with pixels and 256 grayscale. Figures 6 and 7 are the degraded image and the restored image by Projection Filter based on the true degradation matrix A and the true covariance matrix Q. Figure 8 is the restored image by Projection Filter based on the degradation matrix Ã with x = and the covariance matrix Q. Moreover, Fig. 9 is the restored image by Parametric Projection Filter with γ = 0.1 based on Ã with x = and Q. From these figures, we see that neither the restored image by B PF 0 nor one by B PPF 0 (γ)approaches the true restored image though the difference between A and Ã is small. Figure 10 is the restored image by Regularized Parametric Projection Filter with γ = 0.1 and δ = based on Ã with x =0.001 and Q. We see that the restored image by B RP 0 P F (γ,δ)is close to the true restored image.

532 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH 1999 Fig. 8 The restored image by B PF 0 based on Ã and Q. Therefore, we need the optimum filter which is stable to deviations in matrices.

6 532 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH 1999 Fig. 8 The restored image by B PF 0 based on Ã and Q. Therefore, we need the optimum filter which is stable to deviations in matrices. In this paper, we propose new optimum filters for digital image restoration based on the regularization technique named the family of Regularized Parametric Projection Filters. Moreover, we evaluate a difference between an optimum filter belonging to the family of PFs (or PPFs)and one belonging to the family of RPPFs. We also show numerical examples of an actual image restoration. From these results, we show that an optimum filter belonging to the family of RPPFs are stable to deviations in matrices. However, it is difficult to determine the regularization parameters of the filters. Thus, it is necessary to develop the method for deciding the optimum regularization parameters. References Fig. 9 Fig. 10 Ã and Q. The restored image by B PPF 0 (0.1) based on Ã and Q. The restored image by B RP 0 P F (0.1, ) based on 5. Conclusions In practical image restoration problem, the degradation process and the property of noise are estimated on the basis of empirical knowledge. Thus, it appears that they differ from the true ones. When we restore a degraded image by an optimum filter belonging to the family of PFs and PPFs, it is shown that small deviations in a degradation matrix and covariance matrices can cause a large deviation in the restored image. [1] H. Imai, A. Tanaka, and M. Miyakoshi, The family of parametric projection filters and its properties for perturbation, IEICE Trans. Inf. & Syst., vol.e80-d, no.8, pp , Aug [2] H. Imai, A. Tanaka, and M. Miyakoshi, Properties of the family of projection filters for a perturbation of operators, IEICE Trans., vol.j80-d-ii, no.5, pp , May [3] Y. Koide, Y. Yamashita, and H. Ogawa, A unified theory of the family of projection filters for signal and image estimation, IEICE Trans., vol.j77-d-ii, no.7, pp , July [4] J.R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics, Wiley, New York, [5] H. Ogawa, Operator equations related to the restoration problems, IEICE Technical Report, PRU86-60, Nov [6] H. Ogawa, Image and signal restoration [II] Traditional opitimum restoration filters, J. IEICE, vol.71, no.6, pp , June [7] H. Ogawa, Image and signal restoration [III] A family of projection filters for optimum restoration, J. IEICE, vol.71, no.7, pp , July [8] A.N. Tikhonov and V.Y. Arsenin, Solutions of Ill-posed Problems, Wiley, Washington, D.C., [9] V.N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, [10] Y. Yamashita and H. Ogawa, Mutual relations among optimum image restoration filters, IEICE Trans., vol.j75-d- II, no.5, pp , May [11] Y. Yamashita and H. Ogawa, Optimum image restoration and topological invariance, IEICE Trans., vol.j75-d-ii, no.2, pp , Feb [12] Y. Yamashita and H. Ogawa, Optimum image restoration filters and generalized inverses of operators, IEICE Trans., vol.j75-d-ii, no.5, pp , May Appendix Proof of Lemma 2 Since P R(A) A = A and P R(A) = AA + hold, we obtain (A A + δi n ) 1 A

7 IMAI et al: THE FAMILY OF REGULARIZED PARAMETRIC PROJECTION FILTERS 533 Thus, =(A A + δi n ) 1 A (AA + ) = A + δ(a A + δi n ) 1 A (AA ) + = A + δ(a A + δi n ) 1 A (AA + )(AA ) + = A + δa + (AA ) + + δ 2 (A A + δi n ) 1 A + (AA ) +. (A A + δi n ) 1 A A + 2δ (AA ) + A + holds because (A A + δi n ) 1 1 δ. Proof of Lemma 3 Let A =(Ã Ã + δi n ) (A A + δi n ), then, A 2 A ε + ε 2 holds. Thus, we obtain if (Ã Ã + δi n ) 1 =(A A + δi n + A) 1 =(A A + δi n ) 1 +(A A+δI n ) 1 ( 1) k ( A(A A+δI n ) 1 ) k k=1 A(A A + δi n ) 1 A δ holds. Therefore, < 1, (Ã Ã + δi n ) 1 Ã (A A + δi n ) 1 A ε δ + 1 ( ) k A ( A + ε) δ δ k=1 ε δ + ε (2 A + ε)( A + ε) δ 2 1 ε δ (2 A + ε) ( ε ε )( ( ε )) = δ +2 A 2 δ 2 1+O as ε δ δ 0, holds if 2 A ε + ε 2 <δ. Proof of Lemma 4 Since γ 1 and max( R F Ã R F A, Q Q ) ε, hold, we obtain Ũ(γ) U(γ) (2 R F A +1)ε + ε 2. Thus, applying Lemma 2 and Lemma 3, we obtain T (γ,δ) U(γ) + 2 U + 3 δ γ 3, and T (γ,δ) T { (γ,δ) (2 R F A +1) ε δ +(4 U 2 R F A +2 U 2 ) ε }( ( ε )) δ 2 1+O δ as ε δ 0. Applying the triangular inequality, we complete the proof. Proof of Theorem 1 Since R 2 F Ã R 2 F A ( R F + R F A )ε + ε 2, holds, we obtain R F 2 Ã RF 2 A as ε 0. Thus, by Lemma 4, BPPFs 0 (γ) B RP 0 P F s (γ,δ(ε)) = R F 2 Ã T (γ,δ(ε)) RF 2 A U(γ) + 0asε 0, holds under the conditions 1 and 2. Moreover Ũ(γ) T (γ,δ(ε)) U(γ)U(γ) + as ε 0, holds under the same conditions. Therefore, we obtain B PPFs (γ) B RP P F s (γ,δ(ε)) 0asε 0. Proof of Theorem 2 If the conditions 1, 2, 3, and 4 are satisfied, B 0 RP P F s (γ(ε),δ(ε)) B0 PPFs (γ(ε)) 0asε 0, holds. Thus, under these conditions, we get B RP 0 P F s(γ(ε),δ(ε)) BPFs 0 B RP 0 P F s (γ(ε),δ(ε)) B0 PPFs (γ(ε)) + BPPFs(γ(ε)) 0 B PFs 0asε 0, by Lemma 1. Moreover, Ũ(γ(ε)) T (γ(ε),δ(ε)) UU + as ε 0, holds since R(U) =R(U(γ(ε))) holds and UU + is the orthogonal projector onto R(U). Therefore, B RP P F s (γ(ε),δ(ε)) B PFs 0asε 0, holds, and we complete the proof.

8 534 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH 1999 Hideyuki Imai received the M.E. degree from Hokkaido University in He joined the Faculty of Engineering, Hokkaido University. His research interests include statistical inference. Akira Tanaka received the M.E. degree from Hokkaido University in He has been in the Graduate School of Engineering, Hokkaido University. His research interests include digital image processing. Masaaki Miyakoshi was received the D.E. degree from Hokkaido University in He joined the faculty of Engineering, Hokkaido University. His research interests include fuzzy theory.

Optimum Sampling Vectors for Wiener Filter Noise Reduction

58 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 50, NO. 1, JANUARY 2002 Optimum Sampling Vectors for Wiener Filter Noise Reduction Yukihiko Yamashita, Member, IEEE Absact Sampling is a very important and