Interpretation of the Multi-Stage Nested Wiener Filter in the Krylov Subspace Framework

Transcription

1 Interpretation of the Multi-Stage Neste Wiener Filter in the Krylov Subspace Framework M. Joham an M. D. Zoltowski School of Electrical Engineering, Purue University, West Lafayette, IN Abstract In this paper, we show that the Multi-Stage Neste Wiener Filter (MSNWF) can be ientifie to be the solution of the Wiener-Hopf equation in the Krylov subspace of the covariance matrix of the observation an the crosscorrelation vector of the observation an the esire signal. This unerstaning leas to the conclusion that the Arnoli algorithm which arises from the MSNWF evelopment can be replace by the Lanczos algorithm. Thus, the computation of the unerlying basis of the Krylov subspace can be simplifie. Moreover, the settlement of the MSNWF in the Krylov subspace framework helps to erive an alternative formulation of the alreay presente MSNWF composition. Introuction The Wiener filter (WF) is a well known approach to estimate the unknown signal 0 [n] from an observation x 0 [n] an is optimal in the Minimum Mean Square Error (MMSE) sense. The WF is also optimal in the Bayesian sense if the signals 0 [n] anx 0 [n] are jointly Gaussian ranom variables (e. g. [Sch9, MW95]). Therefore, the WF is employe in many applications because it is easily implemente an only relies on secon orer statistics which can be estimate with partial knowlege of the transmitte signal. Since the resulting filter epens upon the inverse of the covariance matrix of the observation x 0 [n] an the neee filter length can be very large, if the observation x 0 [n] is of high imensionality, an alternative approach which operates in a reuce space is of great interest to reuce computational complexity an the number of observations neee to estimate the statistics. The first approach to reuce the imension of the estimation problem is the well establishe Principal Component (PC) metho [Hot33, EY36]. The observation signal is transforme by a matrix constitute by the eigenvectors

2 belonging to the principal eigenvalues, thus, a truncate Karhunen-Loeve transform is applie. The Wiener Filter solution with respect to the new observation can be easily obtaine since the covariance matrix of the new observation is a iagonal matrix with the principal eigenvalues as its entries. However, the PC metho only takes into account the statistics of the observation signal an oes not consier the relation to the esire signal. Therefore, Golstein et. al. [GR97b] introuce the Cross-Spectral (CS) metric which evolve from the Generalize Sielobe Canceller (GSC) [AC76, GJ82] an incorporates the similarity of the crosscorrelation vector of the observation an the esire signal with the respective eigenvector. The CS metho oes not choose the principal eigenvectors, but the eigenvectors which belong to the largest CS metric as was also suggeste by Byerly et. al. in [BR89]. Recently, Golstein et. al. presente the Multi-Stage Neste Wiener Filter (MSNWF) approach [GR97a, GRS98] which can be seen as a chain of GSCs. This funamental contribution showe that the reuction of the imension of the observations signal base on the eigenvectors as propose by the PC an CS methos is suboptimum. The MSNWF oes not nee the eigenvectors of the covariance matrix of the observation signal an is, thus, computationally avantageous (see [GGR99, GGDR98] for an illustrative example). This property le to the application of the MSNWF in etection problems [GRZ99] an CDMA interference cancellation [HG00]. Honig et. al. showe that the number of necessary MSNWF stages even for a heavy loae CDMA system is small compare to the eigenspace base methos [HX99]. Paos et. al. presente a similar result in [PB99] for the Auxiliary Vector (AV) metho which was also evelope from the GSC. In fact the earlier AV publications [KBP98, PLB99] are an implementation of the GSC metho. Their formulation in [PB99] is similar to [HG00] except that the combination of the transforme observation signal to form the estimate of the esire signal is suboptimal. Our contribution is to show that the Multi-Stage Neste Wiener Filter (MSNWF) can be seen as the solution of the Wiener-Hopf equation in the Krylov subspace of the covariance matrix of the observation signal an the crosscorrelation vector of the observation an the esire signal. This conclusion follows from the results in [PB99] an especially in [HG00], but we present the consequences of this connection. First, the Arnoli algorithm which is use to fin the orthonormal basis for the Krylov subspace can be replace by the Lanczos algorithm [Saa96] since the covariance matrix is Hermitian. Secon, we evelop a new formulation of the MSNWF algorithm which gives an expression for the resulting Mean Square Error (MSE), although it only works in the reuce imensional space. Schneier et. al. [SW99] presente a similar algorithm which also inclue the computation of the error variance in an image processing application. However, their formulation assumes only aitive noise an a few ominant eigenvalues of the covariance matrix. In the next section, we briefly review the theory of the Wiener filter to make the reaer familiar with our notation. Before we iscuss the reuce rank MSNWF in Section 4, we concentrate on the original MSNWF approach in 2

3 Section 3 to motivate our reasoning in Section 5, where we show the close relationship between the MSNWF an Krylov subspace base methos. In Section 6 we present a new formulation of the MSNWF algorithm. 2 Wiener Filter Figure shows the principle of a Wiener Filter (WF). The esire signal 0 [n] C which is assume to be a zero-mean white Gaussian process is estimate by applying the linear filter w C N to the observation signal x 0 [n] C N which is a multivariate zero-mean Gaussian process. The error of the estimation can be written as ε 0 [n] = 0 [n] ˆ 0 [n] = 0 [n] w H x 0 [n], () where ( ) H enotes conjugate transpose. The variance of the estimation error is the mean square error MSE 0 =E{ ε 0 2 } = σ 2 0 w H r x0, 0 r H x 0, 0 w + w H R x0 w, (2) with the covariance matrix of the observation x 0 [n] the variance of the esire signal 0 [n] R x0 =E{x 0 [n]x H 0 [n]} C N N, (3) σ 2 0 =E{ 0 [n] 2 }, (4) an the crosscorrelation between the esire signal 0 [n] an the observation signal x 0 [n] r x0, 0 =E{x 0 [n] 0[n]} C N, (5) where ( ) enotes complex conjugate. 0 [n] " 0 [n] x 0 [n] w 0 ^ 0 [n] Figure : Wiener Filter The Wiener filter w 0 is the filter w which minimizes the mean square error, thus, w 0 = arg min MSE 0. (6) w This criterion leas to the Wiener-Hopf equation R x0 w 0 = r x0, 0 (7) an the Wiener filter w 0 = R x 0 r x0, 0 C N. (8) 3

4 Consequently, the minimum mean square error (MMSE) can be written as MMSE 0 = σ 2 0 r H x 0, 0 R x 0 r x0, 0. (9) Before we focus on the Multi-Stage Neste Wiener Filter (MSNWF) in the next section, we have to recite the following theorem (e. g. [GRS98]) which is prove in Appenix A. Theorem If the observation x 0 [n] to estimate 0 [n] is pre-filtere by a fullrank matrix T C N N,i. e.,z [n] =Tx 0 [n], the Wiener filter w z to estimate 0 [n] from z [n] leas to the same estimate ˆ 0 [n], thus, the MMSE is unchange. This Theorem can be generalize to a bigger set of pre-filtering matrices by the following corollary. Corollary If the pre-filtering matrix T C M N,M N has full column rank, i. e. T has a left inverse, the resulting estimate ˆ 0 [n] an the MMSE are unchange. The proof of Corollary is straightforwar, the inverse of T just has to be replace by the left inverse of T in Appenix A. However, if T is tall, i. e., M>N, then the Wiener filter solution is ambiguous, because a vector which is orthogonal to the column space of T canbeaetow z in Equation (58) without changing the estimate ˆ 0 [n] an the MMSE. 3 Multi-Stage Neste Wiener Filter The Multi-Stage Neste Wiener Filter (MSNWF) was evelope by Golstein et. al. [GR97a, GRS98] to fin an approximate solution of the Wiener-Hopf equation (cf. Equation 7) which oes not nee the inverse or the eigenvalue ecomposition of the covariance matrix. The approximation for the Wiener filter is foun by stopping the recursive algorithm after D steps, hence, the approximation lies in a D-imensional subspace of C N. The first step of the MSNWF algorithm is to apply a full rank pre-filtering matrix of the form [ ] h H T = C N N (0) B to get the new observation signal z = T x 0 [n] = [ h H x 0 [n] B x 0 [n] ] = [ [n] x [n] ] C N () which oes not change the estimate ˆ 0 [n] as postulate in Theorem. The rows of B are chosen to be orthogonal to h H, therefore, B h = 0 or B = null(h H )H. (2) 4

5 The intuitive choice for the first row h H is the vector which, when applie to x 0 [n], gives a scalar signal [n] that has maximum correlation with the esire signal 0 [n]. Without loss of generality we assume that h 2 = an force [n] to be in-phase with 0 [n], i. e. the correlation between 0 [n] an [n] is real, which is motivate by the trivial case when [n] = 0 [n]. Note that this criterion is ifferent from the one propose in [GGDR98] since we o not optimize the absolute value of the correlation which woul lea to a loss of information about the phase an, thus, Golstein et. al. ha to use Schwarz inequality to erive the same result. The formulation in [PB99, PLB99] is also base on the absolute value of the correlation, but in the proof of the result the correlation is assume to be real. With our consieration we en up with following optimization problem h = arg max E{Re( [n] 0[n])} or h h = arg max h which results in the normalize matche filter 2 (hh r x0, 0 + r H x 0, 0 h) s.t.: h H h = (3) h = r x 0, 0 r x0, 0 2 C N. (4) Note that [n] contains all information about 0 [n] which can be foun in x 0 [n] since 0 [n] is a scalar, thus, the information about 0 [n] lies in a oneimensional subspace of C N an with the matche filter we pick out only these portions of x 0 [n] which are inclue in this subspace. Moreover, the secon part of z [n], namely x [n], oes not contain any information about 0 [n] because of the orthogonality conition in Equation (2). 0 [n] " 0 [n] x 0 [n] T z [n] w z ^ 0 [n] Figure 2: Wiener Filter with Pre-Filtering Again, we have to solve the Wiener-Hopf equation of the new system epicte in Figure 2 to get w z = R z r z, 0 C N, (5) where the covariance matrix of z [n] can be expresse by the statistics of [n] an x [n], i.e., [ ] σ 2 R z = r H x, C N N, (6) with the variance of [n] r x, R x σ 2 =E{ [n] 2 } = h H R x 0 h, (7) 5

6 the crosscorrelation vector between x [n] an [n] r x, =E{x [n] [n]} = B R x0 h C N (8) an the covariance matrix of x [n] R x =E{x [n]x H [n]} = B R x0 B H CN N. (9) The crosscorrelation vector of the pre-filtere observation signal z [n] an the esire signal [n] reveals the usefulness of the choice of pre-filtering: r z, 0 = T r x0, 0 = r x0, 0 2 e R N, (20) where e i enotes a unit norm vector with a one at the i-th position. Therefore, the Wiener filter w z of the pre-filtere signal z [n] is just a weighte version of the first column of the inverse of the covariance matrix R z in Equation (6). After applying the inversion lemma for partitione matrices (e. g. [Sch9, MW95]) we en up with [GR97a, GRS98] [ ] w z = α R C N, (2) x r x, where α = r x0, 0 2 (σ 2 r H x, R x r x, ). (22) Equation (2) is the key equation to unerstan most interpretations of the MSNWF. The first an most important observation in Equation (2) is that the vector in brackets, when applie to z [n], gives the error signal ε [n] of the Wiener filter which estimates [n] from x [n]: ε [n] = [n] ˆ [n] = [n] w H x [n] = [, w H ] z [n] (23) with the Wiener filter w = R x r x, C N. (24) This observation immeiately leas to the next step in the MSNWF evelopment. In the secon step, the output of the Wiener filter w with imension N can be replace by the weighte error signal ε 2 [n] of a Wiener filter which estimates the output signal 2 [n] of the matche filter h 2 from the blockingmatrix output x 2 [n] =B 2 x [n]. Moreover, this observation shows the close relationship of the MSNWF to the Generalize Sielobe Canceller (GSC, [AC76, GJ82, GRZ99]). The GSC can be interprete as a MSNWF after the first step. Secon, the factor α is a scalar Wiener filter to estimate 0 [n] from the scalar ε [n]. The covariance matrix or variance of ε [n] is the MMSE of the Wiener filter w an the crosscorrelation between the scalar observation signal ε [n] an the esire signal 0 [n] is the norm of the matche filter r x0, 0,thus, α = σε r ε, 0 =(σ 2 r H x, R x r x, ) r x0, 0 2. (25) 6

7 0 [n] " 0 [n] x 0 [n] h [n] " [n] ^ 0 [n] x [n] B w ^ [n] Figure 3: MSNWF after the First Step These two interpretations lea to the well known structure in Figure 3. The Wiener filter w 0 is replace by the scalar Wiener filter α which estimates 0 [n] from the error signal ε [n] of the vector Wiener filter w. Thir, the MSNWF can be create without knowlege of the covariance matrix R x0 an, therefore, the implementation of a MSNWF is simplifie, because at each step the Wiener filter is replace by the normalize matche filter an the next stage. Since the matche filter is simply the crosscorrelation between the new observation x i [n] an the new esire signal i [n] ateachsteponly an estimation of this crosscorrelation is neee. Note, however, that estimating all the crosscorrelations an estimating the covariance matrix lea to the same resulting estimate ˆ 0 [n] at the same expense. Fourth, it is straightforwar to see that each new esire signal i [n],i =,...,N, is the output of a length N filter i t i =( B H k )h i C N (26) k= as epicte in Figure 4. This interpretation of the MSNWF will be use in the following sections. Note that all following stages are orthogonal to the first stage, i.e., t H t i = δ,i, i=2,...,n, an δ k,i enotes the Kronecker elta function which is for k = i an 0 for k i. However, the filters t i are not an orthogonal basis of C N in general (cf. Section 5). The Auxiliary Vector metho [KBP98] also evolve from GSC consierations, but ene the iteration after the secon step an is, thus, a rank two MSNWF approximation of the ieal Wiener filter. The extension of the AV metho presente in [PB99] is basically the structure shown in Figure 4, but the propose choice of the scalar filters α i is suboptimal. The very important property of the MSNWF is that the pre-filtere observation vector [n] =[ [n],..., N [n]] T, (27) where ( ) T enotes transpose, has a tri-iagonal covariance matrix [GRS98]. This can be unerstoo with the help of Figures 3 an 4. The matche filter h i is esigne to retrieve all information of i [n] that can be foun in x i [n]. Therefore, the output of h i, i [n], is correlate with i [n] an also with i+ [n], because h i+ is the matche filter to fin i [n]. But i+ [n] inclues 7

8 x 0 [n] t [n] ^ 0 [n] t 2 2 [n] 2 ^ [n] ^ 2 [n] t N N [n] N ^N,[n] Figure 4: MSNWF as a Filter Bank no information about i [n], since the input of h i+ was pre-filtere by the blocking matrix B i+. Consequently, i [n] is only correlate to its neighbours i [n] an i+ [n] leaing to a tri-iagonal covariance matrix. The remaining part of the MSNWF (summers an scalar Wiener filters α i ) is a Wiener filter which estimates 0 [n] of[n]. Because only [n] is correlate with 0 [n], the crosscorrelation vector is simply a weighte version of e,thus, again only the first column of the inverse of the covariance matrix of [n] isof interest. The structure with summers an scalar Wiener filters follows from the tri-iagonal property of the covariance matrix of [n]. 4 Reuce Rank MSNWF The reuce rank MSNWF of rank D is easily obtaine by stopping the evelopment of the MSNWF after D steps an replacing the last Wiener filter w D by the respective matche filter. Thus, the MSNWF of rank is simply the matche filter r x0, 0. We restrict ourselves to the MSNWF interpretation shown in Figure 4, hence, for D<Nwe get the length D observation with a tri-iagonal covariance matrix (D) [n] =T (D),H x 0 [n] C D, (28) where the superscript ( ) (D) inicates that we use a rank D approximation an the transformation matrix T (D) =[t,...,t D ] C N D (29) comprises the first D filters which were alreay efine in Equation (26). Therefore, we have to fin the Wiener filter w (D) which estimates 0 [n] from (D) [n]. Obviously, this Wiener filter reas as ( ) ( w (D) = R (D) (D) r, 0 = T (D),H R x0 T (D)) T (D),H r x0, 0 (30) an the MSNWF rank D approximation of the Wiener filter w 0 can be expresse as ( w (D) 0 = T (D) w (D) = T (D) T (D),H R x0 T (D)) T (D),H r x0, 0 (3) 8

9 an has following mean square error: MSE (D) = σ 2 0 r H x 0, 0 T (D) ( T (D),H R x0 T (D)) T (D),H r x0, 0. (32) 5 MSNWF an Krylov Subspace In Section 3 we state that the filters t i in Figure 4 are not orthogonal in general. It can be easily shown that they are orthogonal for the special choice of the blocking matrices B i to have the property that all singular values unequal to 0 are the same, i. e., σ =...= σ N i = σ an σ k =0,k >N i. In the following we restrict ourselves to this class of blocking matrices, therefore, the filters t i are orthogonal. Moreover, without loss of generality we assume that t i 2 =. Now, recall the MSNWF evelopment (cf. Figure 3). At step i the signal x i [n] at the blocking matrix output of the previous stage was the input for the matche filter h i an the blocking matrix B i. Again, the matche filter was chosen, because its output i [n] has the maximum correlation with the output of the previous stage i [n]. Because we can substitute the chain of blocking matrices an the matche filter h i C N i+ by a filter t i C N (cf. Equation 26) an since we restrict ourselves to use orthonormal filters t i, we can compute the filters t i irectly. At the i-th step we get the aitional output signal i [n] =t H i x 0 [n] which has to be maximally correlate with the output signal of the previous stage i [n] = t H i x 0[n]. Together with the orthogonality conitions this leas to following optimization: t i = arg max E{Re( i [n] i [n])} t t i = arg max t 2 (th R x0 t i + t H i R x0 t) (33) s.t.: t H t = an t H t k =0,k =,...,i. The result which is easily obtaine (e.g. with Langrange multipliers, cf. Appenix B) reas as ( ) k=i P k R x0 t i t i = ( ), (34) k=i P k R x0 t i 2 where P k enotes the projector onto the space orthogonal to t k, i.e., P k = N t k t H k (35) an N enotes the N N ientity matrix. The expression in Equation (34) implies that the best choice for the blocking matrix B i is the projector P i onto the orthogonal space of the matche filter t i, if the filters t i are orthogonal to each other. This result suggests that the or 9

10 reuction of the imension of the solution space at each step of the MSNWF iteration oes not necessarily lea to a reuction of the length of the filter as propose by Golstein et. al. in [GRS98]. This result is consistent since Corollary allows to use a tall pre-filtering matrix T i at each step. Honig et. al. [HX99, HG00] alreay evelope the same result, but they gave no motivation for this special choice of B i. Also Paos et. al. [PB99, LUN99] ene up with a similar expression for the Auxiliary Vector (AV) metho, but i not consier the orthogonality of the filters t i which they incorporate into the optimization, leaing to a more complicate expression. However, both contributions i not make the observation that the recursive algorithm escribe in Equation (34) is the well known Gram-Schmit Arnoli algorithm [Arn5, Saa96]. The Arnoli recursion is the basic algorithm to compute the orthonormal basis of the Krylov subspace K (D) of the square matrix A C M M an the column vector b C M. The Krylov subspace of imension D is efine as follows [Saa96, vv00]: ( ) K (D) =span [b, Ab,...,A D b]. (36) Again, also Honig et. al. [HX99] mae this observation an prove that for the choice B i = P i the filters t i are an orthonormal basis of the Krylov subspace. However, they i not excavate the funamental implications of this result which can be foun in following theorem [Arn5, Saa96]. Theorem 2 If the columns of the matrix T D = [t,...,t D ] were compute using the recursion ) t i = ( k=i P k ( k=i P k At i ) At i 2, t = b b 2, (37) where A C N N is an arbitrary square matrix an b C N is an arbitrary column vector, then the following equality hols: AT (D) = T (D) H (D) + h D+,D t D+ e T D, (38) where H (D) is a D D Hessenberg matrix an t D+ enotes the next vector of the recursion. Obviously, since the vectors t i are orthonormal, Equation (38) can be rewritten to get H (D) = T (D),H AT (D). (39) The proof of Theorem 2 which shows the actual values of the entries of H (D) can be easily obtaine from the recursion in Equation (37) an is outline in Appenix C. To see the value of Theorem 2 we nee to specialize A to be Hermitian. 0

11 Corollary 2 Given an Hermitian square matrix A C N N, i.e., A = A H, the recursion in Equation (37) leas to a transformation matrix T (D) which tri-iagonalizes A, i.e., is a Hermitian tri-iagonal matrix. H (D) = T (D),H AT (D). Proof. Theorem 2 proposes that H (D) is a Hessenberg matrix. Since A is Hermitian, H (D),H = T (D),H A H T (D) = T (D),H AT (D) = H (D), (40) therefore, H (D) is a Hermitian Hessenberg matrix or Hermitian tri-iagonal. Equation (34), Theorem 2, an Corollary 2 isclose the connection between the MSNWF approach an Krylov subspace methos to solve linear equation systems. Our conclusion is that the MSNWF approach, although it is only motivate by statistical reasoning, is simply the solution of the Wiener-Hopf equation by employing the Krylov subspace of the matrix vector pair (R x0, r x0, 0 ), if the filters t i are orthogonal. Furthermore, if a reuce rank MSNWF with rank D is compute, this is fully equivalent to solving the Wiener-Hopf equation in the D-imensional Krylov subspace K (D). An more escriptively, the inverse of is approximate by a matrix polynomial q(r x0 )of orer D. We showe that the MSNWF proceure elivers filters t i which form an orthonormal basis of the Krylov subspace. Henceforth, we can base our reasoning on the Krylov subspace. With Corollary 2 it is straightforwar to unerstan that the covariance matrix R of the pre-filtere observation [n] (cf. Equation 27) is tri-iagonal, because the covariance matrix of the original observation the covariance matrix R x 0 x 0 [n] is Hermitian. Moreover, the Hermitian property of R x0 can be exploite to compute the orthogonal basis t i of the Krylov subspace K (D) of (R x0, r x0, 0 ). In the case of Hermitian matrices the Lanczos algorithm [Lan50, Lan52, Saa96] can be applie to fin the orthogonal basis: t i = = P i P i 2 R x0 t i P i P i 2 R x0 t i 2 R x0 t i t H i 2R x0 t i t i 2 t H i R x0 t i t i R x0 t i t H i 2 R x 0 t i t i 2 t H i R x 0 t i t i 2. (4) Thus, we reuce the complexity of the MSNWF iteration by exploiting the fact that the MSNWF is equivalent to fining the solution of the Wiener-Hopf equation in the Krylov subspace of (R x0, r x0, 0 ). However, the Lanczos algorithm is sensitive to rouning errors, hence, the filters t i are not orthogonal anymore for large i. In the sequel, we assume that the necessary rank D to fin an goo approximation of the Wiener filter is small enough to be able to apply the Lanczos algorithm.

12 6 A New MSNWF Iteration In this section, we evelop a new algorithm which computes the rank D MSNWF, but works only within the Krylov subspace of imension D. Therefore, we assume that the orthonormal basis T (D) C N D of the Krylov subspace K (D) was foun by the Arnoli algorithm (cf. Equation 34) or by the Lanczos algorithm (cf. Equation 4). The resulting rank D MSNWF is given in Equation (3) by the means of the Wiener filter w (D) which is applie to the pre-filtere observation (D) [n] =T (D),H x 0 [n] C D, (42) with the tri-iagonal covariance matrix 0 R (D) = T (D),H R x0 T (D) T = (D ),H R x0 T (D ) 0 T rd,d r D,D r D,D C D D (43) an the crosscorrelation vector with respect to the esire signal 0 [n] [ ] r (D), 0 = T (D) rx0, r x0, 0 = 0 2 R 0 D. (44) If we use the covariance matrix R (D ), the new entries of R (D) are simply r D,D = t H D R x 0 t D an r D,D = t H D R x 0 t D. (45) Because r,0 has the property that only the first element is not equal to 0, only the first column of the inverse of R is neee to compute w (D) = R (D), r (D), 0 C D. (46) Consequently, we are only intereste in the first column c (D) C D of C (D) = R (D), =[c (D),...,c (D) D ] CD D (47) an the inversion lemma for partitione matrices (e. g. [Sch9, MW95]) leas to [ ] C (D) C (D ) 0 = 0 T + β D 0 b(d) b (D),H, (48) where the aitional terms rea as [ ] 0 [ b (D) = C(D ) r D,D rd,d c (D ) ] = D C D (49) an β D = r D,D [0 T,r D,D]C (D ) [ ] 0 = r r D,D r D,D 2 c (D ) D,D D,D (50) 2

13 with c (D ) D,D being the last element of the last column c(d ) D of C (D) at the previous step. Therefore, the new first column c (D) can be written as [ ] c (D) (D ) c = + β 0 D c(d ),,D [ r D,D 2 c (D ) D r D,D ] C D, (5) where c (D ),D enotes the first element of c(d ) D. Obviously, the first column of C (D) an, thus, the Wiener filter w (D) at step D epens upon the first column c (D ) at step D an the new entries of the covariance matrix r D,D an r D,D. However, we also observe a epenency on the previous last column c (D ). Hence, we have to fin an expression for the last column of C(D) an D with Equation (48) we get c (D) D = β D [ rd,d c (D ) D ] (52) which only epens on the previous last column an the new entries of R (D). So, we foun an iteration that only upates two vectors c (D) an c (D) D at each step an, moreover, the mean square error at step D can be expresse with the first entry c (D), of c(d) (cf. Equation 32): MSE (D) = σ 2 0 r x0, 0 2 2c (D),. (53) The iteration in Equation (5) an (52) only operates with scalars an vectors. However, the scalars r D,D an r D,D (cf. Equation 45) are neee which are quaratic forms with the N N covariance matrix R x0. But the matrix vector multiplication R x0 t i with O(N 2 ) which can be foun in the expression for r i,i an r i,i has alreay been use for the Lanczos algorithm in Equation (4) to fin the orthonormal basis T (i). Thus, it is worth to inclue the Lanczos recursion an the resulting algorithm is shown in Table, where we substitute c (i) an c (i) i by c (i) first an c(i) last, respectively. The resulting computational complexity for a rank D MSNWF is O(N 2 D), since a matrix vector multiplication with O(N 2 ) has to be performe at each step. Note that the algorithm in Table is just a version of the Conjugate Graient algorithm [HS52, Saa96]. In fact it is a irect version of the Lanczos algorithm [Saa96] for linear systems. A Wiener Filter with Pre-Filtering The new observation signal after pre-filtering with a full-rank matrix T C N N reas as z [n] =Tx 0 [n] (54) leaingtoanewmean square error MSE z = σ 2 0 w H z r z, 0 r H z, 0 w z + w H z R z w z, (55) 3

14 choose esire MSE: σε 2 choose maximum imension: D t 0 = 0, t = r x0, 0 / r x0, 0 2 u = R x0 t r 0, =0, r, = t H u c () first = r,, c() last = r, MSE () = σ 2 0 r x0, c() first = for i =2,...,D if MSE (i) <σε 2 then = i ; break v = u r i,i t i r i 2,i t i 2 r i,i = v 2 if r i,i =0then =i ; break t i = v/r i,i u = R x0 t i r i,i = t H i u β i = r i,i r i,i 2 c (i ) [ ] last,i [ c (i) (i ) c first = first + β i c (i ), ri,i 2 c (i ) last last, 0 r [ i,i c (i) last = ri,i c (i ) ] β last i MSE (i) = σ 2 0 r x0, c(i) first, T (D) =[t,...,t ] w (D) 0 = r x0, 0 2 T (D) c ( ) first ] Table : Lanczos MSNWF where the covariance matrix of the new observation signal z [n] can be expresse as R z =E{z [n]z H [n]} = TR x 0 T H (56) an the new crosscorrelation vector is r z, 0 =E{z [n] 0[n]} = Tr x0, 0. (57) If we apply the resulting Wiener filter (cf. Equation 8) w z = R z r z, 0 = T,H R x 0 T Tr x0, 0 (58) to the new observation z [n] to get the new estimate ˆ 0,z [n] =w H z z [n] =r H x 0, 0 R x 0 T Tx 0 [n] =w H 0 x 0 [n], (59) we observe that ˆ 0,z [n] = ˆ 0 [n] an, thus, the estimate is unchange. Consequently, the new minimum mean square error MMSE z = σ 2 0 w H z R z w z = σ 2 0 r H x 0, 0 T H T,H R x0 T Tr x0, 0 (60) 4

15 is the same as before (cf. Equation 9) which completes the proof of Theorem. B Basis of MSNWF is Krylov Subspace The Langrange function for the optimization in Equation (33) reas as L(t,λ,...,λ i )= 2 (th R x0 t i + t H i R i x 0 t) λ k t H t k λ i (t H t ). (6) The erivation with respect to the complex conjugate of t must be zero, thus, k= L(t,λ,...,λ i ) t = 2 R i x 0 t i λ k t k λ i t = 0. (62) k= Because t is orthogonal to t k,k =,...,i an the filters t k are orthonormal, i.e., t H k t l = δ k,l, the Langrange multipliers can be expresse as λ k = 2 th k R x 0 t k, k =,...,i, (63) an λ i t = 2 R x 0 t i i t H k R x0 t i t k = 2 2 ( i N t k t H k )R x0 t i. (64) k= If we rewrite this expression by substituting the sum with a prouct of projection matrices (cf. Equation 35), we en up with ( ) t = P k R x0 t i (65) 2λ i k=i an since λ i has to be chosen to give a unit norm vector, we prove the result in Equation (34). k= C Tri-Diagonalization with Arnoli Algorithm The recursion formula of Theorem 2 in Equation 37 can be rewritten to get ( ) h i+,i t i+ = P k At i, (66) where we introuce the abbreviation ( ) h i+,i = P k At i 2. (67) k=i 5 k=i

16 If the prouct of the projectors P k is converte to a sum an the orthogonality of the t i is consiere, Equation (66) reas as an by substituting weenupwith h i+,i t i+ = At i i t H k At i t k (68) k= h k,i = t H k At i (69) At i = h i+,i t i+ + i h k,i t k (70) k= which is the i-th column of the matrix equality in Equation (38). References [AC76] [Arn5] [BR89] S. P. Applebaum an D. J. Chapman. Aaptive Arrays with Main Beam Constraints. IEEE Transactions on Antennas an Propagation, 24(5), September 976. W. E. Arnoli. The Principle of Minimize Iterations in the Solution of the Matrix Eigenvalue Problem. Quarterly of Applie Mathematics, 9():7 29, January 95. K. A. Byerly an R. A. Roberts. Output Power Base Partial Aaptive Array Design. In Proc. 23r Asilomar Conference on Signals, Systems & Computers, volume 2, pages , October 989. [EY36] C. Eckart an G. Young. The Approximation of One Matrix by Another of Lower Rank. Psychometrika, (3):2 28, September 936. [GGDR98] J. S. Golstein, J. R. Guerci, D. E. Dugeon, an I. S. Ree. Theory of Signal Representation. submitte to IEEE Transactions on Signal Processing, 998. [GGR99] [GJ82] [GR97a] J. S. Golstein, J. R. Guerci, an I. S. Ree. An Optimal Generalize Theory of Signal Representation. In Proc. ICASSP 99, volume 3, pages , March 999. L. J. Griffiths an C. W. Jim. An Alternative Approach to Linearly Constraine Aaptive Beamforming. IEEE Transactions on Antennas an Propagation, 30(), January 982. J. S. Golstein an I. S. Ree. A New Metho of Wiener Filtering an its Application to Interference Mitigation for Communications. In Proc. MILCOM 997, volume 3, pages , November

17 [GR97b] J. S. Golstein an I. S. Ree. Subspace Selection for Partially Aaptive Sensor Array Processing. IEEE Transactions on Aerospace an Electronic Systems, 33(2): , April 997. [GRS98] [GRZ99] [HG00] J. S. Golstein, I. S. Ree, an L. L. Scharf. A Multistage Representation of the Wiener Filter Base on Orthogonal Projections. IEEE Transactions on Information Theory, 44(7): , November 998. J. S. Golstein, I. S. Ree, an P. A. Zulch. Multistage Partially Aaptive STAP CFAR Detection Algorithm. IEEE Transactions on Aerospace an Electronic Systems, 35(2):645 66, April 999. M. L. Honig an J. S. Golstein. Aaptive Reuce-Rank Interference Suppression Base on the Multistage Wiener Filter. submitte to IEEE Transactions on Communications, March [Hot33] H. Hotelling. Analysis of a Complex of Statistical Variables into Principal Components. Journal of Eucational Psychology, 24(6/7):47 44, , September/October 933. [HS52] [HX99] [KBP98] [Lan50] [Lan52] [LUN99] [MW95] M. R. Hestenes an E. Stiefel. Methos of Conjugate Graients for Solving Linear Systems. Journal of Research of the National Bureau of Stanars, 49(6): , December 952. M. L. Honig an W. Xiao. Performance of Reuce-Rank Linear Interference Suppression for DS-CDMA. submitte to IEEE Transactions on Information Theory, December 999. A. Kansal, S. N. Batlama, an D. A. Paos. Aaptive Maximum SINR RAKE Filtering for DS-CDMA Multipath Faing Channels. IEEE Journal on Selecte Areas in Communications, 6(9), December 998. C. Lanczos. An Iteration Metho for the Solution of the Eigenvalue Problem of Linear Differential an Integral Operators. Journal of Research of the National Bureau of Stanars, 45(4): , October 950. C. Lanczos. Solution of Systems of Linear Equations by Minimize Iterations. Journal of Research of the National Bureau of Stanars, 49():33 53, July 952. P. Li, W. Utschick, an J. A. Nossek. A New DS-CDMA Interference Cancellation Scheme for Space-Time rake Processing. In Proc. European Wireless 99 & ITG Mobile Communications, pages , October 999. R. N. McDonough an A. D. Whalen. Detection of Signals in Noise. Acaemic Press,

18 [PB99] [PLB99] [Saa96] D. A. Paos an S. N. Batlama. Joint Space-Time Auxiliary-Vector Filtering for DS/CDMA Systems with Antenna Arrays. IEEE Transactions on Communications, 47(9), September 999. D. A. Paos, F. J. Lombaro, an S. N. Batalama. Auxiliary-Vector Filters an Aaptive Steering for DS/CDMA Single-User Detection. IEEE Transactions on Vehicular Technology, 48(6), November 999. Y. Saa. Iterative Methos for Sparse Linear Systems. PWS out of print, saa/books.html. [Sch9] L. L. Scharf. Statistical Signal Processing. Aison-Wesley, 99. [SW99] M. K. Schneier an A. S. Willsky. Krylov Subspace Estimation. submitte to SIAM Journal on Scientific Computing, June 999. [vv00] H. A. van er Vorst. Krylov Subspace Iteration. Computing in Science & Engineering, 2():32 37, January/February