A Recursive Block Incomplete Factorization. Preconditioner for Adaptive Filtering Problem

Size: px

Start display at page:

Download "A Recursive Block Incomplete Factorization. Preconditioner for Adaptive Filtering Problem"

Colleen Warren
5 years ago
Views:

1 Alied Mathematical Sciences, Vol. 7, 03, no. 63, 3-3 HIKARI Ltd, A Recursive Bloc Incomlete Factorization Preconditioner for Adative Filtering Problem Shazia Javed School of Mathematical Sciences Universiti Sains Malaysia 800 Penang, Malaysia shaziafateh@hotmail.com oor Atinah Ahmad School of Mathematical Sciences Universiti Sains Malaysia 800 Penang, Malaysia atinah@cs.usm.my Coyright 03 Shazia Javed and oor Atinah Ahmad. his is an oen access article distributed under the Creative Commons Attribution License, which ermits unrestricted use, distribution, and reroduction in any medium, rovided the original wor is roerly cited. Abstract In this aer a recursive incomlete factorization reconditioner for the iterative solvers of adative filtering roblem is roosed. he reconditioner is able to reduce the eigenvalue sread of the inut signals. he technique involves QR factorization of a toelitz bloc sub matrix of the inut data matrix. Incomlete factorization is realized by bloc-diagonalization of the uer triangular factor. he reconditioner obtained by udate of the incomlete uer triangular factor, with no fill-ins, has low comutational cost as comared with comlete factor udate. With aroriate choice of the number of diagonal blocs, the comutational cost of the resulting reconditioner can be reduced significantly. Simulations results show noticeable decrease in the sectral condition number of the autocorrelation matrix by the alication of resulting reconditioner. Mathematics Subect Classification: 65F08

2 3 Shazia Javed and oor Atinah Ahmad Keywords: Incomlete Cholesy s factor, QRD-RLS, comutational comlexity. Introduction Adative filtering roblem can be considered as an adative least squares roblem of the form: n n i min J ( x ( n = λ en i x ( i= where en ( i, for i=,,, n, are the samles of error estimates obtained when the filter of order has run from i = to i= n, using the set of filter arameters that are comuted at time n, while 0< λ is the forgetting factor []. Methods used for the solution of adative filtering roblem are either direct or iterative[7]. Direct methods include RLS based algorithms and have fast convergence seed, but are comutationally intensive. Iterative methods, including LMS based algorithms, are comutationally simle, but their convergence is highly deendent on the sectral condition number of the inut autocorrelation matrix. wo of the most imortant characteristics to udge the erformance of an algorithm are convergence rate and comutational cost [,3]. If the sectral condition number is high, algorithm taes more iteration to access a steady state solution and converges slowly. In such a situation, reconditioning is necessary to mae the algorithm robust by decreasing sectral condition number. It means that a reconditioned algorithm is required to have convergence rate of direct method, but with low comutational cost. A robust direct method is QRD- RLS algorithm [8] which involves ran one udate of the data matrix. A new bloc- adative QRD-RLS algorithm was resented in [6] by introducing a nonorthogonal transformation into recursive udate. his algorithm adats an uer triangular bloc diagonal matrix. Some well nown reconditioners for adative filtering algorithms include reconditioners derived directly from the autocorrelation matrix [4,5] and clamed lattice reconditioner []. In this aer, we are intending to develo a comutationally simle incomlete factorization reconditioner for the iterative solvers of adative filtering roblem which should be able to reduce the sectral condition number of the inut autocorrelation matrix. he reconditioner is formed by recursive udate of the bloc uer triangular diagonalization of the uer triangular QR-factor. Ran one udate of the incomlete QR factors imroves the robustness of the reconditioner. he sarsity attern with no fill-ins significantly reduces the comutational comlexity of the resulting reconditioner.. Incomlete QR Factorization Consider the standard least squares roblem of the form:

3 A recursive bloc incomlete factorization reconditioner 33 min x R b Ax Where A is a n ( n full ran matrix, with the corresonding normal equation A Ax = A b ( Since the condition number of the coefficient matrix of ( is the square of that of A, therefore a slight increase in condition number of A can mae the normal equation very ill-conditioned. In such a situation, reconditioning is necessary for robustness. Incomlete factorization rovides a good factorization reconditioner to reduce the condition number and imrove the convergence seed of the system. Incomlete QR factorization is an aroximation of comlete QR factorization of the form: A = QR + E n Where Q may not have orthonormal columns, R is an uer n triangular matrix and E is the error matrix which can be made as sarse as we lease. he matrix R here is the Cholesy s factor of the autocorrelation matrix AA. If A QR, then AA RRand the matrix R can be used as a reconditioning matrix for the normal equation, that is, ( R A AR Rx = R A b Or Ax = b, where A = R A AR = AR AR (3 x = Rx, b = R A b = AR b. he closer the incomlete QR decomosition to the comlete QR decomosition, the closer the condition number of A is to. It is imortant to note that Q is totally ignored here, because our interest is in the triangular factor R, which is actually the transose of the Cholesy's factor of the autocorrelation matrix AA. Since reconditioned autocorrelation matrix (3 involves the inverse R of the reconditioner matrixr, therefore comlexity O of inversion of R has great imortance in the comutation of the reconditioner. 3. Formulation Of Adative Least Squares Filtering In order to solve the adative least squares roblem (, we form the vectors a u i in such a way that i using inut signals

4 34 Shazia Javed and oor Atinah Ahmad a ( ( i = u i u i u i + If vector xi is an estimate of the filter coefficient vector, s( i the desired signal, and, y( i = ax i i is the filter outut at time i ; i n, then error incurred from redicting s( i by y( i = ax i is ei = si ax (4 i i Defining en = e(, e(,, e( n and d n = s(, s(,, s( n, error equation (4 can be written in vector form as: where A is the n n en = dn Ax n n data matrix given by: u( i u( i u( i + u( i+ u( i u( i + A n = u( n u( n u( n u( n u( n u( n + Setting λ =, the roblem ( taes the form: J n( x = dn Anx (5 n he toelitz structure of A n allows the ran-one udating of the matrix which is the ey feature of the QRD-RLS algorithm [8]. Overall QRD-RLS udation ste has a comlexity of O(. We begin with a toelitz square bloc submatrix A, of size, of full data matrix A at the th samle instant. o maintain the square structure of A, we are confined to the most recent inut vectors only. In the next section we show that how secial structure of A is maniulated to udate the incomlete QR factorization of A. 4. Recursive Bloc Incomlete QR Factorization Preconditioner (RBIFP A new bloc incomlete QR factorization based reconditioner is derived in this section. he aroach is recursive, similar to the one emloyed in QRD-RLS algorithm, in conunction with sliding windows involving ran-udating and downdating comutations. Beginning with a toelitz square bloc submatrix A, of size, of full data matrix A at the th instant, consider the sliding window

5 A recursive bloc incomlete factorization reconditioner 35 recursive least squares (RLS comutations of A, accomlished by modifying the Cholesy s factor. he toelitz bloc submatrix A, at th instant, can be written as: a + (6 a + A = a he comlete QR factorization of A gives an orthogonal matrix Q and non-singular uer triangular matrix R such that A = Q R (7 Such QR factorization is realized by Givens rotations, so that orthogonality of Q mean rows as well as columns are orthogonal. If q ( i denotes the i th row of Q for i=,,,, then Q = q ( q q ( he uer triangular matrix R is the Cholesy's factor of the autocorrelation matrix of A. We will recursively udate it to get an aroximation of the Cholesy s factor of the autocorrelation matrix of A n. here are have two mains obectives: o get an aroximation of the Cholesy s factor of the autocorrelation matrix of A n, which serves as a reconditioner to increase the convergence rate of the iterative algorithms by decreasing the sectral condition number of the autocorrelation matrix. o imrove the erformance of the reconditioner by udating it according to new inut signals, but at low comutationally cost. o overcome the roblem of high comutational cost of recursion, we use the bloc-adative aroach of [6] to set a sarsity attern of R before starting recursion. Choose ositive integers,,, such that = (8 = hen artition the bloc diagonal of R into uer triangular submatrices in such a way that ( of size ( R ;

6 36 Shazia Javed and oor Atinah Ahmad ( R R ( R R R = ( O R where R is an i matrix, for all. With definitions: i = ( R O ( R U = (9 ( O R and R ( ( 0 R ( 0 R = O 0 We are now in a osition to define the incomlete QR-factorization of A as: A = Q U + Q R = B + E where E = Q R is the error matrix and is highly ill conditioned. he matrix U is sarser and requires less comutations for udate as comared with R. a + A + ( A a + Figure. Sliding Data window - udating and downdating rocess. After + samle udate, A is udated with inut vector a +, using sliding data window of size, as dislayed in figure-. Here udating and downdating taes lace at the same time. Removal of a downdates data matrix + A to

7 A recursive bloc incomlete factorization reconditioner 37 ( matrix A (, while addition of a + udates it to data matrix A +. he corresonding udate of incomlete matrix B = Q U, gives ( B (0 B + = a + where, q q ( 3 B Q U = = U q ( is a truncated version of B obtained by removing its first row. Using value of ( B, equation (0 can be written as: ( Q 0 ( U ( B + = 0 a + It is straight forward from equation ( that, Q 0 U B + = ( 0 ( a + he matrix on the right hand side of ( is artially triangularized in a way that its ( + th row consists of nonzero elements. In order to udate U to U +, we need to zero out the last row of the right hand matrix of (. Such an udate is + + orthogonal transformations ossible [5], since there exists and matrix ( + such that U U + + = a + 0 ( he transformation matrix ( +, used here, is a roduct of Givens rotations and is orthogonal, being the roduct of orthogonal matrices. o reserve the sarsity attern of U no fill-in strategy is used, which means that the transformation ( + is alied in such a way that it does not change the zeros of U. he udate rocedure is summarized in Algorithm. + on left hand side of ( generates the matrix Q +. Since Alication of (

8 38 Shazia Javed and oor Atinah Ahmad Q 0 Q + ( + = 0 ( ( q + is an ( + matrix having transose of matrix equation ( gives: U + ( B+ = ( Q+ q+ 0 ( q + in the last row, After multilication the above equation reduces to B + = Q + U (3 + It is worth noting that Q + is not an orthogonal matrix, hence equation (3 gives an incomlete QR-factorization at ( + th samle instant. Since our interest is in udate U + of the uer triangle only, therefore, to avoid un-necessary comutations, algorithm does not involve the udation stes of Q. ALGORIHM : o fill-in Udate of U : Initializations: U (given by (9 Udate: for i =,,, t Inut vector : ( ( a+ i= a+ i a+ i a + i for =,,, (* for n =,,, ( ( ( denominatior = R+ i nn, + a+ i( n ( ( R+ i nn, ( a+ i( n cn = & sn = denominatior denominatior for l = n, n +,, ( + i(, = n + i (, + n + i ( + i = n + i (, + n + i R nl c R nl s a l a l s R nl c a l End

9 A recursive bloc incomlete factorization reconditioner 39 Continuing this rocess for t inut instants, with < t n, we get an incomlete bloc-diagonalized uer triangular matrix U + t which is an aroximation of the Cholesy s factor R n of the autocorrelation matrix of inut data matrix A n. Since he structure of R n is such that the elements which are far away from the main diagonal are very small as comared with the elements of the diagonal and its uer band. he sarsity attern of U + t is such that maximum number of non-zero elements are at the diagonal. he diagonal of U + t is a fair aroximation of that of R n and diag( U+ t diag ( Rn as t n. he bloc diagonal submatrix U + t is the required incomlete QR-factorization reconditioner (RBIFP for the adative filtering roblem and can reduce the eigenvalue sread of the inut data. 4.. Comlexity Analysis of RBIFP Comlexity of our reconditioner involve comlexity of t udates and the comlexity of the inversion of bloc-diagonalized reconditioner U + t as exlained in recious sections. From equation (8, we have (4 = Since an udate of QRD-RLS algorithm has O comlexity, udate of our reconditioner has O( comlexity. It follows from (4 that = = ( O O Furthermore inversion of an uer triangular matrix of order has O comlexity, and we need to find inverse of our reconditioner, while alying it in some iterative method. In our formulation of diagonal subblocs, this O. Hence it is ustified to say that with comlexity too reduces to ( = aroriate choice of the sizes and number of the uer triangular blocs, the cost of formation of reconditioner, as well as its inversion, can be significantly reduced. he lowest comlexity, equal to O is reached for =,

10 30 Shazia Javed and oor Atinah Ahmad whereas the highest comlexity is reached for =. < is not a suitable choice for our reconditioner because in that case U + t does not have much effect on the sectral condition number of the autocorrelation matrix of inut data. Our simulation results show significant decrease in the sectral condition number for. 5. Simulation Results o chec the erformance of our reconditioner, we use an adative system identification configuration. Consider a finite imulse resonse (FIR filter of order. A white Gaussian inut signal of variance σ = is assed through a colouring filter with frequency resonse [] α H( z = α z where α <, α controls the sectral condition number of the autocorrelation matrix. α = 0 corresonds to the case when condition number is close to. We choose the sizes of the uer triangular blocs R ; ( such that = Lfor =,,,, while <. Setting = 50, t =, we tae such that. Figure- shows the comutational comlexity of multilications used in the formation of the reconditioner as a function of. Considerable decrease in the comutational cost is observed with the increase in the number of blocs. ow we aly our reconditioner to the autocorrelation matrix of inut signal according to equation (3. Figure-3 shows the decrease in the sectral condition number of the autocorrelation matrix by the alication of RBIFP as a function of for. We have considered three tyes of ill conditioned matrices corresonding to α = 0.7, 0.8, 0.9 and have observed significant decrease in the sectral condition number. It is seen that rate of decorrelation increases with the increase in the value of correlation arameter α. For α = 0.9 the inut is highly correlated and 97% decrease in observed corresonding to =.

11 A recursive bloc incomlete factorization reconditioner 3 Figure. Comutational Comlexity of the reconditioner. Figure 3. Percentage of reduction in sectral condition number after alication of reconditioner. 6. Conclusion A reconditioner for the iterative solvers of the adative filtering roblem is develoed by recursively udating the bloc-diagonalized uer triangular QR-

12 3 Shazia Javed and oor Atinah Ahmad factor. he reconditioner RBIFP is comutationally simle and reduces the eigenvalue sread of the inut signals by more than 50%. Our future wor concerns with develoing a RBIFP reconditioned adative filtering algorithm with RLS-lie erformance at a very low comutational cost. Acnowledgement: he authors would lie to acnowledge University Sains Malaysia (USM, Penang, Malaysia and Lahore College for Women University(LCWU, Lahore, Paistan for financial suort. References []. B. Farhang-Boroueny, Adative filters: theory and alications998: John Wiley & Sons, Inc. []. I. Proudler, I. Sidmore, and J. McWhirter. Increasing the erformance of the LMS algorithm using an adative reconditioner. in Euroean Signal Processing Conference (EUSIPCO' [3]. J. Benesty, and. Gansler. A recursive estimation of the condition number in the RLS algorithm [adative signal rocessing alications]. in Acoustics, Seech, and Signal Processing, 005. Proceedings. (ICASSP '05. IEEE International Conference on [4]. J. Sung Lim, and Y. Jong Cho, Circulant-reconditioned bloc adative filtering algorithms based on the nested iteration technique. Signal Processing, (3: [5]. J. Sung Lim, and U. Chong Kwan, Otimum bloc adative filtering algorithms using the reconditioning technique. Signal Processing, IEEE ransactions on, (3: [6]. M. Bhouri, M. Bonnet, and M. Mbou. A new QRD-based bloc adative algorithm IEEE. [7].. A. Ahmad, Comarative Study Of Iterative Search Method For Adative Filtering Problems, in International Conference On Alied Mathematics005: Bedong, Indonesia. [8]. S. Hayin, Adative Filter heory. Vol.. 99: Englewood Cliffs, J: Prentice- Hall. Received: February 6, 03

Improving AOR Method for a Class of Two-by-Two Linear Systems

Improving AOR Method for a Class of Two-by-Two Linear Systems Alied Mathematics 2 2 236-24 doi:4236/am22226 Published Online February 2 (htt://scirporg/journal/am) Imroving AOR Method for a Class of To-by-To Linear Systems Abstract Cuixia Li Shiliang Wu 2 College