Convergence of Stochastic-Approximation-Based Algorithms for Blind Channel Identification

Transcription

1 24 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 [3] S Ohno, Y Inouye, K Tanebe, A numerical algorithm for implementing the single-stage maximization criterion for multichannel blind deconvolution, in Proc EUSIPCO-98, Rhodes, Greece, 998, pp [32] S Ohno Y Inouye, A least-squares interpretation of the singlestage maximization criterion for multichannel blind deconvolution, in Proc ICASSP-98, Seattle, WA, 998, pp 2 24 [33] H Luo R Liu, Second-order statistics blind MIMO-FIR channel equalization for antenna array-based wireless communications systems, in Proc IEEE Sensor Array Multichannel (SAM) Signal Processing Worsh, Cambridge, MA, Mar 2 [34] H H Zeng, L Tong, C R Johnson, Jr, An analysis of constant modulus receivers, IEEE Trans Signal Processing, vol 47, pp , Nov 999 [35] L Tong, M Gu, S Y Kung, A geometrical approach to blind singal estimation, in Signal Processing Advances in Wireless & Mobile Communications, G Giannais, P Stoica, Y Hua, L Tong, Eds Upper Saddle River, NJ: Prentice-Hall, 2, vol, ch 8, pp [36] J K Tugnait, Identification deconvolution of multichannel linear non-gaussian processes using higher-order statistics inverse filter criteria, IEEE Trans Signal Processing, vol 45, pp , Mar 997 [37], Channel estimatin equalization using higher-order statistics, in Signal Processing Advances in Wireless & Mobile Communications, G B Gianniis, P Stoica, Y Hua, L Tong, Eds Upper Saddle River, NJ: Prentice-Hall, 2, vol, ch, pp 4 [38], On linear predictors for MIMO channels related blind identification equalization, IEEE Signal Processing Lett, vol 5, no, pp , Nov 998 [39] J K Tugnait B Huang, On a whitening approach to partial channel estimation blind equalization of FIR/IIR multiple-input multipleoutput channels, IEEE Trans Signal Processing, vol 48, pp , Mar 2 [4] M Kawamoto, Y Inouye, R Liu, Blind equalization algorithms for MIMO-FIR channels driven white but higher-order colored source signals, in Proc 34th Asilomar Conf Signal, Systems, Computers, vol 2, Pacific Grove, CA, 2, pp [4] J G Proais, Digital Communications New Yor: McGraw-Hill, 995 [42] D R Brillinger, Time Series: Data Analysis Theory San Francisco, CA: Holden-Day, 98 [43] M Rosenblatt, Stationary Sequences Rom Fields Boston, MA: Birhäuser, 985 [44] J K Massey M K Sain, Inverse of linear sequential circuits, IEEE Trans Comput, vol C-8, pp , 968 [45] G D Foney, Minimal bases of rational vector spaces, with applications to multivariable linear systems, SIAM J Contr, vol 3, no 3, pp , 975 [46] P P Vaidyanathan, Multiple Systems Filter Bans Eaglewood Cliffs, NJ: Prentice-Hall, 993, p 724 Convergence of Stochastic-Approximation-Based Algorithms for Blind Channel Identification Han-Fu Chen, Fellow, IEEE, Xi-Ren Cao, Fellow, IEEE, Jie Zhu, Member, IEEE Abstract In this correspondence, we develop adaptive algorithms for multichannel (single-input multiple-output, or SIMO) blind identification with both statistic deterministic models In these algorithms, the estimates are continuously improved while receiving new signals Therefore, the algorithms can trac the channel continuously thus are amenable to real applications such as wireless communications At each step, only a small amount of computation is involved The algorithms are based on stochastic approximation methods Convergence properties of these algorithms are proved Simulation examples are presented to show the performance of the algorithms Index Terms Stochastic approximation, wireless communication I INTRODUCTION Because of its potential applications in wireless communication other areas, blind channel identification equalization have become very active areas of research in recent years (see, eg, [4], [8] [2], [4] [7]) The recent surveys [3], [7], [6] contain comprehensive overviews in the area As characterized in [3], there are two major approaches in blind channel estimation: statistical methods deterministic methods In the formal approach, statistic assumptions are made for the input signals; in the latter, input signals are assumed to be deterministic sequences Most algorithms developed so far have been bloc algorithms in nature, ie, a bloc of data sample is collected first then processed together to get the estimation of the channel parameters; in the statistic methods, statistic quantities such as moments, etc, are estimated based on the data sample, in the deterministic methods a set of linear equations usually with large dimensions are established In this correspondence, we develop adaptive algorithms for multichannel (single-input multiple-output, or SIMO) blind identification with both statistic deterministic models Compared with the bloc algorithms, adaptive algorithms have their own advantages Instead of estimating the channel parameters after the entire bloc of data have been received, the parameter estimates are updated when each single signal is received The estimates are continuously improved while receiving new signals Thus, these algorithms are more amenable to real applications such as wireless communications because channels can be traced continuously (even for time-varying systems) Moreover, instead of a large amount of computation when all data are received, the computation is distributed at every step, each step involves only a small amount of computation /2$7 22 IEEE Manuscript received October 5, 999; revised January, 22 The wor of H F Chen was supported by the National Key Project of China the National Natural Science Foundation of China The wor of X-R Cao was supported by the Hong Kong RGC under Grant HKUST 623/98E H-F Chen is with the Institute of Systems Science, Academy of Mathematics Systems Science, Chinese Academy of Sciences, Beijing 8, China ( hfchen@iss3issaccn) X-R Cao is with Hong Kong University of Science Technology, Clear Water Bay, Kowloon, Hong Kong ( eecao@usth) J Zhu was with the Centre for Signal Processing, S2-B4b-5, Nanyang Technological University, Singapore He is now with Communication Department of ESS Technology Inc, Fremont, CA USA ( jiezhu@esstechcom) Communicated by T Fuja, Associate Editor At Large Publisher Item Identifier S (2)392-4

2 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY One distinguishable feature of our algorithms is that at each step of the recursion, the received data are directly used to update the estimation of the channel parameters; statistic quantities, such as moments, are not required The algorithms are based on the deterministic model developed in [5], the development of these algorithms was motivated by stochastic approximation methods [] [3], [5] Therefore, our algorithms differ from the previous ones Algorithms with stochastic models require accurate estimations for stochastic quantities such as moments, which usually need a large data sample, while the deterministic algorithms developed in [5] are bloc in nature they rely on least squares techniques for noisy channels However, as it is well nown, with fixed size of noisy data least squares estimations may not converge to the true values The problem considered in this correspondence is stated in Section II In Section III, we describe the fundamentals of our two algorithms, one for the noise-free case the other for noisy channels The convergence properties of the algorithms are stated in Section IV for different settings In Section IV-A, we assume that the observed data do not contain noise Two subcases are considered In the first subcase, only a finite noise-free data sequence is received; the set of data has to be used repeatedly, ie, after the recursion finishes at the end of the data sequence, it starts again from the beginning of the sequence, so on In the second case, the algorithm is applied to an infinite data sequence obtained from independent input signals The proofs of the convergence of the algorithms are given in the Appendix In Section IV-B, the convergence of the algorithm is given for independent input signals with additive channel noise The proof is presented in the Appendix A few simulation examples are given in Section V to illustrate the performance of the algorithms The results indicate that in addition to the desirable recursive nature, the algorithms perform reasonably well compared with the bloc algorithms One example shows how the algorithms can be used to trac a time-varying system In Section VI, we discuss the pros cons of the proposed approach II THE PROBLEM Consider a system consisting of p finite impulse response (FIR) channels with L being the maximum order of the channels Let s, = ; ; 2; ;N, be the one-dimensional (-D) input signal, x = (x () ;x(2); ;x(p) ), = L; L +; ;N, be the p-dimensional output signal, where N is the number of samples may not be fixed; the superscript denotes transpose, the superscript (i) denotes the ith component, the subscript is the time index Then where Equation () can be written as where L x = h is i; L () h i = h () i ; ;h (p) i : x (i) = h(i) (z)s (2) h (i) (z) = h (i) + h(i) z + + h(i) L zl ; i =; ;p (3) with z being the shift operator zs = s : The observations y may be corrupted by noise n y = x + n where n is a p-dimensional noise vector The problem is to estimate h i, i =; ;L, on the basis of observations fy g Note that s, x, n, y can be complex numbers The channels can be characterized by a p(l+)-dimensional vector h 3 First we define then let h (i) = h (i) ; ;h(i) L h 3 = h () ; ; h (p) : (4) We want to develop an estimate h() for h 3 at time =; 2;,on the basis of fy i;ig (or fx i;igwhen there is no noise) Our goal is to give an adaptive algorithm for h() so that h() is updated on-line on a sample path h()!! h3, where is an arbitrary constant III THE ALGORITHM A comparison between our results the approach presented in [5] will help to explain the significance of our approach Thus, our algorithms are stated in parallel to the approach of [5] Denote (i) = y(i) y(i) L ; '(i) = x(i) x(i) L ; i =; ;p; 2L where y (i) x(i) are the ith component of y x, respectively From (2), we have h (i) (z)x (j) = h (i) (z)h (j) (z)s = h (j) (z)h (i) (z)s = h (j) (z)x (i) ; 8 i; j =; ;p; =2L; 2L +; : (5) Using the observed data (y or x in the noise-free case), the above set of equations can be written in a matrix form X L h 3 = (6) where X L is a (N 2L+)[p(p)=2]2[(L+)p] matrix, N is the number of samples (see [5] for the specific form of X L ) Xu et al [5] proposed to solve (6) for h 3 in the deterministic case (noise-free), or to solve the following constrained least squares problem when the observations are corrupted by noise h 3 =arg min h X Lh 2 ; h = : (7) This approach describes the essential feature of the problem very well The only drawbac of the approach is that the matrix X L is usually very large since N is normally a large number, thus, solving (7) is time-consuming In addition, N is fixed; when new data are available, one has to solve (6) again to obtain new estimates Besides, as mentioned in the Introduction, applying least squares methods requires a large data sample, which maes the computation in the algorithm very complicated In this correspondence, we propose adaptive algorithms in which estimates for h 3 are obtained at every step =2L; 2L +; ;N by updating the estimates obtained at the previous step The estimates converge to h 3 when N goes to infinity or the data is repeatedly used In the noise-free case, this can be viewed as a recursive method for solving (6) For systems with noise, it is a stochastic approximation approach

3 26 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 Our algorithms can be presented as follows First, we define two p(p) 2 2 p(l +)matrices denoted as = (2) (3) (p) () () (2) (3) (2) () (p) (p) (2) (p) : (8) Note that X L in (6) is (N 2L +)times as large as 9 (i) We define (i) 8 as the matrix that has the same structure as (8) with replaced by ' (i) 8 i =; 2; ;p 9 (or 8 ) contains the observation x in the noise-free case (or y for noisy observations) in a window of size L +bac from time instant (ie, ; ; ;L); these are the observations that are related to signal s L It is worth emphasizing that neither 9 nor 8 depends on N in contrast to X L Let fa g be a sequence of step sizes to be specified later In the noise-free case, we tae an initial value h(2l ) 6=, define the adaptive algorithm for h() as follows: h( +)=h() a 8 y + 8 +h(); =2L; 2L +; : In the noisy observation case, let h(2l ) < define h( +) = h() a 9 y EN y + N + h() I [h()a (9 9 EN N )h()<] + h(2l )I [ h()a (9 9 EN N )h()] ; (9) =2L; 2L +; () where I [] is an indicator function, the superscript y denotes transpose with complex conjugate, N =9 8, =2L; 2L +; Without loss of generality, we can choose = In both cases, we will prove that h()!! h 3 where is a scalar multiple In fact, () is a truncated algorithm, which does not allow h() to be greater than or equal to = Once h() reaches the truncation bound, the estimate is pulled bac to the initial h(2l ) This does not mean that it repeats over the estimate from the beginning, because the step size has been reduced This truncation is needed only when the initial step size is not small enough to avoid the blow-up of the estimate Theoretically, we will prove that the number of such truncations is finite; in practice, truncation does not happen in our simulation presented in Section VI The necessity for truncation also depends on the noise When N =for all, truncation will not occur () becomes (9) for the deterministic case In the following two sections, we will study the convergence properties of the above two algorithms From the input sequence fs ;s ; ;s N ;N 2L +g, we form the (N 2L +)2 (2L +)Hanel matrix S N (2L +) S N (2L +) = s s s 2L s s 2 s 2L+ s N2L s N2L+ s N : () It is clear that the maximal ran of S N (2L+)is 2L+IfS N (2L+) is of full ran, then the finite sequence fs i ;; ;Ng in [5] is said to have linear complexity greater than or equal to 2L + We need the following lemma Lemma : Assume that L is nown the following conditions hold A h (i) (z), i =; ;p, given by (3) have no common factor A2 The Hanel matrix S N (2L +)given by () is of full ran (=2L +) Then h 3 is the unique (up to a scalar multiple) nonzero vector simultaneously satisfying 8 h 3 =; =2L; 2L +; ;N: (2) This lemma can be deduced from [5, Theorem ] For completeness, we provide a direct proof in the Appendix IV MAIN RESULTS A Noise-Free Observations We first consider the case where a finite noise-free data sequence fx L ;x L+ ; ;x N g is observed In this case, ' (i), i = ; ;p; 2L N, are available Thus, we can construct 8, =2L; ;N: Since the algorithm will not converge to the true value in a finite number of steps, we need to use these data repeatedly to form a sequence of infinitely many samples To this end, we denote 8 (N2L+)+i =8 i ; i =2L; ;N; =; ; : (3) The sequence can be divided into periods each with a length of N 2L + consisting of the same data as the first one For step sizes fa g, we need the following condition A3 a >, a + a, 8 = ; 2;, a!! = a = Theorem : Assume A A3 hold, L is nown Then, for h() given by (9) (3) with initial value h(2l ) h()!! h 3 : The proof of this theorem is given in the Appendix As shown in the proof, we have = (h3 ) y h(2l ) h 3 2 : However, because h 3 is unnown, this equation does not help us in determining the absolute value of h 3 It is worth noting that (h 3 ) y h(2l ) = is a noninteresting case, because the fact that h() tends to zero gives no information about h 3 So, the initial value h(2l ) should have a nonzero projection on h 3 Theorem shows that our algorithm converges to the true value (with a scaling factor ) if we repeatedly apply the sequence of finite samples Starting with any initial value h(2l ), after applying the adaptive algorithm to the sequence of samples, we obtain an estimate h(n) When N is small, h(n) may not be very accurate; in this case, we can

4 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY use h(n ) as the initial value apply the algorithm again on the same set of data (see (3)) When N is large, h(n) is accurate enough even after applying the adaptive algorithm to the sequence only once; repeatedly using the data is not needed In a sense, the algorithm discussed in this section can be viewed as a recursive approach to solving (6) The recursive nature maes the implementation more flexible: channel estimates can be updated when new samples are received One nice feature of the recursive algorithm is that N is not required to be fixed That is, the algorithm can be applied to an infinitely long sequence As the length of the sequence increases, the estimate improves eventually may converge to the true value under some very mild assumptions Next, we will prove that this is indeed true if the samples are chosen from a system with independent rom input signals We assume that the input signal fs i g is a sequence of infinitely many mutually independent rom variables that the observations do not involve noise; ie, n Similarly to Lemma for deterministic sequences, we have the following Lemma 2: Assume A holds, L is nown, s i is a sequence of mutually independent rom variables with Ejs ij 2 6= Then h 3 = h 3 =h 3 is the unique unit eigenvector corresponding to zero eigenvalue for the matrices j+(+)(2l+) B j; = E8 y i 8 i; 8 j ; 8 (4) j+(2l+) the ran of B j; is p(l +) The proof is given in the Appendix Since we have an infinitely long sequence of 8 y 8, we need some assumptions on the minimal nonzero eigenvalues of B j; Let min () denote the minimal nonzero eigenvalue of B ; Onfs ig we need the following condition A2 fs g is a sequence of mutually independent rom variables with Ejs j 2 6=, sup Ejs 2+ j < for some >, a (j+)(2l+) min (j) =: (5) j= A3 will be strengthened to the following A3 A3 holds a +=2 i < where is given in A2 It is obvious that if fs i g is an independent identically distributed (iid) sequence, then min (j) is a positive constant, hence (5) is automatically satisfied because of A3 Theorem 3: Assume L is nown, A, A2, A3 hold, h() is given by (9) with initial value h(2l ) Then h()! h 3 as where = h h(2l) h Again, the proof is given in the Appendix B Noisy Observations We now consider the general case where the observations are corrupted by noise: y = x + n, fs g is a sequence of mutually independent rom variables We will use the following conditions A3 A3 holds <c, 8, where c is a constant a a A4 A5 fs g fn g are mutually independent each of them is a sequence of mutually independent rom variables such that Ejs j 2 6=, sup fjs j + jn jg <; E 2+ < : min (j; ) >, 8 j, 8, where min (j; ) is the minimal nonzero eigenvalue of B j; defined in (4) Theorem 3: Assume L is nown, A, A3, A4, A5 hold, h() is given by () with initial value h(2l ) Then after a finite number of steps there is no truncation in () h()! h 3 ; as! where is a rom variable specified later by (55) The proof is given in the Appendix V SIMULATION EXAMPLES In this section, we present three computer simulation examples In the first example, we illustrate the convergence of the algorithm in a noisy time-invariant wireless environment In particular, we investigate the impact of the noise covariance matrix on the channel estimation In the second one, we compare the performance of the recursive algorithm with that of the bloc algorithms proposed in [5] In the third one, we illustrate the capability of the algorithm in tracing a time-variant channel In all the examples, the modulation of the input signal is quaternary phase shift eying (QPSK) the channel order is assumed to be nown a priori We repeat the simulation for N different rom environments compute the root-mean-square error (RMSE) as the measure of the accuracy of the channel estimates; the RMSE is defined as RMSE = h 3 N N i^h i h 3 2 (6) where ^h i is the channel estimate in the ith Monte Carlo run a scalar that minimizes the value of i^h i h 3 ; ie, ^h i 3 h 3 i = ^h i 3 ^h : (7) i Example : In this example, we select a transmitter with raisedcosine pulse p(t) whose rolloff factor is : The raised-cosine pulse p(t) is truncated to 4T where T is the baud period A two-ray wireless radio channel with a long delay multipath is used as the channel model The overall channel impulse response is h(t) =p(t) :7( + j)p(t :3T ): (8) The channel is corrupted by a Gaussian noise of 25 db The received signal is oversampled by a factor of 3(p = 3) thus each input symbol corresponds to a three-dimensional (3-D) output signal (a multichannel system with one input three outputs) The number of input symbols is fixed to thus the received signal consists of 3 2 samples All components of the initial value h(2l ) are set to one In each Monte Carlo run, the set of samples is repeatedly used times in the same manner as Theorem Thus, the channel estimates are updated 97 times The step size is initially chosen to be : is reduced by 2% each time the received sequence of signals is repeatedly used

5 28 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 Fig The Average RMSE in Monte Carlo runs Fig 2 Comparison of Xu s least squares the adaptive algorithm for varying symbol size 5 3 To study how sensitive the channel estimate is to the nowledge of RN, we implemented two sets of simulations In the first set, we use the true value for RN, ie, 25 db; in the second set, we simply set RN =in the algorithm (in ()) Each set contains Monte Carlo runs The RMSEs are computed after each update Fig presents the results after Monte Carlo runs It is apparent that for both cases (ie, when RN is nown RN is simply set to zero) the adaptive procedure is convergent When the number of updates increases, the effect of ignoring RN also increases However, the difference of the RMSEs for both cases always remains reasonably small This shows that the algorithm is relatively insensitive to errors in estimating the variance matrix RN Example 2: In the second example, we compare our algorithm with Xu s least squares algorithm in [5] To this end, we use the same channel responses as presented in [5, Table II] In addition, as in [5], we choose SNR to be 2 db let the number of symbols vary from 5 to 3 The step size for our algorithm is chosen as :4 initially is reduced by 8% after each channel estimate update For each case, 5 Monte Carlo runs are conducted In each Monte Carlo run the channel estimate is updated for 6 times Again, two cases using a true RN RN =are simulated Fig 2 shows the RMSEs of the channel estimate from Xu s paper [5] our simulation results It is not surprising that the adaptive algorithm () performs a channel estimate quite close to Xu s

6 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY Fig 3 Comparison of Xu s least squares the adaptive algorithm for varying SNR 4 db Fig 4 Envelopes of 3 uncorrelated radio rays, Doppler frequency = Hz We repeated the comparison under the above conditions This time the number of symbols is ept at but the signal-to-noise ratio (SNR) changes from to 4 db Fig 3 shows the results after 5 Monte Carlo runs Again, the results are quite similar The channel estimate with a nown RN is even better than Xu s result when SNR is poorer than 2 db Example 3: The channel in the above examples is assumed to be time-invariant, which is a common requirement by most blind channel identification methods As the algorithm developed is inherently adaptive, in this example we apply it to a time-varying three-ray wireless radio channel to evaluate its adaptability It is worthwhile to mention that the convergence of the algorithm is still an open problem in such time-varying environment The time-varying simulation channel used in this example is defined as h(t) =E (t)p(t) +E 2 (t)p(t :4T )+E 3 (t)p(t :3T ): (9) where E (t), E 2 (t), E 3 (t) are the envelopes of three radio rays at time t, generated by Jaes model with Doppler frequency Hz Fig 4

7 22 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 Fig 5 Channel finite impulse response, h, SNR = 35 db Fig 6 RMSE for the time-varying system with SNR = 35 db shows the change of these envelopes where E (t), E 2(t), E 3(t) are indicated by No Delay, Delay 4 T, Delay 3 T, respectively When the carrier frequency is 9 MHz in the group special mobile (GSM) context, -Hz Doppler frequency is equivalent to the receiver moving toward the transmitter with velocity 2 m/h Again, the shaping filter p(t) is chosen with raised-cosine pulse truncated to 4T The rolloff factor is :5 p =3 SNR = 35 db The channel order used is fixed to 4 Consequently, there are 5 channel coefficients to be identified As an example, the channel coefficients of subchannels h (2) are drawn in Fig 5 The symbols are unit-variant white QPSK with symbol rate 3 Hz The noise effect on the update is neglected, ie, RN = To study the tracing capability of our adaptive algorithm, we choose the initial channel parameters be their true values The estimated channel parameters are updated once at the arrival of each symbol signal with previous estimated parameters as their current initial values The step size is chosen to be :5 The estimated RMSE is presented by the curve in Fig 6 From the figure, the algorithm has a good tracing capability for the 35 symbols where the RMSE is always less than :6

8 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY Fig 7 RMSE for the time-varying system with SNR = 2 db Fig 7 shows the same results for SNR = 2 db One can notice that the performance of tracing deteriorates as the noise increases This example shows that even for the time-varying system shown in Fig 5, the algorithm can trace the channel reasonably well with one update each symbol The initial parameters in this simulation are chosen to be the true value Thus, a possible application is to combine this algorithm together with some existing identification techniques such as the training signal approach That is, to use training signals to identify the channel at the beginning of each interval, then apply our adaptive algorithm to follow the channel in the internal Only recursive algorithms can achieve such a goal Further research is certainly needed VI CONCLUSION In this correspondence, we proposed some adaptive algorithms for blind channel identification by exploiting channel diversity In these algorithms, the channel estimates are updated every time a new sample is received The convergence of the algorithms is proved Recursive algorithms are easyly implemented in real systems such as wireless channels The algorithms converge to the true channel parameters even when the number of samples is finite if the samples are repeatedly used In this case, the algorithm becomes a recursive version of the approach proposed in [5] for deterministic inputs The algorithm for noisy observations requires nowledge of the covariance RN of the noise In principle, RN can be estimated based on the sequence of the observed data; the implementation details remain a future research topic However, from the simulation results, we may expect that the estimate does not depend so much on RN As shown in Fig, one can simply set RN the additional RMSE induced is reasonably small The channel order L is, in general, unnown in advance However, if Condition A4 holds if each of the sequences are iid, then the covariance matrix of observations R y () =E[(y i Ey i )(y i+ Ey i+ )] is independent of i, it does not equal zero for =; ; 2; ;L is for = L + Using this property replacing R y () by its sampling approximation, we can obtain consistent estimates for L as the data size increases to infinity In practice, one may not estimate L in advance, but simply tae it sufficiently large This ind of overparametrization technique wors quite well in modeling for systems lie () The adaptive nature of the algorithms allow them to be applied to time-varying systems We did one simulation example to explore this property The example shows that the tracing ability is reasonably good with 35 symbols Further research is needed to study the applicability to time-varying systems APPENDIX Proof of Lemma : From (5), it is readily seen that h 3 satisfies (2) It remains to prove the uniqueness Assume that h =[(h () ) ; ; (h (p) ) ] is also a solution to (2) but h 6= h 3, where h (i) =[h (i) (i) ; ; h L ] is (L +)-dimensional, i =; ;p Define h (i) (z) = h (i) + h (i) (i) z + + h L zl : Since h is a solution to (2), we have h (i) (z)x (j) by (2) h (j) (z)x (i) =; 8 i; j =; ;p; =2L; ;N(2L +); h (i) (z)h (j) (z) h (j) (z)h (i) (z) s =; 8 i; j =; ;p; =2L; ;N(2L +): (2) The above set of equations implies h y (i; j)s N (2L +)=; 8 i; j =; ;p where h(i; j) denotes the (2L +)-dimensional vector composed of coefficients of h (i) (z)h (j) (z) h (j) (z)h (i) (z) written in an increasing order of z By A2, h(i; j) = In other words h (i) (z)h (j) (z) h (j) (z)h (i) (z) =; 8 i; j =; ;p: (2)

9 222 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 For a fixed j, (2) is valid for all i =; ;p, i 6= j Therefore, all roots of h (j) (z) should be roots of h (j) (z)h (i) (z) for all i 6= j By A, it follows that all roots of h (j) (z) coincide with roots of h (j) (z) This means that there is a constant j such that h (j) (z) = j h (j) (z); Substituting this into (2) leads to 8 j =; ;p: i h (i) (z)h (j) (z) j h (j) (z)h (i) (z) = hence i = j =, 8 i; j =; ;p Thus, we conclude that h = h 3 Proof of Theorem : First, we decompose h(2l ) h(), respectively, into orthogonal vectors h(2l ) = h 3 + h (2L ); h() = hy ()h 3 h 3 2 h3 + h (); h y ()h 3 =; =2L; 2L; : (22) If h 3 serves as the initial value for (9), then by (2) h() h 3 Again, by (2) h 3y I a 8 y h () =; 8 2L : We then have h ( +)=h () a 8 y + 8 +h () (23) h() =h 3 + h (): (24) Therefore, for proving the theorem it suffices to show h ()! as! Denote A + = I a(+)(n2l+) 8 y (+)(N2L+) 8 (+)(N2L+) I a (+)(N2L+)2 8 y (+)(N2L+) 8 (+)(N2L+) Denote by min the minimal nonzero eigenvalue of N2L 8 y i+ 8i+: Let h be an arbitrary p(l+)-dimensional vector such that h y h 3 = Then h can be expressed by p(l+) h = iu i where u i, i =; ;p(l +), are unit eigenvectors of N2L 8 y i+ 8 i+ corresponding to its nonzero eigenvalues Noticing N2L y h 8 y i+ 8 i+h min h 2 from (25) (26), we have for h g + 2 g y A +g g 2 mina (+)(N2L+) g 2 2 Since g min 2 a i(n2l+) g 2 : (28) a i(n2l+) a i = N 2L + N2L+ it follows that the right-h side of (28) tends to zero as! From (23), it is seen that h () is nonincreasing for Hence, the convergence g!! implies h ()!! Proof of Lemma 2: Since fs i g is a sequence of mutually independent rom variables Ejs ij 2 6=, it follows that where ES () 4L (2L +)S()y 4L (2L +)> ; 8 (29) I a (N2L+) 8 y (N2L+)+ 8 (N2L+)+ s s + s +2L g = h ((N 2L + )): S () 4L = s + s +2 s +2L+ : (3) From (23) it follows that g + = A + g : (25) Since a! 8 is uniformly bounded with respect to, there is a large such that A + I 2 By (3) (+)(N2L+) (N2L+) (+)(N2L+) (N2L+) a i8 y i+ 8i+; A2 A ; 8 y i+ 8i+ = N2L 8 : (26) 8 y i+8i+ (27) which, by Lemma, is of ran p(l+) has the unique (up to a constant) multiple eigenvector h 3 corresponding to the zero eigenvalue s +2L s +2L+ s +4L Proceeding along the same lines as the proof of Lemma, we obtain (2), 8 i; j =; ;p, 8 2L This implies that h y (i; j)es () 4L h y (i; j)s () 4L (2L +)= (2L +)s()y 4L (2L +)h(i; j) =; 8 i; j =; ;p; 8 2L: (3) From (29) (3), it follows that h(i; j) = Following the proof of Lemma, we conclude that h 3 is the unique unit vector satisfying (2) Consequently, h 3 is the unique unit vector satisfying E8 y i 8 ih 3 =; 8 i: j + (2L +) i j +( + )(2L +); ; 8 j : This shows that B j; is with ran p(l+), 8 j, 8, h 3 is its unique unit eigenvector corresponding to zero eigenvalue

10 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY Proof of Theorem 2: We have shown (2) in the proof of Lemma 2 So, we still have (23) (24),, again, it suffices to show that h ()!! With N replaced by 4L in the definitions for A g, we again obtain (25) Since a!!, fe8y + 8 +g is bounded a i (8 y i+ 8 i+ E8 y i+ 8 i+) converges as by A2, there is a large such that (+)(2L+) A + I a i E8 y i+ 8 i+ (2L+) (+)(2L+) a i 8 y i+ 8 i+ E8 y i+ 8 i+ (2L+) + o a (2L+) I 2 (+)(2L+) (2L+) a ie8 y i+ 8i+: Let h be an arbitrary p(l+)-dimensional vector such that h y h 3 = Then, by Lemma 2 hence g + 2 g y A +g h y B ; h min ()h 2 g 2 a (+)(2L+) min () g 2 ; : 2 Therefore, g min(i) 2 a (i+)(2l+) g 2 which tends to zero since min (i)a (i+)(2l+) = by A2 This implies h ()!! Proof of Theorem 3: Set h(2l ) = Without loss generality for convenience in the proof, we tae = 4 Define R i =8 y i 8 i E8 y i 8 i +8 y N i i + N y i 8 i + N y N i i EN y N i i: (32) Then 9 y i 9 i EN y i N i = E8 y i 8 i + R i : (33) Denote by ; = ; 2;, the truncation times, ie, h( ) = h(2l ) From () we have +j h( + j) =h(2l ) a i E(8 y i+ 8 i+h(i) +R i+ )h(i) h( + j) < ; j =; ; + ; 8 : (34) Denote h 3 = h 3 =h 3 Let be a Hermite matrix Denoting U = V h 3 h () = VV y h(); h() = h 3y h() (35) we have Noticing 8 i h 3 =, 8 i by (34), (36) we find that h() =h ()+h()h 3 : (36) VV y E8 y i+ 8i+ = E8y i+ 8i+ +j h ( + j) =h (2L ) a y ie8i+ 8i+h (i) +j a ivv y R i+h(i) (37) +j h( + j) =h(2l ) a 3y ih R i+h(i); We complete the proof in five steps Step : First, we show that j =; ; + : (38) a i R i+ h(i) < as (39) By A4, D i =8i N i + N y 8 i i+ N y N i i EN y N i i is a martingale difference sequence with sup i ED i + < Since it follows that a + i < h(i) < a id i+h(i) < as (4) by the convergence theorem for martingale difference sequences Since 8 y i 8i E8y i 8i is independent of 8y i+2l+ 8i+2L+ E8 y i+2l+ 8 i+2l+ sup E8 y i 8 i E8 y i 8 i + < i we also have a i 8 y i+ 8 i+ E8 y i+ 8 i+ h(i) < as Incorporating the above inequality with (4) yields (39) Step 2: Next, we show that for any j T > with m(j; T ) +, there is a c >, which possibly depends on the sample path but is independent of, j, T, such that h(i+)h(j)c t; 8 i: j im(j; t); 8 t 2 [; T] (4) where m(j; T )=max : j a i T : By A4, there is a c >, possibly depending on the sample path, such that E8 y i+ 8 i+ + R i+ <c ; 8 i: Then, by noticing h(i) <, (4) immediately follows from (34)

11 224 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY 22 Step 3: Third, we show that for any h = VV y h 6=, there exists an N which is large enough, such that m(j; t) h y a i E8 y i+ 8 i+h th 2 ; j 8 j N; 8 t 2 (; T] (42) where = > with c given by A3 A5, 2(2L+)c respectively (j; ) (j; ) Let [u ; ;u p(l+) ; h 3 ] be the Hermite matrix composed of eigenvectors of B j; By Lemma 2, h 3 is its unique eigenvector corresponding to the zero eigenvalue Since h y h 3 =, h can be expressed as p(l+) h (j; ) = i u i : Then p(l+) h y B j; h min (j; ) j i j 2 h 2 ; 8 j; 8 : (43) Noticing that by A4 (43), the fact that E8 y i+8i+ is bounded with respect to i fa i g is nonincreasing, we have m(j;t) h y a y ie8i+ 8i+h j h y [ ] h 2 = [ ] = j+(+)(2l+) j+(2l+) a ie8 y i+ 8i+h a j+(+)(2l+) ; (44) where [x] denotes the integer part of the real number x Since a a c c, from (44) it follows that for a large enough j m(j;t) h y a y ie8i+ 8i+h j h 2 [ ] = h 2 (2L +)c 2L+ 2L + = 2L a j+(+)(2l+) [ ] 2L a j+(2l+)+i h 2 [ ](2L+)+2L = (2L +)c 2L+ a j+i h 2 m(j;t) = (2L +)c 2L+ a i + o() th 2 where o()! as j! Step 4: We now show that the number of truncations in () is finite To this end, let us first assume that the converse is true:!! Then, by (39) h(2l ) =, there exists a large K such that 4 +j a i R i+ h(i) 8 +j h y (2L )h 3 h 3 a i R i+ h(i) < 3 8 ; 8 j ; 8 K: (45) By the definition of +,wehave h(2l ) a i E8 y i+ 8 i+ + R i+ h(i) : Incorporating the above with (45) implies Define h (2L ) a i E8 y i+ 8 i+h (i) 5 8 ; h (2L ) a i E8 y i+ 8 i+h (i) a i VV y R i+ h(i) > 2 : (46) j() = min j: j< + ; h (2L ) a i E8 y i+ 8 i+h (i) a i VV + R i+ h(i) > 2 : (47) By (39), a i!, the boundedness of E8 y i+ 8 i+h (i),wehave a i E8 y i+ 8 i+h (i) a i VV y R i+ h(i)! i! : Thus, j() is well-defined Consequently h( + j) =h(2l ) +j a i (E8 y i+ 8 i+ + R i+ )h(i) (48) h ( + j) =h (2L ) +j Next, we say that the sequence +j a i VV y R i+ h(i); a i E8 y i+ 8 i+h (i) 8 j: j j(); 8 K: (49) fh ( + i) 2 ; l ;l +; ;m g crosses an interval [a; b] with <a<b<,ifh ( + l ) 2 a, h ( + m ) 2 b, a<h ( + i) 2 <b, 8 i: l <i<m, if there is no truncation for any i: l i m From (47) (49) h(2l ) = it is seen that the sequence 4 fh ( + i) 2 ;; ; ;j()gcrosses the interval [ 6 ; 4 ] for each K Without loss of generality, we may assume h ( + l ) converges: h ( + l )! h as! It is clear that h = 4 h y h 3 = By (4), there is no truncation for h(i) i = + l ; + l +; ;m( + l ;T)+

12 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 5, MAY if T is small enough Then by Taylor s expansion (39) (4), it follows that h (m( + l ;T)+) 2 h ( + l ) 2 m( +l ;T) = h y a i +l E8 y i+ 8i+h (i) +VV y R i+h(i) + o(t ) m( +l ;T ) = h y a i E8 y i+ 8 i+(h + h (i) h ) +l m( +l ;T ) h y a i VV y R i+ h(i)+o(t ) +l m( +l ;T ) = h y a y ie8i+ 8i+h (i) +o() + o(t ) (5) +l where o()! as! o(t )! as T! By (42), for a large a small T,wehave h (m( + l ;T)+) 2 h ( + l ) 2 T 2 h 2 : (5) By (4), h (m( + l ;T)+)! T! h ( + l ): Noticing that h ( + l ) 2 6 h ( + m ) 2, by the 4 definition of crossing we see that m( + l ;T)+< + m for small enough T In order words h (m( + l ;T)+) 2 > 6 : (52) Letting! in (5) leads to a contradictory inequality lim sup! h (m( + l ;T)+) 2 < 6 T 2 h 2 (53) since the left-h side of (53) should be greater than or equal to 6 by (52) The obtained contradiction shows that! < (ie, there are only a finite number of truncations) as Step 5: Now we are ready to complete the proof of the theorem We have shown that starting from a rom, the algorithm () suffers from no more truncation Now, let us first show that h () 2 converges If this were not true, then lim inf! h () 2 < lim sup h () 2! Since lim! = <, from (38) it follows that lim! h() =hy (2L )h 3 Finally, by the fact h ()!! a 3y ih R i+h(i) = : (55), from (36) (55) it follows that lim! h() =h 3 : REFERENCES [] A Benveniste, M Metivier, P Priouret, Adaptive Algorithms Stochastic Approximations Berlin, Germany: Springer, 99 [2] H F Chen, Stochastic approximation its new applications, in Proc 994 Hong Kong Int Worsh New Directions of Control Manufacturing, 994, pp 2 2 [3] H F Chen Y M Zhu, Stochastic Approximation (in Chinese) Shanghai, China: Shanghai Scientific Technical Publishers, 996 [4] Z Ding Y Li, On channel identification based on second order cyclic spectra, IEEE Trans Signal Processing, vol 42, pp , May 994 [5] H J Kushner G Yin, Stochastic Approximation Algorithms Applications Berlin, Germany: Springer, 997 [6] R Liu, Blind signal processing: An introduction, in Proc 996 Int Symp Circuits Systems, vol 2, May 996, pp 8 83 [7] H Liu, G Xu, L Tong, T Kailath, Recent developments in blind channel equalization: From cyclostationarity to subspacs, Signal Processing, vol 5, pp 83 99, 996 [8] E Moulines, P Duhamel, J-F Cardoso, S Mayrargue, Subspace methods for the blind identification of multichannel FIR filters, IEEE Trans Signal Processing, vol 43, pp , Feb 995 [9] Y Sato, A method of self-recovering equalization for multilevel amplitude-modulation, IEEE Trans Commun, vol COM-23, pp , June 975 [] D Sloc, Blind fractionally-spaced equalization, perfect-reconstruction filter bans multichannel linear prediction, in Proc IEEE ICASSP, Adelaide, Australia, May 994, pp [] L Tong, G Xu, T Kailath, Blind identification equalization based on second-order statistics: A time domain approach, IEEE Trans Inform Theory, vol 4, pp , Mar 994 [2], Blind channel identification based on second-order statistics: A frequency-domain approach, IEEE Trans Inform Theory, vol 4, pp , Mar 995 [3] L Tong S Perreau, Multichannel blind identification: From subspace to maximum lielihood methods, Proc IEEE, to be published [4] A J van der Veen, S Talwar, A Paulraj, Blind estimation of multiple digital signals transmitted over FIR channels, IEEE Signal Processing Lett, vol 2, no 5, pp 99 2, May 995 [5] G Xu, L Tong, T Kailath, A least-squares approach to blind identification, IEEE Trans Signal Processing, vol 43, pp , Dec 995 [6] J Zhu, X-R Cao, R-W Liu, A blind fractionally-spaced equalizer using higher order statistics, IEEE Trans Circuits Syst, II, vol 46, pp , June 999 [7] J Zhu, Z Ding, X-R Cao, Column anchoring zeroforcing blind equalization for wireless FIR channels, IEEE J Select Areas Commun, vol 7, pp 4 423, Mar 999 h () 2 would cross a nonempty interval [a; b] for infinitely many times As shown above, this is impossible Therefore, h () 2 converges If the limit of h () were not zero, then there would exist a convergent subsequence h ( j)! h 6= Replacing + l in (5) by j, from (5) it follows that h (m( j ;)+) 2 h ( j ) 2 T 2 h : (54) Since h () 2 converges, the left-h side of (54) tends to zero, which maes (54) a contradictory inequality Thus, we have proved h ()!! as