On the Robustness of Lattice Reduction Over. Correlated Fading Channels

Transcription

1 On the Robustness of Lattice Reduction Over Correlated Fading Channels Hossein Najafi, Student Member, IEEE, Mohammad Erfan Danesh Jafari, Student Member, IEEE, and Mohamed Oussama Damen, Senior Member, IEEE, Abstract In multiple-input multiple-output (MIMO) systems, the use of lattice reduction methods significantly improves the performance of suboptimal algorithms. Taking advantage of the temporal correlation of a Rayleigh fading MIMO channel, the robustness of lattice reduction is investigated in this work by adaptively updating the reduced lattice basis. It is shown that by using the result of the previous channel realization, we can achieve the same error performance as the original lattice reduction, but with lower computational complexity. Another method is also considered for lattice reduction that further reduces the complexity at the cost of a degradation in the error performance by employing a measure that offers an appropriate tradeoff. These adaptive methods can be used in conjugation with any lattice reduction algorithm and in any MIMO scenario over correlated channels that requires lattice reduction, such as the MIMO detection and broadcast systems. Index Terms Lattice reduction, multiple-input multiple-output (MIMO) channels, lattice detection, Rayleigh fading channel. This work was supported by an NSERC-Nortel Network industrial reseach chair, IRCSA The authors are with the Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1, {hnajafi,medanesh,modamen}@uwaterloo.ca.

2 2 I. INTRODUCTION In recent years, the use of multiple antenna for communication over wireless fading channels has attracted a large interest. In these systems, the outputs can be described as a linear combination of the inputs corrupted by the additive noise. In point to point MIMO systems, decoding represents a challenging problem. Recently, many researchers have used some of the methods from lattice theory as solutions to this challenging problem [1], [2]. In fact, the MIMO detection problem translates to the closest lattice point search (CLPS) problem in lattice theory. Lattice reduction methods have proved themselves to be powerful tools in solving the CLPS problem. Interestingly, lattice reduction is also a powerful tool for improving the approximate detection techniques [3], [4]. There is no unique definition for lattice reduction, and hence there exist many different methods for lattice reduction. Among these, the LLL algorithm due to Lenstra, Lenstra, and Lovász, [5] is the most practical one due to its efficiency in finding near orthogonal vectors with short norms. Generally, in most of the recent works (e.g, [6] and references therein) the complexity of using the LLL algorithm is ignored. This can be justified in situations where the channel variations are slow enough to make it possible to use the result of the LLL reduction for quite a large number of received signals. In this way, at the beginning of each frame the lattice reduction is performed on the channel, and for the rest of the frame the channel is assumed to be constant. In this work, the channel is not assumed to be constant throughout the frame and the channel realizations have a temporal correlation. This scenario is justified in many practical situations where the channel slowly varies through the frame, or the frame length over which the channel can be considered constant is small. The robustness of lattice reduction is investigated here by adaptively updating the reduced lattice basis. In fact, instead of performing lattice reduction on each channel realization, we consider two methods that takes advantage of this correlation to reduce the complexity of lattice reduction. The main idea is to use the results of the

3 3 previous or the past channel realizations to perform an efficient reduction of the new channel realization. In the first method, we consider the observation that the lattice reduction result of the previous channel realization can still reduce the actual channel if the channel variation is slow enough. Thus, applying the lattice reduction algorithm on the nearly reduced channel matrix which is computed from the previous result should cost much less than the original unreduced channel matrix. This method achieves the same error performance as the original lattice reduction method but significantly reduces the computational complexity. If further saving in computational complexity is needed, the part that performs the lattice reduction algorithm on the nearly reduced channel matrix can be removed. This means that if the variations in the channel are small enough, we can use the same transformation matrix to reduce the later channel matrices, and this translates the lattice reduction algorithm to just performing a matrical multiplication. Based on this idea, the second method is introduced by using a measure that offers a tradeoff between the performance and the complexity. By applying the results from [7], it is shown that the second proposed method achieves the maximum receive diversity in a V-BLAST transmission if the updating measure is selected properly. Note that these adaptive methods can be used along with any lattice reduction method. This makes it a powerful tool in many problems in communication theory that use the lattice reduction algorithms over slowly varying channels. This paper is organized as follows. The channel and system model are described in Section II and the lattice reduction aided detection is discussed in Section III. In Section IV, we introduce the adaptive lattice reduction aided detection methods. Section V gives the simulation results and Section VI concludes the paper.

4 4 II. SYSTEM MODEL Consider a MIMO system with M transmit and N receive antennas. If we assume s c = [s c 1,..., s c M ]T, y c = [y1, c..., yn c ]T, w c = [w1, c..., wn c ]T and the N M matrix H c as the transmitted signal, the received signal, the noise vector and the channel matrix, respectively, then one has y c = H c s c + w c (1) where the channel is assumed to be Rayleigh, i.e., the elements of H c, h c i,j, are independent and identically distributed (i.i.d), with zero mean and unit variance complex Gaussian distribution. Note that superscript c denotes complex value here. In addition, the noise in this model is considered to be complex Gaussian. The complex input signal s c is composed of components, s c i, chosen from a Q 2 -QAM constellation with energy ρ, in which ρ can be interpreted as the M signal-to-noise ratio (SNR) observed at any receive antenna. The vector s c can be obtained by Gray mapping from the vector of data bits, to the QAM constellation points. Note that the number of transmitted bits in each channel use is 2 log 2 (Q) M. This system can be transformed to its real counterpart by using the following transformations defined for vectors and matrices, [ ] u c u = R{u c } T I{u c } T H c H = R{H c } I{H c } I{H c } R{H c } (2) in which, now H is a n m real matrix with m = 2M and n = 2N and the components of s are chosen with uniform probability from a Q-PAM constellation, with energy ρ 2M. We can further simplify the model by mapping the equivalent PAM signals to integers using the following mapping. s = κc + v (3)

5 5 in which, elements of c are in {0, 1,..., Q 1}, κ is a constant related to the PAM constellation energy, and v is a constant vector. Substituting (3) into (2) our system simplifies to, y = H(κc + v) + w (4) where vector c has integer elements. Hence, our problem over MIMO channel changes to the detection of a lattice point transmitted over a linear channel with additive white Gaussian noise [8]. In this work, we assume a Rayleigh fading channel with a power spectral density (PSD) limited by the maximum Doppler frequency. In the Rayleigh fading model, the variation of the channel is captured by its auto correlation function (ACF). The second order statistics generally depend on the geometric properties of the area, the pace that the mobile user is moving at, and the characteristics of the antennas. A common assumption is that the propagation path consists of a two dimensional isotropic scattering, with vertical monopole antennas at the receiver [9]. In this case, the theoretical PSD of a path is of the form S(f) = 1 πf d 1 ( f f d ) 2 (5) where f d is the maximum Doppler frequency. The in-phase and quadrature Gaussian processes (of the channel) each should have the autocorrelation sequence R[n] = J 0 (2πf m n ) (6) in which f m = f d T s is the maximum Doppler frequency normalized by the sampling rate 1/T s. Furthermore, in this model the in-phase and quadrature components are zero mean and independent of each other.

6 6 III. LATTICE REDUCTION AIDED DETECTION Following the system model in section II, the maximum-likelihood (ML) solution to the MIMO system in (1) is given by, ŝ c = arg min s c X M y c H c s c 2 (7) where X M denotes the M-dimensional hyper-cube with components from Q 2 -QAM constellation. Using the mentioned transformations which resulted in (4), the minimization in (7) can be rewritten as, ĉ = arg min y Hv Hκc 2 c U = arg min c U y H c 2 (8) in which U refers to the hyper-cube {0, 1,..., Q 1} m R m, y = y Hv, and H = κh. For a general H, finding the optimal solution to this problem has exponential complexity and hence approximate methods such as zero forcing (ZF) and decision feedback equalizer (DFE) are used in practical systems. However, the error performance of ZF and DFE algorithms is far from ML specially in high SNRs. Therefore, further efforts have been done to develop methods with low complexity and error performance near the ML solution. If the boundaries of the search region in (8) are relaxed to be the whole cubic lattice Z m instead of U, lattice decoding or detection is performed which is related to the Closest Lattice Point Search (CLPS). Solution to this problem is vastly investigated in lattice theory (e.g., [10], [6], [11], [1], and [2]). In fact, efficient lattice detection methods perform lattice reduction followed by a closet point search algorithm such as sphere decoder. However, the complexity of the closest point search problem is shown to be NP-hard in general. Therefore, approximate lattice decoders with lower complexity were proposed. Interestingly, lattice reduction methods are powerful tools for improving the error performance of approximate detection techniques. Thus, they are used in conjunction with ZF and DFE decoders. The idea was first proposed

7 7 by Babai in [12] and then employed in [3] and [4] for detection and precoding in MIMO systems. Performing the lattice reduction on H results in B = H G (9) where B is the reduced basis and G is a unimodular transformation matrix. 1 In the lattice reduction aided detection, the minimization (8) can be approximated as Using the reduced matrix in (9), one can equivalently solve ĉ arg min c Z m y H c 2. (10) where c = G 1 c. ĉ arg min c Z m y H GG 1 c 2 = G arg min c Z m y Bc 2 (11) The goal of the lattice basis reduction is to find a new basis, that the columns of the new generator matrix B have small norms and they are as orthogonal as possible. This concept was proposed more than a century ago. There is no unique definition for lattice reduction. In fact, the problem of finding the shortest vector in general is considered to be NP-hard. In 1982, Lenstra, Lenstra, and Lovász (LLL) [5] proposed a breakthrough algorithm for lattice reduction. A further improved version was developed by Schnorr and Euchner [13] which is called deep insertion LLL. This modification gives significantly shorter vector in comparison to the original LLL algorithm. The complexity of the original LLL is polynomial in lattice dimension. The complexity of the deep insertion LLL in worst case can be exponential, but simulations show that on average it does not require much more iterations than the original LLL [14]. 1 Unimodular matrices and their inverses have integer elements GZ m = Z m.

8 8 To measure how reduced a basis is, the orthogonality defect factor is used. It is defined as δ(b) ( b 1 2 b 2 2 b m 2 ), (12) det B H B where b i s are the columns of the basis B. Clearly, δ(b) 1 with equality for an orthogonal basis. In fact, the goal of lattice reduction is to determine a basis with smaller orthogonality defect factor. Therefore, for a lattice with bases B 1 and B 2, we can say that B 1 is better reduced than B 2 if δ(b 1 ) < δ(b 2 ). When different bases of the same lattice are compared, the product of the norms can also be used because the determinant is equal for all of them. We define this product as D(B) b 1 b 2 b m. (13) IV. ADAPTIVE LATTICE REDUCTION AIDED DETECTION A. Method I Here we assume that the Rayleigh fading channel is slowly varying during the transmission frame. Our method hinges on the observation that a lattice basis transformation (i.e., the unimodular matrix associated with the LLL) of the previous channel realization can still reduce the actual channel (though not in the sense of LLL) if the channel variation is slow enough. Then, applying the LLL reduction on the already reduced channel matrix should cost much less than applying the LLL on the original unreduced channel matrix. More details are given in the following. The output of the LLL algorithm is a reduced matrix and a transformations matrix. The transformation matrix is used to convert the initial channel matrix to the reduced one. This relation can be expressed as, B 1 = H 1 G 1 (14) in which B 1, H 1, G 1 are the reduced matrix, channel matrix, and the transformation unimodular matrix, respectively.

9 9 Without loss of generality, assume the complex channel model as H k = ah k 1 + Z k (15) where 0 a < 1 and Z k has i.i.d entries distributed as circular complex Gaussian with zero mean and variance equal to ɛ 2 = σ 2 (Z k ) = 1 a 2. Here, a shows the channel variations between consecutive transmissions. We consider a small value for f m in the MIMO fading channel model and the variations in each element of the channel will be modest through time. Thus, it is assumed that a is close to one and so ɛ is close to zero. Note that this channel model which shows the correlation simply and basically is used here to analyze the Method I. In the simulation part, the Jakes model which was introduced in Section II is used. Considering these small changes, it seems quite reasonable to make use of the previous transformation for H 1 in computing a reduced matrix for the new channel H 2. Using the previous transformation on the new matrix results in, B 2 H 2 G 1 = (ah 1 + Z 2 )G 1 (16) = ah 1 G 1 + Z 2 G 1 = ab 1 + Z 2 G 1 (17) Using the fact that G 1 is a unimodular matrix, B 2 is still a basis for the space spanned by columns of H 2. As B 1 is already LLL reduced, it satisfies all the Lovasz and size reduction conditions. If the other term in equation (17) is small enough, the Gram-Schmidt coefficients of the resulting matrix in the right-hand side of equation (17) are close to those of the B 1. Therefore, performing LLL on B 2, does not require many more basis updates. Roughly speaking, starting from an almost LLL reduced matrix results in less complexity for LLL reduction. Thus, using B 2 as the initial input for the LLL algorithm in comparison with starting from the original channel matrix H 2, reduces the complexity of the LLL algorithm significantly.

10 10 Performing LLL on B 2 results in, B 2 B 2 G 2 = (H 2 G 1 ) G 2 = H 2 G 2 (18) in which G 2 G 1 G2 is the transformation matrix for reducing H 2. Note that the proposed adaptive method of matrix reduction results in a reduced matrix which might be different from the result of applying the LLL algorithm directly on H 2, but as both resulting matrices are LLL reduced, the error performance of the MIMO decoder is the same. In fact, the orthogonality defect factor and the product of the norms for the near reduced basis B 2 is close to the one for the previous reduced basis, B 1, if the channel slowly changes. Assume B 1 = [b 1 b 2 b M ], B 2 = [b 1 b 2 b M ] and G 1 = [g 1 g 2 g M ]. The product of the norms for the near reduced basis, defined in (13), can be written as D(B 2) = b 1 b 2 b M = H 2 g 1 H 2 g 2 H 2 g M = (ah 1 + Z 2 )g 1 (ah 1 + Z 2 )g 2 (ah 1 + Z 2 )g M (19) where the definition of B 2 and H 2 are used from (17) and (15) respectively. By applying the triangle inequality on (19) for the norms, we have M D(B 2) [a H 1 g i + Z 2 g i ] i=1 M [ = H 1 g i a + Z ] 2g i H i=1 1 g i M [ = b i a + Z ] 2g i H i=1 1 g i M [ = D(B 1 ) a + Z ] 2g i H 1 g i in which B 1 = H 1 G 1. Furthermore, the ratio of the norms in (20) can be written as i=1 Z 2 g i H 1 g i = Z 2H 1 1 b i b i (20) Z 2 H (21)

11 11 where the inequality came from the matrix norm definition, A 2 = max x 0 Ax 2 / x 2. By applying (21) in (20), we have in which Z 2 has unit variance. D(B 2) D(B 1 ) M [a + Z 2 H 1 1 ] i=1 = D(B 1 )[a + Z 2 H 1 1 ] M = D(B 1 )[a + ɛ Z 2H 1 1 ] M. (22) To further analyze the upper bound, we need to investigate the norms of the two matrices which is related to the maximum and minimum singular values of a Gaussian distributed matrix. (see [15] and [16]). Here, for our purpose we just compute the upper bound numerically for small changes in fading parameters. Fig. 1 shows the CDF of D(B 2)/D(B 1 ) for the upper bound and also experimental channel matrices and compares it with D(H 2 )/D(B 1 ). A 4 4 system with ɛ =.01 for the channel model in this section is assumed. As it can be seen from the figure and the upper bound (22), in Method I, the norm products for near reduced matrix is very close to the previous one which has an LLL reduced basis with a locally minimum products of the norms. In fact, in the LLL algorithm after each size reduction the product of the norms becomes smaller. Thus, applying the LLL on B 2 = H 2 G 1 with smaller products of the norms instead of H 2 needs fewer reduction steps. In other words, we have already done most of the reduction steps during the previous reduction. B. Method II If further saving in computational complexity is required, one can omit the part that performs the LLL reduction algorithm on B 2, and use B 2 itself as the reduced basis for H 2. This means that if the variations in the channel are slow enough we can use the same unimodular transformation matrix to reduce the later channel matrices, and this translates the

12 12 lattice reduction algorithm to just performing a matrical multiplication. If we generalize the bounds in (22) for small channel variations, we can see that the product of the norms (and orthogonality defect factor) for consecutive realizations are close to each other. In other words, the average value for the product of the norms and orthogonality defect factor are bounded within a time frame in which the tightness of the bounds is related to the frame length. Considering that the method proposed is an approximate one, as the difference between the current channel and the reference channel gets larger with time, the effect of basis reduction using the unimodular transformation matrix decreases. Therefore, it is required to update the unimodular transformation matrix after it no longer produces acceptable results. Note that, the update rate can be used as a tradeoff between complexity and accuracy. We propose to update the unimodular transformation matrix using the orthogonality defect factor defined in (12). To adaptively track the channel variations and be sure that the orthogonality defect factor for the adaptive reduction stays bounded, we use a lower and an upper bound for the orthogonality defect factor in Method II. Therefore, in each channel realization we check the ratio of the near reduced basis to the reference one (the last time the LLL was performed). If it is within the assumed bound, the reduced basis is computed just by multiplying the reference unimodular matrix to the new channel matrix, otherwise the LLL is performed on the new channel and the reference basis and its corresponding unimodular are reset. If we denote the last time we performed LLL as the reference point, the channel at that time by H ref, and the defect factor of the output of the LLL as δ(b ref ), the rule for updating the unimodular transformation matrix can be defined as follows: In the ith channel realization after the reference one, If 1 α δ (H ig ref ) δ(b ref ) α, (23) then keep the unimodular transformation matrix unchanged, Otherwise update the unimodular transformation matrix by performing the lattice reduction algorithm.

13 13 To define the above interval, α is chosen according to f m, computational complexity constraints and the desired error performance. The minimum value for α is 1 which denotes the non-adaptive reduction method. The greater α provides less complexity but worse performance. To investigate when the adaptive lattice reduction in Method II achieves the maximum diversity, the results from [7] are used. It was shown that lattice reduction aided detection achieves the maximum receive diversity which is the number of receive antennas. To show this result, the upper bound for the orthogonality defect factor of the LLL reduced basis is used. If B is an LLL reduced basis then δ(b) is always less than C M(M 1) where C is a constant(see [17]). The main result is presented in the following theorem. Theorem 1 [7]: For a point-to-point MIMO system with the V-BLAST transmission with M transmit antennas and N receive antennas, when we use the LLL lattice aided decoding, lim SNR log P e log SNR = N. (24) Considering the updating condition in (23), the orthogonality defect factor for adaptive reduction Method II is upper bounded by αδ(b ref ). On the other hand, B ref is an LLL reduced matrix and δ(b ref ) is less than C M(M 1). Therefore, for all channel matrices, the adaptively reduced basis has an upper bound as δ(b) Adp LLL αδ(b) LLL αc M(M 1). (25) Applying the above bound and Theorem 1, It can be shown that if α is selected as an increasing function of SNR, then to achieve maximum diversity, it should be in form of o(sn R). We have the following corollary for for adaptive reduction Method II. Corollary 1: The LLL reduction aided detection by the adaptive Method II with the V- BLAST transmission and with N receive antennas achieves the maximum receive diversity if lim SNR log α log SNR = 0.

14 14 Proof : It can be shown that P e c 1α N SNR N (26) which c 1 in a constant. Therefore, we have lim SNR log P e log SNR lim N log SNR log c 1 N log α SNR log SNR = N. (27) Note that there is always a gap in performance between the basic reduction and the adaptive Method II because the adaptive method is an approximate one with lower complexity. In fact, we have a tradeoff between detection complexity and performance in the adaptive reduction Method II which can be controlled by α for different SNRs. Remark: It is also possible to combine Method I and Method II to get even more efficient adaptive algorithm. This means that when it is required to update the unimodular transformation matrix in Method II, we can perform it using the adaptive algorithm we proposed in Method I, i.e. at the time instant i, that it is required to update the unimodular matrix we can perform the LLL algorithm on H i G ref instead of H i. This method makes it possible to save more in computational complexity. In the simulation results, this method is named as Method III. C. Extension As mentioned before, the adaptive lattice reduction can be used in conjugation with any MIMO scenario that requires lattice reduction. In the MIMO broadcast systems, search based precoding scheme is a straightforward method which requires a lattice decoder, [18]. To have a less complex precoder, approximate methods are proposed in [19] by applying the lattice reduction. When the channel realizations are correlated in time, as the one we are using, it is possible to take advantage of this correlation to reduce the complexity of the precoding stage

15 15 in this system. Therefore, the proposed adaptive lattice reduction method can be used in the lattice reduction aided broadcast precoding to reduce the computational complexity. As the next extension of this work, lattice linearity property can be used over slowly varying channels to perform adaptive soft-output detection. It was shown that a serially concatenated scheme of MIMO channel with a channel code as the outer code, can provide impressive performance for high data rate MIMO communications ( [20]). In these systems, the iterative joint detection and decoding can be performed by employing a soft-input, soft-output (SISO) decoder for the outer code and exchanging the soft information between the MIMO detector and the SISO decoder. Using the temporal correlation of channels beside the lattice structure in the MIMO systems, we proposed an adaptive soft-output detection scheme to gain better computational complexity, (see [21] for more details). V. SIMULATIONS RESULTS In this section, the performance and complexity of the adaptive reduction methods for MIMO detection are studied and compared with other conventional reduction methods. For generating the Rayleigh fading channel, we used the model proposed in [22]. The motivation for this was the ability to generate the channel samples sequentially with accurate statistical properties and low computational complexity. The fading parameters of the channels for all the simulations are f d = 100Hz and f s = 270ksps. We consider the MIMO channel with M = N = 4 transmit and receive antennas. A 4-QAM constellation is used to investigate the error performance of the adaptive methods. Figs. 2 and 3 show the bit error rates versus E b /N 0 in which E b is the average energy of a bit at each receiver antenna. In this simulation, the adaptive methods are used to modify the LLL and the deep insertion LLL algorithms. It can be seen from both figures that using the adaptive Method I does not affect the performance of the detection methods. Moreover, the degradation in performance for different value of α is depicted.

16 16 To study the complexity, we compare the average number of required flops for different reduction algorithms where a flop is either a multiplication, a division, an addition or a subtraction. Figs. 4 and 5 show the computational complexity savings by using the adaptive methods. Fig. 4 shows the average number of flop counts for the LLL and the adaptive methods. One can also find the results for the deep insertion LLL in Fig. 5. In these figures, the average number of flops are sketched for different number of antennas where in all cases M = N is considered. As you can see in these figures, there exist an obvious gain in using the adaptive methods. Fig. 6 compares the average number of basis updates for the LLL algorithm and the adaptive algorithms. Also, Fig. 7 shows the average number of required reductions and insertions for both deep insertion LLL and adaptive methods. As it was expected from the adaptive nature of the proposed methods, there is a significant gain in computational complexity. This admits that for the channel we have considered in these simulations, using the transformation matrix of the previous channel realization gives, a quite good reduction for the current channel realizations. In fact, multiplication of the transformation matrix by the new channel matrix becomes the main part of computational complexity in the adaptive methods. Note that reduction, swap and insertion are the basic basis updates used in LLL and deep insertion LLL. Next, in Fig. 8 the behavior of the adaptive deep insertion LLL is investigated for different values of f m and it is compared to deep insertion LLL. In this figure, for each value of f m, number of reductions(insertions) of the deep insertion LLL divided by the number of reductions(insertions) of adaptive deep insertion LLL is plotted. The simulation results show that along with increasing the speed of the moving receiver, the complexity gain from using the proposed method gets smaller. Also it can be seen that the proposed algorithm performs better than the deep insertion LLL for all the velocities considered.

17 17 VI. CONCLUSION In this work, the robustness of lattice reduction results was studied over slowly varying fading channels in MIMO systems. In a Rayleigh fading channel that the channel realizations are correlated in time, it was shown that it is possible to take advantage of the temporal correlation and reduce the complexity of the lattice reduction. Two adaptive methods which improve the complexity of the lattice reduction algorithms were introduced. The first method achieves the same error performance as the original lattice reduction. In the second method which is an approximate one, a measure for tradeoff between detection complexity and performance were proposed. It was shown that if the variations in the channel are slow enough, we can use the past transformation matrices to reduce the future channel matrices and convert the lattice reduction algorithm to just performing a matrical multiplication. REFERENCES [1] E. Viterbo and J. Boutros, A universal lattice code decoder for fading channels, IEEE Trans. on Inform. Theory, vol. 45, pp , July [2] E. Agrell, T. Eriksson, A. Vardy, and K. Zeger, Closest point search in lattices, IEEE Trans. on Inform. Theory, vol. 48, pp , July [3] H. Yao and G. Wornel, Lattice-reduction-aided detectors for MIMO communication systems, In the Proc. IEEE Globecom, Taipei, Taiwan, pp , NOV [4] C. Windpassinger and R. Fischer, Low-complexity near maximum-likelihood detection and precoding for MIMO systems using lattice reduction, In the Proc. IEEE Information Theory Workshop (ITW), Paris, France, pp , Mar [5] A. K. Lenstra, H. W. Lenstra, and L. Lovász, Factoring polynomials with rational coefficients, Math. Ann, vol. 261, pp , [6] A. D. Murugan, H. E. Gamal, M. O. Damen, and G. Caire, A unified framework for tree search decoding: redisovering the sequential decoder, IEEE Trans. on Inform. Theory, vol. 52 no.3, pp , Mar [7] M. Taherzadeh, A. Mobasher, and A. Khandani, LLL reduction achieves the receive diversity in MIMO decoding, IEEE Trans. on Inf. Theory, vol. 53, pp , Dec [8] M. O. Damen, A. Chkeif, and J. C. Belfiore, Lattice codes decoder for space-time codes, IEEE Commun. Lett., vol. 4 no. 5, pp , May [9] W. C. Jakes, Microwave mobile communications. New York: Wiley, [10] M. O. Damen, H. E. Gamal, and G. Caire, On maximum likelihood detection and the search for the closest lattice point, IEEE Trans. on Inform. Theory, vol. 49, no 2, pp , Oct

18 18 [11] W. H. Mow, Maximum likelihood sequence estimation from the lattice viewpoint, IEEE Trans. on Inform. Theory, vol. 40, pp , Sep [12] L. Babai, On lovász lattice reduction and the nearest lattice point problem, Combinatorica, vol. 6 no. 1, pp. 1 13, [13] C. P. Schnorr and M. Euchner, Lattice basis reduction: Improved practical algorithms and solving subset sum problems, Math. Programming, vol. 66, pp , [14] H. Cohen, A course in computational algebraic number theory. Springer, [15] A. Edelman, Eigenvalues and condition numbers of random matrices, SIAM J. Matrix Anal. Appl., vol. 9, pp , Oct [16] S. Szarek, Condition numbers of random matrices, J. Complexity, vol. 7, no. 2, pp , [17] H. Napias, A generalization of the LLL algorithm over euclidean rings or orders, J. Thorie Des Nombres de Bordeaux, vol. 8, p , [18] C. B. Peel, B. M. Hochwald, and A. L. Swindlehurst, A vector-perturbation technique for near capacity multiantanna multiuser communication-part ii, IEEE Trans. on Inform. Theory, pp , Mar [19] C. Windpassinger, R. F. H. Fischer, and J. B. Huber, Lattice-reduction-aided broadcast precoding, IEEE Trans. on Commun., pp , Dec [20] B. Hochwald and S. ten Brink, Achieving near-capacity on a multiple-antenna channel, IEEE Trans. on Commun., vol. 51, no. 3, pp , Mar [21] H. Najafi, M. E. D. Jafari, and M. O. Damen, Adaptive soft-output detection in MIMO systems, In Proc. 46th Annual Allerton Conference on Communication, Control, and Computing, pp , Sep [22] K. E. Baddour and N. C. Beaulieu, Autoregressive modeling for fading channel simulations, IEEE Trans. on Wireless Commun., vol. 4 no. 4, pp , July 2005.

19 19 1 Empirical CDF F(x) D(B 2 ) / D(B 1 ), Experimental results D(B 2 ) / D(B 1 ), Upper bound D(H 2 ) / D(B 1 ), Experimental results x Fig. 1. CDF of the norms product for the channel and the near reduced basis

20 ML MMSE LLL DFE Adaptive MMSE LLL DFE (Method I) Adaptive MMSE LLL DFE (Method III), α=1.15 Adaptive MMSE LLL DFE (Method III), α=3 Adaptive MMSE LLL DFE (Method III), α=5 BER E b /N 0 Fig. 2. MIMO system. Bit error performance of the adaptive hard-output detection methods using the LLL reduction for a 4 4, 4-QAM ML MMSE DILLL DFE Adaptive MMSE DILLL DFE (method I) Adaptive MMSE DILLL DFE (method III), α=1.15 Adaptive MMSE DILLL DFE (method III), α=3 Adaptive MMSE DILLL DFE (method III), α=5 BER E b /N 0 Fig. 3. MIMO system. Bit error performance of the adaptive hard-output detection methods using the DILLL reduction for a 4 4, 4-QAM

21 Average Number of Flops LLL Adaptive LLL (Method I)!=1.15 (Method III)!=3 (Method III)!=5 (Method III) Dimension Fig. 4. Average number of flops for the LLL and the adaptive methods for different number of antennas Average Number of Flops DILLL Adaptive DILLL (Method I)!=1.15 (Method III)!=3 (Method III)!=5 (Method III) Dimension Fig. 5. Average number of flops for the DILLL and the adaptive methods for different number of antennas.

22 Average Number of Reductions Dimension LLL Adaptive LLL (Method I) α=1.15 (Method III) α=3 (Method III) α=5 (Method III) Average Number of Swaps Dimension LLL Adaptive LLL (Method I) α=1.15 (Method III) α=3 (Method III) α=5 (Method III) (a) Average Number of Size Reductions (b) Average Number of Swaps Fig. 6. Average number of basis updates for the LLL and the adaptive methods for different number of antennas Average Number of Reductions DILLL Adaptive DILLL (Method I) α=1.15 (Method III) α=3 (Method III) α=5 (Method III) Dimension Average Number of Insertions DILLL Adaptive DILLL (Method I) α=1.15 (Method III) α=3 (Method III) α=5 (Method III) Dimension (a) Average Number of Size Reductions (b) Average Number of Insertions Fig. 7. Average number of basis updates for the DILLL and the adaptive methods for different number of antennas

23 23 (# of DILLL basis updates) / (# of Adaptive DILLL basis updates) Insertions Reductions f m Fig. 8. rate). Performance of the Adaptive DILLL for different f m (the maximum Doppler frequency normalized by the sampling