Performance Analysis of Modified Gram-Schmidt Cholesky Implementation on 16 bits-dsp-chip

Transcription

1 Int. J. Com. Dig. Sys., No., -7 (03) International Journal of Computing and Digital 03 UOB CSP, University of Bahrain Performance Analysis of odified Gram-Schmidt Implementation on 6 bits-dsp-chip Rabah aoudj, Luc Fety and Christophe Alexandre 3 Lab. CEDRIC / LAETITIA, CNA, Paris, France rabah.maoudj@cnam.fr, luc.fety@cnam.fr, 3 christophe.alexandre@cnam.fr Received 9 Oct. 0, Revised 0 Nov. 0, Accepted 5 Dec. 0 Abstract: This paper focuses on the performance analysis of a linear system solving based on decomposition and QR factorization, implemented on 6bits fixed-point DSP-chip (TS30C6474). The classical method of decomposition has the advantage of low execution time. However, the modified Gram-Schmidt QR factorization performs better in term of robustness against the round-off error propagation. In this study, we have proposed a third method called odified Gram-Schmidt Decomposition. We have shown that it provides a compromise of the two performance criterias cited above. A joint theoretical and experimental analysis of global performance of the three methods has been presented and discussed. Keywords: fixed-point; cholesky; qr; system solving; dsp; signal processing. I. INTRODUCTION This paper presents the implementation results on the 6bitfixed-point DSP chip (TS 30C6474 which is optimized for 6bits arithmetic operations), [] of a linear system solving performed by decomposition and modified Gram- Schmidt process. The algorithms that contain a matrix inversion or a leastsquares problem are nowadays frequently used in a vast range of domains. But wireless telecommunications is the main user of linear system solving, as attest the several works published in the IEEE. For the field of wireless telecommunications, linear system solving are found mainly in the algorithmic part of the propagation channel equalization and interference cancellation for IO (ulti Input ulti Output) systems where the least square problem is often encountered [, 3] as result of solving SE (inimum ean Square Estimation ) or LSE (Least Square Estimation) algorithms. In the case of least square problem, the most popular techniques are the classical decomposition and QR factorization as will be detailed in section. We can show that the lower triangular matrix in the classical decomposition can be obtained by the QR factorization and vice versa. This property can be used for performing a solving system using only the upper triangular matrix issued from QR factorization without using the orthogonal matrix Q as in the classical method. We call this method Gram- Schmidt- (GS-). In this work, we try to evaluate the efficiency versus the round-off error of this method compared to the classical method of. Essentially it is shown that the QR factorization using the modified Gram-Schmidt process (GS) is less sensitive to the round-off error than the classical Gram-Schmidt process (GS) when using the fixed-point calculation format [4, 5]. Therefore, the study will focus on evaluating the robustness of methods versus their round-off errors in intermediate calculations. In the second section, we give an overview of the classical decomposition and GS-QR factorization algorithms that are implemented along with the analytical aspects. In the third section, the 6-bits implementation of solving systems based on decomposition, QR factorization and GS- are detailed. In the fourth section, a theoretical study followed by an experimental evaluation is performed. The performance in sense of time execution and robustness against the round-off error in intermediate calculations due to 6-bit resolution are discussed. II. LINEAR SYSTE SOLVING BASED ON CHOLESKY DECOPOSITION AND QR FACTORIZATION The decomposition is among the well-known methods and most popular for the linear system solving. This method is applied only if the matrix is Hermitian positive definite which is often encountered when dealing with least

2 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified square problems. The classic decomposition can be achieved in two ways as we shall develop below. A. Least square problem The least square problem results often from an overdetermined linear system [6]. It means that the number of equations, is greater than the number N of unknowns. This situation occurs in the case of pilot-aided propagation channel estimation with assumption of low rank channel [, 7]. The following mathematical expressions can summarize the situation. We assume a linear system given by, Ax b = ε (.) A C N, b C and ε C The two most popular methods to resolve the equation (.) for min ε are given by the following sections and. ) Normal equation method The normal equation method is based on the search of the minimum modulus of the residual error ε as expressed in the following equation, min ε d Ax b H Ax b dx = 0 Therefore, A H Ax = A H b (.) If we set, Γ = A H A and β = A H b then, Γx = β and x = Γ β (3.) Γ C N N being Hermitian positive definite matrix and β C N. Γ can be decomposed by the method, and the equation (3.) becomes, x = LL H β (4.) C N N, a lower triangular matrix ) QR factorization It consists in a QR decomposition of A. Then equation (.) becomes, QRx = b + ε x = R Q H b, ε = 0 (5.) A = QR R C N N, an upper triangular matrix Q C N, an orthogonal matrix From section A and section B we have, Γ = A H A = L H L = RR H L = R H Q can be obtained from the decomposition by the following system solving, A = QL H Q = AL H (6.) The operation in equation (6.) is commonly called orthogonalization of matrix A. We conclude in this case that decomposition is a way to obtain QR decomposition and vice versa. This observation confirms that the QR factorization requires more execution time than the decomposition since QR can be considered as a decomposition, plus construction of matrix Q. In the rest of the paper, we focus on the experimental comparison of robustness versus round-off errors introduced by the intermediate calculations in 6-bit fixed-point of the decomposition. For this purpose, we evaluated the classical method and the Gram-Schmidt process, by taking Gram-Schmidt QR decomposition as reference. B. Classical decomposition This method needs to compute initially the Hermitian matrix Γ = A H A before performing the classical decomposition process as given below. Let s denote γ ij the elements of matrix Γ, and l ij the elements of matrix L which is the decomposition of Γ with i, j N. Then the elements l ji are obtained by the following algorithm. for i = to N i = γ ii k= l ik for j = i + to N l ji = C. odified Gram-Schmidt process This second method does not require to pre-compute the Hermitian matrix Γ, but uses the matrix A directly as input. atrix Γ is implicitly obtained in a sequential manner in the process of QR decomposition. This decomposition uses the Gram-Schmidt process as given by the following algorithm. i γ l ij k= l ik l jk ii

3 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified 3 We denote, r ij the elements of matrix R and q ij the elements of matrix Q. Q = A for i = to N q H mi q m = mi = for j = i + to N r ij = q H mj q H m = mi for m = to q mj = q mj The number of mathematical operations required in floating point, for and QR decompositions, associated to the solving systems proposed in (4.) and (5.), is given in the following tables. The number of square roots is not included in the tables because it is the same for different methods, and equal to N. It should be noted that, multiplication and addition are complex operations. All the division operations used in the different algorithms are divisions of a complex number by a real number. Table. Cost in number of multiplication and addition operation ultiplications and additions QR GS- A H A N N A H b N - N Classical N N - - factorization. odified Gram - N N Schmidt process x = LL H A H b N N - N N x = R Q H b - N + N - q mi r ij Table. Cost in number of division operation Divisions QR GS- A H A A H b Classical factorization. N N - - odified Gram - N + N Schmidt process x = LL H A H b N - N x = R Q H b - N - III. DSP IPLÉENTATION N N This section is dedicated to the practical 6bit fixed-point implementation of the system solving algorithms described in the last section. In order to compare results of robustness of these various algorithms, we give the exact code embedded in the DSP by flowcharts to 3 shown below. The main functions are given first, followed by the final DSP implementation of each algorithm. The Q x/y notation of fixed point format is widely used in fixed-point arithmetic [8] and briefly explained in [9]. We note that Q x/y do not mean the classical representation Q a b (where a is the number of integer bits and b the number of fractional bits). The Q x/y representation means here that x is the number of fractional bits and y the number of total available bits including signed bit. These two representations are related by this expression, y = + a + b and x = b The initial and final vectors and matrices are expressed on Q 5/6 format. Intermediate vectors and matrices are in Q Z/6 format in order to prevent over flow in multiplication process. In the following flowcharts, the real positive number Z is calculated by the following equation: Z = 5 round log (7.) In the following, Figures and 3 show respectively the classical decomposition and odified Gram-Schmidt factorization processes as implemented on the DSP. Once these processes are performed the triangular matrices resulted are carried to the final step of the system solving where the solution is obtained as depicted by figure.

4 4 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified A Q 5/6 Q = A i = Q 5/6 Γ = A H A i = i N triangular system solving i N triangular system solving j = Q 5+z/3 = Z Γ i, i j = j N j j < i L j, i = Z Γ i, j k = Q 5+z/3 Q 30/3 = + Q j, i Q H j, i Q 30/3 = L i, j L H i, j k < i Q 5/6 = j = i + Q 5/6 = j = i + L j, i = L j, i L i, k L H j, k Q 30/3 j N k = k = L j, i = L j, i Q 5/6 k k Figure. Flowchart () of Classical decomposition Q 5/6 β i = i N end system solving R i, j = R i, j + Q k, j Q H k, i R i, j R i, j = ρ = 5 R i, j Q k, j = Q k, j Q k, i ρ 5 Q k, i = 5 Q k, i Q 5/6 Q 5+Z/3 x = Z β j = i Figure 3. Flowchart (3) of modified Gram-Schmidt QR factorization Q 30/3 Q 5/6 j x i = x i L i, j x j x i x i = Z Finally, the three system solving implemented in the DSP are given as follows, A. System solving using classical method, as depicted in fig. (4-a) B. System solving using GS- method, as depicted in fig. (4-b) C. System solving using QR method, as depicted in fig. (4-c) Figure. Flowchart () of triangular system solving

5 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified 5 Flowchart Flowchart Flowchart w + ε w = x + ε x y + ε y = x + ε x y + ε y a. Classical Flowchart 3 Flowchart Flowchart b. GS- By Taylor development of y +ε y around y, we have y + ε y y ε y y ε w y ε x + xε y Flowchart 3 Q H b Flowchart ε w y t c. QR Figure 4. System solving algorithms 4) Square root w + ε w = x + ε x IV. NUERICAL RESULTS A. Theoretical analysis of round-off errors In this section, we assume that the absolute value of all real and imaginary parts are less than. All calculations are performed without overflow. The additions and subtractions are made with t + Z bits (t = 5). It means that the following error model for the arithmetic operations used to process the various algorithms given in last section is accurate [0]. Let s denote x, y, and w to be complex variables with absolute values of their real and imaginary components inferior to. In addition ε x, ε y, ε w are the corresponding complex errors. ) Addition and subtraction ) ultiplication w + ε w = x + ε x + y + ε y, ε x t, ε y t ε w t w + ε w = x + ε x y + ε y ε w t In the case of scalar product of two vectors the model is, 3) Division w + ε w = K k= x k + ε xk y k + ε yk ε w t By Taylor development of x + ε x around x, x + ε x x ε x x ε w ε x x ε w x t Now we apply this error model to the decomposition and QR factorization. 5) decomposition According to the flowchart, Γ is obtained by, Γ + E Γ = A + E A H A + E A ε A i,j t We denote by: E Γ, and E A are the error matrices corresponding to Γ and A respectively. Therefore, ε Γ i,j t, i, j N The diagonal elements of matrix are given by the following equation, i + ε lii = γ ii +ε γii l ik + ε lik k= For i =, the round-off error is ε lii t For i >, ε lii t i + k= ε lik, ε lik is calculated iteratively as shown below. The non-diagonal element l ji of the matrix L is given by,

6 6 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified i l ji + ε lji = γ + ε ij +ε γij l ik + ε lik l jk + ε ljk lii k= i q mj + ε qmj = q mj + ε qmj + ε rii q mi + ε qmi r ij + ε rij ε lji ε γij + γ ij l ik l jk ε lji k= t + ε The upper limit of the round-off error of the components of the matrix L is too difficult. But if we assume the calculation with only two recursive iterations, we find the following result. And, ε lii t ε lji 6) QR Factorization + l i i + t + ε ε lii l i i According to the flowchart 3, the diagonal elements of matrix R can be written as, + ε rii = q mi + ε qmi H q mi + ε qmi m= Then the round-off error ε rii can be bounded by, ε rii t For the non-diagonal element of the matrix R, the round-off error is given as follows, r ij + ε rij = q + ε mj + ε H qmj q mi + ε qmi rii m= H ε qmj ε qmj + q mi ε rij + r ij ε qmi + q mi r ij ε rii ε qmj 3 + t 7) Observation From the sections e and f, we observe that the round-off error is greater in the case of Cholseky decomposition than in the case of QR factorization. This round-off error depends on the values of or, but the decomposition is more sensitive to than QR factorization to. B. Simulation results The fixed point implementation is done according to flowcharts,, and 3. The target is TS 30C6474 with GHz clock rate. The experimental results depicted by figures 5, 6 and 7 are obtained using onte Carlo statistical method of solving system Ax = b with well-conditioned A H A matrices i.e.,( condition number A H A 30 ). Simulations are performed for a fixed row number = 6 and several values of column number N = 4, 6, 8, 0,, 4. Figure 5 depicts the relative norm of rounding errors on the triangular matrix L, L L 0 L 0. The reference triangular matrix L 0 is obtained with full precision floating-point (IEEE float-point) format. Figure 6 depicts the square root of the mean square error Ax b. These experimental results confirm the theoretical ones and show that the GS-QR is less sensitive to the round off errors than Classical and GS-. However GS- is more robust than Classical. This classification criteria based on round-off error immunity is inverted when applying the time execution criteria. Classical requires twice less execution time than GS-QR and around.5 less than GS- as depicted in figure 7. ε rij q mj H ε qmj + q mi ε mi m= ε rij ε t + ε rii q mj H q mi m= Finally we calculate the round-off error of matrix Q. The matrix Q is obtained by performing the following equation, Figure 5. -norm of error triangular matrix L, L L 0 L 0

7 R. aoudj, L. Fety, C. Alexandre: Performance Analysis of odified 7. However this robustness has a cost in terms of execution time. Classical requires twice less execution time than GS-QR and around.5 less than GS-. With the introduction of the GS-, we give an alternative to GS-QR if we search a good robustness of the system solving against additive round-off error in the final solution with a slight degradation in execution time performance. V. CONCLUSION Figure 6. ean square error, Ax b Figure 7. Execution time (us) In this paper, we have analyzed, theoretically and experimentally using atlab simulations, the solving system using classical, GS-QR and GS-. Both analyses are concordant. As expected through the theoretical analysis, the experimental analysis has confirmed that the GS-QR is more robust than the classical and GS- REFERENCES [] TS30C6474 ulticore Digital Signal Processor Data anual (Rev. H), April 00. [] Rabah aoudj, Christophe Alexandre, Denis Popielski, ichel Terré, DSP Implementation of Interference Cancellation Algorithm for a SIO System, ISWCS 0, Paris 0. [3] Saqib Saleem, Qamar-ul-Islam, Optimization of LSE and LSE Channel Estimation Algorithms based on CIR Samples and Channel Taps, IJCSI International Journal of Computer Science Issues, vol. 8, Issue, January 0. [4] A. Bjorck, Numerics of Gram-Schmidt Orthogonalization, Linear Algebra and Its Applications, vol. 98, pp , Feb [5] Chitranjan K. Singh, Sushma Honnavara Prasad, and Poras T. Balsara, VLSI Architecture for atrix Inversion using odified Gram-Schmidt based QR Decomposition, 6th International Conference on Embedded Systems, 0th International Conference on, pp , Jan [6] Charles L. Lawson, Richard J. Hanson, Solving Least Squares Problems, Printice-Hall, Inc., Englewood Cliffs, N. J, 974. [7] Chang Dong Lee, Jin Sam Kwak, and Jae Hong Lee, Low- Rank Pilot-Symbol-Aided Channel Estimation for IO- OFD Systems, IEEE VTC 004, vol. 7, pp [8] Israel Koren, Computer Arithmetic Algorithms, Brookside Court Publishers, 998. [9] Ali Irturk, Shahnam irzaei and Ryan Kastner, FPGA Implementation of Adaptive Weight Calculation Core Using QRD-RLS Algorithm, Technical repport CS , ar [0] J.H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University Press Inc, New York, 965..