Least-Squares Performance of Analog Product Codes

Size: px

Start display at page:

Download "Least-Squares Performance of Analog Product Codes"

Eleanor Williams
5 years ago
Views:

1 Copyright 004 IEEE Published in the Proceedings of the Asilomar Conference on Signals, Systems and Computers, 7-0 ovember 004, Pacific Grove, California, USA Least-Squares Performance of Analog Product Codes Olivia emethova, Markus Rupp Institute for Communications and Radio-Frequency Engineering University of Technology Vienna Gusshausstrasse 5/389, 040 Wien, Austria Web: prototyping Abstract In this article the least squares method is utilized for the decoding of analog product codes In particular it is shown, (i) how the LS solution is constructed for arbitrary image block sizes, (ii) that the theoretical achievable SR improvement after LS decoding is inverse proportional to the code rate, and finally (iii) the LS results are compared with a known iterative method The convergence of the iterative method is analyzed, allowing for optimal behavior and a consistent proof of convergence is provided Theoretical results are compared to simulations on the example of a picture transmission I ITRODUCTIO Analog codes are error correcting codes continuous in value Such codes may be especially suitable for applications with analog nature, supposed to run in real-time, such as audio or video transmission because of the low processing time requirements Different types of analog codes have been investigated in the past: in [ RS and BCH analog codes were proposed; [ introduced analog convolutional codes; [3 analyzed the features of analog product codes and proposed an iterative decoding scheme, claiming its convergence to the Least- Squares (LS) solution In the following, the principle of analog product codes will be described The D-picture sequence that has to be encoded is subdivided into blocks, each represented by the following matrix X: x, x, X () x, x, To protect information, analog parity check symbols are added to the rows and columns of the X matrix: b, b, b,+ B b, b, b,+, () b +, b +, b +,+ so that the sum of all elements in each row or column equals zero, b i,j x i,j i, j,,, (3) b i,+ x i,j, (4) b +,j b +,+ j x i,j, (5) i i j x i,j (6) Let b vec(b) and x vec(x) The sequence r received after the transmission over an AWG channel with some additive white noise n has the form r b + n (7) The task is to recover the information represented by x as exact as possible without any a-priori information available We shall assume the data as well as the noise to be IID In Section the LS solution of this approximation problem is shown and the resulting limit of SR improvement is discussed Section 3 provides a comparison of the proposed LS solution to the iterative scheme presented in [3 The argument for LS convergence in [3 was correct but not conclusive We therefore present a new convergence analysis for this iterative scheme In Section 4 the results of both schemes are visualized by means of simulation results of a picture transmission over a noisy channel and its reconstruction Some concluding remarks about the implications of this method can be found in Section 5 II LS SOLUTIO To provide an LS solution we first need to define the linear basis for the encoding of analog codes Let us define an ( + ) matrix P, the columns of which are formed by linearly independent basis vectors p, p,, p of dimension ( + )

2 Such vectors p k can be obtained as follows Consider an matrix A(k) that has only one element equal to one at the position (i, j) (i ) + j k; i, j, and all remaining elements equal to zero: A(k) i,j { ; (i ) + j k 0 ; otherwise (8) The index k runs from to Every matrix A(k); k has to be further encoded by the described product code, resulting in an ( +) ( +) matrix A C (k) By vectorizing each such encoded matrix A C (k), we finally obtain the p k basis vectors: p k vec(a C (k)); k,, (9) Using the just obtained basis, the encoding process may be simply described as: b Px (0) Minimizing the square r b r Pˆx of the approximation error based on the received r leads to the following system of linear equations: P T Pˆx P T r () Solving this system of linear equations to find the optimum coefficients ˆx leads to the LS solution represented by the estimation of the transmitted vector ˆb Pˆx The estimated vector ˆb and the corresponding estimation error e may thus be written as the following projections onto the received vector r: ˆb Pˆx P(P T P) P T }{{} r, () P P e r ˆb r P(P T P) P T r (I (+) P(P T P) P T )r (I (+) P P )r (3) Matrix P P defines a projection with the idempotent property P P P P The same applies for the matrix I (+) P P To evaluate the performance of the proposed LS scheme, the Signal-to-oise Ratio (SR) is utilized, defined as follows: E [ b SR 0 log 0 E [ n [db (4) The SR after the decoding can be written in the same way: E [ b SR D 0 log 0 E [ ˆn [db, (5) with ˆn Pn We define the SR improvement c SR after the decoding as c SR SR E [ n SR D E [ ˆn [db (6) Under the assumption of n being the vector of size ( + ) with the statistically independent white noise elements having the variance σ n, we can further write: E [ n ( + ) σ n (7) The expression for the expectation of noise after decoding can be also further reduced: E [ ˆn E [ P P n E n T P T P P P n }{{} Y (+) (+) E n l y l,k n k (+) k l,k l y l,k E [n l n k (8) The expression E [n l n k corresponds to the diagonal correlation matrix R nn σn I (+) Taking into account the properties of the projection matrix P P and the trace operator, we can further write: E [ ˆn σ n (+) l y l,l σ n trace(pt P P P ) σ ntrace(p(p T P) P T ) σ n trace(i ) σ n (9) Thus, the SR improvement c SR by decoding with LS as described above, can be explicitly expressed as follows: c SR log 0 ( + ) [db (0) ote that the code rate is exactly /( + ), ie, the SR improvement is inverse proportional to the code rate For about 35dB improvement is obtained ote further that by concatenating several of this schemes almost arbitrary SR improvements can be achieved at the expense of a corresponding data rate expansion

3 III ITERATIVE SCHEMES In [3 the following iterative scheme has been defined: ˆb (0) r and ˆb (k) Φˆb (k ), () Φ I (+) w(m + M ), () + w ( + ) ( + ) matrices M and M denote the operations of computing every element of the received matrix R from the remaining elements in the same row or column respectively The parameter w is in the range [0, / as will be shown further ahead These matrices can also be expressed by means of Kronecker products and sums [4: M I + Ī+, M Ī+ I +, (3) Ī + E + I + (4) is a ( +) ( +) matrix and E + is a ( +) ( +) matrix with all elements equaling one Φ I (+) w(i + Ī+ + Ī+ I + ) + w I (+) w(ī+ Ī+) (5) + w This iterative method does not diverge if λmax(φ), (6) λmax(φ) being the greatest eigenvalue of the matrix Φ Knowing the eigenvalues of E + and I + λ i (I + ) i,, + (7) λ i (E + ) { + ; i 0; i,, + for eigenvalues of Ī+ we can thus write: λ i (Ī+) { ; i ; i,, + (8) (9) The eigenvalues of Ī+ Ī+ are obtained by utilizing the property of the Kronecker sum that the eigenvalues of the Kronecker sum matrix can be obtained by all possible sums of eigenvalues of the operand matrices: ; i λ i (Ī+ Ī+) ; i,, + ; i +,, ( + ) The matrix Φ has following form: Φ αi (+) + β(ī+ Ī+), (30) α + w ; β w + w (3) Therefore, we obtain the eigenvalues of matrix Φ as follows: w +w ; i w( ) λ i (Φ) +w ; i,, + (3) ; i +,, ( + ) After evaluating the condition (6) with the eigenvalues of Φ in (3), we obtain the region this iterative method does not diverge: w [0, ote that even if the lower and upper values are excluded for the range of w, the convergence of the scheme would not be able to be shown here since some eigenvalues always remain one However, it can be shown that this iterative method converges to the described LS solution, ie that Φ lim k Φk P P (33) Our method to proof this is to show that the eigenvalues and eigenvectors are identical for Φ and P P A Eigenvalues and Vectors of Φ Knowing the eigenvalues of Φ, we obtain the eigenvalues of Φ as follows: λ i (Φ k ) ( ) { k w 0; w 0 lim k +w ; w 0 i ( ) k lim w( ) k +w 0; i,, + (34) lim k () k ; i +,, ( + ) The next task is to find the eigenvectors q i of Φ Consider them forming a matrix Q: Λ Φ Q T ΦQ αq T IQ + βq T (Ī+ Ī+)Q, (35) In [3an incorrect interval w [0, was given

4 so we need to know matrix Q that diagonalize Ī+ Ī+ Knowing that eigenvectors of a Kronecker sum matrix can be obtained as all possible Kronecker products of the eigenvectors of operand matrices, we further need to know only the eigenvectors of Ī+ E + I + being the same as those of E + only: u i { e+ ; i ε i ; i,, +, (36) ε i are arbitrary mutually orthonormal vectors, orthonormal also on e + [,,, T Thus, the eigenvectors of Φ have the following form: q i u k u j ; k, j,, +, (37) i k + j( + ) being an index from to ( + ) These eigenvectors are also the eigenvectors of Φ because if Λ Q T ΦQ, then is also true that Λ k Q T Φ k Q for every k B Eigenvalues and Vectors of P P Matrix P defined by its basis vectors in (9) has for an arbitrary following structure: C Thus, we can write I e T P C C, (38) P P P(P T P) P T ; e T [,, (39) (C C)((C C) T (C C))(C C) T (C C)((C T C) (C T C) )(C C) T (C(C T C) C T ) (C(C T C) C T ) P C P C (40) Matrix P C has following form: P C I e T I e T (I + E ) [ I e (I + E ) [ I e I + E e + + e e T + + et + (4) and can be thus written as P C be + + (a b)i +, (4) a + ; b + (43) Knowing its structure, and combining (7) and (8), we find following eigenvalues for P C : λ i (P C ) { 0 ; i ; i,, (44) The eigenvalues of P P are obtained by using the property of Kronecker product that the eigenvalues of the Kronecker product matrix can be obtained by all possible products of eigenvalues of the operand matrices [4: λ i (P P ) { 0 ; i,, + ; i +,, ( + ) (45) The eigenvectors we can get by considering again the form (4) of matrix P C Obviously, the eigenvectors of P C do have the same form as those of Ī+ As P P P C P C, we can conclude, that P P has the same eigenvectors (37) as Φ Comparing further (34) and (45) we conclude that P P Φ, and thus that the iterative method proposed in [3 converges indeed to the LS solution IV SIMULATIO RESULTS To visualize the achievable SR improvement and the mentioned iterative method, both of them were simulated in MATLAB In Figures and a picture reconstruction by the proposed LS method can be seen For this simulation an additive white gaussian noise with variance σ n 9, resulting in SR39dB was added to the picture (quantized by 8 bits per pixel per color), encoded block by block, each of them having size The resulting SR improvement was 35dB which is in excellent agreement with the calculated SR improvement of 35dB In Figure 3 the SR for the iterative method is visualized, as function of the number of iterations The speed of the convergence depends on the choice of the weight parameter w The optimal convergence can be found by minimizing the maximum eigenvalue of the matrix Φ, wopt arg min max λ(φ) w (46) As already shown in the previous section, the iterative method converges to the LS solution However, the complexity of the iterative method is much higher than the direct LS method: Utilizing the LS method, only one operation (multiplication with precomputed projection matrix P P from ()) is necessary to obtain the solution; using the iterative method one needs to perform many (simple) matrix multiplications with the matrix of the same size as for the direct LS method

4 35 SR improvement [db 3 5 5 w00 w05 w0 w09 05 Fig The noisy picture 0 5 0 5 0 5 umber of iterations Fig 3 SR improvement over number of iterations for various values of w 00, 0, 05, 09, V COCLUSIO

5 4 35 SR improvement [db w00 w05 w0 w09 05 Fig The noisy picture umber of iterations Fig 3 SR improvement over number of iterations for various values of w 00, 0, 05, 09, V COCLUSIO The LS solution to the decoding of analog product codes was provided as well as an explicit relation for the achievable SR improvement A detailed and consistent proof for the described iterative method converging to the LS solution is given It can be concluded that the iterative method does not provide any benefit over the proposed direct LS solution and indeed does not save complexity at all REFERECES Fig Picture after applying the LS method SR improvement 35dB An advantage of the iterative method compared to the exact LS solution is thus at least doubtful [ Wolf, JK, Analog Codes, IEEE International Conference on Communication (ICC 83), Boston, MA, USA, vol, pp 30-, June 9-, 983 [ Chen, B; Wornell, GW, Analog Error-Correcting Codes Based on Chaotic Dynamical Systems, IEEE Transactions on Communications, vol 46, no 7, pp , July 998 [3 Mura, M; Henkel, W; Cottatelluci, L, Iterative Least-Squares Decoding of Analog Product Codes, Proceedings IEEE International Symposium on Information Theory, pp44, June 9 - July 4, 003 [4 Moon, T; Stirling, W, Mathematical Methods and Algorithms for Signal Processing, Prentice Hall, Engelwood Cliffs (ew Jersey), 000

Basic Principles of Video Coding

Basic Principles of Video Coding Introduction Categories of Video Coding Schemes Information Theory Overview of Video Coding Techniques Predictive coding Transform coding Quantization Entropy coding Motion