Distributed Source Coding Using LDPC Codes

Transcription

1 Distributed Source Coding Using LDPC Codes Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete May 29, 2010 Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

2 Outline 1 Problem Statement 2 Distributed Source Coding Using LDPC Codes Encoding Decoding 3 Source Splitting Encoding Decoding 4 Results Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

3 Problem Statement Consider two correlated binary sources X and Y and their joint distribution designated by p X,Y (x, y). If the encoder has access to both x and y, it can easily encode both x and y at an overall rate satisfying R > H(X, Y ). It has been shown that this possible even if the encoders for x and y do not communicate, with rates satisfying R x > H(x y) R y > H(y x) R x + R y > H(x, y), also known as the Slepian-Wolf bound. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

4 Rate Region For two correlated sources, from the above inequalities which the rates have to satisfy, we get the following rate region: Corner points can easily be achieved by encoding one source independently at rate H(X ) (or H(Y )) and the other at rate H(Y X ) (or H(X Y )). More general points can be achieved either by joint decoding or by source-splitting. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

5 Distributed Source Coding Using LDPC Codes LDPC codes have been shown to be able to achieve corner points for both memoryless sources and sources with Markov memory [1]. Non-corner points can be achieved by splitting the source into three virtual sources ( x = x (1), x (2)) and y to form virtual corner points, or by jointly decoding the two encoded sources. It is proposed in [2] to do both source-splitting and joint decoding, overcoming certain problems. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

6 Encoding Consider the source sequences x, y with x i i.i.d. and y i i.i.d. The joint pmf will then be f Xi,Y i (x i, y i ) for all bit positions i. Consider the parity check matrices H (x) and H (y). Encoding is performed by calculating s (x) = xh (x) and s (y) = yh (y) and transmitting these syndromes. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

7 Decoding Decoding is done using the SPA with some slight modifications. The parity check values are no longer required to be equal to the zero vector, but equal to s (x) and s (y) respectively. The prior probabilities for the variable nodes are calculated as follows: For l = 0: p yi = x i p Xi,Y i (x i, y i ) and p xi = y i p Xi,Y i (x i, y i ) For l > 0: p yi = x i p Xi,Y i (x i, y i )p (E) X i and p xi = y i p Xi,Y i (x i, y i )p (E) Y i where p (E) X i and p (E) Y i is the extrinsic information obtained through the decoding process. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

8 Decoding The correlation between x and y is expressed as a factor node corresponding to p Xi,Y i (x i, y i ) which connects each x i to is corresponding y i. The overall graph looks as follows: Figure: Circles are variable nodes, black squares are function nodes corresponding to p Xi,Y i (x i, y i ) and white squares are parity checks. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

9 A note on joint decoding Consider a joint distribution p X,Y (x, y) defined as: { 0.49, x = y = 0, x = y = 1, p X,Y (x, y) = 0.01, x y. Note that H(Y X ) = H(X Y ) = 0.141, meaning that this distribution is a good candidate for Slepian-Wolf coding. However, marginals are equiprobable, meaning that all priors at variable nodes will be exactly equal, so decoding fails to start! Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

10 A note on joint decoding This can be surpassed by applying source-splitting as well as joint decoding. Split the source into x = (x 1, x 2 ) and y. Encode x 1 independently with some conventional entropy achieving source coding technique. At the decoder, knowledge of x 1 ensures that decoding can start. There is some loss in the overall rate. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

11 Source Splitting We use a random sequence t {0, 1} n, which is available to both the encoder and decoder, where φ : Pr(t i = 1) is the source splitting parameter. If t i = 0 then x 1i = x i and x 2i = E If t i = 1 then x 2i = x i and x 1i = E E values can be discarded as they are not needed, only the non-erasure values are encoded. In our case, since x i s are iid, we chose the first n(1 t) bits of x to be x 1 and the remaining bits to be x 2, where n is the length of x and we assume that n(1 t) is an integer. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

12 Source Splitting The graph, after source-splitting has been applied, looks as follows: Figure: x 1 is completely known at the decoder and provided to the subgraph of y. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

13 Encoding Encoding is done as follows: x 1 is encoded independently in an arbitrary rate-achieving manner. y is encoded using an LDPC code with parity-check matrix H y and rate R y > H(Y X 1 ). x 2 is encoded using an LDPC code with parity-check matrix H x2 and rate R x2 > H(X 2 Y, X 1 ) iid = H(X 2 Y ). Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

14 MAP Decoding Rule for x 2i x MAP 2i (x 1, s (x), s (y) ) = arg max p(x 2i s (x), s (y), x 1 ) (x 1 ind. of x 2 ) x 2i = arg max p(x 2 s (x), s (y) ) (Bayes rule) x 2i x 2i = arg max p(x 2, s (x), s (y) ) x 2i x 2i = arg max p(s (y) )p(x 2 s (y) )p(s (x) s (y), x 2 ) x 2i x 2i = arg max p(x 2 s (y) ) 1 x 2i }{{} [x T 2 H x =s (x) ] x 2i =A Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

15 MAP Decoding Rule for x 2i If we split y into two parts y = [ y 1 A, we will have: y 2 ], corresponding to x1 and x 2, for p(x 2 s (y) ) = y 2 p(y 2, x 2 s (y) ) = y 2 p(y 2, x 2, s (y) ) p(s (y) ) (p(s (y) )constant w.r.t. x 2i ) = y 2 p(y 2, x 2 )p(s (y) y 2, x 2 ) (x 2 ind. of s (y) given y 2 ) = y 2 j p(y 2j, x 2j ) p(s (y) y 2 ) = y 2 j p(y 2j, x 2j ) 1 [y T H y =s (y) ] Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

16 MAP Decoding Rule for x 2i So, the overall rule is: x2i MAP = arg max x 2i y2 x 2i j p(y 2j, x 2j ) 1 [y T H y =s (y) ] 1 [x T 2 H x =s (x) ] By proceeding accordingly for y i, we get: yi MAP = arg max p(y 1j x 1j )1 y i [y T H Y =s (y) ] y i j p(y 2j, x 2j )1 [x T 2 H x =s (x) ] x2 j Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

17 Decoding In [2] decoding is done jointly for the whole graph. The values of x 1 are exactly known at the decoder and they are provided to the subgraph of y. Provides greater rate flexibility than serial decoding and some slight performance gain. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

18 Decoding We initiliaze the sum-product decoding algorithm by calculating the messages from the function nodes corresponding to x 1 to the variable nodes y. Using this information, we calculate the messages from y to the check nodes corresponding to s (y). We constructed the code in such a way that the information from x 1 goes to variable nodes y of high degree, so that this initial information is distributed to as many check nodes as possible. Variable nodes y not connected to x 1 send messages which act like erasures and are treated as such by the checks. The messages from check nodes of y return to the variable nodes y and are then sent to x 2 through the corresponding function nodes. Decoding proceeds jointly. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

19 Results Using density evolution and differential evolution, for a memoryless source with joint pmf: { 0.473, xi = y p Xi,Y i (x i, y i ) = i, 0.027, x i y i, the following degree sequence was found by setting φ = 0.2: λ (x) = [ ] ρ (x) = [ ] λ (y) = [ ] ρ (y) = [ ] Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

20 Results The rates of the above codes are: R x = R y = Assuming x 1 is encoded at entropy (in this case, 1 bit per symbol as p x is uniform), the overall rate is: R = R y + R x φ + R x1 (1 φ) = = which is 4% higher than the joint entropy. Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

21 Simulation Using the above distributions and Progessive Edge Growth, we created a code of length n y = 2500 for y and a code of length n x = 500 with the appropriate rates, which yielded the following results Offset vs BER X Y 10 2 BER Offset Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22

22 References 1 D. Schonberg, K. Ramchandran, and S. S. Pradhan, LDPC codes can approach the Slepian Wolf bound for general binary sources, Proc. 40th Allerton Conf. on Commun., Control, and Computing, Monticello, IL, Andrew W. Eckford and Wei Yu, Density evolution for the simultaneous decoding of LDPC-based Slepian-Wolf source codes, in Proceedings ISIT 2005, Adelaide, Australia, Telecommunications Laboratory (TUC) Distributed Source Coding Using LDPC Codes May 29, / 22