LDPC Codes. Slides originally from I. Land p.1

Transcription

1 Slides originally from I. Land p.1 LDPC Codes Definition of LDPC Codes Factor Graphs to use in decoding Decoding for binary erasure channels EXIT charts Soft-Output Decoding Turbo principle applied to LDPC-codes

2 Slides originally from I. Land p.2 LDPC Codes Invented by Robert G. Gallager in his PhD thesis, MIT, 1963 Re-invented by David J.C. MacKay from Cambridge, England in 1999 Defined by the parity-check matrix (which has a low density of ones) Iteratively decoded on a factor graph of the check matrix Advantages Good codes and low decoding complexity

3 Slides originally from I. Land p.3 LDPC Codes Linear block codes. Is defined by the parity check matrix. Used as a big block code. Advantage: error correction is less complex.

4 Slides originally from I. Land p.4 Regular LDPC Codes Remember: For every linear binary (N,K) code C with code rate R = K/N: There is a generator matrix G F K N 2 such that codewords x F N 2 and info words u FK 2 are related by x = ug There is a parity-check matrix H F M N 2 with rankh = N K, such that x C xh T = 0

5 Slides originally from I. Land p.5 Regular LDPC Codes Definition A regular (d v,d c ) LDPC code of length N is defined by a parity-check matrix H F M N 2, with d v ones in each column and d c ones in each row. The dimension of the code (info word length) is K = N rankh. Example 1 H = Parameters: d v = 3, d c = 4, N = 8, M = 6, K = 4 (!), R = 1/2 (!)

6 Slides originally from I. Land p.6 Regular LDPC Codes Example 2 H = Parameters:?

7 Slides originally from I. Land p.7 Regular LDPC Codes Design Rate The true rate R and the design rate R d are defined as and they are related by Proof R := K/N, R d := 1 d v /d c, R R d The number of ones in the check matrix is Md c = Nd v. Some check equations may be redundant, i.e., M N K, and thus K N = 1 N K N 1 M N = 1 d v d c

8 Slides originally from I. Land p.8 Regular LDPC Codes The check matrices can be constructed randomly or deterministic Encoding LDPC codes are usually systematically encoded, i.e., by a systematic generator matrix G = [ I K A ] The matrix A can be found by transforming H into another check matrix of the code, that has the form H = [ A I N K ] by row operations and column permutations

9 Slides originally from I. Land p.9 Encoding LDPC Codes Practical Issues We would like long codes, however since G is not low density, complexity is an issue. Encoding is an issue. After error correction, the codes are decoded as a linear block code.

10 Slides originally from I. Land p.10 Factor Graphs Definition A factor graph of a code is a graphical representation of the code constraints defined by a parity-check matrix of this code: xh T = 0 The factor graph is a bipartite graph with a variable node for each code symbol, a check node for each check equation, an edge between a variable node and a check node if the code symbol participates in the check equation. Notice that each edge corresponds to one 1 in the check matrix.

11 Slides originally from I. Land p.11 Factor Graphs Example xh T = [x 0 x 1... x 7 ] x 0 + x 3 + x 4 + x 5 = 0 x 0 + x 2 + x 4 + x 5 = 0 x 0 + x 2 + x 3 + x 5 = 0 x 1 + x 3 + x 6 + x 7 = 0 x 1 + x 4 + x 6 + x 7 = 0 x 1 + x 2 + x 6 + x 7 = 0 T = 0

12 Slides originally from I. Land p.12 Factor Graphs Example (cont.) xh T = 0 x 0 + x 3 + x 4 + x 5 = 0 (chk 0 ) x 0 + x 2 + x 4 + x 5 = 0 (chk 1 ) x 0 + x 2 + x 3 + x 5 = 0 (chk 2 ) x 1 + x 3 + x 6 + x 7 = 0 (chk 3 ) x 1 + x 4 + x 6 + x 7 = 0 (chk 4 ) x 1 + x 2 + x 6 + x 7 = 0 (chk 5 ) X 0 X 1 X 2 X 3 X 4 X 5 chk 0 chk 1 chk 2 chk 3 chk 4 X 6 chk 5 X 7

13 Slides originally from I. Land p.13 Message Passing Algorithm The factor graph represents a factorization of the global code constraint xh T = 0 Variable nodes and check nodes represent local code constraints This is made explicit by the edge interleaver X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 chk 0 chk 1 chk 2 chk 3 chk 4 chk 5

14 Slides originally from I. Land p.14 Message Passing Algorithm LDPC codes can be iteratively decoded on the factor graph: Nodes perform local decoding operations Nodes exchange extrinsic soft-values called messages Iteration is terminated when code symbol estimates form a valid codeword, i.e., if ˆxH T = 0 Advantages The overall decoding complexity is only linear (!) in the code length The decoder performs close to the ML decoder But nevertheless: This decoding algorithm is sub-optimal due to cycles in the graph

15 Slides originally from I. Land p.15 Message Passing Algorithm Assume an LDPC code of length N defined by the M N check matrix ] H = [H m,n M,N Ingredients of the MPA: Sets of nodes describing the connections of nodes Messages that are exchanged between the nodes Node operations that combine incoming messages to outgoing messages

16 Slides originally from I. Land p.16 Variable Node Sets All variable nodes N := {0,1,...,N 1} All variable nodes that are connected to check node m N(m) := {n N : H m,n = 1} All variable nodes that are connected to check node m, but excluding variable node n (leads to extrinsic ) N(m)\n := N(m)\{n}

17 Slides originally from I. Land p.17 Check Node Sets All check nodes M := {0,1,...,M 1} All check nodes that are connected to variable node n M(n) := {m M : H m,n = 1} All check nodes that are connected to variable node n, but excluding check node m M(n)\m := M(n)\{m}

18 Slides originally from I. Land p.18 Example for Node Sets Variable node sets N = {0,1,...,7} N(0) = {0,3,4,5} N(0)\3 = {0,4,5} Check node sets M = {0,1,...,5} M(1) = {3,4,5} M(1)\4 = {3,5} X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 chk 0 chk 1 chk 2 chk 3 chk 4 chk 5

19 Slides originally from I. Land p.19 Messages and Node Operations Initial message to variable node X n : µ ch (n) = fct ( y n ) Message from variable node X n to check node chk m : µ vc (n,m) = fct ( µ ch (n),µ cv (m,n) : m M(n)\m ) Message from check node chk m to variable node X n : µ cv (m,n) = fct ( µ vc (n,m) : n N(m)\n ) Variable-to-check messages and check-to-variable messages are extrinsic messages

20 Slides originally from I. Land p.20 Example for Variable Node Variable node X 0 Incoming messages: µ ch (0),µ cv (0,0),µ cv (1,0),µ cv (2,0) Outgoing messages: µ vc (0,0),µ vc (0,1),µ vc (0,2) Example operation: µ vc (0,0) = fct ( µ ch (0),µ cv (1,0),µ cv (2,0) ) X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 chk 0 chk 1 chk 2 chk 3 chk 4 chk 5

21 Slides originally from I. Land p.21 Example for Check Node Check node chk 1 Incoming messages: µ vc (0,1),µ vc (2,1),µ vc (4,1),µ vc (5,1) Outgoing messages: µ cv (1,0),µ cv (1,2),µ cv (1,4),µ cv (1,5) Example operation: µ cv (1,4) = fct ( µ vc (0,1),µ vc (2,1),µ vc (5,1) ) X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 chk 0 chk 1 chk 2 chk 3 chk 4 chk 5

22 Slides originally from I. Land p.22 Messages and Node Operations (cont.) Final message generated by variable node X n : µ v (n) = fct ( µ ch (n),µ cv (m,n) : m M(n) ) Estimate for code symbol X n {0 if Pr ( X n = 0 µ v (n) ) > Pr ( X n = 1 µ v (n) ) ˆx n = 1 otherwise

23 Slides originally from I. Land p.23 Optimality, Cycles, and Girth Nodes assume that incoming messages are independent Independency assumption holds only for first few iterations due to cycles in the graph Length of smallest cycle is called the girth For N, the girth tends to and the MPA becomes optimal X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 chk 0 chk 1 chk 2 chk 3 chk 4 chk 5

24 Slides originally from I. Land p.24 MPA for the BEC If the communication channel is a BEC, the MPA becomes very simple. All (!) messages are from the set {0,1, } ( means erasure ). Message updating rules: Variable node operation An outgoing message is only if all relevant incoming messages are Check node operation An outgoing message is different from only if all relevant incoming messages are different from The iteration is terminated if all code symbols are recovered.

25 Slides originally from I. Land p.25 L-values, repetition Boxplus Operator I For two L-values l 1 and l 2, the boxplus operation is defined as l 1 l 2 = 2tanh 1( tanh l 1 2 tanh l ) 2 2 It can be approximated by l 1 l 2 sgn(l 1 )sgn(l 2 ) min { l 1, l 2 }

26 Slides originally from I. Land p.26 L-values, repetition Boxplus Operator II Meaning of boxplus: let X 1 + X 2 = X 3 mod 2; then L(X 3 ) = L(X 1 ) L(X 2 ) For more than two L-values, the operator can be evaluated successively, or as l i = 2tanh 1( i i i tanh l ) i 2 sgn(l i ) min i { li }

27 Slides originally from I. Land p.27 Single Parity-Check Codes Soft-output decoding rules when using L-values Extrinsic L-values l e,n = N 1 i=0 i n l ch,i A-posteriori L-values l p,n = l ch,n + l e,n

28 Slides originally from I. Land p.28 L-values, repetition Binary symmetric channel (BSC) with crossover probability ǫ L(y X) = { +ln 1 ǫ ǫ for y = 0 ln 1 ǫ ǫ for y = 1 Binary erasure channel (BEC) with erasure probability δ L(y X) = + for y = 0 0 for y = for y = 1 Binary-input AWGN channel with SNR E s /N 0 (0 +1, 1 1) L(y X) = 4 E s N 0 y

29 Slides originally from I. Land p.29 MPA based on L-values I The symbol µ (for message) is commonly replaced by l (for L-value) when the messages are L-values. Initial messages: l ch (n) = L(X n y n ) Variable-node operation l vc (n,m) = L ( X n l ch (n),l cv (m,n) : m M(n)\m ) = l ch (n) + Check-node operation m M(n)\m l cv (m,n) l cv (m,n) = L ( X n l vc (n,m) : n N(m)\n ) = n N(m)\n l vc (n,m)

30 Slides originally from I. Land p.30 MPA based on L-values II Final variable-node operation l v (n) = L(X n l ch (n),l cv (m,n) : m M(n)) = l ch (n) + l cv (m,n) m M(n) Hard decision ˆx n = { 0 if l v (n) > 0 1 if l v (n) < 0 Termination criterion ˆxH T = 0

31 Slides originally from I. Land p.31 Some Remarks The MPA gives the a-posteriori probabilities (APPs) of the code symbols if the graph is cycle free Otherwise, it approximates decoding for the maximum-likelihood (ML) codeword. For the BEC, the MPA may get stuck: A set of code symbols which is not resolvable is called a stopping set. The performance of a code of infinite length can be determined by tracking the evolution of the probability density function of the messages with respect to the iterations. This method is called density evolution.