ECEN 655: Advanced Channel Coding Spring Lecture 7 02/04/14. Belief propagation is exact on tree-structured factor graphs.

Size: px

Start display at page:

Download "ECEN 655: Advanced Channel Coding Spring Lecture 7 02/04/14. Belief propagation is exact on tree-structured factor graphs."

Magdalen Robbins
5 years ago
Views:

1 ECEN 655: Advaced Chael Codig Sprig 014 Prof. Hery Pfister Lecture 7 0/04/14 Scribe: Megke Lia 1 4-Cycles i Gallager s Esemble What we already kow: Belief propagatio is exact o tree-structured factor graphs. A reasoable assumptio: For a fixed esembles of codes, it is better to have fewer cycles. Gallager s costructio: What is the expected umber of bits ivolved i 4-cycles? Cycles i (, k-ldpc Code Cosider the umber of bits ivolved i 4-cycles. Sice the Gallager esemble is ivariat uder colum permutatio, we have [ ] E I (bit i i 4-cycle = P (bit 1 i 4-cycle i=1 This observatios allows oe to focus o 4-cycles ivolvig the first bit of the code. For ay H from the Gallager esemble, oe ca always permute the colums so that the first /k rows are equal to Q. For example, multiplyig H o the right by P1 T does this because P 1P1 T = I. This holds becaue a permutatio preserves all p-orms ad hece P 1 must be uitary. We ca also permute the rows of the secod Q-block (i.e., rows /k + 1 to /k so that the first row of the d Q-block (i.e., row /k + 1 has a oe i the first positio. Let H = P 0 HP1 T be the parity-check matrix after these permutatios (i.e., P 0 defies the ecessary row permutatio. For a (, k code, the first code bit will be ivolved i a 4-cycle iff row /k + 1 of H has a oe is colum positios, 3,..., k. H = a a k a k+1 a From the above diagram, we observe that row /k + 1 is a biary vector with a oe i the first positio ad k 1 oes distributed uiformly across the other positios. The first bit will be 1

2 ivolved i a 4-cycle iff a l = 1 for l {, 3,..., k}. Whe oe observes a permutatio of a biary vector, there are may permutatios that are idistighuishable. The equivalece classes ca be idetified simply by where the oes are placed i the resultig vector. I this case, there are ( 1 ways to choose the a,..., a values that satisfy the costraits. Of these choices, there are ( k ways that have a = a 3 = = a k = 0. Thus, we fid that P (bit 1 ot i 4-cycle = = # way to choose k 1 oes from k positios # way to choose k 1 oes from 1 positios ( k k ( ( k k i 1 k+i = = 1 i 1 i+1 ( 1 Usig the Taylor expasio, oe ca verify that 1 ax 1 bx = 1 ax + bx + O( x (1 ax(1 bx = 1 ax bx + O ( x. Hece, for a (, k-ldpc code, the probability that bit 1 is ivolved i a 4-cycle is give by P (bit 1 i 4-cycle = 1 P (bit 1 ot i 4-cycle ( k 1 k+i = 1 = 1 k 1 i+1 ( 1 k O( k ( k 1 = + O( = (k 1 + O (. Hece, we fid that the expected umber of bits ivolved i 4-cycles is ( (k 1 P (bit 1 i 4-cycle = + O ( (k 1. Sice the umber coverges to a costat, this gives a hit that a clever choice of permutatios might avoid all 4-cycles Cycles i (j, k-ldpc Code By iductio, we ca compute P (bit 1 i 4-cycle i (j, k-ldpc Code: j 1 P (bit 1 i 4-cycle = 1 P (bit 1 ot i 4-cycle = 1 i=1 ( i( 1 ( 1

3 This is agai based o coutig the equivalece classes of permutatios that avoid 4-cycles for the first bit. Usig the same approach as above, oe ca verify that [ ] ( j lim E I (bit i i 4-cycle = (k 1. i=1 Oe ca see this heuristically by oticig that there are ( j ways to choose two sub-blocks of the parity-check matrix ad form a (, k LDPC code. Each of these will cotribute (k 1 ivolved bits o average. 1.3 Fidig 4-Cycles Let H be the parity-check matrix for LDPC code with its i-th colum h i. The dot(h i, h j = # overlappig oes i h i ad h j If H is a sparse matrix i Matlab, the oe ca compute Z = H * H - diag ( sum(h, [i,j,v] = fid(z>= matrix cotaiig # overlappig oes betwee colums of H gives all pairs of colums that cotai 4-cycles 1.4 Removig 4-Cycles H = Radomly swap bad (sub-colum with ay other (sub-colum i same block to remove 4-cycles. If the block legth is sufficietly large, this will reduce the umber of 4-cycles o average ad evetually remove al 4-cycles. The followig simulatio results for a (3, 6 Gallager code with = 3600 shows that effect of removig 4-cycles is larger i the error floor tha i the waterfall regio. 3

4 Performace of (3, 6 LDPC code ( = BER SNR (db Code with 4-cycles (has error floor Code after removig 4-cycles Note: While the above figure was geerated by a actual experimet, the reader is wared that there ca be sigificat variatio i the performace of radomly chose codes with ad without 4-cycles. 1.5 Matlab Code for Fidig & Removig 4-Cycles fuctio H = samplegallager(j,k, % Sample a radom parity check matrix from Gallager s (j,k regular % esemble % Iput: % j The degree of every variable ode % k The degree of every check ode % The legth of LDPC code % Output: % H The parity check matrix of the code col = zeros(j,1; row = zeros(j,1; for i = 0:j 1 col(( i +1:((i+1 = 1:; row((i +1:((i+1 = i /k + ceil(radperm(/k; H = sparse(row,col,1,j /k,; fuctio H = remove 4Cycle(H % G radom parity check matrix i Gallager s (j,k regular esemble, ad % try to remove all 4 Cycle % Iput: % H The parity check matrix before removig 4 Cycle % Output: 4

5 % H The parity check matrix after removig 4 Cycle = size(h,; k = sum(h(1,:; max iter = 3000; iter = 0; while iter < max iter % Fid 4 Cycles Z = H H diag(sum(h,1; [ col1, col ] = fid(triu(z>=; um cycle = legth(col1; % Check if all 4 Cycles are removed if um cycle == 0 fpritf( All 4 cycles have bee removed after %d iteratio\, iter break; for i = 1:legth(col1 % Fid first same elemet idex = fid(h(:,col1(i. H(:,col(i==1,1, first ; if isempty(idex cotiue; % Fid which level is the elemet i level = ceil(idex/(/k; % Compute row idex of that level rows = ( ( level 1 /k + 1 : ( level /k ; % Radomly choose aother sub colum i that level c = radsample( fid(h(idex,: == 0, 1; % Swap the first sub colum ad the radom oe H(rows,[col1(i c] = H(rows,[c col1(i ]; iter = iter + 1; if iter == max iter [ col1, col ] = fid(triu(h H diag(sum(h,1>=; um cycle = legth(col1; fpritf( There are %d 4 cycles remaiig after %d iteratio\, um cycle, max iter LLR Messages i Decodig The simplest approach is to use uiform quatizatio over some bouded regio. I particular, we ca irestrict rage of LLRs to [ L max, L max ] ad use a quatizatio step size of = Lmax #bits 1. This leads to symmetric 0-cetered quatizatio. While the implemetatio is more complex, o-uiform quatizatio allows oe to achieve very low error rates with oly a few bits of storage. For example, see 5

6 .1 Simple Update Rules Bit ode: ˆL i a = L i + b F (i\a L b i Saturate ˆL to [ L max, L max ] (e.g., a value of L max = 15 is sufficiet i may cases. Check ode: where ˆL a i = h j V (a\i h ( L i a j V (a\i sg (L i a ( ( z h(z = l tah L max, z < h 1 (L max = h(l max h(z = 0, z > L max h(z, otherwise This fuctio is plotted for the case of L max = 3 where h(l max h(z/h(z 1 h(z h(z z Note: While this approach to the check update rule is simple ad ca be made to work well i the waterfall regio, the dyamic rage of the h( fuctio ca be problematic whe tryig to achieve low error floors.. Forward-Backward Approach to Check-Node Rule To use o-uiform quatizatio or improve umerical stability, oe ca use a forward-backward approach to the check-ode update. This approach is based o factorig a degree-d check ode ito a chai of degree-3 check odes. 6

7 f x 1 x 3 f 1 f x 1 x x 3 x 4 x y x 4 o observatio L L 3 L 4 x x 3 x 4 f 1 f f 3 x 1 y 1 y x 4 L L 5 Forward-Backward Approach Usig this approach, oe ca have arbitrary o-uiform quatizatio by usig a lookup table for ( ( a b a b tah (tah 1 tah I the example, oe ca compute the output LLR L 3 associated with the iput LLR L 3 usig L y1 = L 1 L L y = L 4 L 5 L 3 = L y1 L y 7

8 .3 Data Storage x 1 x x 3 x 4 π f 1 f f 3 1 Assume messages from the checks are stored i order by check ad the edge umber (i.e., the 1st edge of the 1st check followed by d edge of 1st check, etc... (a Let c[i] be the i-th edge message i this order (b We will refer this edge orderig as the atural order for the checks (c Iitialize all check messages to 0 (i.e., c[i] = 0, i 1,..., E Assume messages from the bits are stored i order by bit ad the edge umber (i.e., the 1st edge of the 1st bit followed by d edge of 1st bit, etc... (a Let b[i] be the i-th edge message i this order (b Likewise, this edge orderig is called the atural order for the checks 3 Decodig proceeds as follows: (a Reorder check messages ito atural order for bit messages: b[i] = c[π(i], i 1,..., E (b Do bit message updates o b[i] (ote: all operatios are memory local (c Reorder bit messages ito atural order check bit messages: c[π(i] = b[i], i 1,..., E (d Do check message updates o c[i] (ote: all operatios are memory local 4 Other advatages (a For a fixed bit ad check ode degrees, oe ca geerate a radom code simply by choosig π to be a uiform radom permutatio. 8

OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES

OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES Peter M. Maurer Why Hashig is θ(). As i biary search, hashig assumes that keys are stored i a array which is idexed by a iteger. However, hashig attempts to bypass