Codes designed via algebraic lifts of graphs

Transcription

1 p./40 Codes designed via algebraic lifts of graphs Clemson Mini-Conference on Discrete Mathematics Oct. 3, Christine A. Kelley Department of Mathematics University of Nebraska-Lincoln

2 p.2/40 Outline Channel coding and LDPC codes Random vs. algebraic constructions Protograph codes Voltage graphs Guidelines for choosing voltage assignments Construction Voltage assignment scheme Examples of codes and their performance Extensions and conclusions

3 p.3/40 Channel Coding Goal: Communication in the presence of noise.

4 y p.3/40 x Channel Coding Goal: Communication in the presence of noise. x =(x,x,...,x ) 2 k Message x = (x,x 2,...,x k ) of length k. Components from alphabet F 2 = {0,}.

5 y p.3/40 x Channel Coding Goal: Communication in the presence of noise. x =(x,x 2,...,x k ) c =(c,c 2,...,c n) Encoder Encoder maps x to codeword c = (c,c 2,...,c n ) of length n, (n > k).

6 y p.3/40 x Channel Coding Goal: Communication in the presence of noise. x =(x,x,...,x ) c =(c,c,...,c n ) r =(r,r,...,r ) 2 k 2 2 Encoder Channel n Received word r = (r,r 2,...,r n ) of length n obtained by from the channel. r = c+error vector

7 y p.3/40 x Channel Coding Goal: Communication in the presence of noise. x =(x,x 2,...,x k ) c =(c,c 2,...,c n ) r =(r,r 2,...,r n) x =(x,x 2,...,x k ) Encoder Channel Decoder Decoder estimates codeword ĉ = (ĉ,ĉ 2,...,ĉ n ) to get estimate ˆx of x. ˆx? = x

8 y p.3/40 x Channel Coding Goal: Communication in the presence of noise. x =(x,x 2,...,x k ) c =(c,c 2,...,c n ) r =(r,r 2,...,r n) x =(x,x 2,...,x k ) Encoder Channel Decoder Code is image of all messages in F k 2 in the space Fn 2. Code is linear if code forms linear subpace of F n 2.

9 p.4/40 Linear Block Codes An [n,k,d] linear block code is a code having codewords of length n dimension k rate r = n k distance d = min c,c C {i c i c i } = min c C {i c i 0}.

10 p.4/40 Linear Block Codes An [n,k,d] linear block code is a code having codewords of length n dimension k rate r = n k distance d = min c,c C {i c i c i } = min c C {i c i 0}. Generator matrix (Encoder): G Mat k n (F 2 ) C = {c = xg x F k 2}.

11 p.4/40 Linear Block Codes An [n,k,d] linear block code is a code having codewords of length n dimension k rate r = n k distance d = min c,c C {i c i c i } = min c C {i c i 0}. Generator matrix (Encoder): G Mat k n (F 2 ) C = {c = xg x F k 2}. Parity-check matrix H is m n matrix such that GH T = 0. C = {c ch T = 0}.

12 p.5/40 Graph representation of a code Let C be a linear block code defined by the following parity-check matrix H. H = parity \bit x 0 x x 2 x 3 x 4 x 5 x 6 p p p p ch T = 0 (x 0,x,...,x 6 )H T = 0

13 p.6/40 Graph representation of a code We can represent H by the following bipartite graph. x 0 x x 2 p 0 x 3 x 4 x 5 p p 2 p 3 ch T = 0 (x 0,x,...,x 6 )H T = 0 x 6

14 p.6/40 Graph representation of a code We can represent H by the following bipartite graph. x 0 x x 2 p 0 p x 3 p 2 x 4 x 5 p3 p 0 : x 0 + x + x 2 + x 5 = 0. x 6 Code is the set of all binary vectors that, when input to the variable nodes, satisfy all the check equations

15 p.6/40 Graph representation of a code We can represent H by the following bipartite graph. x 0 x x 2 p 0 p x 3 p 2 x 4 x 5 p3 p : x 0 + x 2 + x 3 + x 4 = 0. x 6 Code is the set of all binary vectors that, when input to the variable nodes, satisfy all the check equations

16 p.6/40 Graph representation of a code We can represent H by the following bipartite graph. x 0 x x 2 p 0 p x 3 p 2 x 4 x 5 p3 p 2 : x + x 4 + x 5 + x 6 = 0. x 6 Code is the set of all binary vectors that, when input to the variable nodes, satisfy all the check equations

17 p.6/40 Graph representation of a code We can represent H by the following bipartite graph. x 0 x x 2 p 0 p x 3 p 2 x 4 x 5 p3 p 3 : x + x 3 + x 4 + x 6 = 0. x 6 Code is the set of all binary vectors that, when input to the variable nodes, satisfy all the check equations

18 p.7/40 Low Density Parity Check (LDPC) codes LDPC codes are a class of linear block codes characterized by having a sparse parity check matrix H. LDPC codes may be represented graphically via Tanner graphs. LDPC codes provide near-capacity performance while simultaneously admitting implementable decoders.

19 p.8/40 Graph properties that influence iterative decoder. Girth The girth is the smallest number of edges in the graph that form a cycle. 2. Diameter The diameter is the farthest distance between any pair of vertices 3. Expansion 4. Other: Stopping sets, trapping sets, etc.

20 p.9/40 Low Density Parity Check (LDPC) codes: History Gallager in 963 Margulis in the 970s Tanner in 98 MacKay and Neal in 995 Richardson, Urbanke, Shokrollahi in 996

21 p.0/40 Construction Techniques π Random. Regular random LDPCs have been shown to be asymptotically good codes their minimum distance d min grows linearly with block length. Irregular random LDPCs have been shown to perform very close to the theoretical limits at poor channel conditions. Degree profile of the LDPC constraint graph determines convergence behavior of the BP decoder.

22 p.0/40 Construction Techniques π Algebraic. Have structure. Efficient representation. Lower encoding complexity.

23 p./40 Protograph codes Protograph codes are codes obtained by taking lifts of a suitably chosen base graph and using the lift (or, derived graph) as the graph for the code. Idea: the structure of the base graph influences the structure of the graphs representing the codes.

24 p./40 Protograph codes Protograph codes are codes obtained by taking lifts of a suitably chosen base graph and using the lift (or, derived graph) as the graph for the code. Idea: the structure of the base graph influences the structure of the graphs representing the codes. Existing protograph constructions use random lifts, which can be interpreted as replacing s in the parity-check matrix by permutation matrices (array-based LDPC codes), and 0-matrices elsewhere. Other code families (e.g. Repeat-accumulate codes) also designed in this way.

25 Graph Lifts Sketch of graph with a degree 5 lift p.2/40

26 p.3/40 Constructing codes using algebraic lifts Idea: Start with a base graph and use algebraically obtained lifts of the base graph to represent codes. The lifts are determined by algebraic voltage assignments that are associated with the edges in the base graph. This voltage graph framework unifies several families of existing codes [with Walker]. Benefits Ability to prove properties of the resulting codes. Potential to analyze random protograph codes algebraically.

27 p.4/40 Voltage graphs [Gross and Tucker, 977] LetG = (V G,E G ) be a graph where each edge has an orientation, G a finite group, and α : E(T) G is a mapping from the positively oriented edges to G. Then G is an (ordinary) voltage graph and G α = (V α,e α ) is the derived graph, where V α = V G = {v g v V,g G}. E α = E G = {e g e E,g G}. If e + = (u,v) E and α(e + ) = a G, then e + g = (u g,v ga ) E α.

28 p.4/40 Voltage graphs [Gross and Tucker, 977] LetG = (V G,E G ) be a graph where each edge has an orientation, G a finite group, and α : E(T) G is a mapping from the positively oriented edges to G. Then G is an (ordinary) voltage graph and G α = (V α,e α ) is the derived graph, where V α = V G = {v g v V,g G}. E α = E G = {e g e E,g G}. If e + = (u,v) E and α(e + ) = a G, then e + g = (u g,v ga ) E α. If α(e + ) = a G, then α(e ) = a G. G is called the ordinary voltage group, and α is an ordinary voltage assignment.

29 p.5/40 Example: Ordinary voltage graph and derived graph The ordinary voltage group is Z/5Z. u 0 c 0 d 0 v 0 c 4 c u 0 v d e 2 u c d v e 4 e 0 e 2 e e 3 v 4 v 2 v 3 d 4 c 3 u 4 d 2 d 3 u 2 u 3 Remark: G α is a G -degree lift ofg, with natural projection map ρ :G α G. v g v, e g e c 2

30 p.6/40 Permutation voltage graph Let G be a permutation group, i.e. a subgroup of the symmetric group S n. LetG = (V G,E G ) be a graph where each edge has an orientation, and α : E(T) G is a mapping from the positively oriented edges to G. ThenG is a permutation voltage graph and G α = (V α,e α ) is the permutation derived graph where V α = V {,2,...,n} E α = E {,2,...,n}. If τ S n is a permutation voltage on the edge e + = (u,v) ing, then (u i,v τ(i) ) is an edge ing α.

31 p.6/40 Permutation voltage graph Let G be a permutation group, i.e. a subgroup of the symmetric group S n. LetG = (V G,E G ) be a graph where each edge has an orientation, and α : E(T) G is a mapping from the positively oriented edges to G. ThenG is a permutation voltage graph and G α = (V α,e α ) is the permutation derived graph where V α = V {,2,...,n} E α = E {,2,...,n}. If τ S n is a permutation voltage on the edge e + = (u,v) ing, then (u i,v τ(i) ) is an edge ing α. If α(e + ) = σ G, then α(e ) = σ G. Note thatg α is an n-degree lift ofg.

32 Example: Permutation voltage graph and derived graph The permutation voltage group is S 3. u 3 y 2 v3 z 3 x 3 x 2 u 2 y 3 v 2 x z 2 z u y v ( 2 3) x u ()(2 3) y v z ( 2)(3) Remark: G α is a degree 3 lift ofg, with natural projection map ρ :G α G. v i v, e i e, for i =,2,3 p.7/40

33 p.8/40 Walks in the base graph A walk ing is a sequence of oriented edges in the order they are traversed: W = e σ eσ 2 2 eσ n n, with σ i = {±} and e,...,e n E. The length of W is the number of edges in the walk. The net voltage of a walk W is the voltage group product of the voltages on the edges in W in the order and direction of W.

34 p.9/40 Net voltage example The walk W = z + y x + has net voltage ( 2)(3) ()(2 3) ( 2 3) = ()(3)(2) S 3 u 3 y 2 v3 z 3 x 3 x 2 u 2 y 3 v 2 x z 2 z u y v ( 2 3) x u ()(2 3) y v z ( 2)(3)

35 p.20/40 Example: SFT [55,64,20] code as a voltage graph- based code Set q = 3, k = 5, and j = 3. Choose a = 2 and b = 5 in F 3. o(2) = 5 and o(5) = 3 The parity-check matrix H is: H = I I 2 I 4 I 8 I 6 I 5 I 0 I 20 I 9 I 8 I 25 I 9 I 7 I 4 I 28 (93 55), where I x is a 3 3 identity matrix shifted to the left by x positions.

36 p.2/40 Example: Sridhara-Fuja-Tanner codes H = Example of a sparse parity check matrix representing an LDPC code: A 3 5 array of permutation matrices.

37 Example: Sridhara-Fuja-Tanner codes p.2/40

38 Example: Sridhara-Fuja-Tanner codes p.2/40

39 p.22/40 Previous work on choosing voltages Previous work: We classified subgraphs of base graphs that force cycles in derived graph under any abelian voltage assignments [K. and Walker, 2008]. IfG contains an (a,b,c)-theta graph or an (a,a 2 ;b)-dumbbell graph, then for any abelian voltage assignment α : E(G) G, a cycle will result in the derived graphg of length 2(a+b+c) for the theta-graph case. [see also, Exoo] 2(a + a 2 )+4b for the dumbbell-graph case. If a derived graph has an abelian-inevitable cycle (i.e. from abelian voltages), then the graphg contains either a theta graph or a dumbbell graph.

40 p.23/40 Choosing voltages (previous insights) Commuting permutations have been shown to limit girth: For j k array of commuting permutations, the code has girth 2 [Tanner, Sridhara, Fuja, 200.] Inevitable cycles result in derived graphs from (most) base graphs with any abelian voltage assignment to any abelian group. Commuting permutations have also been shown to limit distance: For j k array of commuting permutations, the code has distance ( j+ )! [MacKay and Davey, 999.] Guideline : Choose a nonabelian voltage group (and non-commuting voltages).

41 p.24/40 Cycle Decomposition of Permutations Every permutation has a unique cycle decomposition, for example σ = may be written as σ = (572)(34)(6) The cycle structure of a permutation π in S n is the vector (c,c 2,...,c n ) where c j is the number of j-cycles in the cycle decomposition of π. The cycle structure of σ is (,,0,,0,0,0)

42 p.25/40 Voltages to give good cycle structure Theorem (Gross and Tucker, 977) Let C be a cycle of length k in the base graph of a permutation voltage graph with net voltage π, and let (c,c 2,...,c n ) be the cycle structure of π. Then the pre-image of C in the derived graph has c + c 2 + +c n components, including for each j =,...,n, exactly c j cycles of length k j. The above result also extends to closed walks in G. Guideline 2: Choose voltages so that the net voltages of short cycles (and short closed walks) in G do not have small cycles in their cycle decompositions.

43 How to incorportate Guideline 2 in a design i v i i c v 2 c c 2 i v 3 d Choose voltages so that the net voltages of the 4-cycles ing and avoid cycles of length 3: c c cd dc d d Similarly, we check the closed walks of length 6 and avoid net voltages with cycles of length or 2, and for cycles or walks of length 8 or 2, we avoid fixed points in the net voltages. c 2 (c ) 2 d 2 (d ) 2 cd d c dc c d cdc c d c dcd d c d p.26/40

44 p.27/40 Voltages and connectivity Let α be a voltage assignment fromg to G. Then for any spanning treet ofg, there is an assignment α such that the derived graphg α = G and the edges oft are each assigned the identity element (Gross and Tucker). The edges outside of T form the co-tree, and their voltages generate the local voltage group, G.

45 p.27/40 Voltages and connectivity Let α be a voltage assignment fromg to G. Then for any spanning treet ofg, there is an assignment α such that the derived graphg α = G and the edges oft are each assigned the identity element (Gross and Tucker). The edges outside of T form the co-tree, and their voltages generate the local voltage group, G. The number of components of a permutation derived graphg α with such an assignment is equal to the number of orbits of the action of G on {,2,...,n}. Guideline 3: Choose a permutation group whose action yields one orbit, and assign identity element to the edges of a spanning tree and generating voltages to the edges of the co-tree.

46 p.28/40 Steps of the construction Our examples will use the complete bipartite graph K j,k as a base graph, though method may be applied to any base graph.. Choose base graph and orient edges from variables to checks, and choose a nonabelian group G of order m, and label the elements of G from to m.

47 p.28/40 Steps of the construction Our examples will use the complete bipartite graph K j,k as a base graph, though method may be applied to any base graph.. Choose base graph and orient edges from variables to checks, and choose a nonabelian group G of order m, and label the elements of G from to m. 2. Let G act on itself by left multiplication to obtain an isomorphic group P, where P is a permutation group of order m. The only element in P with a fixed point is the identity permutation.

48 p.28/40 Steps of the construction Our examples will use the complete bipartite graph K j,k as a base graph, though method may be applied to any base graph.. Choose base graph and orient edges from variables to checks, and choose a nonabelian group G of order m, and label the elements of G from to m. 2. Let G act on itself by left multiplication to obtain an isomorphic group P, where P is a permutation group of order m. The only element in P with a fixed point is the identity permutation. 3. Assign voltages Choose a spanning tree of the base graph and assign each edge in the tree the identity permutation i. For edges in co-tree, choose the nontrivial voltages so that each belongs to a distinct cyclic subgroup of the group P, and, in addition, the generators of the group should be chosen among this set.

49 p.29/40 Construction step Choose m = pq such that p and q are prime, q < p, and q (p ). We construct the nonabelian group G generated by elements c and d such that c = p and d = q and dc = c s d, where s (mod p) and s q (mod p).

50 p.29/40 Construction step Choose m = pq such that p and q are prime, q < p, and q (p ). We construct the nonabelian group G generated by elements c and d such that c = p and d = q and dc = c s d, where s (mod p) and s q (mod p). Example: Let p = 3 and q = 2 with relation c 2 d = dc. Then G = {,c,c 2,d,cd,c 2 d = dc}. Order the elements according to their position.,c 2,c 2 3,d 4,cd 5,c 2 d 6.

51 p.30/40 Step 2: Obtaining the permutation group Let each element of G act on G from the left to obtain a permutation group P of order m. Example: The action of c on G = {,c,c 2,d,cd,c 2 d = dc} yields the set {c g g G} = {c,c 2,,cd,c 2 d,d} = {2,3,,5,6,4}. This correponds to the permutation c = (23)(456)

52 p.30/40 Step 2: Obtaining the permutation group Let each element of G act on G from the left to obtain a permutation group P of order m. Example: The action of c on G = {,c,c 2,d,cd,c 2 d = dc} yields the set {c g g G} = {c,c 2,,cd,c 2 d,d} = {2,3,,5,6,4}. This correponds to the permutation c = (23)(456) Note: The action of P on {,2,...,n} has just one orbit by orbit-counting lemma: P σ P fix(σ).

53 p.3/40 Step 3: Assigning the voltages. Assign the identity permutation to the edges of a chosen spanning tree,t. We chooset to contain the eges corresponding to the first row and column of j k array. 2. Choose remaining ( j )(k ) nontrivial voltages so that Each is from a different cyclic subgroup of P. They generate P (for connectivity). Net voltages of short cycles are checked. Note that due to construction, the 8-cycles and 2-cycles will not have fixed points, unless the net voltage is the identity.

54 p.32/40 Code Construction with Order 55 To avoid 3-cycles or smaller in the net voltages, p,q > 3. Let G be the nonabelian group of order m = 55, with p = and q = 5. The relation is c 3 d = dc, where c = and d = 5. We generate the group, label the elements, and let G act on itself to form the isomorphic permutation group P with generators: c : (,2,3,4,5,6,7,8,9,0,) (2,6,7,8,9,20,2,22,23,24,25) (3,26,27,28,29,30,3,32,33,34,35) (4,36,37,38,39,40,4,42,43,44,45) (5,46,47,48,49,50,5,52,53,54,55). d : (,2,3,4,5)(2,8,34,40,49,2) (3,2,32,42,53)(4,24,30,39,46) (5,6,28,44,50)(6,9,26,38,54) (7,22,35,43,47)(8,25,33,37,5) (9,7,3,42,55)(0,20,36,48,0) (,23,27,4,52)

55 p.33/40 Code example I: (2,3) LDPC code Let i denote the identity permutation. H = i i i i c d The voltages are chosen from distinct cyclic subgroups of P and do not commute. The code has block length 65 and code rate The girth is 20. i v i i c v 2 c c 2 i v 3 d

56 p.34/40 Code example II: (3,5) LDPC code H = i i i i i i c c 2 d 2 c 5 d c 8 d 3 i d c 3 d 3 c 4 d 4 c 6 d 2 The code has block length 275 and code rate 0.4. The voltages are chosen from distinct cyclic subgroups of P and do not commute.

57 p.35/40 Simulation Results: (2,3) LDPC codes Performance of (2,3) LDPC codes over the memoryless additive white Gaussian noise (AWGN) channel New construction Array/SFT construction Random construction Randomly chosen permutations 50 sum product iterations Bit error rate (BER) SNR (db) Performance of block length 65, code rate 0.33, (2,3) LDPC codes on the binary-input additive white Gaussian noise channel under sum-product decoding.

58 p.36/40 Simulation Results: (3,5) LDPC codes Performance of (3,5) LDPC codes over the memoryless additive white Gaussian noise (AWGN) channel New construction Array/SFT construction Random construction Randomly chosen permutations 0 3 (50 sum product iterations) Bit error rate (BER) SNR (db) Performance of block length 275, code rate 0.4, (3,5) LDPC codes on the binary-input additive white Gaussian noise channel under sum-product decoding.

59 p.37/40 Extensions: code ensembles Next step: Use optimal base graph and generalize the voltage assignment to an arbitrary nonabelian group of order m = pq. Then prove asymptotic results about the ensemble.

60 p.38/40 Summary and Ongoing work We presented a construction that demonstrates that codes based on algebraic lifts have the potential to outperform random codes, as well as those based on random lifts. Find an explicit relationship between the voltages and distance. Use other nonabelian groups Wider range of permutation groups to make more block lengths possible. Groups with more generators will make the choices of voltages easier. Use ordinary voltage assignments. How voltages affect other parameters important for decoding: trapping sets, stopping sets. Extend to constructions of repeat-accumulate codes, interleavers, etc.

61 References [] J.L. Gross and T.W. Tucker, Topological graph theory, Wiley, New York, 987. [2] G. Exoo, Voltage graphs, group presentations, and cage", The Electronic Journal of Combinatorics, vol. (), [3] R. M. Tanner, D. Sridhara, and T. E. Fuja, A class of group-structured LDPC codes", in Proceedings of International Symposium on Communication Theory and Applications, Ambleside, U.K., pp , July 200. [4] L. Brankovic, M. Miller, J. Plesnik, J. Ryan, and J. Siran, Large graphs with small degree and diameter: A voltage assignment approach", Journal of Combinatorial Mathematics and Combinatorial Computing, vol. 24, pp. 6-76, 997 [5] R. M. Tanner, On quasi-cyclic repeat accumulate codes," in Proc. 37th Allerton Conf. Communication, Control, and Computing, Sept. 999, pp [6] J. Thorpe, LDPC codes constructed from protographs", IPN progress report, pp , JPL, [7] J. L. Fan, Array codes as low-density parity-check codes," in Proc. 2nd. Int. Symp. Turbo codes and related topics, pp , Brest, France, Sept. 4-7, [8] O. Milenkovic, N. Kashyap, D. Leyba, Shortened array codes of large girth", preprint on archive, [9] J. Thorpe, K. Andrews, and S. Dolinar, Methodologies for designing LDPC codes using protographs and circulants.", in Proceedings of the IEEE International Symposium on Information Theory, p. 236, Chicago, June [0] D. J. C. MacKay and M. C. Davey, Evaluation of Gallager codes for short block length and high rate applications", in Codes, Systems, and Graphical Models, B. Marcus and J. Rosenthal, Eds., vol. 23 of IMA Volumes in Mathematics and its Applications, pp Springer, New York, [] R. Smarandache and P.O. Vontobel, On regular quasi-cyclic LDPC codes from binomials", in Proceedings of the IEEE International Symposium on Information Theory, Chicago, IL, June [2] K. Yang and T. Helleseth, On the minimum distance of array codes as LDPC codes", in IEEE Transactions on Information Theory, vol. 49, No. 2, December p.39/40

62 p.40/40 Example: the Binary Erasure Channel (BEC) Input 0 p p Output 0 Binary input B E C Ternary output p p Note: There are no errors but only erasures or losses. The Internet is a real-world example for this channel.

63 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0

64 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0 0

65 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0 0 0

66 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0 0 0

67 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,)

68 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0 0 0

69 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) 0 0 0

70 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,)

71 p.40/40 Example: the Binary Erasure Channel (BEC) Received word = (0,,,,,,,) Estimate=(0, 0,,, 0,, 0, )