Graph-covers and iterative decoding of finite length codes

Transcription

1 Graph-overs and iterative deoding of finite length odes Ralf Koetter and Pasal O. Vontobel Coordinated Siene Laboratory Dep. of Elet. and Comp. Eng. University of Illinois at Urbana-Champaign 1308 West Main Street, Urbana, IL Abstrat: Codewords in finite overs of a Tanner graph G are haraterized. Sine iterative, loally operating deoding algorithms annot distinguish the underlying graph G from any overing graph, these odewords, dubbed pseudo-odewords are diretly responible for sub-optimal behavior of iterative deoding algorithms. We give a simple haraterization of pseudoodewords from finite overs and show that, for the additive, white Gaussian noise hannel, their impat is aptured in a finite set of minimal pseudoodewords. We also show that any (j, k)-regular graph possesses asymptotially vanishing relative minimal pseudo-weight. This stands in sharp ontrast to the observation that for j > 2 the minimum Hamming distane of a (j, k)-regular low-density parity-hek ode typially grows linearly with the length of the ode. 1. Introdution While iterative, message-passing deoding algorithms have had unparalleled suess, it is fair to say that their behavior for the ase of finite length odes is, at present, not well understood. Nevertheless, in some ases speialized tehniques give some insight into the problem. The ase of iterative deoding for the erasure hannel was investigated by Di et al. [8] utilizing the notion of stopping sets. On the other hand, the omputation tree and pseudoodewords were the basis of a finite length analysis introdued by Wiberg [3] developed in [5,6]. Finally, the idea of near-odewords was used by MaKay and Postol [10] to empirially haraterize problemati situations for iterative deoding. The goal of this paper is to ontinue the study of iterative deoding algorithms for finite length odes. It turns out that finite graph overs (in ontrast to the universal over) provide a powerful tool to haraterize the behavior of loally operating, messagepassing deoding algorithms. Not only does our analysis give a risp and quantifiable design riterion for iteratively deodable odes but it also elegantly reflets and unifies the notions of stopping sets, pseudoodewords and near-odewords. We show that the performane of iterative deoding shemes is, even in the high SNR regime, largely dominated not by minimum distane onsiderations but by the notion of pseudo-weight whih, loosely speaking, measures the minimum weight of an error pattern that will ause nononvergene in the iterative deoder. This minimum pseudo-weight is shown to grow sublinearly for sequenes of regular low-density parity-hek (LDPC) odes, whih stands in sharp ontrast to the fat that their expeted minimum distane grows as a linear funtion of the ode length. This paper is organized as follows: In Setion 2 we give some basi notation relating to iterative deoding and we give an illustrative example. Setions 3 and 4 lay out the basi theory behind our analysis. Setion 5 gives bounds on the effetive pseudo-weight of any LDPC ode. Setion 6 skethes algorithmi approahes to omputing the minimum pseudo-weight of a ode. While many fats are stated as theorems, propositions et. in this paper, proofs are generally omitted due to lak of spae. For proofs of the laims we refer to a forthoming paper on these issues [12]. 2. Basis and an Example Let F 2 denote the binary field. A binary, linear ode C of type [n, k] is a k-dimensional subspae of the binary Hamming spae F2 n. Any ode of type [n, k] may be speified as the nullspae of an n (n k) parity-hek matrix H, i.e. C = { F2 n : H T = 0}. We an assoiate a bipartite graph G H, the soalled Tanner graph [1,2,4], with a given parity-hek matrix H in the following way: a vertex f i, i = 0, 1,..., n k 1 is reated for eah row in the parityhek matrix and a vertex j, j = 0, 1,..., n 1 is reated for eah odeword position. Moreover, we reate an (undireted) edge {f i, j } between f i and j if and only if the entry H i,j of the parity-hek matrix is nonzero. The set of parity-hek verties f i is denoted as V f = {f i : i = 0, 1,..., n k 1} and the set of odeword position verties i is denoted as V = { i : i = 0, 1,..., n 1}. We will simultaneously refer to odeword position, odeword position verties and the value of a odeword position by i. The edge set of G H is denoted as E {{v, u} : v V f, u V }. The set of neighbors of a vertex v is

2 λ a) b) λ Fig. 1 a) Tanner graph of a trivial ode of length 3 onsisting of only the zero odeword. b) Convergene regions for the ode illustrated in a). The value of λ 0 is fixed to defined as Γ(v) = {u : {v, u} E} and the degree δ(v) of a vertex v is defined as δ(v) = Γ(v). Let a odword C be transmitted over a noisy, memoryless hannel and let a vetor y be reeived. We an summarize y in form of a vetor λ = (λ 0, λ 1,..., λ n 1 ) of log-likelihood ratios λ i = ln Pr(i=0 yi) Pr(. i=1 y i) The deoding problem onsists of finding the most likely odeword given the vetor λ. The Tanner graph of a ode is the appropriate framework to desribe message-passing deoding algorithms. By now, a variety of suh algorithms is known, all of whih may be seen as instanes of the same underlying priniple [2,3,9]. Most of the development in subsequent setions applies to any loally operating algorithm and is, thus, independent of the partiular hoie of message-passing algorithm. However, whenever we give experimental results we will usually use the so-alled min-sum algorithm [3]. The differene between this algorithm and the more ommon sum-produt algorithm is relatively small and the min-sum algorithm is more amenable to analysis. Before we develop the theory of our approah in the next setion, the following example sheds some light on some basi onepts involved: Example 1 We onsider a trivial ode C of length n = 3 and dimension k = 0 with parity-hek matrix H = and Tanner graph depited in Figure 1a. While it, at first, may seem strange to onsider a zero-rate ode, it is indeed an ideal andidate to investigate problemati behavior of iterative deoding. Under an optimal deision rule the deoding algorithm must output the all-zero word independently of the reeived log-likelihood vetor λ. On the other hand, a simple experiment reveals that the behavior of the deoding algorithm is dependent of the reeived vetor λ. Figure 1b depits the onvergene behavior of an iterative deoding algorithm for a fixed value of λ 0 = as both λ 1 and λ 2 range from 5 to 5. The algorithm fails to onverge after 100 iterations in the blak region of the image while it onverges to the zero ode- λ 1 λ { { { Fig. 2 A ubi over of the graph in Figure 1a. word in the gray olored areas. The speed of onvergene is indiated by the shade of gray. Moreover, we note that this behavior is independent of the algorithm in question (min-sum, sum-produt, et.) and it is found for virtually any loally operating deoding method. A loser study shows that the region of onvergene to the zero word is empirially well desribed (up to numerial auray) by the ondition λ 0 + λ 1 + λ 2 0. In other words, a message-passing algorithm realizes the deoding region of a repetition ode of length 3. In order to understand this behavior we onsider the graph in Figure 2. Figure 2 depits a so-alled ubi over of the graph G H in Figure 1. The graph is obtained by repliating every node in G H three times and introduing edges so that the loal adjaeny relationships between repliated nodes is preserved. (A onise definition of a finite graph over is given shortly). We emphasize two ruial observations: In priniple, loally operating deoding algorithms annot distinguish if they are operating on a Tanner graph G H or any finite over of this graph as, for example, the ubi over depited in Figure 2. The binary odes in finite overs support odewords that do not have an equivalent in the original graph. Suh a odeword is indiated in Figure 2 for the ubi over of G H. It is lear, that any loally operating message-passing algorithm will automatially take into aount all possible odewords in all possible overs of the original graph. In other words, the binary onfiguration indiated in Figure 2 will ompete for the best solution along with all other valid onfigurations in the union of all overs. In the ase of our example ode, the existene of nonzero odewords in finite overs of the original graph explains the behavior of iterative deoding algorithms sine it ats with respet to a reeived word as the all-one onfiguration. In other words, the odeword indiated in Figure 2 is loser than the all-zero word to a reeived word in a region that would orrespond to a virtually present all-one word. Moreover, it an be shown that any nonzero odeword in a finite over of G H has the same effet as a virtually present, all-one odeword.

3 Example 1 shows how odewords in graph overs impat the performane of message-passing algorithms. At first glane it seems a formidable task to haraterize all possible odewords being introdued by the union of finite overs of any degree. (The number of finite overs of a graph grows faster than exponential with the overing degree). However, it turns out that this beomes an objet that itself is elegantly desribed and ompatly represented in the original fator graph of Fig Finite Graph Covers Let a graph G = (V, E) be given with vertex set V = {v 0, v 1,..., v l 1 } and edge set E. Definition 1 A finite degree m over of G = (V, E) is a graph Ĝ with vertex set ˆV = l 1 ˆV i=0 i where eah set ˆV i = {ˆv i,0, ˆv i,1,..., ˆv i,m 1 } ontains exatly m verties. The edge set Ê of Ĝ is hosen as a subset of {{ˆv i,s, ˆv j,r } : {v i, v j } E, s, r {0, 1,..., m 1}} suh that, for eah vertex ˆv i,s ˆV, δ(ˆv i,j ) equals δ(v i ) and Γ(ˆv i,s ) ontains preisely one vertex ˆv j,r for all j suh that v j Γ(v i ) holds. If a graph G is a Tanner graph for a ode C of length n, a degree m over Ĝ is a Tanner graph of a ode Ĉ of length mn. (Any objet relating to a finite over of an underlying graph is distinguished by a ˆ symbol). Verties in ˆV i are denoted as ĉ i,0, ĉ i,1,..., ĉ i,m 1 for lifted nodes i V or ˆf i,0, ˆf i,1,..., ˆf i,m 1 for lifted nodes in f i V f. Any odeword in C an be lifted to a odeword in Ĉ by assigning the value of i to all ĉ i,l. In partiular, the all-zero word in C will be lifted to the all-zero word in Ĉ. In order to haraterize the effet of any nonzero word in Ĉ we repliate the reeived values and loglikelihood ratios to obtain ŷ i,l = y i and ˆλ i,l = λ i for l = 0, 1,..., m 1, thus obtaining vetors ŷ and ˆλ. Let ĉ be a odeword in Ĉ and let ω i(ĉ) be defined as ω i (ĉ) def = {l : ĉ i,l = 1}, m i.e. the fration of times a variable in ˆV i assumes the value 1. The vetor ω(ĉ) = (ω 0 (ĉ), ω 1 (ĉ),..., ω n (ĉ)) plays a ruial role in haraterizing the behavior of odewords in Ĉ. Let the inner produt of two vetors a, b be defined as a, b def = a i b i. Proposition 1 Let a vetor of log-likelihood values λ and its lifting ˆλ be given. Moreover let two words ĉ and ĉ in Ĉ be given. We have Pr{ĉ ˆλ} > Pr{ĉ ˆλ} if and only if ω(ĉ), λ < ω(ĉ ), λ holds. The most important property of Proposition 1 is that odewords in Ĉ an be effetively haraterized by the vetors ω(ĉ). Assume that C and its lifted version ĉ are the all-zero odeword in Proposition 1. It follows that pairwise deisions between and a ompeting nonzero odeword ĉ Ĉ will partition the spae of λ into two regions separated by the hyperplane ω(ĉ ), λ = 0. For any partiular hannel model we an ompute the distane of this hyperplane from the transmitted signal point in signal spae, thus effetively haraterizing a type of minimum distane, the so-alled pseudo-distane [3,5,6]. Let x q = ( i xq i ) 1 q denote the L q norm of a vetor. For the binary antipodal signaling on an additive, white, Gaussian noise (AWGN) hannel we have the following definition [3,5,6]: Definition 2 (Pseudo-odewords) Let ĉ Ĉ be a odeword in a over of the Tanner graph G. We all ω = ω(ĉ) a pseudo-odeword of C. Its pseudoweight w p (ω) on an additive, white, Gaussian noise hannel is given by w p (ω) def = ( ) 2 ω 1. (1) ω 2 Let wp min (C) denote the minimum pseudo-weight of all nonzero pseudo-odewords of C taken over all finite degree overs of G. Remark 1 Note that if is a odeword with Hamming weight w H (), then ω =, 1 = w H () and 2 = w H (). It follows that w p () = 2 1 / 2 2 = w H () 2 / w H () = w H (). The pseudo-weight measures the distane of the all-zero odeword in signal spae to a pairwise deision boundary aused by a pseudo-odeword ω. Proposition 2 Let a binary ode be used on an additive, white, Gaussian noise hannel with antipodal signaling with signal alphabet {±1}. Let a nonnegative vetor ω be given. The squared Eulidean distane in the signal spae between the signal point 1, orresponding to the all-zero word, and the hyperplane ω, y = 0 is given as w p (ω). Remark 2 Proposition 1 is independent of the partiular hannel. In the spae of log-likelihood ratios λ the pseudo distane is always proportional to the pseudo-weight of Definition 2. However, signal spae is, in general, not linearly related to λ and we get different pseudo-distane expression for non-awgn hannels. Expressions for the pseudo-distane in the ontext of nonbinary signaling, the binary symmetri

4 hannel and the binary erasure hannel an be found in [5]. Proposition 1, in onjuntion with the fat that loally operating deoding algorithms annot distinguish between G and Ĝ motivates our subsequent task to haraterize vetors ω(ĉ) for the union of all possible finite overs. While this, at first, appears to be a diffiult task, we will see in the next setion that it is elegantly solved by the original Tanner graph G. 4. The Fundamental Polytope We start this setion by onsidering a simple parityhek ode C δ of length δ and its Tanner graph onsisting of a single parity-hek node f 0 of degree δ and δ variable nodes 0, 1,..., δ 1. Any finite over of degree m of G is simply an m-fold opy of the original graph G. It is partiularly simple to desribe the pseudo-odeword indued by these m-fold repetitions. We onsider a odeword in the original parity-hek ode desribed by G as a odeword over the real numbers with elements {0, 1}. Sine any individual opy of G an support any odeword from C δ, the possible set of words ω(ĉ) originating from the m-fold over an be desribed as the set of vetors { m i=1 } i m : i C δ. Let a matrix P δ be defined as the 2δ 1 δ matrix ontaining all binary even weight vetors. As we onsider overs over larger and larger degree m, we have the following proposition: Proposition 3 Let a Tanner graph G δ be given onsisting of a single parity-hek node of degree δ and δ variable nodes. Consider the set P of pseudoodewords ω(ĉ) taken over the union of all overs of G of all degrees m = 1, 2,.... The losure of P in the real numbers is desribed by the polytope (G δ ) def = {ω R n : ω = xp δ, x R2δ 1, 0 x i 1, i x i = 1} Example 2 We onsider the Tanner graph of a parityhek ode of length three. The polytope (C 3 ) of all possible vetors ω(ĉ) is depited in Figure 3. It is atually possible to extend Proposition 3 to a nontrivial Tanner graph G. To this end, let the restrition of a vetor ω to a set V of variable nodes be denoted as ω V. ω 3 (0,1,1) (1,0,1) ω (1,1,1) 2 (1,1,0) ω 1 Fig. 3 The pseudo-odeword polytop for a [3,2] parityhek ode. Theorem 4 Let a Tanner graph G be given with parity-hek nodes f 0, f 1,..., f l 1 and variable nodes 0, 1,..., n 1. Let P be the set of pseudo-odewords ω(ĉ) taken over the union of all overs of G of all degrees m = 1, 2,.... The losure of P in the real numbers is desribed by the polytope (G) = {ω R n : ω Γ(fi) (G δ(fi)), i = 0, 1,..., l 1}. Theorem 4 gives a ompat and elegant haraterization of the possible vetors ω for any given Tanner graph G. In fat, the polytope (G) is itself ompatly representable in G by hoosing for variable nodes the alphabet R and assoiating with node f i the indiator funtions of the parity-hek polytopes (G δ(fi)). (G) is a onvex body entirely inside the positive orthant and with one orner of (G) loated in the origin. To any vetor ω in (G) we an find at least one (in general there are many) odewords ĉ in some finite over of G suh that ω = ω(ĉ). Moreover, this pseudo-odeword has pseudo-distane w p (ω) from the all-zero odeword. Note that all multiples of the vetor ω have the same pseudo-weight. Hene, provided we relate our future disussion to the all-zero odeword we an restrit our attention to the (onvex) one that is generated by (G). We all this objet the fundamental one of the graph G. Definition 3 Let a Tanner graph G be given with assoiated polytope (G). The fundamental one (G) assoiated with G is defined as (G) = {µω R n : ω (G), µ 0} Assuming that the all-zero word was transmitted, Proposition 1 motivates the definition of a region 0 in R as 0 = {λ R : ω, λ > 0, ω (G)}.

5 ω 3 (0,1,1) (1,1,1) reeived vetor. Thus, the shape of 0 for general hannels is not neessarily a polytope. 0 (1,0,1) ω 2 ω 1 (1,1,0) Fig. 4 Deision region in binary on-off keying due to the orners of the pseudo-odeword polytope for a [3,2] parity-hek ode. 0 is the region where the all-zero word is more likely than any ompeting odeword ĉ in a finite over. The pseudo-weight of a vetor ω may be expressed as w p (ω) = n(os( (ω, 1))) 2 where (ω, 1) denotes the angle between the vetor ω and the all-one vetor. Hene, the minimum pseudo-weight wp min (C) is ahieved by a orner of the onvex one that enloses the maximal angle with the all-one vetor. Let U(G) be the set of orner points in (G). For the AWGN hannel we have the following theorem: Theorem 5 Let (G) be the fundamental one of a Tanner graph G. For the AWGN hannel, the region 0 may be desribed by the ornerpoints of (G) alone, i.e. 0 = {λ R : ω, λ > 0, ω U(G)} Remark 3 Theorem 5 allows us to ompatly represent the region 0. Maximum likelihood deision regions on an AWGN hannel are determined by soalled minimal odewords whih are the subset of odewords that ontribute a fae to the maximum likelihood deision region polytope. Here we have a quite similar situation where for the AWGN hannel again a finite set of minimal pseudoodewords, i.e. the set U(G), ontributes faes to the polytope 0. Example 3 Theorem 5 gives a risp haraterization of the region 0. We an use this haraterization to investigate LDPC odes and their parameters. A partiularly nie LDPC ode was onstruted by Tanner et al. [11]. The ode is a regular (3,5)-LDPC ode (all variable nodes in the Tanner graph have degree three and all hek nodes have degree five), of length 155, dimension 64 and minimum Hamming distane 20. Its parity-hek matrix of size would atually suggest a R = 2/5 ode, but beause of rank loss, the atual rate is slightly higher, namely R = 64/155 = The underlying graph G has a girth of 8 and a diameter of 6 whih, together with the relatively large minimum distane of twenty (the best known ode with the the same length and dimension has minimum Hamming distane 28), makes this ode an outstanding andidate for iterative deoding. However, it is relatively easy to find a pseudo-odeword in U(G) whih has pseudo-weight only Thus the large minimum distane of the ode is largely irrelevant for iterative deoding and does not determine the performane of the ode. In partiular, based on the automorphism group of the graph, the multipliity of pseudo-odewords of weight is, at least, 155. We onlude this setion with a theorem for the well understood ase that the Tanner graph of a ode C is a tree. In this ase iterative deoding realizes the optimal deoding algorithm. This is niely refleted in the shape of the fundamental one (G). Theorem 6 Let (G) be the fundamental one of a Tanner graph G. Moreover, assume that G is a tree. Let M be the set of minimal odewords of C. The fundamental one F(G) is generated by the set M, i.e. (G) = {ω R : ω = α(), 0 α() R}. M Thus, if G is a tree, 0 is exatly the maximum likelihood deision region of the all-zero odeword. Remark 4 For the AWGN hannel, Theorem 5 translates diretly into a desription of the set 0 in signal spae sine λ depends in an affine way on the reeived vetor y. For example, the region 0 for binary onoff signaling and the fundamental one of Figure 3 is indiated in Figure 4. However, we note that, λ does, for other hannels, in general, not depend in an affine way on a signal spae representation of the 5. An Upper Bound on the Minimal Pseudo-Weight In this setion we investigate the asymptoti behavior of the minimum pseudo-weight of a Tanner graph G. Let g(g) be the girth of G, and let (G) be its diameter. Given any variable node v in G let v (G) denote the maximal distane from v that any

6 node in G an have. The ode C is alled a (j, k)- regular ode if the uniform olumn weight of parityhek matrix H is j and the uniform row weight of H is k. Definition 5 We denote an arbitrary variable node v of G to be the root. We lassify the remaining variable and hek nodes aording to their (graph) distane from the root, i.e. the root is a tier 0, all nodes at distane 1 from the root will be alled nodes of tier 1, all nodes at distane 2 from the root node will be alled nodes of tier 2, et.. We all this ordering breadth first spanning tree ordering with root v. Beause of the bipartiteness of G, it follows easily that the nodes of the even tiers are variable nodes whereas the nodes of the odd tiers are hek nodes. Furthermore, a hek node at tier 2t + 1 an only be onneted to variable nodes in tier 2t and possibly to variable nodes in tier 2t + 2. Note that the last variable node tier is tier v (G) and that the symbol nodes are at tiers 0, 2,..., 2 v (G)/2. Remark 6 Let the Tanner graph of a binary (j, k)- regular ode C be given and let v be an arbitrary bit node. We perform breadth first spanning tree ordering with respet to v aording to Def. 5. Let N t (C) be the number of nodes at tier t and let Nt max = Nt,j,k max be the maximal number of nodes possible at tier t. It is not diffiult to see that N max 0 = 1, N max 1 = j, N max 2 = j(k 1), N max 3 = j(k 1)(j 1), N max 4 = j(k 1)(j 1)(k 1). In general, N2t max = j(j 1) t 1 (k 1) t for t > 0 and N2t+1 max = j(j 1) t (k 1) t for t 0. Definition 4 (Canonial ompletion) Let the Tanner graph of a binary (j, k)-regular ode C be given and let v be an arbitrary symbol node. After performing the breadth first spanning tree ordering with root v we onstrut a pseudo-odeword ω in the following way. If bit i orresponds to a variable node in tier 2t, then ω i def = 1 (k 1) t. (2) We all this the anonial ompletion with root v. Proposition 7 The anonial ompletion with root v yields a vetor ω suh that ω is in the fundamental one (G). The vetor ω has pseudo-weight w p (ω) = ω 2 1 / ω 2 2, where ω 1 = v(g)/2 t=0 v(g)/2 ω 2 2 = t=0 1 N 2t (G) (k 1) t, (3) ( N 2t (G) 1 (k 1) t ) 2. (4) Fig. 5 Tanner graph for the [7,4,3] Hamming ode For a given G, one an alulate the pseudo-weight of the pseudo-odeword given by the anonial ompletion for any given root; this will always yield an upper bound on wp min (C). Example 4 We onsider the Tanner graph of the [7, 4, 3] Hamming ode given in Figure 5. The anonial ompletion with root 0 orresponds to a vetor ω = (1, 1 9, 1 9, 1 3, 1 3, 1 9, 1 3 ). It is easy to hek that this pseudo-odeword is indeed inside the fundamental polytope for this graph. The pseudo-weight in this ase equals ( )2 = We note that the Tanner graph of Figure 5 also supports a pseudo-odeword ω of type ω = (1, 0, 0, 1 3, 1 3, 0, 1 3 ). The pseudo-weight of ω equals only three and is thus at minimum distane for this ode. The anonial ompletion with a given root is not only a generally good andidate in order to find a pseudo-odeword of low weight but it is also a poweful enough tehnique to show the asymptoti behavior of the pseudo-weight by properly bounding ω 1 and ω 2 2. Theorem 7 Let C be a (j, k)-regular LDPC ode with 3 j < k. Then the minimum pseudo-weight is upper bounded by where β j,k = j(j 1) j 2 w min p (C) β j,k n β j,k, (5) 2, β j,k = log (j 1) 2 log (j 1)(k 1) < 1. (6) Corollary 8 Consider a sequene of (j, k)-regular LDPC odes whose length goes to infinity. The relative minimum pseudo-weight (i.e. the fration of minimum pseudo-weight to ode length) must go to zero.

7 Remark 9 Note that Corollary 8 is in sharp ontrast to the fat that the relative minimum weight of a randomly generated (j, k)-regular LDPC ode is lower bounded by a nonzero number with probability one for n [7]. a) b) Remark 10 The different nature of pseudo-weight with respet to different hannels is underlined by the fat that the anonial ompletion with respet to any given root yields a small pseudo-weight in the AWGN ase while its normalized pseudo-weight on the erasure hannel equals one. Nevertheless, the fundamental one still haraterizes the set of pseudo-odewords it is the worst ase pseudo-odeword within the fundamental one that is different. 6. Relations to Stopping Sets and Near Codewords Stopping Sets Stopping sets were introdued in [8] as a means to understand the suboptimal behavior of iterative deoding tehniques for the erasure hannel. It has been observed later that stopping sets seem to also reflet, to some degree, the performane of iteratively deoded odes for other hannels. Let S be a subset of variable nodes and onsider the subgraph G of G indued by S and the neighbors of S. S is alled a stopping set if G does not ontain any hek nodes of degree one. Theorem 11 Let x be a vetor that equals one in a stopping set S and whih is zero otherwise. There exists an α with 0 < α 1 suh that ω = αx is a pseudo-odeword of pseudo-weight S. While the notion of stopping set is well suited to the erasure hannel where the pseudo-weight is defined as the support of a pseudo-odeword [5], it is not refined enough to apture the situation for the AWGN hannel. Figure 6 shows two Tanner graphs that only allow the all-zero word as valid odeword. Both graphs admit a pseudo-odeword ω = (2/3, 2/3, 2/3, 2/3) in the orresponding fundamental ones that has an interpretation as stopping set. However, in addition to this pseudo-odeword, the fundamental one F(G) of one of the two graphs ontains a pseudo-odeword of pseudo-weight only three. Near-Codewords MaKay and Postol [10] introdued the notion of near-odewords. These are vetors x with x i = 0 or x i = 1 for all 1 i n suh that the syndrome s = xh T has low Hamming weight. Espeially interesting are the low-weight nearodewords. 2/3 2/3 2/3 2/3 0 2/3 2/3 2/3 Fig. 6 Two stopping sets of size four. Pseudo- Codewords ω are indiated that ahieve different minimum pseudo-weight on an AWGN. While the notion of near odewords is helpful in understanding potential problems in the design of iteratively deodable odes it suffers from being quantifiable in a preise sense. For example, a single one in a (j, k)-regular ode may be onsidered as a near odeword with syndrome weight j. In order to make a preise statement on how problemati this near odeword is, one an find a orner in the fundamental one that is lose to the vetor ontaining a single one. Note that any near odeword an be ompleted into a pseudo-odeword with a proedure similar to the anonial ompletion (now rooted at the near odeword). This gives a preise measure of the effet of a near odeword. 7. Algorithmi Issues Theorem 5 gives a risp haraterization of the minimal pseudo-odewords, i.e. the set of pseudoodewords that determine the shape of the region 0. In this setion we investigate algorithmi issues to find pseudo-odewords of small pseudo-weight. In this ontext it is interesting to note that the fundamental one is readily represented in the original Tanner graph by re-interpreting the funtion nodes and the variable nodes. To this end let a matrix P δ δ be defined as the 2 δ matrix ontining all binary weight two vetors. For a real valued vetor of length δ, let an indiator funtion I δ (ω) be defined as { 1 x R I δ (z) = δ : z = xp δ, x i 0 0 otherwise. Membership in the fundamental one (G) an thus be tested by heking the indiator funtion I G (ω) = l i=0 I δ(fi)(ω Γ(fi)). The fator graph [2] that is obtained by assigning the indiator funtions I δ(fi)(ω Γ(fi)) to the individual funtion nodes f i and by letting the variable alphabets be R gives, in fat, a suitable framework for an iterative algororithm to find pseudo-odewords. While there is some oneptual appeal to this approah it is essentially similar to a gradient desent algorithm.

8 Permutation Π high weight. However, hoosing a vetor v whih ontains a single one in a position and is zero otherwise will yield pseudo-odewords of smaller weight. The same is true in general if the support of v is hosen aording to a near-odeword. Fig. 7 The permutation in an LDPC ode In the sequel we desribe a linear programming approah to finding pseudo-odewords of small pseudoweight. For simpliity we restrit the subsequent desription to (j, k)-regular LDPC odes. The generalization to irregular odes is straightforward. LDPC odes may be desribed by a permutation that maps edges in G = (V, E) whih are inident with variable nodes to edges that are inident to funtion nodes (see Fig. 7). Let Π be the orresponding E E permutation matrix. Let a (j 1) j matrix F j be defined as F j = [ 1 : I j 1 ] where I j 1 is a j 1 j 1 identity matrix. F 1 is defined as the empty matrix. Let A be a m n matrix and let the Kroneker produt of two matries A, B be defined as A B = a 1,1 B a 1,2 B... a 1,n B a 2,1 B a 2,2 B a 2,n B.. a m,1 B a m,2 B... a m,n B. We have the following proposition haraterizing the fundamantal one: Proposition 8 Let a length n, (j, k)-regular LDPC ode be given with assoiated graph G and permutation matrix Π. Let matries W, Z be defined as W = I nj P k and Z = WΠ(I nk F j ). Moreover, k j let for a given vetor x, x :j denote the sub-sampled vetor (x 0, x j, x 2j,...) The fundamental one may be desribed as (G) = {ω R n : ω = (xwπ) :j, xz = 0, x i 0} While the desription of the fundamental one in Proposition 8 seems umbersome at first, it is well suited to formulate a linear program to find pseudoodewords of small pseudo-weight: Linear Program: Given v and the graph G Minimize v, (xwπ) :j Subjet to: xz = 0, x, 1 = 1, x i 0. The above linear program an be used to hek a given graph G for pseudo-odewords in the set U(G). For a random hoie of the vetor v we will typially get a pseudo-odeword in U(G) of relatively REFERENCES [1] R. M. Tanner, A reursive approah to low-omplexity odes, IEEE Trans. on Inform. Theory, vol. IT 27, pp , Sept [2] F. R. Kshishang, B. J. Frey, and H.-A. Loeliger, Fator graphs and the sum-produt algorithm, IEEE Trans. on Inform. Theory, vol. IT 47, no. 2, pp , [3] N. Wiberg, Codes and Deoding on General Graphs. PhD thesis, Linköping University, Sweden, [4] N. Wiberg, H.-A. Loeliger, R. Kötter, Codes and Iterative Deoding on General Graphs, European Transations on Teleommuniations, 6(5), pp , September [5] G. D. Forney, Jr., R. Koetter, F. Kshishang, and A. Reznik, On the effetive weights of pseudoodewords for odes defined on graphs with yles, in Codes, systems, and graphial models (Minneapolis, MN, 1999), vol. 123 of IMA Vol. Math. Appl., pp , New York: Springer, [6] B. J. Frey, R. Koetter, and A. Vardy, Signalspae haraterization of iterative deoding, IEEE Trans. Inform. Theory, vol. 47, no. 2, pp , [7] R.G.Gallager, Low-Density Parity-Chek Codes. M.I.T. Press, Cambridge, MA, 1963, available online under people/gallager.html. [8] C. Di, D. Proietti, I. E. Telatar, T. J. Rihardson, and R. L. Urbanke, Finite-length analysis of low-density parity-hek odes on the binary erasure hannel, IEEE Trans. on Inform. Theory, vol. 48, no. 6, pp , [9] S. M. Aji and R.J. MEliee The Generalized Distributive Law, IEEE Trans. Inform. Theory, vol. 46, no. 2, pp , Marh [10] D. J. C. MaKay and M. S. Postol, Weaknesses of Margulis and Ramanujan-Margulis low-density parity-hek odes, preprint, [11] R. M. Tanner, D. Sridhara, and T. Fuja, A lass of group-strutured LDPC odes, Pro. of ICSTA 2001, Ambleside, England, [12] R. Koetter, P.O. Vontobel, Graph-overs and iterative deoding of finite length odes, in preparation, 2003