A Branch and Cut Algorithm to Design. LDPC Codes without Small Cycles in. Communication Systems

Transcription

1 A Branch and Cut Agorithm to Design LDPC Codes without Sma Cyces in Communication Systems arxiv: v1 [cs.it] 28 Sep 2017 Banu Kabakuak 1, Z. Caner Taşkın 1, and Ai Emre Pusane 2 1 Department of Industria Engineering, Boğaziçi University, İstanbu, Turkey 2 Department of Eectrica and Eectronics Engineering, Boğaziçi University, İstanbu, Turkey In a digita communication system, information is sent from one pace to another over a noisy communication channe using binary symbos (bits). Origina information is encoded by adding redundant bits, which are then used by ow density parity check (LDPC) codes to detect and correct errors that may have been introduced during transmission. Error correction capabiity of an LDPC code is severey degraded due to harmfu structures such as sma cyces in its bipartite grapepresentation known as Tanner graph (TG). We introduce an integer programming formuation to generate a TG for a given smaest cyce ength. We propose a branch-and-cut agorithm for its soution and investigate structura properties of the probem to derive vaid inequaities and variabe fixing rues. We introduce a heuristic to obtain feasibe soutions of the probem. Our computationa experiments show that our agorithm can generate LDPC codes without sma cyces in acceptabe amount of time for practicay reevant code engths. Keywords: Teecommunications, LDPC code design, integer programming, branch and cut agorithm. Corresponding author. Te.: ; fax: E-mai addresses: banu.kabakuak@boun.edu.tr (B. Kabakuak), caner.taskin@boun.edu.tr (Z. C. Taşkın), ai.pusane@boun.edu.tr (A. E. Pusane).

2 1 Introduction and Literature Review Teecommunication is the transmission of messages from a transmitter to a receiver over a potentiay unreiabe communication environment. In a digita communication system, binary code symbos (bits) represent the messages. In parae to the rapid deveopments in technoogy, digita communication systems find severa appication areas: messaging via digita ceuar phones, fiber optic internet, TV broadcasting or agricutura monitoring through digita sateites, and receiving high quaity images of Jupiter under NASA s Juno mission [1] are some exampes of digita communication. In practice, numerous transmitter receiver pairs share the same communication environment such as air or space. Hence, radio waves, eectrica signas, and ight waves over fiber optic channes accumuate some amount of noise on the medium. The noise in the environment can cause transmission errors or faiures. Channe coding is the term used for the coection of techniques that are empoyed in digita communications to ensure that a transmission is recovered with minima or no errors. These techniques encode the origina information by adding redundant bits. When the receiver receives information, the decoder estimates the origina information by detecting and correcting errors in the received vector with the hep of redundant bits. Among the codes that are used in the decoding process at receiver, ow density parity check (LDPC) code famiy has received attention thanks to its high error detection and correction capabiities. LDPC codes were first proposed by Gaager in 1962 and today they are used in wireess network standard (IEEE n), WiMax (IEEE e), and digita video broadcasting standard (DVB-S2) [2]. They have sparse parity check matrices, i.e., H matrix, and can aternativey be represented by bipartite graphs known as Tanner graphs (TG) [3]. A TG (or LDPC code) is said to be (J, K) reguar if a nodes at one side of the bipartite graph have degree J and a other nodes have degree K (see Section 2 for a forma definition). Otherwise, a TG (or LDPC code) is irreguar and degrees of the nodes can be expressed with a degree distribution. Iterative decoding agorithms, which have ow compexity and ow decoding atency due to the sparsity property of parity check matrix, have been deveoped on TG [4, 5]. Iterative decoding agorithms decide on whether each code symbo is 0 or 1 by cacuating probabiities for the code symbos to estimate the origina information. The cacuated probabiities are dependent on each other if there are cyces on the TG. In order to minimize code symbo estimation errors, designing LDPC codes to maximize the smaest cyce ength, i.e., girth, is usefu. There are different approaches in the iterature for obtaining a TG with arge girth. 2

3 One approach is to eiminate the cyces with ength smaer than the target girth from a given TG. In [6], certain edges are exchanged within TG to eiminate sma cyces without simutaneousy creating any others. In the edge deetion agorithm of [7], an edge that is common for the maximum number of cyces is seected. These methods are heuristic approaches and they change the degree distribution of the nodes in the TG. It is known that the degree distribution affects the error correction capabiity of an LDPC code [8]. Hence, it is important to eiminate as few edges from TG as possibe. There are studies based on optimization techniques in the iterature to find the best degree distribution of an irreguar TG in terms of error correction capabiity [8, 9]. Another way of designing an LDPC code is to construct a TG from scratch. Bit Fiing heuristic in [10] starts with a arge girth target and decreases target as it inserts edges to TG one by one. The heuristic terminates when a prescribed girth is met. A randomized approach in [11] can create irreguar LDPC codes by introducing new edges in a zig zag pattern. Progressive Edge Growth (PEG) heuristic in [12] is based on adding edges to the TG iterativey without constructing sma cyces. PEG agorithm is adjusted to generate a reguar LDPC code in [13] and an irreguar LDPC code in [14] for improving the error correction performance. Independent tree based heuristic of [15] can iterativey construct reguar LDPC codes whose girth vaues are better than the ones obtained by PEG. A protograph is a TG with a reativey sma number of nodes. Design of LDPC codes with simpe protographs is investigated in [16] to obtain infinite dimensiona LDPC codes. Different studies in the iterature focus on the design of LDPC codes with arge girth using the protograph [17, 18]. Agebraic construction is to construct structured LDPC with agebraic and combinatoria methods. Turbo LDPC (T LDPC) codes are structured reguar codes whose TG incudes two trees connected by an intereaver. In [19], authors design the intereaver to avoid sma cyces and obtain T LDPC codes with high girth. Quasi cycic LDPC (QC LDPC) codes consist of identity matrices whose coumns are shifted by a certain amount. A method that can buid QC LDPC codes with girth at east 6 using Vandermonde matrices is introduced in [20]. A technique to generate irreguar QC LDPC codes with girth at east 8 is given in [21]. Quasi cyce constraints are added to PEG agorithm in order to obtain reguar and irreguar QC LDPC codes in [22]. Other studies aso use PEG agorithm for this code famiy [23] [25]. For the same code famiy, a ifting method is given in [26] and generaized poygones are used in [27]. Patent [28] describes a method for QC LDPC codes, that guarantees a girth of at east 8. The above mentioned methods are heuristic approaches and they may fai to generate a TG for a given dimension with a target girth vaue. On the other hand, optimization techniques are capabe 3

4 of finding a TG for a given girth vaue, or proving that there cannot be such a TG. Combinatoria approaches to design QC LDPC codes are utiized in [29] to find the best degree distribution of the nodes in a TG. Authors obtain the degree distribution by evauating a aternatives witespect to some performance metrics and choosing the most promising one. Then, authors construct a TG for the seected degree distribution. In [30], the seection criteria of PEG agorithm to ocate an edge in a TG is modified in order to have a better girth vaue than PEG. The generated TG does not necessariy have the argest girth vaue, since their method is a TG constructive heuristic. There are other LDPC code constructive heuristics in the iterature that avoid sma cyces [31] [33]. A genetic agorithm to design a TG with a sma number of nodes is given in [34]. In [35] a modified shortest path agorithm is used to construct a TG. Our contribution to the iterature can be isted as foows: We investigate the LDPC code design probem, which seeks a TG of desired dimension with a target girth vaue, from an optimization point of view. We propose an integer programming formuation to generate LDPC codes with a given girth vaue and deveop a branch and cut agorithm for its soution. We investigate structura properties of the probem for (J, K) reguar codes to improve our agorithm by appying a variabe fixing scheme, adding vaid inequaities and utiizing an initia soution generation heuristic. Our computationa resuts indicate that our proposed methods significanty improve sovabiity of the probem. We aso iustrate how our method can be used to find the smaest dimension n that one can generate a (J, K) reguar code (see Tabe 7). The remainder of the paper is organized as foows: we formay define the probem and introduce our mathematica formuation in the next section. Section 3 expains the proposed branch and cut method and techniques to improve its performance. We test the efficiacy of our methods via computationa experiments in Section 4. Some concuding remarks and comments on future work appear in Section 5. 2 Probem Definition Figure 1 shows information fow in a digita communication system. In Figure 1, et the origina information be a binary vector u = (u 1 u 2...u k ) of k bits, i.e., u i {0, 1}. Encoder adds redundant 4

5 parity check bits to vector u by utiizing a k n generator matrix G. That is codeword w = (w 1 w 2...w n ) of n bits, where n k and w i {0, 1}, is obtained through the operation w = ug. In u Digita Source a codewordencoder w, there are k information bits and (n k) parity check bits, which are used to test whether there are H = A errors in the transmission. For integrity of the communication, codeword w shoud be in the nu space Noise of the (n k) n parity check matrix H, i.e., wh T = 0 B(mod 2) hods. After transmission, the receiver gets vector v of n bits as shown in Figure 1. Decoder detects whether û Digita Sink the received vector v incudes errors or not by checking whether the expression vh T Decoder is equa to vector 0 in (mod 2) or not. In the case that v is erroneous, the decoder attempts to determine error ocations Generator Matrix G Parity-Check Matrix and fix them [36]. As a resut, the information u sent from the source is estimated as û at the sink. H v Binary Symme Channe r Generator Matrix G Digita Source u Encoder w Noise Coding Channe Digita Sink û Decoder v Parity-Check Matrix H Figure 1: Digita communication system diagram Üreteç Matrisi (G) In this work, we focus on the binary symmetric channe (BSC) for modeing the noisy communication u channe. As shown in Figure Mesaj 2, Kaynağı in a BSC, an error occurs Kana Kodayıcı with probabiity p and the transmitted bit v fips, i.e., if a bit is 0, it becomes 1 and vice versa. The transmission is competed without any errors with probabiity 1 p [37]. The decoder aims Gürütü to find the ocations İetişim of Kanaı the errors in BSC. Once the decoder detects a bit is erroneous, it corrects the error by fipping the bit s vaue. LDPC codes are members of inear bock û codes that can be represented by a sparse parity check Kana Kod Mesaj Hedefi Çözücüsü matrix H, i.e., the number of ones at every row and coumn of the H matrix is forced to be very sma. An LDPC code is reguar, if there are constant number of ones at each coumn and row of the matrix. Eşik-Denetim As given in Figure 3, a (3, 6) reguar LDPC code has ony Matrisi 3 (H) ones at each coumn and 6 ones at each r 5

6 0 1 1 p p p 1 p 0 1 Figure 2: Binary symmetric channe row independent from the dimension of the H. This impies that for (3, 6) reguar LDPC code with dimension , ony 0.2% of the matrix eements are nonzero H = Figure 3: A parity check matrix from (3, 6) reguar LDPC code famiy An LDPC code can aternativey be represented as a TG, which is a sparse bipartite graph, corresponding to the H matrix [3]. On one part of the TG there is a variabe node j (v j ), j {1,..., n}, for each bit of received vector. Eacow of the H matrix represents a parity check equation and corresponds to a check node i (c i ), i {1,..., n k}, on the other part of the TG. A check node is said to be satisfied if its parity check equation is equa to zero in (mod 2). The degree of v j (c i ) is the number of adjacent check nodes (variabe nodes) on the TG. Hence, H matrix is the bi adjacency matrix of the TG. This representation of LDPC codes is practica due to the advantage of appying iterative decoding agorithms easiy. Figure 4 shows the TG representation of the H matrix defined in Figure 3. v1 v2 v3 v4 v5 v6 v7 v8 v9 v c1 c2 c3 c4 c5 Figure 4: TG representation of the parity check matrix given in Figure 3 6

7 It is known that iterative decoding agorithms may fai to decode in the existance of sma cyces (such as (v 1, c 3, v 4, c 4 ) in Figure 4) [38]. The ength of a smaest cyce is known as the girth of the graph [39]. In this work, we wi focus on designing LDPC codes whose TGs do not contain sma cyces. In particuar, we aim to construct a TG with girth no smaer than a given target girth vaue. 3 Soution Methods In this section, we introduce our integer programming formuations and propose a branch and cut agorithm for the soution of the probem. We investigate additiona methods to improve the performance of our branch and cut agorithm. We summarize the terminoogy used in this paper in Tabe 1. Tabe 1: List of symbos Parameters k ength of the origina information n ength of the encoded information, number of coumns in H m n k, number of rows in H G generator matrix H parity check matrix p error probabiity in BSC T target girth v j variabe node j c i check node i dv j target degree of v j dc i target degree of c i ρ(i, j) cyce region of (i, j) Decision Variabes X ij (i, j) entry of the H matrix dvj s sack for degree of v j dc s i sack for degree of c i 3.1 Mathematica Formuations In our Girth Feasibiity Mode (GFM), our aim is to generate an H matrix of dimensions (m, n), where m = n k, with girth no smaer than a given vaue T. In the GFM mode given beow, X ij variabe represents the (i, j) entry of the H matrix, dv j is the degree of variabe node j, and dc i is the degree of check node i. Constraints (2) and (3) aow generation of an irreguar code with the given degree vaues. As a specia case, one can obtain a (J, K) reguar H matrix by picking dv j = J for a j and dc i = K for a i. We introduce cyce breaking constraints (4) for the cyces with ength ess than the target girth T. In GFM, the objective is a constant, since the target girth T is a given vaue. Hence, any feasibe soution 7

8 of the mode wi be optima. Girth Feasibiity Mode (GFM): max T (1) m s.t.: X ij = dv j, j = 1,..., n (2) i=1 n X ij = dc i, i = 1,..., m (3) j=1 (i,j) C X ij C 1, C cyce with C < T (4) X ij {0, 1}, i = 1,..., m, j = 1,..., n. (5) An aternative modeing approach is to assume dv j and dc i as the target degrees of v j and c i, respectivey. In Minimum Degree Deviation Mode (MDD), the objective is to minimize the degree deviations dvj s of v j and dc s i of c i from the target vaues. Minimum Degree Deviation Mode (MDD): n m min dvj s + dc s i (6) s.t.: j=1 i=1 m X ij + dvj s = dv j, j = 1,..., n (7) i=1 n X ij + dc s i = dc i, i = 1,..., m (8) j=1 (4) (5) (9) dvj s, dc s i 0, i = 1,..., m, j = 1,..., n. (10) One can observe that MDD is aways feasibe, since X ij = 0 for a (i, j), dvj s = dv j for a j, and dc s i = dc i for a i is a trivia soution. Moreover, if the optimum objective function vaue of MDD is zero, which means constraints (7) and (8) are satisfied without deviation, we get a feasibe (optimum) soution of GFM. As we expain in Proposition 3, GFM can be infeasibe depending on the vaue of the target girth T. Hence, in our study, we work with the MDD mode. Since there can be an exponentia number of cyces in a TG, we can have exponentia number of constraints (4) in the corresponding MDD mode. In order to obtain a soution in an acceptabe amount of time, we add the constraints (4) in a cutting pane fashion to MDD. This gives rise to our branch and cut agorithm expained in the next section. 8

9 3.2 Branch and Cut Agorithm The main steps of our Branch and Cut (BC) agorithm are isted in Agorithm 1. In the BC agorithm, we are given a target girth vaue T and the dimensions of H matrix as (m, n). We initiaize our agorithm by reaxing constraints (4) from MDD, to obtain reaxed mode MDD r. Steps (I.1) (I.3) are our improvement techniques (see Section 3.3) to the BC agorithm. Agorithm 1: (Branch and Cut) Input: Target girth vaue T, (m, n) 0. Obtain MDD r by removing constraints v (4) from MDD, set x = nu and z 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 =. v 9 v 10 (I.1) Appy Agorithm 4 to fix some X ij variabes, update x and z. (I.2) Add vaid inequaities given in Proposition 5 to MDD r. (I.3) Appy Agorithm 6 to generate a feasibe soution, update x and z. add MDD r to ist L. 1. Whie ist L is not empty 2. Seect and remove a probem from L. 3. Sove LP reaxation of the probem. 4. If the soution is infeasibe, Then prune the branch and go to Step Ese et the current soution be x with objective vaue z. c 1 c 2 c 3 c 4 c 5 6. End If 7. If z z, Then prune the branch and go to Step 1. v 8. If x is an integer soution, 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 v 9 v If Agorithm 2 finds cyces smaer than T, Then add cuts (4) and go to Step 3. Ese set z z, x x. End If 9. Ese If Agorithm 3 generates any cuts, Then add cuts (4) and go to Step Ese branch to partition the probem into subprobems. Add these probems to L and go to Step End If 12. End Whie Output: H matrix with girth at eastc 1 T c 2 c 3 c 4 c 5 c 6 v v We can find either an integra or a fractiona soution after soving the reaxed MDD. In the case we d find an integra soution, we test its feasibiity witespect to the reaxed constraints (4) with Agorithm 2. The integra soution is separated from the soution space by adding required constraints from (4) if the soution is not feasibe. Simiary, we try to separate a fractiona soution from the soution space 1 2 with Agorithm 3, in order to strengthen the inear reaxation of MDD. v 1 v 2 v 3 v 4 v c c 1 c 2 c 3 Figure 5: An exampe TG for Agorithm 2 9

10 In the integra soution separation probem, we find a cyces in the TG whose ength is ess than T with a depth first search agorithm running in O( V + E ) time using Agorithm 2. In Figure 6, we iustrate Agorithm 2 with T = 6 on the TG given in Figure 5. In Figure 6a, the search agorithm starts with v 1 at eve 0, i.e., = 0, and it is abeed. We abe c 1 at = 1, v 2 at = 2 and c 2 at = 3, since they are the first untracked neighbors of their predecessors. At = 4, we visit v 1 but it has been previousy abeed. This means that we have a cyce of ength 4 consisting of nodes stored in nodet rack array and we add this cyce to C set, which keeps a cyces whose ength is ess than T in the current integra H matrix. Agorithm 2: (Integra Soution Separation) Input: A soution of MDD r with integra X ij vaues, T target girth 1. Let set of cyces C = and nodet rack be an array 2. For Each variabe node j, et = 1 3. Whie > 0, Do set nodet rack[0] = j and abe node j 4. For Each eve from 1 to T 2 5. Set nodet rack[] to first untracked neighbor of nodet rack[ 1] 6. If nodet rack[] is abeed, Then a cyce of ength is added to C unabe nodet rack[] and go to next untracked neighbor of nodet rack[ 1] If no such neighbor, Then 1 7. Ese abe nodet rack[] and + 1, End If 8. For Each 9. End Whie 10. End For Each Output: Set of cyces C In Figure 6b, we consider other untracked neighbors of c 2 at eve 4. After observing that none of v 3, v 4 and v 5 form a cyce, we unabe them and return to eve 3. At = 3, we see that there are no other untracked neighbors of c 2 and backtrack to eve 2. In Figure 6c, we see v 3 is untracked and we abe it at = 2. We abe c 2 at = 3 and v 1 at = 4. This means we found another cyce of ength 4 and add this to set C. The time to find an optima soution of MDD can be improved by reducing the feasibe region using cuts for fractiona soutions. In such a case, we have fractiona X ij vaues in the TG. We consider finding a maximum average cost cyce in the TG with X ij as cost vaues. If this cyce vioates constraints (4) and its ength is ess than T, then we can add the corresponding vioated constraint. Minimum mean cost cyce is a we known network probem in the iterature and there is a poynomia time soution agorithm for the directed graphs [40]. The probem simpy aims to find a directed cyce C with the smaest mean cost (i,j) C X ij/ C in a graph. However, we cannot impement this agorithm directy, since a TG is undirected. For the soution, we can update best known mean cost by impementing a negative cyce detection agorithm repeatedy. Beman Ford agorithm can detect 10

11 (a) (b) = 0 1 = 0 1 = 0 1 = 1 1 = 1 1 = 1 1 = 2 2 = 2 2 = = 3 2 = 3 2 = = 4 c 1 = 4 c = 4 c 1 c 1 nodetrack = (v1, c1, v2, c2, v1) (a) Move to = 3 and then to = 2 (b) nodetrack = (v1, c1, v3, c2, v1) (c) Figure 6: Depth first search in integra soution separation negative cyces whie searching 1 to many shortest paths for directed graphs. Beman Ford agorithm is aso appicabe for undirected graphs in O( V E ) time, if for an edge (i, j) the agorithm updates distance abe of node j when it is not the predecessor of node i [40]. If the agorithm detects a negative cyce, we can track the predeccessor ist to form the cyce. In the fractiona soution separation probem, we use the undirected Beman Ford agorithm to detect negative cyces within a mean cost update method. We first set edge costs as X ij to turn our maximization probem to minimization. Let µ represent an estimation on the minimum mean cost, and µ denote the (unknown) optima vaue of µ. Then, given a µ vaue, we update the edge costs to ( X ij µ) and check for the existance of a negative cyce. If we start with a µ that is an upper bound for µ, we can face with one of these cases for the minimum mean cost µ. Case 1: G has a negative cyce C. In this case, (i,j) C ( X ij µ) < 0. This means, µ > (i,j) C X ij C > µ. (11) Hence, µ is a strict upper bound on µ (i,j) C. We can update µ as µ = Xij C Case 2: G has a zero cost cyce C. In this case, (i,j) C ( X ij µ) = 0. This means, µ = (i,j) C X ij C Hence, µ = µ and C is a minimum mean cost cyce. in the next iteration. = µ. (12) 11

12 Agorithm 3: (Fractiona Soution Separation) Input: A soution of MDD r with fractiona X ij vaues, T target girth 1. Let µ = 0, set cost of edge (i, j) as ( X ij µ) 2. Whie we can detect negative cyce C with undirected Beman Ford 3. If C < T and C is vioating (4), Then add corresponding cut (4) 4. Update µ 5. End Whie (i,j) C X ij C Output: Cuts added to MDD r mode Fractiona soution separation agorithm is summarized in Agorithm 3. We set initia µ = 0, since it is an upper bound on µ. If we can find a negative cyce with ength C < T, we can add a cut to MDD if it is vioated. This means that C is a cyce with (i,j) C X ij > C 1. We continue updating µ vaues unti we find a minimum mean cyce. 3.3 Improvements to the Branch and Cut Agorithm In this section we propose some improvements to the BC agorithm given in the previous section. We first observe that the soution space of MDD incudes symmetric soutions. Hence, we consider a variabe fixing approach to decrease the adverse effect of symmetry. Secondy, we introduce some vaid inequaities to improve the inear reaxation of MDD. Finay, we adapt an agorithm from the teecommunications iterature, i.e., PEG, to provide an initia soution to the BC agorithm Symmetry in the MDD Soution Space In combinatoria optimization probems such as scheduing, symmetry among the soutions is an important issue, which directy affects the performance of appied soution methods [41, 42]. We observe that the feasibe region of MDD contains symmetric soutions. That is, there can be isomorphic representations of a TG by permuting the variabe and check nodes. As an exampe, the variabe nodes are in the order of {v 1, v 2, v 3, v 4 } in Figure 7a and the names of v 2 and v 4 are swapped in Figure 7b (a) (b) Figure 7: Symmetry in MDD soution space 12 H¹ =

13 In Figure 8, H 1 and H 2 are the parity check matrices for TGs in Figures 7a and 7b, respectivey. We see that athough TGs are isomorphic, their H matrix representations are not identica. In the MDD soution space H 1 and H 2 are considered as two different soutions, which increases the compexity of the soution agorithm. H 1 = H 2 = Figure 8: Parity check matrices for the TGs in Figure 7 We can cacuate the number of symmetric soutions for a TG as (n!)(m!), since we can permute n variabe nodes as (n!) and m check nodes as (m!) different ways Symmetry Breaking with Variabe Fixing In the iterature, ordering the decision variabes, adding symmetry breaking cuts to the formuation and reformuating the probem are some of the techniques to eiminate symmetric soutions from the feasibe region [42, 43]. In our case, we propose a fixing scheme for nonzero X ij entries of H matrix that breaks symmetry and does not form any cyces in TG. In our variabe fixing method (given as Agorithm 4) we consider (J, K) reguar H matrices and two modes, i.e., basic and extended. In the basic mode, we fix first K entries in the first row to 1 and first J entries in the first coumn to 1. The remaining entries in the first row and coumn are set to 0, since constraints (2) for j = 1 and constraints (3) for i = 1 are satisfied. We iustrate the basic and extended modes in Figure 9 for a (3, 6) reguar code of dimensions (30, 60) beow. Bod entries in Figure 9 are fixed with the basic mode. Agorithm 4: (Variabe Fixing) Input: (m, n) dimensions, (J, K) vaues, mode type 0. Let r cr = (n 1)/(K 1) and c cr = (m 1)/(J 1) Set X 1j = 0, j = 1,..., n, X i1 = 0, i = 1,..., m If mode = extended For i = 2,..., r cr, j = 1,..., n, set X ij = 0 For i = r cr + 1,..., m, j = 2,..., c cr, set X ij = 0 End If 1. Set X 1j = 1, j = 1,..., K and X i1 = 1, i = 1,..., J 2. If mode = extended 3. For i = 2,..., r cr + 1, j = 1,..., K 1, 4. If 1 + (i 1)(K 1) + j n, Then set X i,1+(i 1)(K 1)+j = End For 6. For i = 1,..., J 1, j = 2,..., c cr + 1, 7. If 1 + j(j 1) + i m, Then set X 1+j(J 1)+i,j = End For 9. End If Output: Some X ij vaues are fixed 13

14 20 In the extended mode, we extend variabe fixing further as dimensions (m, n) of the H matrix aow. In Figure 9, the abes on the rows and coums show the sum of the vaues in that row and coumn, respectivey. We observe that for r cr = (n 1)/(K 1) many rows the sum is equa to 6 and c cr = (m 1)/(J 1) many coumns the sum is equa to 3. Hence, for c cr coumns constraints (2) and for r cr rows constraints (3) are satisfied. We remain with a reduced rectange of size (m r cr ) (n c cr ), constraints which incudes (2) the andunfixed for r X ij variabes shown as dots. Agorithm 4 runs O(nc cr ) time. cr rows constraints (3) are satisfied. We remain with a rectange of In practica appications, for a (J, K) reguar code J < K < n reationship is vaid. In Proposition size (m r cr ) (n c cr ) which incudes the unfixed entries shown as dots. 1, we use this reationship to compare r cr and c cr. c cr r cr Figure 12: 9: Variabe fixing on a (3, (3, 6) reguar 6) reguarh Hmatrix of of dimensions (30, (30, 60) 60) Some characteristics of cyces in a Tanner graph can be visuaized by considering the Tanner graph given in Figure 9a and corresponding parity check matrix H 1 in Figure 10. We can 14 see that C 1 = (v 1, c 1, v 3, c 2 ) and C 2 = (c 1, v 1, c 2, v 2, c 3, v 3 ) are two cyces in Tanner graph in Figure 9a. We can represent cyces C 1 and C 2 on H 1 as in Figures 13a and 13b, respectivey.

15 Proposition 1. Let J < K < n. For a (J, K) reguar code of dimensions (m, n), r cr c cr where r cr = (n 1)/(K 1) and c cr = (m 1)/(J 1). Proof. Let J K = a (0, 1), then mk = nj = m = na. We can write, m 1 J 1 = na 1 Ka 1 = a(n 1)+a 1 a(k 1)+a 1 > n 1 K 1, since a < 1. From here we obtain n 1 K 1 m 1 J 1 = r cr c cr. c cr In Proposition 2, we show that any (J, K) reguar H matrix of dimensions (m, n) that has sufficienty n - c cr arge girth T can be expressed as in Figure 10 by reordering its rows and coumns. m - rcr rcr Proposition 2. Let H be a (J, K) reguar code of dimensions (m, n). Let R be the reduced rectange of size (m r cr ) (n c cr ) and R S be the region between the two extending 1 bocks as in Figure 10. Let ρ(i, j) be the ength of a smaest cyce that is formed when X ij = 1, and τ = max (i,j) S {ρ(i, j)}.then, nonzero entries of H can be represented as two extending 1 bocks as in Figure 10 by reordering its rows and coumns if it has a girth T > τ. Remaining nonzero entries are in the reduced rectange R. c cr n - c cr c cr n - c cr m - rcr rcr R m - rcr rcr S R Figure 10: Reordered (J, K) reguar H matrix with girth T > τ US US USS SS SS Proof. Let H be (J, K) reguar matrix of dimensions (m, n) with girth T > τ. Let us appy the foowing reordering agorithm with time compexity O(c cr ) on the H Agorithm 5: (Reordering) Input: H, (m, n) dimensions, (J, K) vaues, T vaue = = Pick 0 = 1 + row , reorder coumns 0 = 0 + such that 0 = 0 a + 0 ones are + 1 in 0 first = 1 + K0 + coumns. 1 Pick coumn 1, reorder rows such that a ones are in first J rows. (a) (b) 2. For s {2,..., r cr} 3. Pick row s, reorder coumns such that (K 1) ones are in first avaiabe coumns. Pick coumn s, reorder rows such that (J 1) ones are in first avaiabe rows. 4. End For 5. For s {r cr + 1,..., c cr} 6. Pick coumn s, reorder rows such that (J 1) ones are in first avaiabe rows. 7. End For Output: Reordered H matrix 15

16 At step 1 of Agorithm 5, J many ones are ocated in the first coumn. For the second row, i.e., s = 2, first avaiabe (K 1) coumns to ocate ones are the coumns (K + 1,..., 2K 1), since otherwise a cyce with ength ess than T exists. Simiary for the second coumn, i.e., s = 2, first avaiabe (J 1) rows are the rows (J + 1,..., 2J 1) without creating a cyce. The agorithm continues in this fashion for r cr rows and coumns. Since we see in Proposition 1 that r cr c cr, we continue to ocate ones for the remaining (c cr r cr ) many coumns. Using Proposition 2, we can give a ower bound on the dimension n of a (J, K) reguar code with girth at east T as in Proposition 3. Proposition 3. Consider a (J, K) reguar H matrix having girth at east T. Let ρ(i, j) be the ength of a smaest cyce that is formed when X ij = 1. The foowing statements are vaid on dimensions (m, n): (1) n = 2m if K = 2J, (2) Consider Figure 10 and et (i, j) R S. Let r cr be the row such that i r cr we have ρ(i, j) < T and j with ρ(r cr + 1, j) T. Then n (K 1)(r cr + 1). (13) Proof. For a (J, 2J) reguar H matrix, each variabe node has J neighbors and each check node has 2J neighbors in the TG. Since tota variabe degree shoud be equa to tota check degree in a bipartite graph, we have nj = m(2j) = n = 2m. Since H is a (J, K) reguar matrix with girth at east T, we can reorder its rows and coumns as in Figure 10. Let (i, j) R S. According to Proposition 2, the maximum dimension n that this reordering is possibe is such that r cr = n 1 K 1 and i r cr we have ρ(i, j) < T and j n with ρ(r cr + 1, j) T. From r cr = n 1 K 1 we can write r cr + K 2 K 1 n (K 1)(r cr + 1). n 1 K 1 to maximize n. This gives We can cacuate ρ(i, j) of an entry (i, j) by carrying out a breadth first search starting from the variabe node v j. The smaest depth which we revisit v j is ρ(i, j). From Proposition 3, we can provide ower bound on n for a (3,6) reguar code as r cr = 3, n 20 for T = 6, r cr = 13, n 70 for T = 8, r cr = 33, n 170 for T = 10 (see Figure 14 for ρ(i, j) vaues). Some characteristics of the cyces in a TG can be visuaized by considering the TG given in Figure 7a and the corresponding parity check matrix H 1 in Figure 8. It can be seen that C 1 = (v 1, c 1, v 3, c 2 ) and C 2 = (c 1, v 1, c 2, v 2, c 3, v 3 ) are two cyces in the TG in Figure 7a. Figures 11a and 11b visuaize 16

17 c h c c 3 (a) (b) cyces C 1 and C 2 on H 1, respectivey. v 1 v 2 v 3 v 4 v 1 v 2 v 3 v 4 c c 1 h c 2 v u h c 2 v u c c (a) (b) Figure 11: Cyces C 1 and C 2 on H 1 We observe that a cyce is an aternating sequence of horizonta and vertica movements between ces having vaue 1. In particuar, cyce C 1 is a sequence of horizonta right ( ), vertica down ( ), horizonta eft (h ) and vertica up (v u ) movements. Simiary, cyce C 2 can be expressed with the sequence (,,,, v u, h ). Moreover, we deduce that a cyce shoud incude at east one from each of the h u, h d, v u and movements. Proposition 4. Variabe fixing on H matrix with the extended mode does not form any cyces in the TG. Proof. Assume we appy variabe fixing with the extended mode and consider ces whose X ij vaues have been fixed to 1. There are four cases to have an aternating sequence among variabe and check nodes as given in Figures 12 and 13. v u h (a) (b) h Figure 12: Aternating variabe and v u check nodes, cases 1 and 2 In Figure 12a, the sequence of case 1 is (v,,,,...) and d in Figure 12b for case 2, we have the sequence (,,,,...). Both of the sequences do not incude v u and h movements. Hence, there 17 (a) (b)

18 cannot be any cyces in these cases. (a) (b) h v u (a) (b) Figure 13: Aternating variabe and check nodes, cases 3 and 4 In Figure 13a (case 3), we have two options to start, i.e., or h movements. Then the sequence wi be ( or h,,,,,...), which does not incude v u movement. In Figure 13b (case 4), or v u are candidates to begin the sequence. In this case, the sequence wi be ( or v u,,,,,...), which does not incude h movement. Hence, there are no cyces in these cases either. We can use the partia soution obtained with Agorithm 4 to generate a feasibe soution of MDD. Since partia soution does not incude any cyces (see Proposition 4), setting the nonfixed entries to zero gives a feasibe soution (an upper bound). Step (I.1) of Agorithm 1 impements variabe fixing with the basic or extended mode and updates the initia upper bound Vaid Inequaities for Cyce Regions After appying extended fixing, MDD probem reduces to ocating ones in the reduced rectange R of ( ) size (m r cr ) (n c cr ). That is probem size reduced by 1 (m rcr) (n ccr) m n 100%. We can further improve the performance of BC agorithm by introducing vaid inequaities. We add the generated vaid inequaities to MDD r at step (I.2) of Agorithm 1. We observe that for given dimensions (m, n), the reduced rectange R appears between the two extending 1 bocks as given in Figure 10. For a (J, K) reguar code, we divide the region R S into subbocks with (J 1)(K 1) rows and (K 1) coumns as given in Figure 14. For each entry (i, j) in a subbock, we investigate the ength of a smaest cyce ρ(i, j) (see Proposition 2) when there is a singe 1 at entry (i, j). For exampe, in Figure 14, we observe that ρ(i, j) is common for a (i, j) entries in a subbock except the subbocks at the boundaries of the extending 1 bocks. Hence, we can define 18

19 Cyce 4, Cyce 6, Cyce 8, and Cyce 10 regions, which have repeating pattern due to (J, K) reguarity. Figure 14: Subbocks and cyce regions with J = 3 and K = 6 In particuar, when there is a 1 in a Cyce 4 region (dotted area), we have a cyce of ength 4 as in the case of cyces C 1 and C 2 in Figure 15. We note that, Cyce 4 regions repeat both horizontay and verticay. Figure 15: Cyce 4 regions with J = 3 and K = 6 Simiar horizonta and vertica repeating patterns can be seen for Cyce 6 and Cyce 8 regions in 19

20 Figure 16: Cyce 4, Cyce 6, and Cyce 8 regions with J = 3 and K = 6 Figure 16. Making use of these patterns, one can express ρ(i, j) of an entry (i, j) as a function. We introduce vaid inequaities for MDD based on the cyce region information of the entries in the reduced rectange R. Proposition 5. Let (i, j) R, i.e., i {m r cr,..., m} and j {n c cr,..., n} and et ρ(i, j) represent the cyce region of the entry. Let S denote the number of subbocks that intersects with R and et B s, s {1,..., S} represent the set of (i, j) entries in subbock s. (1) If ρ(i, j) < T, then constraint X ij = 0 (14) is vaid. (2) If T = 8 and (i, j) B s with ρ(i, j) = 8 or 10, then constraints J 1 i=1 ((k 1)(J 1)+i,j) B s X (k 1)(J 1)+i,j 1, k {1,..., K 1} (15) are vaid. (3) If T = 10 and (i, j) B s with ρ(i, j) = 10, then constraint 20

21 (i,j) B s X ij 1 (16) is vaid. Proof. Let us consider each caim separatey. (1) There cannot be cyces of ength smaer than the girth T. If X ij = 1, then we have a cyce of ength ρ(i, j) < T, which is not desired. Hence, X ij = 0 in this case. (2) If T = 8, then there shoud not be any cyces of ength 6. Let us consider a subbock with cyce region 8 or 10, which is subdivided into (K 1) equa subpieces each incudes (J 1) rows. In Figure 17, we give an exampe for Cyce 8 subbock with J = 3 and K = 6 where we have (K 1) = 5 subpieces each having (J 1) = 2 rows. As seen in figure, a cyce of ength 6 forms when there is more than one nonzero entry in a subpiece. Figure 17: A cyce of ength 6 on Cyce 8 region with J = 3 and K = 6 A simiar case appears for Cyce 10 subbocks. Hence, constraints (15) are vaid, since they force to have at most one nonzero entry in each subpiece when cyce region of the subbock is either 8 or 10. (3) A cyce of ength 8 is not aowed when T = 10. However, when there is more than one nonzero entry in a subbock with cyce region 10, there is a cyce of ength 8 as given in Figure 18. Con- 21

22 Figure 18: A cyce of ength 8 on Cyce 10 region with J = 3 and K = 6 straint (16) is vaid, since it bounds the number of nonzero entries from above with 1. Proposition 6. Let z be the optimum objective vaue of MDD and z f be the optimum objective vaue of MDD when variabes are fixed with the extended mode. Let τ be defined as in Proposition 2. Assume there exists a (J, K) reguar code with dimensions (m, n), then (1) 0 = z = z f (2) 0 = z z f if T > τ, if T τ. Proof. For any dimensions (m, n), we have z z f, since we fix some X ij variabes in the extended mode. If there exists a (J, K) reguar code, then there is an optima soution with objective vaue z = 0. We know from Proposition 2 when T > τ, a (J, K) reguar code can be expressed as in Figure 10, which coincides with the case in the extended mode. Hence, we have zf = z = 0. In MDD if ρ(i, j) T, then X ij can be nonzero without harming the girth T. When T τ, there are (i, j) S in Figure 10 with ρ(i, j) T and they are fixed to zero, since we fix a entries in the region S to zero in the extended mode. Then, we have 0 = z z f in this case. 22

23 3.3.4 Modified Progressive Edge Growth Agorithm The ast improvement to our BC agorithm is to introduce a starting soution for an initia upper bound. For this purpose, we adapt an existing agorithm from the iterature known as Progressive Edge Growth (PEG) agorithm [44]. We modify this agorithm for our probem by starting PEG from a partia initia soution generated by our fixing agorithm given in Agorithm 4. We aso update PEG such that the generated soution has girth at east T. Time compexity of Agorithm 6 is the same with the origina PEG, which is O( V E + E 2 ). In Agorithm 1, we set an upper bound by appying Agorithm 6 at step (I.3). Agorithm 6: (Modified PEG) Input: (m, n) dimensions, dv and dc vectors, T vaue 0. Initiaize X 0, dv c 0, dv s dv and dc s dc, I 0 1. Appy Agorithm 4 and update sacks dv s j dvs j i X ij for a j and dc s i dcs i j X ij for a i and current degrees dv c j i X ij for a j 2. For j {1,..., n} set I 0 3. For k {0,..., dv c j } 4. If k = 0, Then set X i j = 1 for i = argmax i {dc s i } 5. Ese appy BFS from v j to reach check nodes, et tree has depth 6. If 2 T or N j m, et I is incidence vector for N j set X i j = 1 for i = argmax i {(1 I ci )dc s i } 7. End If 8. Update dvj c, dvs j, dcs i as in Step 1 9. End For 10. End For Output: An initia soution for MDD In Agorithm 6, dv and dc are the target degree vectors for variabe and check nodes, respectivey. Let deviation from the target degrees for variabe and check nodes be given by sack vectors dv s and dc s, and the current degrees of variabe nodes be isted in vector dv c. Moreover, Nj represents the set of a check nodes that can be reached from v j with a tree of depth. Hence, the set N j \ N 1 j coects the check nodes that are reached at the th step from v j for the first time. We can represent the check nodes in the set Nj with an incidence vector I as I c i = 1 if c i Nj and zero otherwise. Starting from the soution provided by Agorithm 4, PEG adds an edge (i, j), i.e., X ij = 1, if this edge does not form a cyce ( Nj m) or the ength of the cyce created is greater or equa to T (Step 6). For edge assignment, the agorithm picks c i having the maximum sack vaue dc s i in order to fit the target degree dc i. The generated soution is feasibe for MDD, since it has girth at east T. 23

24 4 Computationa Resuts The computations have been carried out on a computer with 2.0 GHz Inte Xeon E processor and 46 GB of RAM working under Windows Server 2012 R2 operating system. In the computationa experiments, we use CPLEX to test the performance of BC agorithm and evauate how different improvement strategies to BC agorithm given in Section 3.3 affect the resuts. We impement a agorithms in the C++ programming anguage. We summarize the soution methods in Tabe 2. Tabe 2: Summary of soution methods Method Mode Vaid Inequaities Modified PEG BC 0 BC 1 basic BC 2 extended BC 3 extended BC 4 extended In BC 0, we appy the BC agorithm in Agorithm 1 without improvement techniques, i.e., we excude steps (I.1) (I.3). Agorithm 1 incudes Agorithm 2 and 3 to separate integra and fractiona soutions, respectivey. In CPLEX, we impement Agorithm 2 using LazyConstraintCaback and Agorithm 3 with UserConstraintCaback routines. We utiize defaut branching settings of CPLEX. In BC 1 method, we appy step (I.1) to fix the first row and coumn of H matrix in the basic mode. In BC 2 method, step (I.1) fixes r cr rows and c cr coumns in the extended mode (see Section 3.3.2). In BC 3 method, we appy step (I.1) in the extended mode and step (I.2) adds vaid inequaities that are expained in Section Finay in BC 4 method, step (I.1) runs in the extended mode, step (I.2) adds vaid inequaities and step (I.3) provides an initia soution with modified PEG (Agorithm 6). We ist the parameters used in the computationa experiments in Tabe 3. We generate (3, 6) reguar H matrices with girth vaues T = 6, 8 or 10 in our experiments. We try nine different (m, n) dimensions from n = 20 to We report the resuts that CPLEX found in 3600 seconds time imit. Tabe 3: List of computationa parameters Parameters (J, K) (3, 6) reguar codes (m, n) (10, 20), (15, 30), (20, 40), (30, 60), (40, 80), (100, 200), (150, 300), (250, 500), (500, 1000) T 6, 8, 10 Time Limit 3600 secs From Tabe 4 to 6, coumn z is the objective function vaue of MDD and coumn z is the best known ower bound found by CPLEX within the time imit. For each of the methods, we have an initia 24

25 feasibe soution (an upper bound) with objective vaue zu. i In BC 0 method, H = 0 is a trivia soution providing an initia upper bound. In methods from BC 1 to BC 4 an initia feasibe soution is obtained from variabe fixing (see Section 3.3.2) or modified PEG heuristic (see Section 3.3.4). Computationa time in seconds is given with coumn CPU (secs) and percentage difference among z and z is under coumn Gap (%). In coumn Lazy we show number of cuts added to MDD using Agorithm 2, whereas coumn User is the number of cuts added to MDD with Agorithm 3. As discussed in Section 3.1, we have a (J, K) reguar code if z = z = 0. We can concude that it is not possibe to have a (J, K) reguar code with given (m, n) and the girth T when we have z z > 0 (see Proposition 3). In Tabe 4, we can see that BC 0 can find a (3, 6) reguar code for 8 instances when T = 6. As T and n increase, BC 0 method cannot improve initia upper bound zu. i For T = 8 and T = 10, we observe that the number of azy and user cuts added to MDD gets smaer as n gets arger. This is because adding a cut takes more time as n increases, which causes the agorithm to generate fewer cuts within the given time imit. Tabe 4: Computationa resuts for BC 0 CPU Gap # Cuts T n z z zu i (secs) (%) Lazy User time time time time time time time time time time time time time time time time time time

26 Tabe 5 shows our computationa resuts for BC 1 and BC 2. We have better initia upper bound (z i u) vaues compared to BC 0 when we impement variabe fixing with the basic mode in BC 1. We improve zu i vaues more in BC 2 with the extended mode, since we fix more entries compared to the basic mode. We observe that z = 1 for T = 6 and n = 20 in BC 1, which means it is not possibe to have a (3, 6) reguar code for this dimension. BC 1 method is abe to sove 9 instances out of 27 instances to optimaity, i.e., Gap (%) vaue is zero. Tabe 5: Computationa resuts for BC 1 and BC 2 BC 1 BC 2 CPU Gap # Cuts CPU Gap # Cuts T n z z z i u (secs) (%) Lazy User z z z i u (secs) (%) Lazy User time time time time time time time time time time time time time time time time time time time time time time time time time time time time In Tabe 5, we observe that we can sove 17 instances to optimaity with BC 2 method. BC 2 finds z > 0 for 11 instances indicating that there are no (3, 6) reguar codes for those dimensions. There are 7 instances such as T = 10 and n = 80 that we have z = z > 0. This means that for n = 80 dimension, the best possibe code with the girth T = 10 incudes z/2 = 236/2 = 118 fewer ones than a (3, 6) reguar code (having X ij = 1 improves MDD objective by 2). Comparing Tabe 5 and 6, we can see that zu i vaues for BC 2 and BC 3 are the same, since we appy the extended mode for both. On the other hand, feasibe soution of Agorithm 6 (see Section 3.3.4) provides better zu i vaues in BC 4. Resuts show that z vaues get better, the number of cuts added to MDD gets smaer and computationa time improves on the average as we have tighter initia soutions. 26