Multi-Edge Low-Density Parity-Check Coded Modulation. Lei Zhang

Transcription

1 Multi-Edge Low-Density Parity-Check Coded Modulation by Lei Zhang A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Edward Rogers Sr. Department of Electrical and Computer Engineering University of Toronto Copyright c 2011 by Lei Zhang

2 Abstract Multi-Edge Low-Density Parity-Check Coded Modulation Lei Zhang Master of Applied Science Graduate Department of Edward Rogers Sr. Department of Electrical and Computer Engineering University of Toronto 2011 A method for designing low-density parity-check (LDPC) codes for bandwidth-efficient high-order coded modulation is proposed. Code structure utilizes the multi-edge-type LDPC code ensemble to achieve an improved match between codeword bit protection capabilities and modulation bit-channel capacities over existing LDPC coded modulation techniques. The multi-dimensional EXIT vector field for the specific multi-edge parameterization is developed for the analysis and design of code ensembles. A multi-dimensional EXIT decoding convergence condition is derived to enable efficient optimization. Code design results in terms of ensemble thresholds and finite-length Monte-Carlo simulations indicate that the new technique improves on the state-of-the-art performance, with significantly lower design and implementation complexity. ii

3 Acknowledgements This thesis would not have been written but for the great number of people from whom I have received sage advice, unparalleled friendship and unconditional love. At the forefront of this amazing group is my advisor, professor Frank Kschischang, whose overarching perspective, originality of ideas, seemingly boundless knowledge and meticulous attention to detail have ensured a smooth and thoroughly enjoyable research experience for me. In the last few years I ve experienced tremendous academic and professional growth under his guidance, inspired by his dedication and excellence in all aspects of an academic career, from research, teaching, to professional commitments. It has been an honour and a pleasure to work with professor Kschischang. A special thank you to Dr. Benjamin Smith, whose wealth of knowledge, insights and advice have greatly facilitated my research. I ve also enjoyed the many stimulating conversations we ve had regarding careers and the minutiae of academia. It is one of the many aspects which allowed him to become a valuable role model to an incipient researcher such as myself. To all my friends, I honestly believe that without your encouragement, company, and coffee breaks, I would not have been able to overcome several particularly trying stages throughout this endeavour. Thank you all. I hope to have the opportunity to repay each and every one of you in kind. Finally, to my parents, I dedicate this thesis to you as a small token of my appreciation for your unconditional love and support. I love you both from the bottom of my heart. iii

4 Contents 1 Introduction Improving BICM-LDPC Literature Review Thesis Outline Technical Background Target System IID channel adapter Non-iterative vs. Iterative demapping Shaping Analysis and Design of Binary LDPC Codes Ensemble-based design and density evolution Extrinsic information transfer charts Multi-edge-type LDPC Codes Multi-edge LDPC Coded Modulation Multi-edge Parameterization Check degree edge-type assignment Multi-edge Optimization Multi-dimensional EXIT vector field Design of multi-edge coded modulation iv

5 4 Results Threshold Discussion High code rate designs Finite-length Rate 3/4 Gray-labelled 16-QAM Rate 1/2 Gray-labelled 16-QAM Fixed BER performance comparison Conclusion 76 Bibliography 78 v

6 List of Tables 3.1 Density evolution thresholds for the (3,6) regular LDPC check degree edgetype split pairings under Gray-labelled 16-QAM Density evolution thresholds for the (3,9) regular LDPC check degree edgetype split pairings under Gray-labelled 16-QAM Density evolution thresholds for irregular LDPC check degree edge-type split pairings under Gray-labelled 16-QAM Bit-channel capacities for Gray-labelled 2 n -QAM at rate (n 1)/n ME-LCM-OPT optimized ensembles for 2 n -QAM rate (n 1)/n codes ME-LCM-OPT optimized ensembles of various code rates for 16-QAM ME-LCM-OPT optimized ensembles of various code rates for 16-QAM (continued) Legend for data points in Fig vi

7 List of Figures 1.1 Decoding block diagrams of BICM and MLC-MSD with 4 bit-levels Tanner graph for parity check matrix in Eqn Gray-labelled 16-QAM constellation with labels corresponding to bits b 0 b 1 b 2 b Histogram of LLRs for Gray-mapped 16-QAM bit-levels b 0, b 1, b 2, b Target system block diagram for the baseband-equivalent discrete-time complex AWGN channel Bit-channel and symbol-channel capacities of 16-QAM for set-partition labelled MLC/MSD and Gray-labelled BICM EXIT chart of optimized rate = 0.33 ensemble at threshold of db E s /N A possible multi-edge-type representation of the Tanner graph in Fig Tanner graph of the 2 edge-type specified MET parameterization, node degrees are illustrative and not meant to be realistic Multi-dimensional EXIT vector field for 2 edge-types at threshold σ = Multi-dimensional EXIT vector field for 2 edge-types at above threshold σ = Backward-difference vector field of 2 edge-types at threshold σ = vii

8 3.5 Backward-difference vector field of 2 edge-types at above threshold σ = ME-LCM-OPT optimized ensembles for 2 n -QAM rate (n 1)/n codes Probability of bit and errors for n = 4096 rate 3/4 code and Gray-labelled 16-QAM Probability of bit and errors for n = 8192 rate 3/4 code and Gray-labelled 16-QAM Probability of bit and errors for n = rate 3/4 code and Gray-labelled 16-QAM Probability of bit and errors for n = rate 1/2 code and Gray-labelled 16-QAM Comparison of E s /N 0 required to achieve BER of 10 5 at different spectral efficiencies for Gray-labelled 16-QAM viii

9 Chapter 1 Introduction In many communication systems, the modulation and channel code are designed separately. Several factors motivate this paradigm. A complicated modulation system can often be encapsulated by a simple channel model, such as the binary symmetric channel (BSC), to greatly simplify the channel code design without incurring a significant loss in performance. For applications with high error tolerance, the uncoded modulation error rate may be sufficiently low to obviate the need for a channel code. Even the pedagogical tradition in undergraduate and graduate digital communication courses dictates separating modulation and coding. The most dominant reason, however, is the complexity of designing the modulation and channel code in conjunction. The trade-off between the cost to design the more complex system and the performance gains from doing so often results in the pragmatic engineering solution of the distinct modulation-coding architecture. For applications where bandwidth is a limiting resource, the gain in combining modulation and channel coding significantly outweighs the increase in system complexity. A well-known example of coded modulation is Trellis Coded Modulation (TCM) [1]. TCM combines the Euclidean distance properties of the modulation signal constellation with the Hamming distance properties of the error-correcting convolutional (trellis) code in its 1

10 Chapter 1. Introduction 2 design process. Design rules map trellis transitions of the convolutional code to subsets of the constellation of different Euclidean distances. In the early 1980 s, telephone line modem designers considered 9600 kbit/s to be the limiting throughput under standard bandwidth and power constraints. With the introduction of TCM in 1984, telephone modems achieved 14.4 kbit/s and higher [2], which helped usher in the meteoric rise of personal dial-up Internet services. After the discovery of capacity-approaching turbo and low-density parity-check (LDPC) codes in the early 90 s [3, 4], coded modulation using these modern codes evolved following two dominant approaches based on modulation bit-level capacities. Bit-Interleaved Coded-Modulation (BICM) [5] uses an interleaver between the channel code encoder and the bit-to-symbol mapper. BICM averages the Euclidean distances of different modulation bit-levels so that the underlying error-correcting code experiences an average noise degradation from the channel. The BICM bit-channel capacities, I(b i ; Y ), can be shown to be an approximation of the expanded symbol channel mutual information using the chain rule I(X; Y ) = I(b 0,..., b n 1 ; Y ) n 1 = I(b i ; Y b 0,..., b i 1 ) (1.1) i=0 n 1 I(b i ; Y ). (1.2) i=0 Even though capacity is lost in the process, it has been shown in [5] that BICM with Gray labelling can closely approximate the finite-constellation constrained channel capacity. A capacity-approaching code can then be designed for each bit-channel capacity. An alternative method of coded modulation is Multi-Level Coding (MLC) [6, 7]. In MLC, for a fixed constellation label, the bit-level capacities are given by (1.1) and achieves the symbol channel capacity. Again, capacity-approaching codes can be designed for each bit-channel. At the receiver, decoding progresses in a level-by-level fashion called Multi-

11 Chapter 1. Introduction 3 Stage Decoding (MSD) [7]. Initially only the decoder for bit-level b 0 is active. Assuming all b 0 bits are correctly decoded, the receiver uses this side-information along with received information to decode b 1, and so on. A block diagram comparison between BICM and MLC-MSD coded modulation is shown in Fig Even though MLC/MSD can achieve channel capacity, it has significant disadvantages compared to BICM. Decoding latency of MSD is a major issue for low latency systems. Error propagation from lower to higher bitlevels may increase error rates. Furthermore, passing lower bit-level soft information to higher bit-levels requires iterating through the symbol demapper which increases receiver complexity. Taking into account the factors of power efficiency, decoding latency, and implementation complexity, BICM outclasses MLC/MSD as the more suitable capacity-approaching coded modulation technique. On-going standardization activity lends supports to this conclusion. LDPC-based BICM (LDPC-BICM) is included in the 2nd-generation satellite television standard (DVB-S2) [8] and the 2nd-generation digital cable television standard (DVB-C2) [9]. LDPC-BICM is defined in the multiple access mode of the wireless metropolitan area networks standard (WiMax) [10] as a high performance option. Protograph-based LDPC-BICM is proposed for deep space communication in the Consultive Committee for Space Data Systems (CCSDS) O-2 standard [11]. The complexity of LDPC-BICM is minimal since bit-interleaving can be built into the LDPC parity-check matrix. However, in each of these standards except CCSDS, the LDPC code used is based on an Irregular Repeat-Accumulate (IRA) code [12] initially designed for a low-rate power-limited applications. There is currently no bandwidth-limited application that uses high-rate LDPC codes specifically designed for coded modulation. It appears that combining an available LDPC design with BICM has become the de facto standard. However, it may be imprudent to accept LDPC-BICM as the optimal LDPC coded modulation technique. In the next section we provide an argument for developing true

12 Chapter 1. Introduction 4 Y b0 decoder b 0 b1 decoder b 1 b2 decoder b 2 b3 decoder b 3 (a) BICM Y b0 decoder b 0 b1 decoder b 1 b2 decoder b 2 b3 decoder b 3 (b) MLC-MSD Figure 1.1: Decoding block diagrams of BICM and MLC-MSD with 4 bit-levels.

13 Chapter 1. Introduction 5 LDPC coded modulation that accounts for modulation bit-level differences in LDPC code design. Significant performance gains maybe achievable with such an improvement to LDPC-BICM. 1.1 Improving BICM-LDPC To introduce the argument for an improved BICM-LDPC design technique, we introduce a few necessary details of LDPC codes. To keep the treatment brief, we relegate additional technical details to Ch. 2. LDPC codes are linear codes with sparse parity-check matrices. Each column of the LDPC parity-check matrix represents a codeword bit. The non-zero entries in a column denote the parity-check equations to which the particular codeword bit belongs. The non-zero entries in a row of the parity-check matrix denote the codeword bits checked by that parity-check equation. The parity-check matrix can be visually represented by a bipartite graph called Tanner graph [13]. An example is shown in Fig. 1.2 for the following parity-check matrix H = (1.3) The circles in Fig. 1.2 are called variable nodes and represent the columns of H. The squares are called check nodes and represent the rows of H. The edges of the Tanner Figure 1.2: Tanner graph for parity check matrix in Eqn. 1.3.

14 Chapter 1. Introduction 6 graph connect codeword bits to the parity-check equations to which they belong. The total number of edges that each variable/check node possesses is called the degree of the variable/check node, corresponding to the number of non-zero entries in a column/row of the parity-check matrix. If all variable nodes have the same degree, the LDPC code is called regular. LDPC codes with variable nodes of different degrees are called irregular codes. All capacity-approaching LDPC codes are irregular [14, 15]. Decoding of LDPC codes uses the sum-product algorithm [16]. At the start, each variable node receives a reliability measure from the demapper and sends it to neighbouring checks. At the check nodes, the reliabilities are updated according to how well they satisfy the parity-check constraints. After a complete iteration, variable nodes re-evaluate their reliabilities according to repetition code constraints. This message passing action continues until the variable node reliabilities are sufficiently high for a hard decision or a maximum number of iterations has been reached. We now outline the argument for seeking to improve the design of LDPC codes for BICM systems. As repetition codes, LDPC variable nodes of different degrees offer different levels of error correction capability. High-degree variable nodes behave as very long repetition codes and therefore are extremely reliable. However their low rates decrease the overall code rate significantly. On the other hand, degree 2 variable nodes offer essentially no error correction but have the highest rate of all repetition codes. In a capacity-approaching irregular LDPC code, there exists an inherent variation among the different codeword bits. Interestingly, high-order modulation also produces differences in the reliabilities of received codeword bits. Distinct bit-levels in the symbol-labelling experience different amounts of noise corruption due to Euclidean distance differences. For the constellation and labelling given in Fig. 1.3, we plot the histogram of bit-channel output log-likelihood ratios (LLR) in Fig The system signal-to-noise ratio is 9.32 db E s /N 0. From the empirical means of the LLR distributions (black markers), we may conclude

15 Chapter 1. Introduction Quadrature In Phase Figure 1.3: Gray-labelled 16-QAM constellation with labels corresponding to bits b 0 b 1 b 2 b Normalized occurance b1,b b0,b Log likelihood ratio Figure 1.4: Histogram of LLRs for Gray-mapped 16-QAM bit-levels b 0, b 1, b 2, b 3.

16 Chapter 1. Introduction 8 that bit-levels b 0 and b 2 are of higher quality than the remaining two bits. Other measures of bit-channel quality such as probability of error and mutual information also point to this conclusion. Therefore, high-order modulation inherently produces different qualities of bit-levels hence varying reliabilities over channel codeword bits. BICM averages over these different bit-channel output reliabilities. For codes which do not have varying levels of protection over codeword bits, this is a good coded modulation technique. However, irregular LDPC codes implement varying levels of protection over the codeword bits. We believe that the performance of high-order LDPC-BICM coded modulation can be improved if the differences in bit-level output reliabilities are integrated into the LDPC code design procedure. Much like the classical TCM scheme, LDPC coded modulation must exploit bit-level reliability differences by optimizing the variable node degree distributions and mapping assignments simultaneously. 1.2 Literature Review While the design of capacity-approaching LDPC codes for binary-input memoryless symmetric channels such as the BSC and the additive white Gaussian channel (BIAWGNC) has been thoroughly studied [14, 17, 18], much less is known about the design of LDPC codes for high-order coded modulation. The few available references on improving LDPC coded modulation focus on two general methodologies: the bit-to-symbol interleaving of a fixed LDPC code [19 21], and code design incorporating differences in bit-level reliabilities [22 26]. The first method is nothing more than finding a particular interleaver for BICM without code design. Only the second method can be truly considered to be the design of LDPC codes for coded modulation. In search of an improved interleaver, the authors of [19] proposed a mapping scheme where less-protected variable nodes were mapped to low-reliability bit-levels while moreprotected variable nodes were mapped to highly reliable bit-levels. The mapping provided

17 Chapter 1. Introduction db of improvement at no complexity increase. Intuitively, the improvement may have been the result of allowing the most-reliable messages to propagate widely from highdegree variable nodes. In [20], a mapping was proposed to minimize the connections of each check node to variable nodes with low-reliability channel output, resulting in db of improvement. Finally, [21] proposed an improved interleaver for the DVB-S2 LDPC code after taking into account the bit-level reliability differences. These mapping-based methods certainly improved LDPC-BICM performance, but since the underlying LDPC code was fixed the improvement was limited. The most significant work on the design of LDPC codes for coded modulation has been [22]. The work applies density evolution [14] to design LDPC codes for the distinct bit-channels of MLC and BICM. The problem is reduced to several binary LDPC code designs. In [23] a powerful class of low-complexity, low error-floor LDPC codes based on protographs are applied to high-order modulation with impressive performance. Together, the references [22, 23] provide the most significant references for our work. In [27, 28] LDPC codes are only used for low to medium quality bit channels, while very high quality bit-channels are either uncoded or protected by very simple classical codes. In [24 26], the multi-edge-type concepts are used in LDPC coded modulation. Multi-edge-type LDPC codes [29, pp ] can incorporate the distinct bit-channel reliabilities into code optimization. They also give the designer flexible control over code structure to trade-off between complexity and performance. Although [24 26] did allude to multi-edge ideas, they fall short of providing a specific multi-edge parameterization with efficient analysis and design techniques. The key contribution of this thesis is the design of multi-edge-type LDPC codes for LDPC-BICM.

18 Chapter 1. Introduction Thesis Outline Ch. 2 provides all the necessary technical background used in the rest of this thesis. The target system model, fundamentals of LDPC design, density evolution and extrinsic information transfer (EXIT) charts are a few of the topics reviewed in the chapter. Ch. 3 develops the multi-edge LDPC code design technique for coded modulation and forms the main body of the thesis. The development follows from the initial specialization of the multi-edge parameterization, to the multi-dimensional EXIT chart method for analyzing such ensembles, to the innovative technique for code design based on the multidimensional EXIT chart. Results of the new LDPC coded modulation design technique are given in Ch. 4 in terms of both the ensemble thresholds and finite-length simulations. Ch. 5 concludes the thesis and provides directions for future work.

19 Chapter 2 Technical Background The goal of this chapter is to provide a comprehensive review of the technical knowledge required for understanding the rest of the thesis. Sec. 2.1 describes the target system. Several alternatives are discussed and justifications are given for choosing to limit the scope of the thesis to one system. Binary LDPC analysis and design techniques are reviewed in Sec A thorough understanding of the details and intuition of these techniques is essential since the solutions developed in this thesis are based on these binary design techniques. Lastly, Sec. 2.3 provides details on multi-edge-type LDPC codes. Throughout this thesis, bold font always denotes a vector quantity of length clear from context. A bold constant denotes a vector of repeated entries, all of which are equal to the indicated value. For example, 1 = (1,..., 1). A bold variable either denotes a vector of variables, for example x = (x 1,..., x n ) or a vector field function f(x) = (f 1 (x),..., f n (x)). The difference between them will be clear from context. 2.1 Target System We now describe the system at which the design techniques given in this thesis are aimed. As illustrated in Fig. 2.1, the source of the target system generates a sequence of 11

20 Chapter 2. Technical Background 12 n m ENC c MAP x y DMAP L DEC ĉ Figure 2.1: Target system block diagram for the baseband-equivalent discrete-time complex AWGN channel. uniformly distributed independent and identically distributed binary random variables. The channel code encoder takes k source bits per input block and maps them to a length n (where n k) codeword c from the channel codebook C. The code rate is r = k/n. A mapping function µ maps each codeword c to a sequence of modulation symbols x, where each symbol x i is selected from the constellation X. In M bits per symbol modulation, there are 2 M distinct points in the constellation. Assuming n is a multiple of M, the sequence x contains n/m symbols. For an arbitrary constellation labelling scheme, let m = (0,..., M 1) index the bit-levels of the labelled constellation and let b m i denote the m-th bit in the label of the transmitted symbol x i. The sequence of symbols x, in discrete-time baseband-equivalent representation of bandpass transmission, is corrupted by complex additive white Gaussian noise n = n I + jn Q of variance σ 2 = N 0 /2 per dimension, where N 0 /2 is the two-sided power spectral density of the Gaussian noise. Note that all simulations in this thesis are performed by assuming unity average symbol energy while scaling noise variance to the desired signal-to-noise ratio. At the receiver, the received sequence y = x + n is demapped by taking the bit-wise log-likelihood ratio (LLR) L Mi+m = log P (y i b m i = 0) P (y i b m i = 1) (2.1) where i = (0,..., n/m 1) indexes the received sequence y. The uncoded maximum-likelihood decision rule is to decide 0 if L Mi+m > 0, decide 1 if L Mi+m < 0 and decide 0 or 1 at random with equal probability if L Mi+m = 0.

21 Chapter 2. Technical Background 13 For complex additive white Gaussian channel with N 0 = σ 2 the LLR is given by L Mi+m = log a X 0 m b X 1 m ) exp ( 1N0 y i a 2 ). (2.2) exp ( 1N0 y i b 2 where X 0 m and X 1 m partition the signalling constellation X into sets of points where b m = 0 and b m = 1, respectively. The length n sequence L is decoded by the channel code decoder to the decoded word ĉ. A bit error occurs if c i ĉ i for some i, a frame error occurs if one or more bit errors occur in the decoded word. After correct decoding, the transmitted message m can be extracted from ĉ if C is systematic. In this thesis, we judge the system error performance only by comparing ĉ to c IID channel adapter As will be explained in Sec. 2.2, it is highly desirable for the bit-wise channels from the transmitted codeword bits c i to the demapped LLR bit reliabilities L i to be output symmetric. By definition, a binary-input channel is output symmetric if P (L i c i = 0) = P ( L i c i = 1). (2.3) High-order modulation systems are in general not output symmetric. A work-around to this difficulty was introduced in [22] by inserting independent and identically distributed (iid) channel adapters into the system. At the transmitter, the iid channel adapter XORs codeword c with a random binary sequence u generated from identically distributed, uniform Bernoulli random variables. At the receiver, the sequence L is multiplied bit-wise by 1 2u. It is easy to verify that these two operations produce bit-channel output symmetry while maintaining the same bit-channel capacity as the original system. For proof please see the reference [22]. Note that iid channel adapters can be easily

22 Chapter 2. Technical Background 14 implemented in practice using synchronized pseudo-random binary sequence generators at the transmitter and receiver Non-iterative vs. Iterative demapping The system in Fig. 2.1 performs the single demap and decode operation used in BICM systems. In MLC/MSD, the decoders for the lower bit-levels pass decoded bit informations back to the demapper to assist the next bit-level decoder. As mentioned, MSD can achieve a capacity higher than non-iterative demapping. However, it has been shown in [5] that if binary-reflected Gray labelling (BRGL) is used to label the constellation, then the difference between the iterative and non-iterative demapping schemes is extremely small at high code rates. Fig. 2.2 plots the bit-channel and symbol-channel capacities for 16-QAM under iterative and non-iterative decoding. The iterative scheme is labelled using set-partition while non-iterative scheme uses BRGL. Note the small difference ( db) between MLC/MSD and BICM at rate 3/4. The three types of capacities shown in Fig. 2.2 should be carefully distinguished. The ultimate Shannon limit (red curve) is the capacity achieved under a continuous, capacity-achieving input distribution with iterative demapping. The iterative demapping capacity (dashed blue curve) can be achieved under discrete, uniformly distributed 16-QAM constellation with iterative demapping. The non-iterative demapping capacity (solid blue curve) can be achieved under discrete, uniformly distributed 16-QAM constellation without iterative demapping. We will always refer to the non-iterative demapping capacity in this thesis unless otherwise noted. Iterative demapping requires higher receiver complexity and latency. In light of the negligible loss in capacity at the operating point marked in Fig. 2.2, we are justified to focus on the low-complexity non-iterative Gray-labelled BICM scheme.

23 Chapter 2. Technical Background 15 bit/channel use b 0 GY b 1 GY b 2 GY b 3 GY b 0 SP b 1 SP b 2 SP b 3 SP sym GY sym SP C Sh 1.53 db shaping gain db E s /N 0 (db) Figure 2.2: Bit-channel and symbol-channel capacities of 16-QAM for set-partition labelled MLC/MSD and Gray-labelled BICM.

24 Chapter 2. Technical Background Shaping The gap between the ultimate Shannon limit and MLC/MSD capacity is due to the use of a discrete, uniformly-distributed M-QAM constellation for the continuous-input complex AWGN channel. From information theory we know the capacity-achieving input distribution for this channel is a two-dimensional circularly symmetric Gaussian distribution. The technique of approximating this ideal input distribution using a discrete distribution is called shaping [30, 31]. An asymptotic shaping gain of 1.53 db can be achieved as shown in Fig We neglect shaping in our system to limit the scope of our research. This simplification has also been made in all of prior works cited in Sec We believe that shaping techniques can be applied to our code designs without affecting the coded modulation gains. 2.2 Analysis and Design of Binary LDPC Codes In this section we review well-known design and analysis techniques for capacity-approaching irregular LDPC codes. Asymptotic ensemble analysis using density evolution and its approximation using extrinsic information transfer (EXIT) charts is explained in detail Ensemble-based design and density evolution An irregular LDPC code of length n is fully specified by the number of variable nodes and their degrees, the number of check nodes and their degrees, and the edge connections between variable and check nodes. The number of variable and check nodes and their degrees can be conveniently represented by using degree distribution polynomials Λ(x) = d v d c Λ i x i, P (x) = P i x i (2.4) i=1 i=2 where d v and d c are maximum variable and check node degrees, Λ i and P i are the number of variable and check nodes of each degree. Note that check degrees are greater or equal

25 Chapter 2. Technical Background 17 to 2 since a parity check equation of 1 term is useless. The total number of variable nodes is Λ(1) = n and of check nodes is P (1) = (1 r)n. It is more useful to normalize (2.4) by the number of variable and check nodes. We define the node-perspective normalized degree distribution polynomials L(x) = Λ(x) P (x), R(x) = Λ(1) P (1). (2.5) Node-perspective indicates that the coefficient in each term of the degree polynomial denotes the fraction of nodes of that degree. An alternative edge-perspective degree distribution polynomial indicates that the coefficients refer to the fraction of edges (in total number edges) that are connected to nodes of that degree. It is easy to convert node-perspective to edge-perspective degree distribution polynomials by where denotes differentiation. λ(x) = L (x) L (1), ρ(x) = R (x) R (1) (2.6) Converting back to node-perspective degree distribution polynomials is achieved by L(x) = x λ(s)ds 0, R(x) = λ(s)ds 1 0 x ρ(s)ds 0 (2.7) ρ(s)ds. An ensemble is the set of all parity-check matrices (equivalently Tanner graphs) that satisfy the degree distribution polynomials. By satisfy, we mean satisfy to within some small tolerance, since in finite block-length it is often impossible to exactly satisfy the distribution polynomials. Consider the degree of a node to be the number of sockets it has available for edges to plug into. An edge-count constraint L (1) = R (1) is placed on the variable and check degree distribution polynomials to ensure an equal number of sockets on both sides. Let π be a permutation for a particular connection of edges between variable node sockets and check node sockets. An ensemble is defined to be the collection of all Tanner graphs which satisfy the degree distribution polynomials, under all possible permutations π, and all possible channel outputs [18]. 1 0

26 Chapter 2. Technical Background 18 The channel outputs are assumed to be independent between all codeword bits. The bit-channel is assumed to be output symmetric as defined in (2.3). For a symmetric channel, one can show that assuming only the all-zeros codeword is sent is equivalent in performance to assuming all possible codewords are sent [29, pp ]. Hence under the symmetric channel assumption, the ensemble also encompasses all possible transmitted codewords. This is why iid channel adapters are necessary in our target system. Ensemble-based analysis cannot be used for code design if the bit-channels are not output symmetric. The sum-product algorithm Ensemble-based analysis evaluates the expected probability of message errors for some decoding algorithm, averaged over all Tanner graphs and channel outputs in the ensemble. The decoding algorithm for LDPC codes in AWGN is a specialized instance of the message-passing sum-product algorithm [16] called Belief Propagation (BP) [4]. For any variable node v i in a LDPC code Tanner graph, denote its channel output LLR by µ i. Let J denote the indices of its neighbouring check nodes. For any check node c j, let I denote the indices of its neighbouring variable nodes. Let µ v i j and µ c i j represent the messages passed from variable and check node of index i to a node of index j. Initially, all µ c i j = 0 and all µ v i j = µ i. In subsequent iterations, the variable node update equation is µ v i j = j J \j The check node update equation is i I\i µ c j i + µ i, j J. (2.8) µ c j i = 2 tanh 1 ( µ v ) i tanh j, i I. (2.9) 2 A full decoding iteration includes one execution of variable and check updates. At

27 Chapter 2. Technical Background 19 the end of every iteration, a hard decision is made for variable v i based on µ v i j + µ c j i using the decision rule from Sec Decoding ends when the hard-decision codeword passes parity check or a maximum number of decoding iterations has been reached. Concentration, decoding-tree and density evolution Several key results justify ensemble-based analysis and its main tool, density evolution. The concentration theorem [18] states that if P n (l) is the expected fraction of incorrect messages passed during the l-th decoding iteration for a block-length n code ensemble, then the probability of the actual fraction of incorrect messages for a sample code of the ensemble being outside of (P n (l) δ, P n (l) + δ) tends to 0 exponentially with n, for any δ > 0. Given the concentration theorem, the problem of analyzing the error performance of a particular code Tanner graph is converted to analyzing the expected performance of all Tanner graphs in the ensemble. At first this appears to be an even more difficult problem, but the expansion of a code to its ensemble allows for a second simplifying theorem to be applied. A second key theorem in [18] states that the ensemble expected fraction of incorrect messages P n (l) converges to P (l) as n tends to infinity, where P (l) is the fraction of incorrect messages passed during iteration l assuming the decoding neighbourhood of depth l is cycle-free. The decoding neighbourhood of depth l for a variable node v i is the recursive expansion of edges and neighbouring nodes of v i in l decoding iterations. An additional level of check and variable nodes is added with every iteration. A length 2l cycle exists if v i appears in its own decoding neighbourhood of depth l. Intuitively, the presence of cycles means the message received by v i after l decoding iterations is necessarily dependent on previous messages from v i, hence messages are correlated. Correlated message passing is extremely difficult to analyze. The convergence of ensemble expectation to the cycle-free

28 Chapter 2. Technical Background 20 case allows for the assumption that all messages are independent, greatly simplifying analysis. The results for the cycle-free graph directly apply to the ensemble expectation, since they are equal as n tends to infinity. The fraction of error messages of the cycle-free graph is analyzed using Density Evolution (DE) [18]. DE tracks all messages in the BP decoding algorithm for the cycle-free graph realization of the code ensemble, over all possible channel outputs and transmitted codewords. As the name suggests, DE operates on probability densities of the channel outputs and messages. Let P 0 denote the density of the channel output LLR. Initially, the variable nodes all send their channel output, therefore the density of the µ v i j messages is P 0. At the check nodes, let Γ() and Γ 1 () be a transform and its inverse that implements (2.9) and allows for the message calculation in the transform domain to be a sum. Such a transform is given in [18]. For a degree i check node, with independent incoming messages, the output message density is given by Γ 1 (Γ(P 0 ) (i 1) ) (2.10) where denotes convolution and exponentiation is a shorthand for repeated convolutions. Averaging over all check node degrees and their respective edge-perspective distribution coefficients, we obtain the µ c j i message density after 1 iteration as ( dc ) Q 1 = Γ 1 (ρ(γ(p 0 ))) = Γ 1 ρ i Γ(P 0 ) (i 1). (2.11) In the cycle-free decoding neighbourhood, all messages remain independent after node updates. Therefore the variable message density after 1 iteration is i=2 P 1 = P 0 λ(q 1 ) = P 0 d v i=1 λ (i 1) i. (2.12) For any iteration l, the recursive density update for µ v i j messages is

29 Chapter 2. Technical Background 21 P l = P 0 λ(γ 1 (ρ(γ(p l 1 )))). (2.13) The density evolution threshold for the AWGN channel is defined to be σ such that P l 0 as l for all σ < σ. In [15] a quantized version of density evolution named discrete density evolution is given with good implementation and numerical stability characteristics. All thresholds reported in this thesis are evaluated using discrete density evolution. To summarize, in ensemble-based LDPC code analysis, for a given pair of variable and check node degree distributions, the goal is to evaluate the expected fraction of incorrect messages for the ensemble of all code Tanner graphs, channel outputs and transmitted codewords. At large block-lengths, the faction of incorrect messages of any specific code Tanner graph is concentrated around the ensemble expectation. The expectation is shown to be asymptotically equal to the fraction of incorrect messages of the cycle-free decoding neighbourhood, which can be analytically determined using density evolution. Gaussian-approximated density evolution Density evolution is an effective analytical tool for finding the threshold of LDPC code ensembles parameterized by degree distribution pairs. However, as a synthesis method to find good degree distribution pairs it is overly complex to be useful in optimization. Originally, [14] used the genetic algorithm differential evolution [32] to optimize degree distribution pairs. Such heuristic algorithms are prone to being trapped in local minima and does not give any convergence guarantee. In addition, since every optimization iteration requires many threshold evaluations, this leads to extremely high runtimes. The complexity of density evolution can be reduced if the message densities (P l, Q l ) are approximated by using symmetric Gaussian densities. Intuitively, since variable node updates are convolutions of independent input message densities, by the central limit theorem [33] the output message density is approximately Gaussian for high node degrees.

30 Chapter 2. Technical Background 22 For the purpose of this discussion, we define a symmetric Gaussian density to be a Gaussian density with σ 2 = 2µ [29]. Consequently, only one parameter µ is needed to fully specify the density function. This Gaussian-approximated density evolution (GA-DE) was introduced in [34] where the authors found degree distribution pairs with thresholds within 0.02 db of full density evolution designs. The key simplifying aspect of GA-DE is the node update equations are no longer operations on densities but on the single representative parameter. The convolution of symmetric Gaussian densities with mean µ at a degree i variable node simply results in an output Gaussian of mean (i 1)µ l 1 + µ 0 where µ 0 is the mean of the channel output. The check node update can be similarly condensed into an expression involving the input message means only. An entire GA-DE iteration can be expressed from the perspective of the average variable output message mean µ v l as µ v l = f(µ v 0, µ v l 1) (2.14) where f is the single-variable function model of (2.13). It has been shown in [34] that the condition f(µ v 0, µ v l 1 ) > µv l 1 is necessary and sufficient to ensure convergence to zero incorrect messages in GA-DE. More importantly, this convergence condition is linear, thus allowing optimization to be achieved by using efficient linear programming. GA-DE was an early instance of single-parameter approximations of density evolution. The idea of single-parameter approximation is to apply the symmetric Gaussian approximation to reduce the cumbersome operations of (2.13) to iterated functions of one parameter. Powerful optimization tools can then be applied to the iterated functions. The single-parameter approximation design technique was further studied in [35] using a semi-gaussian approximation. The modification improved threshold accuracy and design flexibility as the original GA-DE did not work well for variable degrees greater than 10 [34].

31 Chapter 2. Technical Background Extrinsic information transfer charts The most widely used single-parameter approximation of density evolution is the extrinsic information transfer (EXIT) technique. In essence, EXIT uses the extrinsic information parameter as the single-parameter approximation for GA-DE. The definition of extrinsic information is based on the extrinsic processing principle of iterative decoding algorithms. Extrinsic processing is evident in BP update equations (2.8), (2.9) where the out-going message from node i to node j always excludes the incoming message from node j to node i. On a cycle-free decoding graph, the extrinsic processing principle ensures that a node will never receive messages dependent on itself. Assume a variable node initially receives an incorrect channel output LLR, to correct this bit it must eventually receive sufficiently correct extrinsic messages. This means the mutual information between the extrinsic messages of bit i and the value of bit i must eventually converge to 1 as l. The mutual information is defined in [36] as I(X; L) = H(X) H(X L) = 1 e (ξ σ2 /2) 2 /2σ 2 2πσ log 2 2 [1 + e ξ ]dξ (2.15) J(σ) where X is the uniform binary random variable representing a codeword bit and L is the demapped LLR output from a symmetric AWGN channel of variance σ 2. Using this conversion between σ and extrinsic mutual information, the update equations for EXIT-based GA-DE are I v l = d v i=1 λ i J ( ) (i 1)[J 1 (Il c)]2 + σch 2, (2.16) d c ) Il c = 1 ρ i J ( (i 1)[J 1 (1 I vl 1 )]2 (2.17) i=2 where v, c superscripts indicate the extrinsic mutual information updates due to variable or check nodes, l is the decoding iteration, and σch 2 is the channel output LLR variance.

32 Chapter 2. Technical Background 24 The functions J(σ) and J 1 (I) can be pre-calculated or approximated as in [36]. Note the extrinsic informations are averaged over different variable and check node degrees. An approximation is made in (2.17) to find the check node mutual information update based on duality between parity-check and repetition codes. For a full justification please refer to [36, 37]. The condition for successful decoding is I v l 1 as l. Successful decoding is defined to be the existence of a sequence of codes of block-length n such that the probability of bit error goes to 0 as n and l. Conversely, decoding is unsuccessful for a code ensemble if the probability of bit error is bounded away from 0 as n and l [29]. Combining (2.16) and (2.17) into a function f, the equivalent convergence condition for successful decoding is [35] f(i v, σ 2 ch) > I v, I v [J(σ ch ), 1) (2.18) Observe that (2.16) is a linear combination with coefficients λ i, so that (2.18) can be re-written as I v l = d v i=1 λ i f i (I v l 1, σ 2 ch), (2.19) where f i captures the extrinsic mutual information transfer of only degree i variable nodes and one particular check degree. We call these functions elementary EXIT functions. The code design optimization problem is a linear programming problem that maximizes the code rate 1 ( ρ j /j/ λ i /i) over variable node degree distributions λ i given by λ i max λ i i i 1 λ i 0 i 1 λ i = 1 i 1 λ if i (Il 1 v ) > Iv l 1, Iv l 1 [J(σ ch), 1). (2.20)

33 Chapter 2. Technical Background 25 Fig. 2.3 shows the EXIT chart of an optimized ensemble from [35]. The dotted lines are the elementary EXIT functions for d v = (2,..., 30) and a check node of degree 6. The EXIT curve, in solid blue, is the linear combination of the elementary EXIT functions weighted by the optimized variable node degree distribution. The black reference line demarcates improving and degrading variable node mutual information. Notice that the EXIT curve always lies above the reference line, satisfying the convergence condition, thus allowing successful decoding as shown by the staircase line. A final point on EXIT optimization. The problem setup assumes a concentrated check degree. The negligible performance impact of this simplification has been justified in past literature [15, 16, 36]. We will also follow this simplification in our work I l v v I l 1 Figure 2.3: EXIT chart of optimized rate = 0.33 ensemble at threshold of db E s /N 0.

34 Chapter 2. Technical Background Multi-edge-type LDPC Codes The key motivation for the development of multi-edge-type LDPC codes is to impose structure over the random single-edge-type code ensemble defined by pairs of variable and check node degree distribution polynomials. Here we describe two examples where prudently imposed structure leads to complexity reduction or performance improvement over completely structureless code ensembles. Given a maximum variable node degree, it has been observed that under density evolution optimization, variable node degree distributions heavily utilize variable nodes of the highest degree in order to achieve capacity-approaching performance [14]. In practical implementation, the complexity of the decoder hardware scales directly with the maximum variable degree. Therefore, it is desirable to reduce the maximum variable degree while maintaining the capacity-approaching performance. High degree variable nodes are appealing since they help to propagate any reliable intrinsic information and extrinsic information that they are likely to produce. Under the purely random socket assignment of single-edge-type LDPC codes, a very high variable node degree is required to achieve this spreading effect with sufficiently high probability. However, if a code designer imposes structure on maximum degree variable nodes, for example by avoiding connections to many degree 2 nodes, then the same effect can be achieved with high probability for a lower maximum variable degree [29, pp ]. A second example concerns degree 1 variable nodes in LDPC code ensembles. Since a degree 1 variable node only sends its channel observation during message-passing decoding, if it receives an erroneous channel observation then any check node it is connected to is likely to pass on the erroneous message to its neighbouring variable nodes. It is easy to see that if two or more degree 1 variable nodes are connected to the same check node and a few receives erroneous channel observations, they will never be corrected under message-passing decoding. Density evolution on a single-edge-type ensemble with degree 1 variable nodes correctly gives a bit error probability bounded away from zero.

35 Chapter 2. Technical Background 27 For multi-edge-type code ensembles, the code designer can explicitly impose structure on degree 1 variable nodes to eliminate the case where two or more are connected to the same check node. With the extra structure, the bit error probability can be made to go to zero for infinite block-length [29, pp ]. The inclusion of degree 1 variable nodes brings many benefits such as lower error floor, improved decoding threshold, and simpler implementation. Technically, the main difference between multi-edge and single-edge LDPC code ensembles is that the edges between variable and check nodes are assigned to more than one type. Refer to the single-edge-type Tanner graph example in Fig. 1.2, if we assign the edges of all degree 1 variable nodes to the red edge-type, and the edges of all other variable nodes to the blue edge-type, then we obtain the multi-edge-type Tanner graph representation of Fig Recall the useful concept of variable and check node sockets from Sec In multiedge-type ensembles, sockets are also assigned different edge-types. For example, in Fig. 2.4, the left-most check node has 1 red socket and 3 blue sockets, while the right-most variable nodes has 0 red sockets and 3 blue sockets. Only sockets of the same type can be connected by an edge of that type. The tremendous range of code structure can be appreciated by considering both the most general case of defining only one edge-type which is exactly the same as traditional code ensembles, and the most specific case where each edge is assigned a different edgetype, resulting in the definition of a single Tanner graph. The code structures between these two extremes are of the highest interest in applications of multi-edge-type LDPC Figure 2.4: A possible multi-edge-type representation of the Tanner graph in Fig. 1.2

36 Chapter 2. Technical Background 28 codes. In the rest of this section we overview the notation used to work with multi-edge-type code ensembles, closely following the treatment in [29, pp ]. The emphasis will be on the distinguishing features between single and multi-edge-type notations. It may be helpful for the reader to review the single-edge-type notations in Sec. 2.2 to compare with those introduced here. The notation used to distinguish edge-types in multi-edge-type (MET) degree distributions extends the placeholder variable x and node degree d to vectors x and d, where each vector component refers to an edge-type. Whereas a single-edge-type check node of degree 4 in the node-perspective is denoted by x 4, a check node of 3 edge-types of degree d = (2, 3, 4) is denoted in the node-perspective by x 2 1x 3 2x 4 3. In Fig. 2.4, the left-most check node would be denoted by x 1 1x 3 2. A variable node in an MET ensemble has the additional vector r specifying the channel output densities to which it is connected. A variable node with 2 type 1 sockets and 2 type 2 sockets receiving the channel output density 1 is denoted by x 2 1x 2 xr 1 while the same variable node receiving channel output density 2 is denoted by x 2 1x 2 2r 2. In addition to imposing structure, the ability to assign different channel output densities to different variable nodes is another reason for using MET ensembles in this work. Our goal is to exploit the different output densities of bit-channels using inherent differences in LDPC codeword bit protection. Throughout this thesis, the total number of edge-types is denoted by T and indexed by k = (1,..., T ). The total number of distinct channel output densities is denoted by S and indexed by s = (0,..., S). Note that there are S + 1 channel output densities, however the s = 0 density corresponds to the channel output of a punctured variable node which is not used in this work. All edge-type specific quantities such as the maximum variable or check degree, or edge-perspective degree distribution, will be distinguished by a superscript. For example d 1 v or λ 1 are quantities of the edge-type 1. Finally, let x be a shorthand for the partial differentiation operator x.

37 Chapter 2. Technical Background 29 The most general node-perspective degree distribution pair for an MET ensemble is given by L(x, r) = L(x 1,..., x T, r 0,..., r S ) = R(x) = R(x 1,..., x T ) = d 1 v d 1 =1 d 1 c d 1 =1 d T v S d T =1 s=0 d T c d T =1 L d 1,...,d T,s x d x dt T r s R d 1,...,d T xd x dt T (2.21) To make sure the number of sockets of the each type is kept equal between variable and check nodes, the degree distribution pair (2.21) must satisfy the socket-count constraints xk L(1, 1) = xk R(1), k = (1,..., T ). (2.22) Futhermore, (2.21) must also maintain the correct fraction of distinct channel output densities by satisfying the channel-ratio constraints rs L(1, 1) = π s, (2.23) where π s is the fraction of channel output density s over all channel output densities. The code rate is given by r = L(1) R(1). (2.24) Note that all constraints and the code rate are linear in the coefficients of degree distribution polynomials. The edge-perspective degree distributions used by density evolution can be calculated by taking partial derivatives with respect to each edge-type and normalizing

38 Chapter 2. Technical Background 30 ( λ 1 (x 1 ), λ 2 (x 2 ),..., λ T (x T ) ) = ( ρ 1 (x 1 ), ρ 2 (x 2 ),..., ρ T (x T ) ) = ( x1 L(x, r) x1 L(1, 1), x 2 L(x, r) x2 L(1, 1),..., ) x T L(x, r) (2.25) xt L(1, 1) ( x1 R(x) x1 R(1), x 2 R(x) x2 R(1),..., ) x T R(x). (2.26) xt R(1) There are T edge-perspective variable (check) node degree distribution polynomials. Practically, this means density evolution now tracks T message densities to determine the infinite block-length ensemble threshold. Since EXIT chart analysis is a one-dimensional approximation of density evolution, the extrinsic mutual information that must be considered in the MET EXIT chart is also expanded to a vector of T components. One of the main contributions of this thesis is to develop an analytical and design technique based on multi-dimensional EXIT vector fields for a specific MET ensemble defined for LDPC coded modulation.

39 Chapter 3 Multi-edge LDPC Coded Modulation In this chapter we develop the main contributions of this thesis. In Sec. 3.1 the general MET ensemble is reduced to a specific parameterization for LDPC coded modulation. Thorough justifications are given for all simplifications. Sec. 3.2 develops the main analytical tool for the specified MET ensemble: the multi-dimensional EXIT vector field. Several properties of the vector field are proved. Code design using the multi-dimensional EXIT vector field is accomplished after deriving the multi-edge-type convergence criterion based on the fixed points of the iterated system. 3.1 Multi-edge Parameterization We seek to leverage two important properties unique to multi-edge-type (MET) LDPC framework in our coded modulation design. MET ensembles allow, as input, more than one channel output density at variable nodes. This is precisely the desired property for incorporating bit-level differences into the ensemble optimization process. Furthermore, the expanded number of edge-types offers flexible control over the structure of the LDPC ensemble. Structural features can be defined in the ensemble definition before optimiza- 31

40 Chapter 3. Multi-edge LDPC Coded Modulation 32 tion. Several reasons exist for imposing code structure, most common are to reduce design and implementation complexity or to lower the error floor. In this work, we specify a MET structure for complexity reduction. The number of free coefficients in the general MET variable and check degree distribution polynomials (2.21) grows exponentially with the number of edge-types. Taking into account the plausible number of distinct channel output densities, for example 5 in the case of Gray-labelled 1024-QAM, optimizing the general degree distributions quickly becomes intractable. We would like to simplify the parameterization to a manageable complexity without sacrificing the desired properties of the MET framework. This can be achieved by first assigning one edge-type to each distinct bit-channel output density. For example, in Gray-labelled 16-QAM there are 4 bit-channels but only 2 distinct bit-channel output densities, therefore only 2 edge-types are used; whereas for set-partition labelled 16- QAM there are 4 distinct bit-channel output densities, requiring 4 edge-types in the MET parameterization. In addition, each variable node is restricted to have sockets of only one edge-type, while different edge-type sockets are present at check nodes. We are inspired to make this simplification by the MLC scheme, where a distinct LDPC code is optimized for each bit-level in order to approach the capacity of the overall symbol channel. We extend the idea by allowing variable node messages of different edge-types to interact at check nodes, and more importantly, by optimizing the code across all bit-channels simultaneously. With these two restrictions on the general MET ensemble, a flexible trade-off between designing one code for an averaged channel (BICM) and designing distinct codes for distinct bit-channels (MLC) is achieved by our specific MET parameterization. The single, optimized code under our MET parameterization will not only be properly matched to each bit-channel, but also to the overall high-order modulation symbol channel. Fig. 3.1 (b) illustrates the specified MET parameterization for the case of 2 distinct bit-channels such as Gray-labelled 16-QAM.

41 Chapter 3. Multi-edge LDPC Coded Modulation 33 π σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 (a) Single-edge-type LDPC Tanner graph σ 2 2 σ 2 2 σ 2 2 σ 2 2 π 2 π 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 σ 2 1 (b) Multi-edge-type LDPC Tanner graph Figure 3.1: Tanner graph of the 2 edge-type specified MET parameterization, node degrees are illustrative and not meant to be realistic.

42 Chapter 3. Multi-edge LDPC Coded Modulation 34 The variable degree distribution for the specific MET parameterization is d T k v L(x 1,..., x T, r 1,..., r T ) = L i,k x i kr k. (3.1) k=1 i=1 The index k serves the dual purposes of indexing the edge-types and channel densities, since they are the same under our specification. The bit-channel density of index k = 0 is removed since puncturing is not considered in this work. From (3.1) and Fig. 3.1 it is clear that the MET parameterization from the variable node perspective is identical to a single-edge-type parameterization. In fact, if the check node degree distributions are specified such that no mixing of edge-types can occur, the specific MET parameterization degenerates to the MLC scheme. However, we do allow different edge-types at the check nodes, which allows messages from different bit-channels to mix. As a final simplification, we require the total check node degree d c be concentrated to only one value. The total check node degree is the number of all sockets at a check node, regardless of type. Under this simplification, (2.21) can be written as R(x 1,..., x T ) = {d 1,...,d T d 1 + +d T =d c} R d1,...,d T x d x d T T. (3.2) Even after concentrating to one total check degree, the check degree distribution remains overly complex. The additional complication is in choosing the assignment of check node sockets to different edge-types, under the same total degree. For 2 edge-types and a total check degree of d c there are d c + 1 possible edge-type assignments. For more than 2 edge-types the number of possible assignments grows rapidly, and is related to the partition function P [d c ] from number theory [38]. In order to gain insight into this problem and to explore the possibility of concentrating to only one check node edge-type assignment, we undertook an empirical study using simple MET LDPC ensembles with 2 edge-types.

43 Chapter 3. Multi-edge LDPC Coded Modulation Check degree edge-type assignment The goal of this study is to justify further simplifying the check node degree distribution to a single term, given by R(x 1,..., x T ) = R d 1 c,...,d T c xd1 c 1... x dt c T, (3.3) where d 1 c + + d T c = d c is a chosen check degree edge-type assignment. Note the direct use of d k c to denote the number of check sockets of type k, since only one term of the sum is present. The first code under study is the regular (3,6) ensemble, where the notation corresponds to (d v,d c ), of rate 1/2. For Gray-labelled 16-QAM with 2 distinct bit-channels, the degree 6 check node can be split into (d 1 c, d 2 c) = {(0, 6), (1, 5),..., (6, 0)} over the 2 edge-types. However, since the variable node parameterization imposes a constraint (2.22) on the number of edges of each type, we focus on the case of pairs of check degree edge-type splits where the distribution coefficients can be directly found. For example, the check degree distribution polynomial for the pair of assignments (0,6), (4,2) is R(x 1, x 1 ) = R 1 x R 2 x 4 1x 2 2, (3.4) where R 1,R 2 are obtained by substituting (3.4) into (2.22) and solving the system. The thresholds for all pairs of check node splits are given in Table 3.1, the pairs that do not satisfy the edge-count constraint are marked by -. The check node split pairing with the highest density evolution threshold is the single symmetrical (3,3) split. The next highest threshold belongs to the pair of nearlysymmetrical (2,4), (4,2) splits. From this simple example it appears that when the check degree polynomial is restricted to pairs of edge-type assignments as in (3.4), concentrating to a single symmetrical split gives the highest threshold. To see if the same property can be observed for an odd total check degree, we repeated

44 Chapter 3. Multi-edge LDPC Coded Modulation 36 Table 3.1: Density evolution thresholds for the (3,6) regular LDPC check degree edgetype split pairings under Gray-labelled 16-QAM. σ (0, 6) (1, 5) (2, 4) (3, 3) (4, 2) (5, 1) (6, 0) (0, 6) - (1, 5) - - (2, 4) (3, 3) (4, 2) (5, 1) (6, 0) the above study for the (3,9) rate 2/3 regular LDPC code. Since the total check degree is odd, a symmetrical split is not possible. It is hypothesized that the near-symmetrical pair of (4,5), (5,4) will give the highest threshold. Table 3.2 shows the thresholds for the (3,9) code. The simulated thresholds confirm the near-symmetrical pairing (4,5), (5,4) to be the best check degree splits. Again note the drop in threshold as the edge-type splits move away from symmetry. Finally, our last empirical example uses an irregular LDPC ensemble defined in singleedge-type by λ(x) = 1/3x 1 + 2/3x 3, ρ(x) = x 5 of rate 1/2. The check degree edge-type splits are exactly the same as the (3,6) regular case. The thresholds are given in Table 3.3. The same pattern of threshold/split-pair correspondence as Table 3.1 can be observed, with the only difference being a slightly higher threshold due to the variable node irregularity. The above empirical studies of check degree edge-type splits between 2 edge-types point to the conclusion that a concentrated symmetric (or near-symmetric pairing in the

45 Chapter 3. Multi-edge LDPC Coded Modulation 37 Table 3.2: Density evolution thresholds for the (3,9) regular LDPC check degree edgetype split pairings under Gray-labelled 16-QAM. σ (0, 9) (1, 8) (2, 7) (3, 6) (4, 5) (5, 4) (6, 3) (7, 2) (8, 1) (9, 0) (0, 9) - (1, 8) - - (2, 7) (3, 6) (4, 5) (5, 4) (6, 3) (7, 2) (8, 1) (9, 0) case of odd total check degree) split of total check degree is optimal for all pairs of splits. The fact that the irregular ensemble also shows the same property is highly encouraging in extending this concentration to symmetrical edge-type assignments observation to more complicated irregular ensemble parameterizations. We conjecture that by concentrating the check node edge-type assignment to a single symmetric or near-symmetric split for Gray-labelled 16-QAM, the performance loss from a more general linear combination of check degree splits is minimal. Therefore, for subsequent code design of Gray-labelled 16-QAM, we will assume the concentrated symmetrical edge-type split. For higher-order coded modulation design, an extension technique will be introduced in Sec to alleviate the high complexity of optimizing the check degree edge-type assignment for a concentrated total degree. In summary, the specific MET parameterization for coded modulation assigns a different edge-type to each distinct bit-channel output density. A variable node can only

46 Chapter 3. Multi-edge LDPC Coded Modulation 38 Table 3.3: Density evolution thresholds for irregular LDPC check degree edge-type split pairings under Gray-labelled 16-QAM. σ (0, 6) (1, 5) (2, 4) (3, 3) (4, 2) (5, 1) (6, 0) (0, 6) - (1, 5) - - (2, 4) (3, 3) (4, 2) (5, 1) (6, 0) belong to one edge-type and check nodes are concentrated to one total degree. Furthermore, check node edge-type assignments are concentrated to one particular vector of check degrees (d 1 c,..., d T c ). The multi-edge-type ensemble parameterization used in this thesis is given by the pair of variable and check node degree distribution polynomials (3.1) and (3.3). 3.2 Multi-edge Optimization We seek to optimize the MET ensemble by using single-parameter based LDPC design. Single-parameter LDPC design refers to all methods that approximates full density evolution by assuming symmetric Gaussian intermediate message densities, which can be fully characterized using a single parameter such as the mean, variance, probability of error, or extrinsic mutual information [35]. Recall from Sec that the EXIT chart technique tracks the change of the average extrinsic mutual information (2.15) through variable and check node updates during decoding iterations. The EXIT functions were scalar valued thus analysis and design can be easily organized by plotting both variable and

47 Chapter 3. Multi-edge LDPC Coded Modulation 39 check transfer curves on one coordinate plane and solving a curve fitting problem [35,36] Multi-dimensional EXIT vector field In multi-edge-type ensembles, the MET density evolution as given by (2.25),(2.26) contains as many distinct densities as the number of edge-types, T. Therefore, the singleparameter EXIT approximation of MET density evolution uses a vector of mutual informations to keep track of all edge-type message densities. In other words, the EXIT chart is now multi-dimensional. The key contribution of this thesis is developing efficient and accurate analysis and design methods for a specific MET ensemble based on multi-dimensional EXIT charts. For illustrative purposes, we focus on variable mutual information in the EXIT update equations. A full iteration of the EXIT update equations maps the variable node extrinsic mutual information in the previous iteration Il 1 v to the output extrinsic mutual information of the current iteration Il v, while the check node update is implicitly nested into the update as shown Il v = f v (f c (Il 1), v I0 v ). (3.5) This expression can be fully determined if the check node degree distribution has been given. This can be satisfied either by concentrating the check node to one degree, as we have done in our parameterization, or by an iterative design procedure where one of the check or variable node distributions is assumed to be fixed while the other is being optimized [29, pp ]. It is not difficult to derive the check node mutual information analogue of the analysis and design procedures. However, in our development we shall only focus on the variable mutual informations. With this understanding, we drop the v superscript to avoid excessive notation. In general, optimization based on the multi-dimensional EXIT chart is as difficult as directly optimizing using MET density evolution. The mixing of different edge-type

48 Chapter 3. Multi-edge LDPC Coded Modulation 40 densities at both variable and check nodes complicates the EXIT chart and prohibits an efficient optimization procedure. An additional edge-type exponentially increases the number of EXIT functions in the optimization problem. This may be why prior work on single-parameter analysis and design of MET ensembles has been scarce, where as EXIT techniques for single-edge ensembles have flourished. A review of literature revealed only [25,39] as attempts at EXIT-based MET ensemble optimization. Only [39] explicitly defined multi-dimensional EXIT charts but fell short of providing an effective optimization procedure. Keeping design complexity low while retaining the desired properties of MET ensembles has been a guiding principle throughout this work. It is this disciplined approach that allows for the reduction in complexity of multi-dimensional EXIT charts to allow for an efficient optimization procedure. The key simplification in the MET parameterization of Sec. 3.1 is to restrict variable nodes to only one edge-type, determined by its assigned bit-channel. Under this restriction, from the variable node perspective the EXIT charts are exactly the same as the single-edge case, as shown by the variable node update equations of the multi-dimensional EXIT chart for the vector of variable mutual informations (Il 1,..., IT l ) where σ 2 k d k v [ ] Il k = λ k i Ĵ (i 1)Ĵ 1 (I c,k l ) + σk 2 i=2 (3.6) is the LLR variance of the bit-channel output assigned to edge-type k. Unless specifically noted, all expressions in this section containing the index k are to be understood as the set of T expressions over all edge-types (1,..., T ), indexed by k. The mutual information conversion functions Ĵ(σ2 ), Ĵ 1 (I) denote the composite functions J( σ 2 ) and [J 1 (I)] 2 respectively. The check node mutual information update is slightly more complex since all edgetypes mix at check nodes. Given the concentrated check node edge-type assignment vector (d 1 c,..., d T c ) the multi-dimensional EXIT check node update expression is

49 Chapter 3. Multi-edge LDPC Coded Modulation 41 I c,k l = 1 Ĵ (d k c 1)Ĵ 1 (1 I k l 1) + T d t cĵ 1 (1 Il 1) t (3.7) t=1 t k where the coefficients {ρ k } have been removed since the concentrated check degree edgetype assignment in (3.3) means {ρ k 1}. The check node update expression is similar to the single-edge type version, with the main difference being the input mutual informations now come in T types, and the output consists of T simultaneous mutual information updates. For a full iteration update of the variable vector mutual information, substitute the appropriate edge-type output of (3.7) into (3.6) to obtain I k l = d k v λ k i Ĵ (i 1)Ĵ 1 1 Ĵ T (d k c 1)Ĵ 1 (1 Il 1) k + d t cĵ 1 (1 Il 1) t + σk 2. i=2 t=1 t k (3.8) To clarify the functional relationships between all input and output mutual information components in (3.8), it is more convienent to encapsulate the expression within function representations. Let I c,k l (3.7) for mutual information of edge-type k. Let I v,k l = f c,k (Il 1 1,..., IT l 1 ) denote the check node update node update (3.6) for mutual information of edge-type k, where I k 0 = f v,k (I c,k l, I0 k ) denote the variable = Ĵ(σ2 k ) is the mutual information of the LLR density from the bit-channel corresponding to edge-type k. Expression (3.8) can now be written as or in bold-font notation Il 1 = f v,1 (f c,1 (Il 1 1,..., IT l 1 ), I1 0) Il 2 = f v,2 (f c,2 (Il 1 1,..., IT l 1 ), I2 0)... Il T = f v,t (f c,t (Il 1 1,..., IT l 1 ), IT 0 ), (3.9)

50 Chapter 3. Multi-edge LDPC Coded Modulation 42 I l = f(i l 1 ). (3.10) We see that the multi-dimensional EXIT update (3.10) is a vector field in T dimensional space R T. The domain D of f is the Cartesian product of all closed real intervals between bitchannel output mutual informations I0 k and 1 for each edge-type Here we state a property of the vector field f. D = [I 1 0, 1] [I 2 0, 1] [I T 0, 1]. (3.11) Proposition 3.1. For any set of edge-perspective variable degree distributions {λ k i } dk v i=2, f : D D Proof. By definition all bit mutual informations are between 0 and 1. Looking at expression (3.6), the inner-most check node output I c,k l = f c,k (I l ) is greater or equal to 0 for all I l D since it is a mutual information. Since the function Ĵ 1 (I c,k l ) returns the standard deviation of the symmetric Gaussian density corresponding to the check output mutual information, it is also greater than 0, therefore (i 1)Ĵ 1 (I c,k l )+σk 2 σ2 k. Using the fact that Ĵ(σ 2 ) is monotonically increasing in σ 2 [40] we have Ĵ((i 1)Ĵ 1 (I c,k l ) + σk 2) Ĵ(σ2 k ), i. Since the degree distribution coefficients {λ k i } dk v i=2 equal to 0, we have are probabilities thus greater than or d k v [ ] I0 k λ k i Ĵ (i 1)Ĵ 1 (I c,k l ) + σk 2 1 (3.12) i=2 where Ĵ(σ2 k ) is replaced by Ik 0 since they are equivalent. Since I l D and each component of the vector field update f(i l ) is in [I k 0, 1], we have f : D D. Fig. 3.2 illustrates a 2 edge-type Gray-labelled 16-QAM multi-dimensional EXIT vector field f for an optimized multi-edge ensemble at the threshold noise level. The

51 Chapter 3. Multi-edge LDPC Coded Modulation 43 two-component input mutual information vector is plotted on both subplots and labelled as Il 1 1 and I2 l 1. The domain D is the x-y plane. The vector field expression for this example is Il 1 = f v,1 (f c,1 (Il 1 1 f(i l 1 ) =, I2 l 1 ), I1 0) Il 2 = f v,2 (f c,2 (Il 1 1, I2 l 1 ), I2 0). (3.13) The components of the vector field output have been separated into two subplots with Il 1 corresponding to the left subplot and Il 2 corresponding to the right subplot. Each subplot consists of two surfaces and an EXIT decoding path. Take for example the left subplot which describes the mutual information transfer characteristic for every point in D. At the initial point (I0, 1 I0) 2 = (0.610, 0.905), the top surface is the EXIT transfer function f v,1 (f c,1 (Il 1 1, I2 l 1 ), I1 0), referred to here as F 1. The bottom surface is the no-improvement reference where the output type 1 mutual information is equal to the input type 1 mutual information, referred to here as R 1. At the beginning of decoding, surface F 1 is above R 1, indicating that the mutual information of type 1 improves with the initial EXIT update. Similarly, at the same point in the right subplot, the EXIT transfer function f v,2 (f c,2 (Il 1 1, I2 l 1 ), I2 0), denoted by F 2, is also above the noimprovement reference of type 2 mutual information R 2. Since both edge-type mutual informations improve with EXIT update, the decoding path can take a step forward as shown by the staircase progression of the two blue decoding curves in both subplots. Decoding progresses to a new mutual information input vector (I1, 1 I1). 2 The same reasoning applies in all subsequent EXIT vector field decoding iterations. Right after iteration l 1, the multi-dimensional EXIT vector field can be assumed to have reached the point (Il 1 1, I2 l 1 ) somewhere in D. The position of the surfaces F 1 and F 2 relative to their respective reference surfaces R 1 and R 2 determines whether or not decoding can proceed. Intuitively, if F 1 is above R 1 and F 2 is above R 2 decoding will proceed, and if F 1 is equal or below R 1 and F 2 is equal or below R 2 then decoding will not

52 Chapter 3. Multi-edge LDPC Coded Modulation 44 Figure 3.2: Multi-dimensional EXIT vector field for 2 edge-types at threshold σ =

53 Chapter 3. Multi-edge LDPC Coded Modulation 45 proceed. To contrast with the successful decoding case of Fig. 3.2, Fig. 3.3 illustrates the unsuccessful decoding case where the noise power is increased to just above the threshold for the same optimized ensemble. The decoding path stops when it reaches a point in D where either F 1 is below R 1 or F 2 is below R 2 or both. Recall that if decoding does not reach the point (1, 1) in extrinsic mutual information, the probability of error is bounded away from zero asymptotically in block-length and decoding iterations. Analysis of MET ensembles Figs. 3.2 and 3.3 visually illustrates the analytical power of the multi-dimensional EXIT vector field. Even though no visual representations exist for more than 2 edge-types, conceptually the analysis is straightforward. The procedure for analyzing multi-edge coded modulation ensembles using the multi-dimensional EXIT vector field is as follows. Given a set of edge-perspective variable degree distributions {λ k i } dk v i=2, the check degree edge-type assignment vector (d 1 c,..., d T c ), and channel noise power σ 2, iteratively evaluate I l = f(i l 1 ), 1 l L, with I 0 being the initial bit-channel mutual information vector and L the maximum number of allowed iterations. If I l = 1 for some l then σ is below the threshold σ, increment noise power and repeat. If l = L and I l < 1 then σ is considered to be above the threshold σ, decrement and repeat. An outer binary search can be used to efficiently determine the threshold, since the AWGN threshold is monotonic with respect to channel degradation [29, pp. 224]. However, since multi-edge density evolution already exists as an efficient method to determine thresholds of LDPC coded modulation ensembles, using the multi-dimensional EXIT vector field is less accurate and redundant. The main reason for its development is to design of multi-edge LDPC ensembles. For this reason we conclude the study of the analytical uses of this technique and move on to code design.

54 Chapter 3. Multi-edge LDPC Coded Modulation 46 Figure 3.3: Multi-dimensional EXIT vector field for 2 edge-types at above threshold σ =