Lecture 9 Polar Coding

Transcription

1 Lecture 9 Polar Coding I-Hsiang ang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 29, / 25 I-Hsiang ang IT Lecture 9

2 In Pursuit of Shannon s Limit Since 1948, Shannon s theory has drawn the sharp boundary between the possible and the impossible in data compression and data transmission. Once fundamental limits are characterized, the next natural question is: How to achieve these limits with acceptable complexity? For source coding, soon after Shannon s 1948 paper, information and coding theorists found optimal compression schemes with low complexity: Huffman Code (1952): optimal for memoryless source Lempel-Ziv (1977): optimal for stationary ergodic source On the other hand, for channel coding, it turns out be a much harder problem. It has been the holy grail for coding theorist to find a coding scheme that achieves Shannon s limit with low complexity. 2 / 25 I-Hsiang ang IT Lecture 9

3 In Pursuit of Capacity-Achieving Codes Two barriers in pursuing a low-complexity capacity-achieving codes: 1 Lack of explicit construction. In Shannon s proof, it is only proved that there exists coding schemes that achieve capacity. 2 Lack of structure to reduce complexity. In the proof of coding theorems, complexity issues are often neglected, while codes with structures are hard to prove to achieve capacity. Since 1990 s, there are several practical codes found to approach capacity, including turbo code, low-density parity-check (LDPC) code, etc. These codes perform very well empirically, but still in lack of theoretical investigation on the performances and even proof of optimality. The first provably capacity-achieving coding scheme with acceptable complexity is polar code, introduced by Erdal Arıkan in Later in 2012, spatially coupled LDPC codes were also shown to achieve capacity (Shrinivas Kudekar, Tom Richardson, and Rüediger Urbanke). 3 / 25 I-Hsiang ang IT Lecture 9

4 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 7, JULY : A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels Erdal Arıkan, Senior Member, IEEE Abstract A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity of any given binary-input discrete memoryless channel (B-DMC). The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of independent copies of a given B-DMC, a second set of binary-input channels such that, as becomes large, the fraction of indices for which is near approaches and the fraction for which is near approaches. The polarized channels are well-conditioned for channel coding: one need only send data at rate through those with capacity near and at rate through the remaining. Codes constructed on the basis of this idea are called polar codes. The paper proves that, given any B-DMC with and any target rate, there exists a sequence ofpolar codes suchthat hasblock-length, rate, and probability of block error under successive cancellation decoding bounded as independently of the code rate. This performance is achievable by encoders and decoders with complexity for each. A. Preliminaries e write to denote a generic B-DMC with input alphabet, output alphabet, and transition probabilities. The input alphabet will always be, the output alphabet and the transition probabilities may be arbitrary. e write to denote the channel corresponding to uses of ; thus, with. Given a B-DMC, there are two channel parameters of primary interest in this paper: the symmetric capacity and the Bhattacharyya parameter The paper wins the 2010 Information Theory Society Best Paper Award. These parameters are used as measures of rate and reliability, Index Terms Capacity-achieving codes, channel capacity, channel polarization, Plotkin construction, polar codes, Reed respectively. is the highest rate at which reliable communication is possible across using the inputs of with equal Muller (RM) codes, successive cancellation decoding. 4 / 25 I-Hsiang ang frequency. IT Lecture 9 is an upper bound on the probability of max-

5 Overview hen Arıkan introduced polar codes in 2007, he focus on achieving capacity for the general binary-input memoryless symmetric channels (BMS), including BSC, BEC, etc. Later, polar codes are shown to be optimal in many other settings, including lossy source coding, non-binary-input channels, multiple access channels, source coding with side information (yner-ziv problem), etc. Instead of giving a comprehensive introduction, we shall introduce channel polarization and polar coding for BMS, in the following order: 1 First we introduce the concept of channel polarization. 2 Then we explore polar coding. 5 / 25 I-Hsiang ang IT Lecture 9

6 Notations Recall in channel coding, we use the DMC N times with N being the blocklength of the coding scheme. Since the channel is the main focus, we shall use the following notations throughout this lecture: to denote the channel p Y X P to denote the input distribution p X I (P, ) to denote I (X ; Y ). Beside, since we focus on BMS channels, and it is not difficult to prove that X Ber ( 1 2) achieves the channel capacity of any BMS, we shall use I ( ) (abuse of notation) to denote I (P, ) when the input P is Ber ( 1 2). In other words, the channel capacity of the BMS channel is I ( ). 6 / 25 I-Hsiang ang IT Lecture 9

7 1 Polarization 7 / 25 I-Hsiang ang IT Lecture 9

8 Single Usage of Channel X Y N Usage of Channel X 1 Y 1 M ENC X 2. Y 2 DEC ˆM X N Y N 8 / 25 I-Hsiang ang IT Lecture 9

9 Arıkan s Idea U 1 X 1 Y 1 V 1 U 2 Pre- Processing X 2. Y 2 Post- Processing V 2 U N X N Y N V N Apply special transforms to both input and output 9 / 25 I-Hsiang ang IT Lecture 9

10 Arıkan s Idea U 1 V 1 1 U 2 2. V 2 U N N V N 10 / 25 I-Hsiang ang IT Lecture 9

11 Arıkan s Idea Roughly NI ( ) channels with capacity 1 U 1 V 1 1 U 2 2. V 2 U N N V N 11 / 25 I-Hsiang ang IT Lecture 9

12 Arıkan s Idea Roughly NI ( ) channels with capacity 1 U 1 V 1 1 U 2 2 V 2. Roughly N (1 I ( )) channels with capacity 0 U N N V N Equivalently some perfect channels and some useless channels Polarization Coding becomes extremely simple: simply use those perfect channels for uncoded transmission, and throw those useless channels away. 12 / 25 I-Hsiang ang IT Lecture 9

13 1 Polarization 13 / 25 I-Hsiang ang IT Lecture 9

14 Arıkan s Consider two channel uses of : X 1 Y 1 X 2 Y 2 14 / 25 I-Hsiang ang IT Lecture 9

15 Arıkan s Consider two channel uses of : Apply the pre-processor: U 1 Y 1 X 1 = U 1 U 2, X 2 = U 2, where U 1 U 2, U 1, U 2 Ber ( 1 2). U 2 Y 2 e now have two synthetic channels induced by the above procedure: : U 1 V 1 (Y 1, Y 2 ) + : U 2 V 2 (Y 1, Y 2, U 1 ) The above transform yields the following two crucial phenomenon: I ( ) I ( ) I ( + ) I ( ) + I ( + ) = 2I ( ) (Polarization) (Conservation of Information) 15 / 25 I-Hsiang ang IT Lecture 9

16 Example: Binary Erasure Channel Example 1 Let be a BEC with erasure probability ε (0, 1), and I ( ) = 1 ε. Find the values of I ( ) and I ( + ), and verify the above properties. sol: Intuitively is worse than and + is better than : For, input is U 1, output is (Y 1, Y 2 ). Only when both Y 1 and Y 2 are not erased, one can figure out U 1! = is BEC with erasure probability 1 (1 ε) 2 = 2ε ε 2. For +, input is U 2, output is (Y 1, Y 2, U 1 ). As long as one of Y 1 and Y 2 are not erased, one can figure out U 2! = + is BEC with erasure probability ε 2. Hence, I ( ) = 1 2ε + ε 2 and I ( + ) = 1 ε / 25 I-Hsiang ang IT Lecture 9

17 Example: Binary Symmetric Channel Example 2 Let be a BSC with crossover probability p (0, 1), and I ( ) = 1 H b (p). Find the values of I ( ) and I ( + ). 17 / 25 I-Hsiang ang IT Lecture 9

18 Basic Properties Theorem 1 For any BMS channel and the induced channels {, + } from Arıkan s basic transformation, we have I ( ) I ( ) I ( + ) with equality iff I ( ) = 0 or 1. I ( ) + I ( + ) = 2I ( ) pf: e prove the conservation of information first: I ( ) + I ( + ) = I (U 1 ; Y 1, Y 2 ) + I (U 2 ; Y 1, Y 2, U 1 ) = I (U 1 ; Y 1, Y 2 ) + I (U 2 ; Y 1, Y 2 U 1 ) = I (U 1, U 2 ; Y 1, Y 2 ) = I (X 1, X 2 ; Y 1, Y 2 ) = I (X 1 ; Y 1 ) + I (X 2 ; Y 2 ) = 2I ( ). I ( + ) = I (X 2 ; Y 1, Y 2, U 1 ) I (X 2 ; Y 2 ) = I ( ), and hence the first property holds. (Proof of the condition for equality is left as exercise.) 18 / 25 I-Hsiang ang IT Lecture 9

19 Extremal Channels I( + ) I( )[bits] BSC BEC If we plot the information stretch I ( + ) I ( ) versus the original information I ( ), it can be shown that among all BMS channels: BEC maximizes the stretch BSC minimizes the stretch Lower boundary: I() [bits] (Taken from Chap of Moser[4].) 2H b (2p(1 p)) 2H b (p), where p = H b 1 (1 I ( )). Upper boundary: 2I ( ) (1 I ( )). 19 / 25 I-Hsiang ang IT Lecture 9

20 1 Polarization 20 / 25 I-Hsiang ang IT Lecture 9

21 Recursive Application of Arıkan s Transformation Duplicate, apply the transformation, and get and / 25 I-Hsiang ang IT Lecture 9

22 Recursive Application of Arıkan s Transformation Duplicate, apply the transformation, and get and +. Duplicate (and + ). 22 / 25 I-Hsiang ang IT Lecture 9

23 Recursive Application of Arıkan s Transformation Duplicate, apply the transformation, and get and +. Duplicate (and + ). Apply the transformation on, and get and / 25 I-Hsiang ang IT Lecture 9

24 Recursive Application of Arıkan s Transformation Duplicate, apply the transformation, and get and +. Duplicate (and + ). Apply the transformation on, and get and +. Apply the transformation on +, and get + and / 25 I-Hsiang ang IT Lecture 9

25 Recursive Application of Arıkan s Transformation Duplicate, apply the transformation, and get and +. Duplicate (and + ). Apply the transformation on, and get and +. Apply the transformation on +, and get + and ++.. e can keep going and going, until the desired blocklength is reached. 25 / 25 I-Hsiang ang IT Lecture 9