The PPM Poisson Channel: Finite-Length Bounds and Code Design

Size: px
Start display at page:

Download "The PPM Poisson Channel: Finite-Length Bounds and Code Design"

Transcription

1 August 21, 2014 The PPM Poisson Channel: Finite-Length Bounds and Code Design Flavio Zabini DEI - University of Bologna and Institute for Communications and Navigation German Aerospace Center (DLR) Balazs Matuz, Gianluigi Liva, Enrico Paolini, Marco Chiani

2 Motivation Poisson channel is a common channel model for deep space optical links for direct detection Under peak and average power constraints, slotted modulation schemes, such as pulse position modulation (PPM), allow operating close to capacity For finite-length blocks bounds/benchmarks on the block error probability (BLEP) are barely available in literature Binary low-density parity-check (LDPC) codes show a noticeable gap to capacity Non-binary protograph LDPC codes where field and PPM order are matched

3 Outline 1 Preliminaries 2 Finite-Length Benchmarks 3 Code Design 4 Conclusions

4 Outline 1 Preliminaries 2 Finite-Length Benchmarks 3 Code Design 4 Conclusions

5 Slide 1/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Communication System LDPC encoder C q (N,K) c x y P(c y) Modulator Poisson Demodulator (q-ary PPM) channel LDPC decoder encoded symbol symbol probability mass function The code C q (N, K) is used to generate code symbols c q-ary code symbols c are mapped to q-ary PPM symbols x For q-ary PPM one slot out of q slots is pulsed PPM symbols x transmitted on the Poisson channel Given y we get the probabilities P(c y) The decoder provides a decision on the transmitted symbols

6 Slide 2/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Channel Model TX RX PPM symbol PPM symbol ( ) P y [t] X [t] = P = (n s + n b ) y[t] e (n s+n b) y [t]! ( ) P y [t] X [t] = P = n b y[t] e nb y [t]! for a pulsed slot for a non-pulsed slot n s : average number of signal photons in pulsed slot n b : average number of noise photons per slot t: time slot index.

7 Slide 3/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Channel capacity for Poisson with q-ary PPM: { [ q 1 ( ) ]} Y [t] Y [0] ns C PPM = log 2 q E Y X [0] =P log [bits/channel use] n b Capacity in absence of background noise, i.e., for n b = 0 t=0 C PPM EC =log 2 q ( 1 e ns) [bits/channel use] Capacity of q-ary erasure channel (EC) with ɛ = exp( n s )

8 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Generic example P e : Block error probability R : Transmission rate [bits/channel use]

9 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Generic example P e : Block error probability R : Transmission rate [bits/channel use]

10 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Generic example P e : Block error probability R : Transmission rate [bits/channel use]

11 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Generic example P e : Block error probability R : Transmission rate [bits/channel use]

12 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Generic example P e : Block error probability R : Transmission rate [bits/channel use]

13 Slide 4/17 Flavio Zabini The PPM Poisson Channel Preliminaries August 21, 2014 Preliminaries Capacity Channel capacity provides the limit for infinite length codes, but... P e : Block error probability R : Transmission rate [bits/channel use]

14 Outline 1 Preliminaries 2 Finite-Length Benchmarks 3 Code Design 4 Conclusions

15 Slide 5/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Goal..we want to evaluate P e (N, r, n s, n b ): R = r log 2 q [bits/channel use] the minimum block error probability achievable by a code with FINITE length N and rate r [symbols/channel use] over a q-ary PPM Poisson channel, given n s and n b

16 Slide 6/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Random Coding Bound for Poisson PPM Gallager s random coding bound (RCB): Error exponent: P e 2 NE PPM(r) E PPM (r) = max 0 ρ 1 { log 2 [Υ q,n s,n b (ρ)] + [ρ(1 r) + 1] log 2 q}+ n s + qn b ln(2) Υ q,ns,n b (ρ) + k T =0 n kt b k T k T i 1 i 1=0 i 2=0... k T q 2 n=1 in i q 1=0 [ q 1 q 1 ] 1+ρ n=1 ain ρ + a kt n=1 in ρ q 1 n=1 i n! (k T q 1 n=1 i n)! ( ) 1 a ρ 1 + ns 1+ρ n b For q > 16 computation becomes numerically challenging

17 Slide 7/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Erasure Channel Approximation Use q-ary EC as a surrogate for Poisson PPM channel By matching the capacities of both channels we obtain: ɛ = 1 C PPM log 2 q Gallager s error exponent for the q-ary EC: ( ) 1 ɛ log 2 q + ɛ r log 2 q 0 < ɛ < ɛ c E qec (r) = D(B 1 r B ɛ ) ɛ c ɛ 1 r 0 ɛ > 1 r 1 r r(q 1)+1 ɛ c B 1 r and B ɛ are two Bernoulli distributions with parameters 1 r and ɛ.

18 Slide 8/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Singleton Bound via EC Approximation Singleton bound, denoting the BLEP of idealized maximum distance separable code on an EC where S N B{N, ɛ}. Again we require: P (S) e = P (S) e N i=n K+1 ( ) N ɛ i (1 ɛ) N i i = Pr (S N > N(1 r)) ɛ = 1 C PPM log 2 q

19 Slide 9/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Dispersion Approach - Definitions Let us consider a generic discrete memoryless channel (DMC). ( ) PY X (y X=x) Information density: i(x, y) log 2 P Y (y) { Mutual information: I(P X ; P Y X ) E X EY X=x {i(x, y)} } Conditional information variance: V(P X ; P Y X ) E X { EY X=x { [i(x, y)] 2 } [E Y X=x {i(x, y)}] 2} Capacity: C max PX { I(PX ; P Y X ) } Dispersion: V min PX Π { V(PX ; P Y X ) } where Π = {P X : I(P X ; P Y X ) = C}.

20 Slide 10/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Dispersion Approach - Applications Capacity: R < C lim N P e = 0 Shannon (1948) Dispersion: [ P e Q (C R) V N Strassen (1962), recently [1,2] [1] Hayashi, Information Spectrum Approach to Second-order Coding Rate in Channel Coding, IEEE Trans. Inf. Theory 2009 [2] Polyanskiy et al., Channel Coding Rate in the Finite Blocklength Regime, IEEE Trans. Inf. Theory 2010 ]

21 Slide 11/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Converse Theorem Gaussian Approximation For the Poisson PPM channel the dispersion V PPM results in V PPM = E Y X [0] =P { (*** novel result ***) log 2 2 [ q 1 t=0 ( ) ]} Y [t] Y [0] ns + 1 [log n 2 q C PPM ] 2 b Thus: { } N P e (N, r, n s, n b ) Q [C PPM (q, n s, n b ) r log 2 q] V PPM (q, n s, n b )

22 Slide 12/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Converse Theorem EC Approximation For the q-ary EC we have V qec = ɛ(1 ɛ) log 2 2 q; [bits2 /channel use 2 ] [ ] P (qec) N(1 r) Nɛ e Q Nɛ(1 ɛ) = Pr (G N > N(1 r)) where G N N {Nɛ, Nɛ(1 ɛ)} and ɛ = 1 C PPM log 2 q We obtain an approximation of the Singleton bound by replacing the binomial distribution B{N, ɛ} by a Gaussian distribution N {Nɛ, Nɛ(1 ɛ)}.

23 Slide 13/17 Flavio Zabini The PPM Poisson Channel Finite-Length Benchmarks August 21, 2014 Finite-Length Benchmarks Results BLEP N = 2736 N = 684 Gallager RCB RCB via q-ec approx. Converse Converse via q-ec approx. Singleton via q-ec approx. Different bounds/benchmarks: q = 16 n b = 0.02 N = {684, 2736} r = ns/q [db]

24 Outline 1 Preliminaries 2 Finite-Length Benchmarks 3 Code Design 4 Conclusions

25 Slide 14/17 Flavio Zabini The PPM Poisson Channel Code Design August 21, 2014 Code Design Surrogate Erasure Channel Design For non-binary LDPC codes extrinsic information transfer (EXIT) analysis on the Poisson PPM channel is rather complex We propose therefore a surrogate design for the q-ary EC by protograph EXIT analysis Non-binary protograph: Each VN a GF(q) symbol Edges possess edge labels

26 Slide 15/17 Flavio Zabini The PPM Poisson Channel Code Design August 21, 2014 Code Design Selected Protographs Fixing the size of base matrix to 3 5 with one punctured column, we obtain B 1 = with iterative decoding threshold of the ensemble of ɛ = We consider the base matrix of a (binary) LDPC code from [3] B 2 = with ɛ = [3] Barsoum et al., EXIT Function Aided Design of Iteratively Decodable Codes for the Poisson PPM Channel, IEEE TCOM 2010

27 Slide 16/17 Flavio Zabini The PPM Poisson Channel Code Design August 21, 2014 Code Design Performance curves CER, BER nb =0.002 nb =0.2 nb =2 Converse Converse via q-ec approx CER code C1 64 CER code C2 64 BER code C2 64 BER code C ns/q [db] 3 LDPC codes: C 64 1 (1368, 684) over q = 64 from B 1 C 64 2 (1368, 684) over q = 64 from B 2 C 2 3(8192, 4096) over q = 2 from B 2 2 benchmarks: Converse bound normal approx. EC approx. of converse bound

28 Outline 1 Preliminaries 2 Finite-Length Benchmarks 3 Code Design 4 Conclusions

29 Slide 17/17 Flavio Zabini The PPM Poisson Channel Conclusions August 21, 2014 Conclusions Several finite-length benchmarks for the PPM-modulated code performance on the Poisson channel have been developed Converse bound Gaussian approximation: less complex than RCB approach but more accurate for the PPM Poisson channel with realistic parameters (very close to Singleton bound when an EC-equivalent channel is considered) EC-approximation: valid also for high background noise A surrogate erasure channel design for protograph non-binary LDPC codes on the Poisson PPM channel has been proposed The density evolution analysis turns to be extremely simple The code performance is consistently close to the benchmarks also with stronger background radiation (robust design)

30 Thank you for your attention! Questions? Contact: Dr.-Ing. Flavio Zabini Department of Electrical, Electronic and Information Engineering (DEI) University of Bologna

31 Code Design Decoding Thresholds Decoding thresholds on the Poisson PPM channel computed via Monte Carlo density evolution Thresholds in terms of n s /q [db] for protograph code ensembles specified by B 1, B 2, q = 64, and various n b : Background noise n b B B Limit

32 Numerical Results LDPC Code vs. Serially Concatenated Pulse Position Modulation (SCPPM) CER Random coding bound Code C4(2720, 1360) Serially concatenated PPM scheme (2520,1260) Highly irregular code C 4(2720, 1360) over finite field of order q = 64 Background noise n b = RCB as a reference γ [db]

POLAR CODES FOR ERROR CORRECTION: ANALYSIS AND DECODING ALGORITHMS

POLAR CODES FOR ERROR CORRECTION: ANALYSIS AND DECODING ALGORITHMS ALMA MATER STUDIORUM UNIVERSITÀ DI BOLOGNA CAMPUS DI CESENA SCUOLA DI INGEGNERIA E ARCHITETTURA CORSO DI LAUREA MAGISTRALE IN INGEGNERIA ELETTRONICA E TELECOMUNICAZIONI PER L ENERGIA POLAR CODES FOR ERROR

More information

ECEN 655: Advanced Channel Coding

ECEN 655: Advanced Channel Coding ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History

More information

Channel Codes for Short Blocks: A Survey

Channel Codes for Short Blocks: A Survey 11th International ITG Conference on Systems, Communications and Coding February 6, 2017 Channel Codes for Short Blocks: A Survey Gianluigi Liva, gianluigi.liva@dlr.de Fabian Steiner, fabian.steiner@tum.de

More information

Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes

Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel 2009 IEEE International

More information

Successive Cancellation Decoding of Single Parity-Check Product Codes

Successive Cancellation Decoding of Single Parity-Check Product Codes Successive Cancellation Decoding of Single Parity-Check Product Codes Mustafa Cemil Coşkun, Gianluigi Liva, Alexandre Graell i Amat and Michael Lentmaier Institute of Communications and Navigation, German

More information

An Improved Sphere-Packing Bound for Finite-Length Codes over Symmetric Memoryless Channels

An Improved Sphere-Packing Bound for Finite-Length Codes over Symmetric Memoryless Channels An Improved Sphere-Packing Bound for Finite-Length Codes over Symmetric Memoryless Channels Gil Wiechman Igal Sason Department of Electrical Engineering Technion, Haifa 3000, Israel {igillw@tx,sason@ee}.technion.ac.il

More information

Bandwidth Efficient and Rate-Matched Low-Density Parity-Check Coded Modulation

Bandwidth Efficient and Rate-Matched Low-Density Parity-Check Coded Modulation Bandwidth Efficient and Rate-Matched Low-Density Parity-Check Coded Modulation Georg Böcherer, Patrick Schulte, Fabian Steiner Chair for Communications Engineering patrick.schulte@tum.de April 29, 2015

More information

Capacity of a channel Shannon s second theorem. Information Theory 1/33

Capacity of a channel Shannon s second theorem. Information Theory 1/33 Capacity of a channel Shannon s second theorem Information Theory 1/33 Outline 1. Memoryless channels, examples ; 2. Capacity ; 3. Symmetric channels ; 4. Channel Coding ; 5. Shannon s second theorem,

More information

Modulation codes for the deep-space optical channel

Modulation codes for the deep-space optical channel Modulation codes for the deep-space optical channel Bruce Moision, Jon Hamkins, Matt Klimesh, Robert McEliece Jet Propulsion Laboratory Pasadena, CA, USA DIMACS, March 25 26, 2004 March 25 26, 2004 DIMACS

More information

Graph-based codes for flash memory

Graph-based codes for flash memory 1/28 Graph-based codes for flash memory Discrete Mathematics Seminar September 3, 2013 Katie Haymaker Joint work with Professor Christine Kelley University of Nebraska-Lincoln 2/28 Outline 1 Background

More information

On Third-Order Asymptotics for DMCs

On Third-Order Asymptotics for DMCs On Third-Order Asymptotics for DMCs Vincent Y. F. Tan Institute for Infocomm Research (I R) National University of Singapore (NUS) January 0, 013 Vincent Tan (I R and NUS) Third-Order Asymptotics for DMCs

More information

LDPC Codes. Intracom Telecom, Peania

LDPC Codes. Intracom Telecom, Peania LDPC Codes Alexios Balatsoukas-Stimming and Athanasios P. Liavas Technical University of Crete Dept. of Electronic and Computer Engineering Telecommunications Laboratory December 16, 2011 Intracom Telecom,

More information

Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels

Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Jilei Hou, Paul H. Siegel and Laurence B. Milstein Department of Electrical and Computer Engineering

More information

EE5139R: Problem Set 7 Assigned: 30/09/15, Due: 07/10/15

EE5139R: Problem Set 7 Assigned: 30/09/15, Due: 07/10/15 EE5139R: Problem Set 7 Assigned: 30/09/15, Due: 07/10/15 1. Cascade of Binary Symmetric Channels The conditional probability distribution py x for each of the BSCs may be expressed by the transition probability

More information

Turbo Code Design for Short Blocks

Turbo Code Design for Short Blocks Turbo Code Design for Short Blocks Thomas Jerkovits, Balázs Matuz Abstract This work considers the design of short parallel turbo codes (PTCs) with block lengths in the order of (a few) hundred code bits.

More information

THIS paper provides a general technique for constructing

THIS paper provides a general technique for constructing Protograph-Based Raptor-Like LDPC Codes for the Binary Erasure Channel Kasra Vakilinia Department of Electrical Engineering University of California, Los Angeles Los Angeles, California 90024 Email: vakiliniak@ucla.edu

More information

EE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes

EE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check

More information

National University of Singapore Department of Electrical & Computer Engineering. Examination for

National University of Singapore Department of Electrical & Computer Engineering. Examination for National University of Singapore Department of Electrical & Computer Engineering Examination for EE5139R Information Theory for Communication Systems (Semester I, 2014/15) November/December 2014 Time Allowed:

More information

RCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths

RCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths RCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths Kasra Vakilinia, Dariush Divsalar*, and Richard D. Wesel Department of Electrical Engineering, University

More information

Practical Polar Code Construction Using Generalised Generator Matrices

Practical Polar Code Construction Using Generalised Generator Matrices Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:

More information

Iterative Quantization. Using Codes On Graphs

Iterative Quantization. Using Codes On Graphs Iterative Quantization Using Codes On Graphs Emin Martinian and Jonathan S. Yedidia 2 Massachusetts Institute of Technology 2 Mitsubishi Electric Research Labs Lossy Data Compression: Encoding: Map source

More information

Revision of Lecture 5

Revision of Lecture 5 Revision of Lecture 5 Information transferring across channels Channel characteristics and binary symmetric channel Average mutual information Average mutual information tells us what happens to information

More information

An Introduction to Low Density Parity Check (LDPC) Codes

An Introduction to Low Density Parity Check (LDPC) Codes An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,

More information

Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras

Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras e-mail: hari_jethanandani@yahoo.com Abstract Low-density parity-check (LDPC) codes are discussed

More information

Capacity-approaching codes

Capacity-approaching codes Chapter 13 Capacity-approaching codes We have previously discussed codes on graphs and the sum-product decoding algorithm in general terms. In this chapter we will give a brief overview of some particular

More information

On Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels

On Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels 2012 IEEE International Symposium on Information Theory Proceedings On Generalied EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels H Mamani 1, H Saeedi 1, A Eslami

More information

Constellation Shaping for Communication Channels with Quantized Outputs

Constellation Shaping for Communication Channels with Quantized Outputs Constellation Shaping for Communication Channels with Quantized Outputs, Dr. Matthew C. Valenti and Xingyu Xiang Lane Department of Computer Science and Electrical Engineering West Virginia University

More information

The Poisson Channel with Side Information

The Poisson Channel with Side Information The Poisson Channel with Side Information Shraga Bross School of Enginerring Bar-Ilan University, Israel brosss@macs.biu.ac.il Amos Lapidoth Ligong Wang Signal and Information Processing Laboratory ETH

More information

Lecture 4 Noisy Channel Coding

Lecture 4 Noisy Channel Coding Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem

More information

One Lesson of Information Theory

One Lesson of Information Theory Institut für One Lesson of Information Theory Prof. Dr.-Ing. Volker Kühn Institute of Communications Engineering University of Rostock, Germany Email: volker.kuehn@uni-rostock.de http://www.int.uni-rostock.de/

More information

X 1 : X Table 1: Y = X X 2

X 1 : X Table 1: Y = X X 2 ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access

More information

APPLICATIONS. Quantum Communications

APPLICATIONS. Quantum Communications SOFT PROCESSING TECHNIQUES FOR QUANTUM KEY DISTRIBUTION APPLICATIONS Marina Mondin January 27, 2012 Quantum Communications In the past decades, the key to improving computer performance has been the reduction

More information

Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme

Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme Hsien-Ping Lin Shu Lin and Khaled Abdel-Ghaffar Department of Electrical and Computer Engineering University of California Davis

More information

ECE Information theory Final (Fall 2008)

ECE Information theory Final (Fall 2008) ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1

More information

Chapter 4. Data Transmission and Channel Capacity. Po-Ning Chen, Professor. Department of Communications Engineering. National Chiao Tung University

Chapter 4. Data Transmission and Channel Capacity. Po-Ning Chen, Professor. Department of Communications Engineering. National Chiao Tung University Chapter 4 Data Transmission and Channel Capacity Po-Ning Chen, Professor Department of Communications Engineering National Chiao Tung University Hsin Chu, Taiwan 30050, R.O.C. Principle of Data Transmission

More information

Mapper & De-Mapper System Document

Mapper & De-Mapper System Document Mapper & De-Mapper System Document Mapper / De-Mapper Table of Contents. High Level System and Function Block. Mapper description 2. Demodulator Function block 2. Decoder block 2.. De-Mapper 2..2 Implementation

More information

Physical Layer and Coding

Physical Layer and Coding Physical Layer and Coding Muriel Médard Professor EECS Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media:

More information

Chapter 9 Fundamental Limits in Information Theory

Chapter 9 Fundamental Limits in Information Theory Chapter 9 Fundamental Limits in Information Theory Information Theory is the fundamental theory behind information manipulation, including data compression and data transmission. 9.1 Introduction o For

More information

Short Polar Codes. Peihong Yuan. Chair for Communications Engineering. Technische Universität München

Short Polar Codes. Peihong Yuan. Chair for Communications Engineering. Technische Universität München Short Polar Codes Peihong Yuan Chair for Communications Engineering July 26, 2016 LNT & DLR Summer Workshop on Coding 1 / 23 Outline 1 Motivation 2 Improve the Distance Property 3 Simulation Results 4

More information

Polar Coding. Part 1 - Background. Erdal Arıkan. Electrical-Electronics Engineering Department, Bilkent University, Ankara, Turkey

Polar Coding. Part 1 - Background. Erdal Arıkan. Electrical-Electronics Engineering Department, Bilkent University, Ankara, Turkey Polar Coding Part 1 - Background Erdal Arıkan Electrical-Electronics Engineering Department, Bilkent University, Ankara, Turkey Algorithmic Coding Theory Workshop June 13-17, 2016 ICERM, Providence, RI

More information

One-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information

One-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information One-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information ZOU Sheng and Brian M. Kurkoski kurkoski@ice.uec.ac.jp University of Electro-Communications Tokyo, Japan University of

More information

where L(y) = P Y X (y 0) C PPM = log 2 (M) ( For n b 0 the expression reduces to D. Polar Coding (2)

where L(y) = P Y X (y 0) C PPM = log 2 (M) ( For n b 0 the expression reduces to D. Polar Coding (2) Polar-Coded Pulse Position odulation for the Poisson Channel Delcho Donev Institute for Communications Engineering Technical University of unich Email: delchodonev@tumde Georg Böcherer athematical and

More information

Efficient Computation of EXIT Functions for Non-Binary Iterative Decoding

Efficient Computation of EXIT Functions for Non-Binary Iterative Decoding TO BE PUBLISHED IN IEEE TRANSACTIONS ON COMMUNCATIONS, DECEMBER 2006 Efficient Computation of EXIT Functions for Non-Binary Iterative Decoding Jörg Kliewer, Senior Member, IEEE, Soon Xin Ng, Member, IEEE,

More information

ECE Information theory Final

ECE Information theory Final ECE 776 - Information theory Final Q1 (1 point) We would like to compress a Gaussian source with zero mean and variance 1 We consider two strategies In the first, we quantize with a step size so that the

More information

Channel capacity. Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5.

Channel capacity. Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5. Channel capacity Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5. Exercices Exercise session 11 : Channel capacity 1 1. Source entropy Given X a memoryless

More information

On Achievable Rates and Complexity of LDPC Codes over Parallel Channels: Bounds and Applications

On Achievable Rates and Complexity of LDPC Codes over Parallel Channels: Bounds and Applications On Achievable Rates and Complexity of LDPC Codes over Parallel Channels: Bounds and Applications Igal Sason, Member and Gil Wiechman, Graduate Student Member Abstract A variety of communication scenarios

More information

Dispersion of the Gilbert-Elliott Channel

Dispersion of the Gilbert-Elliott Channel Dispersion of the Gilbert-Elliott Channel Yury Polyanskiy Email: ypolyans@princeton.edu H. Vincent Poor Email: poor@princeton.edu Sergio Verdú Email: verdu@princeton.edu Abstract Channel dispersion plays

More information

Constructions of Nonbinary Quasi-Cyclic LDPC Codes: A Finite Field Approach

Constructions of Nonbinary Quasi-Cyclic LDPC Codes: A Finite Field Approach Constructions of Nonbinary Quasi-Cyclic LDPC Codes: A Finite Field Approach Shu Lin, Shumei Song, Lan Lan, Lingqi Zeng and Ying Y Tai Department of Electrical & Computer Engineering University of California,

More information

An Introduction to Low-Density Parity-Check Codes

An Introduction to Low-Density Parity-Check Codes An Introduction to Low-Density Parity-Check Codes Paul H. Siegel Electrical and Computer Engineering University of California, San Diego 5/ 3/ 7 Copyright 27 by Paul H. Siegel Outline Shannon s Channel

More information

Constellation Shaping for Communication Channels with Quantized Outputs

Constellation Shaping for Communication Channels with Quantized Outputs Constellation Shaping for Communication Channels with Quantized Outputs Chandana Nannapaneni, Matthew C. Valenti, and Xingyu Xiang Lane Department of Computer Science and Electrical Engineering West Virginia

More information

THE seminal paper of Gallager [1, p. 48] suggested to evaluate

THE seminal paper of Gallager [1, p. 48] suggested to evaluate IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 11, NOVEMBER 2004 2657 Extrinsic Information Transfer Functions: Model and Erasure Channel Properties Alexei Ashikhmin, Member, IEEE, Gerhard Kramer,

More information

Low-density parity-check codes

Low-density parity-check codes Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,

More information

Second-Order Asymptotics in Information Theory

Second-Order Asymptotics in Information Theory Second-Order Asymptotics in Information Theory Vincent Y. F. Tan (vtan@nus.edu.sg) Dept. of ECE and Dept. of Mathematics National University of Singapore (NUS) National Taiwan University November 2015

More information

Shannon s noisy-channel theorem

Shannon s noisy-channel theorem Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for

More information

Iterative Encoding of Low-Density Parity-Check Codes

Iterative Encoding of Low-Density Parity-Check Codes Iterative Encoding of Low-Density Parity-Check Codes David Haley, Alex Grant and John Buetefuer Institute for Telecommunications Research University of South Australia Mawson Lakes Blvd Mawson Lakes SA

More information

Coding Techniques for Data Storage Systems

Coding Techniques for Data Storage Systems Coding Techniques for Data Storage Systems Thomas Mittelholzer IBM Zurich Research Laboratory /8 Göttingen Agenda. Channel Coding and Practical Coding Constraints. Linear Codes 3. Weight Enumerators and

More information

Chapter 7: Channel coding:convolutional codes

Chapter 7: Channel coding:convolutional codes Chapter 7: : Convolutional codes University of Limoges meghdadi@ensil.unilim.fr Reference : Digital communications by John Proakis; Wireless communication by Andreas Goldsmith Encoder representation Communication

More information

Joint FEC Encoder and Linear Precoder Design for MIMO Systems with Antenna Correlation

Joint FEC Encoder and Linear Precoder Design for MIMO Systems with Antenna Correlation Joint FEC Encoder and Linear Precoder Design for MIMO Systems with Antenna Correlation Chongbin Xu, Peng Wang, Zhonghao Zhang, and Li Ping City University of Hong Kong 1 Outline Background Mutual Information

More information

Belief-Propagation Decoding of LDPC Codes

Belief-Propagation Decoding of LDPC Codes LDPC Codes: Motivation Belief-Propagation Decoding of LDPC Codes Amir Bennatan, Princeton University Revolution in coding theory Reliable transmission, rates approaching capacity. BIAWGN, Rate =.5, Threshold.45

More information

Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity

Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity Henry D. Pfister, Member, Igal Sason, Member, and Rüdiger Urbanke Abstract We present two sequences of ensembles of non-systematic

More information

Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes

Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes Takayuki Nozaki 1, Kenta Kasai 2, Kohichi Sakaniwa 2 1 Kanagawa University 2 Tokyo Institute of Technology July 12th,

More information

Bounds on Mutual Information for Simple Codes Using Information Combining

Bounds on Mutual Information for Simple Codes Using Information Combining ACCEPTED FOR PUBLICATION IN ANNALS OF TELECOMM., SPECIAL ISSUE 3RD INT. SYMP. TURBO CODES, 003. FINAL VERSION, AUGUST 004. Bounds on Mutual Information for Simple Codes Using Information Combining Ingmar

More information

Coding theory: Applications

Coding theory: Applications INF 244 a) Textbook: Lin and Costello b) Lectures (Tu+Th 12.15-14) covering roughly Chapters 1,9-12, and 14-18 c) Weekly exercises: For your convenience d) Mandatory problem: Programming project (counts

More information

Introduction to Low-Density Parity Check Codes. Brian Kurkoski

Introduction to Low-Density Parity Check Codes. Brian Kurkoski Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code

More information

CHANNEL DECOMPOSITION FOR MULTILEVEL CODES OVER MULTILEVEL AND PARTIAL ERASURE CHANNELS. Carolyn Mayer. Kathryn Haymaker. Christine A.

CHANNEL DECOMPOSITION FOR MULTILEVEL CODES OVER MULTILEVEL AND PARTIAL ERASURE CHANNELS. Carolyn Mayer. Kathryn Haymaker. Christine A. Manuscript submitted to AIMS Journals Volume X, Number 0X, XX 00X doi:0.3934/xx.xx.xx.xx pp. X XX CHANNEL DECOMPOSITION FOR MULTILEVEL CODES OVER MULTILEVEL AND PARTIAL ERASURE CHANNELS Carolyn Mayer Department

More information

Tightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes and Applications

Tightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes and Applications on the ML Decoding Error Probability of Binary Linear Block Codes and Department of Electrical Engineering Technion-Israel Institute of Technology An M.Sc. Thesis supervisor: Dr. Igal Sason March 30, 2006

More information

Problem 7.7 : We assume that P (x i )=1/3, i =1, 2, 3. Then P (y 1 )= 1 ((1 p)+p) = P (y j )=1/3, j=2, 3. Hence : and similarly.

Problem 7.7 : We assume that P (x i )=1/3, i =1, 2, 3. Then P (y 1 )= 1 ((1 p)+p) = P (y j )=1/3, j=2, 3. Hence : and similarly. (b) We note that the above capacity is the same to the capacity of the binary symmetric channel. Indeed, if we considerthe grouping of the output symbols into a = {y 1,y 2 } and b = {y 3,y 4 } we get a

More information

Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes

Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes Brian M. Kurkoski, Kazuhiko Yamaguchi and Kingo Kobayashi kurkoski@ice.uec.ac.jp Dept. of Information and Communications Engineering

More information

Channel combining and splitting for cutoff rate improvement

Channel combining and splitting for cutoff rate improvement Channel combining and splitting for cutoff rate improvement Erdal Arıkan Electrical-Electronics Engineering Department Bilkent University, Ankara, 68, Turkey Email: arikan@eebilkentedutr arxiv:cs/5834v

More information

Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel

Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Pål Ellingsen paale@ii.uib.no Susanna Spinsante s.spinsante@univpm.it Angela Barbero angbar@wmatem.eis.uva.es May 31, 2005 Øyvind Ytrehus

More information

Graph-based Codes for Quantize-Map-and-Forward Relaying

Graph-based Codes for Quantize-Map-and-Forward Relaying 20 IEEE Information Theory Workshop Graph-based Codes for Quantize-Map-and-Forward Relaying Ayan Sengupta, Siddhartha Brahma, Ayfer Özgür, Christina Fragouli and Suhas Diggavi EPFL, Switzerland, UCLA,

More information

Finite Length Analysis of Low-Density Parity-Check Codes on Impulsive Noise Channels

Finite Length Analysis of Low-Density Parity-Check Codes on Impulsive Noise Channels Received November 25, 206, accepted December 2, 206, date of publication January 9, 207, date of current version January 27, 207. Digital Object Identifier 0.09/ACCESS.207.264957 Finite Length Analysis

More information

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities Vincent Y. F. Tan Dept. of ECE and Dept. of Mathematics National University of Singapore (NUS) September 2014 Vincent Tan

More information

Approaching Blokh-Zyablov Error Exponent with Linear-Time Encodable/Decodable Codes

Approaching Blokh-Zyablov Error Exponent with Linear-Time Encodable/Decodable Codes Approaching Blokh-Zyablov Error Exponent with Linear-Time Encodable/Decodable Codes 1 Zheng Wang, Student Member, IEEE, Jie Luo, Member, IEEE arxiv:0808.3756v1 [cs.it] 27 Aug 2008 Abstract We show that

More information

LECTURE 10. Last time: Lecture outline

LECTURE 10. Last time: Lecture outline LECTURE 10 Joint AEP Coding Theorem Last time: Error Exponents Lecture outline Strong Coding Theorem Reading: Gallager, Chapter 5. Review Joint AEP A ( ɛ n) (X) A ( ɛ n) (Y ) vs. A ( ɛ n) (X, Y ) 2 nh(x)

More information

On the Computation of EXIT Characteristics for Symbol-Based Iterative Decoding

On the Computation of EXIT Characteristics for Symbol-Based Iterative Decoding On the Computation of EXIT Characteristics for Symbol-Based Iterative Decoding Jörg Kliewer, Soon Xin Ng 2, and Lajos Hanzo 2 University of Notre Dame, Department of Electrical Engineering, Notre Dame,

More information

Lecture 12. Block Diagram

Lecture 12. Block Diagram Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data

More information

Recent Results on Capacity-Achieving Codes for the Erasure Channel with Bounded Complexity

Recent Results on Capacity-Achieving Codes for the Erasure Channel with Bounded Complexity 26 IEEE 24th Convention of Electrical and Electronics Engineers in Israel Recent Results on Capacity-Achieving Codes for the Erasure Channel with Bounded Complexity Igal Sason Technion Israel Institute

More information

Achievable Rates for Probabilistic Shaping

Achievable Rates for Probabilistic Shaping Achievable Rates for Probabilistic Shaping Georg Böcherer Mathematical and Algorithmic Sciences Lab Huawei Technologies France S.A.S.U. georg.boecherer@ieee.org May 3, 08 arxiv:707.034v5 cs.it May 08 For

More information

Unified Scaling of Polar Codes: Error Exponent, Scaling Exponent, Moderate Deviations, and Error Floors

Unified Scaling of Polar Codes: Error Exponent, Scaling Exponent, Moderate Deviations, and Error Floors Unified Scaling of Polar Codes: Error Exponent, Scaling Exponent, Moderate Deviations, and Error Floors Marco Mondelli, S. Hamed Hassani, and Rüdiger Urbanke Abstract Consider transmission of a polar code

More information

Error Floors of LDPC Coded BICM

Error Floors of LDPC Coded BICM Electrical and Computer Engineering Conference Papers, Posters and Presentations Electrical and Computer Engineering 2007 Error Floors of LDPC Coded BICM Aditya Ramamoorthy Iowa State University, adityar@iastate.edu

More information

Polar Code Construction for List Decoding

Polar Code Construction for List Decoding 1 Polar Code Construction for List Decoding Peihong Yuan, Tobias Prinz, Georg Böcherer arxiv:1707.09753v1 [cs.it] 31 Jul 2017 Abstract A heuristic construction of polar codes for successive cancellation

More information

Channel Coding 1. Sportturm (SpT), Room: C3165

Channel Coding 1.   Sportturm (SpT), Room: C3165 Channel Coding Dr.-Ing. Dirk Wübben Institute for Telecommunications and High-Frequency Techniques Department of Communications Engineering Room: N3, Phone: 4/8-6385 Sportturm (SpT), Room: C365 wuebben@ant.uni-bremen.de

More information

State-of-the-Art Channel Coding

State-of-the-Art Channel Coding Institut für State-of-the-Art Channel Coding Prof. Dr.-Ing. Volker Kühn Institute of Communications Engineering University of Rostock, Germany Email: volker.kuehn@uni-rostock.de http://www.int.uni-rostock.de/

More information

Lecture 4 : Introduction to Low-density Parity-check Codes

Lecture 4 : Introduction to Low-density Parity-check Codes Lecture 4 : Introduction to Low-density Parity-check Codes LDPC codes are a class of linear block codes with implementable decoders, which provide near-capacity performance. History: 1. LDPC codes were

More information

Efficient LLR Calculation for Non-Binary Modulations over Fading Channels

Efficient LLR Calculation for Non-Binary Modulations over Fading Channels Efficient LLR Calculation for Non-Binary Modulations over Fading Channels Raman Yazdani, Student Member, IEEE, and arxiv:12.2164v1 [cs.it] 1 Feb 21 Masoud Ardaani, Senior Member, IEEE Department of Electrical

More information

Optimizing Flash based Storage Systems

Optimizing Flash based Storage Systems Optimizing Flash based Storage Systems 1 Lifetime Reliability Latency Throughput 2 Projects Reliability/Latency/Throughput: Error Correction Code (ECC) Parallelization and Incremental Redundancy Lifetime:

More information

ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS

ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS EC 32 (CR) Total No. of Questions :09] [Total No. of Pages : 02 III/IV B.Tech. DEGREE EXAMINATIONS, APRIL/MAY- 207 Second Semester ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS Time: Three Hours

More information

Distributed Source Coding Using LDPC Codes

Distributed Source Coding Using LDPC Codes Distributed Source Coding Using LDPC Codes Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete May 29, 2010 Telecommunications Laboratory (TUC) Distributed Source Coding

More information

Channels with cost constraints: strong converse and dispersion

Channels with cost constraints: strong converse and dispersion Channels with cost constraints: strong converse and dispersion Victoria Kostina, Sergio Verdú Dept. of Electrical Engineering, Princeton University, NJ 08544, USA Abstract This paper shows the strong converse

More information

Design of Non-Binary Quasi-Cyclic LDPC Codes by Absorbing Set Removal

Design of Non-Binary Quasi-Cyclic LDPC Codes by Absorbing Set Removal Design of Non-Binary Quasi-Cyclic LDPC Codes by Absorbing Set Removal Behzad Amiri Electrical Eng. Department University of California, Los Angeles Los Angeles, USA Email: amiri@ucla.edu Jorge Arturo Flores

More information

Non-Linear Turbo Codes for Interleaver-Division Multiple Access on the OR Channel.

Non-Linear Turbo Codes for Interleaver-Division Multiple Access on the OR Channel. UCLA Graduate School of Engineering - Electrical Engineering Program Non-Linear Turbo Codes for Interleaver-Division Multiple Access on the OR Channel. Miguel Griot, Andres I. Vila Casado, and Richard

More information

Turbo Codes for Deep-Space Communications

Turbo Codes for Deep-Space Communications TDA Progress Report 42-120 February 15, 1995 Turbo Codes for Deep-Space Communications D. Divsalar and F. Pollara Communications Systems Research Section Turbo codes were recently proposed by Berrou, Glavieux,

More information

On the Application of LDPC Codes to Arbitrary Discrete-Memoryless Channels

On the Application of LDPC Codes to Arbitrary Discrete-Memoryless Channels On the Application of LDPC Codes to Arbitrary Discrete-Memoryless Channels Amir Bennatan and David Burshtein Dept. of Electrical Engineering Systems Tel Aviv University Tel Aviv 69978, Israel Email: abn@eng.tau.ac.il,

More information

Codes on graphs and iterative decoding

Codes on graphs and iterative decoding Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Prelude Information transmission 0 0 0 0 0 0 Channel Information transmission signal 0 0 threshold

More information

Distance Properties of Short LDPC Codes and Their Impact on the BP, ML and Near-ML Decoding Performance

Distance Properties of Short LDPC Codes and Their Impact on the BP, ML and Near-ML Decoding Performance Distance Properties of Short LDPC Codes and Their Impact on the BP, ML and Near-ML Decoding Performance Irina E. Bocharova 1,2, Boris D. Kudryashov 1, Vitaly Skachek 2, Yauhen Yakimenka 2 1 St. Petersburg

More information

Memory in Classical Information Theory: A Brief History

Memory in Classical Information Theory: A Brief History Beyond iid in information theory 8- January, 203 Memory in Classical Information Theory: A Brief History sergio verdu princeton university entropy rate Shannon, 948 entropy rate: memoryless process H(X)

More information

Information Theory - Entropy. Figure 3

Information Theory - Entropy. Figure 3 Concept of Information Information Theory - Entropy Figure 3 A typical binary coded digital communication system is shown in Figure 3. What is involved in the transmission of information? - The system

More information

On the Design of Raptor Codes for Binary-Input Gaussian Channels

On the Design of Raptor Codes for Binary-Input Gaussian Channels 1 On the Design of Raptor Codes for Binary-Input Gaussian Channels Zhong Cheng, Jeff Castura, and Yongyi Mao, Member, IEEE Abstract This paper studies the problem of Raptor-code design for binary-input

More information

Rate-Compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Schemes in Packet Data Communications

Rate-Compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Schemes in Packet Data Communications Rate-Compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Schemes in Packet Data Communications Jing Li (Tiffany) Electrical Engineering Dept, Texas A&M University College Station, TX

More information