Physical Layer and Coding
|
|
- Merilyn Wilkinson
- 5 years ago
- Views:
Transcription
1 Physical Layer and Coding Muriel Médard Professor EECS
2 Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media: basic models Nyquist sampling theorem How we use the channels: modulation Modeling the net effect of channels: transition probabilities and errors Making up for channel errors: principles of coding theoretical limits
3 Signals Channel has a physical signal traversing a physical medium Electromagnetic signal Polarization of the wave TM TE Propagation of the wave
4 Signals What do we mean by frequency? We can express a signal by a superposition of sinusoids, each of them with its own frequency A continuous signal is bandlimited to a bandwidth W at carrier frequency f 0 if it can be expressed in terms of sinusoids in a range of frequencies of width W around a frequency f 0 If we multiply the signal by a frequency shift, then we operate around 0, the baseband representation passband baseband f 0 -W/2 f 0 f 0 +W/2 -W/2 0 W/2
5 Nyquist sampling theorem We can wholly reconstruct a signal from its periodic samples as long as those samples are within 1/W of each other Continuous signal x(t), discrete time signal is x[n] = x(n T) where T = 1/W x(t) 0 T 2T 3T 4T 5T 6T t
6 Modulation: creating signals How we generate signals, by varying amplitude, phase, frequency On-off keyed (OOK): send 1, 0, a special case of amplitude modulation Frequency-shift keyed (FSK), change frequencies: Phase-shift key (PSK): 2 PSK 4 PSK
7 Modulation Pulse position modulation (PPM) Combining phase and amplitude, for instance: PAM 5x5 symbol code for 100BASE-T2 PHY
8 Channels Canonical channel model: signal goes into channel, there is a non-additive effect and addition of noise N[n] Tx X[n] G G(X[n]) Y[n] Rx Very commonly, G is a multiplicative effect: Y[n] = g X[n] + N[n] Y[n] = g -1 X[n-1] + g 0 X[n] + N[n] INTERSYMBOL-INTERFERENCE (ISI)
9 Effect of the channel If we put in a certain string of input signals X[1], X[2],, then we have a certain probability of having a certain output string Y[1], Y[2], The net effect is what is the probability of having a certain output when a given input is given
10 Channel model The channel is described by an input alphabet, an output alphabet, a set of probabilities on the input alphabet and a set of channel transitions, for a memoryless channel (on every bit the channel behaves independently of other times): Tx Rx k m Channel with input alphabet of size k+1 and output alphabet of size m+1 p(y x) is transition probability, probability of getting output y for input x
11 How do we determine the probabilities? We want to get to p(x y) Use Bayes s rule: p(x y) = p( x, y) / p(y) Also p(x, y) = p(y x) p(x) p(y) = p(y x i ) p(x i ) i For the case where all the inputs have the same probability, then p(x y) = p(y, x) / (1/n p (y x i ) ) i = p(y x) / n p (y x) How can we try to make up for the effect of the channel?
12 When do we use codes Two different types of codes: source codes: compression channel codes: error-correction Source-channel separation theorem says the two can be done independently for a large family of channels stream Source encoder s Channel encoder x y Channel decoder Source decoder Modulator, channel, receiver, etc...
13 What is a reasonable way to construct codes? Suppose that errors occur independently from symbol to symbol: a reasonable way to code is to make codewords as dissimilar as possible Number of bits in red: Hamming distance x 1 = x 2 = If we received y = , we would map to the first codeword - we ll make this more precise later
14 What do we mean by constructing a code? Block code: map a block of bits to another, generally larger, block of bits (n, k) linear block code encodes n-bit message into k-bit code vector, rate of code is R = n/k In general, code maps a message from set M= 2 n onto a codeword of length k Every codeword is one-to-one correspondence with a message An error occurs in decoding if we map a received codeword to the wrong message or to more than one message
15 What do we mean by decoding? We choose most likely input to have yielded observed output - Maximum Likelihood Decoding Given that we observe the output vector y, the probability that x 1 was transmitted is p(x 1 y) and the probability that x 2 was transmitted is p(x 2 y) (let s look at 2 codewords) Then we pick x 1 when p(x 1 y) / p(x 2 y) > 1 or equivalently when ln(p(x 1 y) ) - ln(p(x 2 y) ) > 0 in terms of log likelihoods
16 Decoding example Consider the binary symmetric channel (BSC) 0 Tx 1-p Rx p 0 1 p 1 1-p The probabilities are: p(x 1 y) = (1-p) 13 p 13 no errors and 1 error p(x 1 y) = (1-p) 11 p 3 11 no errors and 3 errors The BER is p
17 Hamming distance and code capability Minimum Hamming Distance d min of a code is the minimum Hamming Distance between any two codewords A code can detect up to t errors iff d min t + 1 A code can correct up to t errors iff d min 2t + 1 A code can correct up to t errors and detect up to t > t errors iff d min 2t + 1 and d min t + t + 1 How does d min relate to r = k-n, the number of redundant bits? Singleton bound: r + 1 d min Example take strings of n bits and add a parity check code: r is 1 and d min is 2 These are bounds, most codes don t achieve these bounds
18 Linear codes: how do we build them? Code word x = s G where for (n,k) code G is a matrix with k rows and n columns Linear code because sums of codewords are codewords How can we find d min? The Hamming distance between two binary codewords is the same as the Hamming weight of their sum Thus, the minimum Hamming distance is the minimum weight of non-zero code vectors
19 Syndromes and parity-check For any code, it is always possible to find a systematic code, i.e. a code for which G = [I k x k P k x r ]by manipulating the rows using linear operations We define the parity-check matrix H to be: & P r x k # $ - I % k x k " We have x H = 0 Syndrome s = y H Define error sequence e = y + x Then s = e H Decoding is done by finding the minimum Hamming weight sequence that satisfies s = e H and decoding to the codeword y + z
20 Let s think in 3-D Hamming distance between codes is related to t, the number of errors to correct in a (n, k) linear block code Example: rate (3,1) repetition code: can correct at most 1 error and detect 2 errors
21 Further bounds Hamming bound: r Gilbert bound: there exists a code such that: " # $ $ % & " # $ % & ' ( = t 0 j j n log " # $ $ % & " # $ % & ' ( = 2t 0 j j n log r
22 Cyclic Codes Cyclic codes are codes with cyclic shift property in code words: if (a, b, c, d, e) is a codeword, so is (b, c, d, e, a) A generator matrix is then G = & g $ $ $ $ $ $ m 0. g... m.. g # g... $ %... g m... gm " Can we make use of the regular structure of this matrix to simplify things? 0
23 Polynomials on fields Galois field: a field with a finite number of elements We can define a polynomial over a finite field (Galois Field) GF(q) as f(d) = f n D n + f n-1 D n f 0 D is not a variable in the field, it is an indeterminate - think of the D-transform or the Z-transform For polynomials, addition and multiplication can be defined, they are commutative, associative, closure holds, an addition inverse is defined BUT NO MULTIPLICATIVE INVERSE Instead, we have the following theorem: let f(d) and g(d) be polynomials over GF(q) and let g(d) have degree at least 1. Then there exist unique polynomials h(d) and r(d) over GF(q) for which the degree of r(d) is less than that of g(d) and f(d) = g(d) h(d) + r(d) (Euclidean division algorithm)
24 Relation between polynomials and cyclic codes Represent a vector x = (x n-1 x 0 ) as x(d) = x n-1 D n x 0 We say that x(d) is a codeword when the coefficients for the letters of a codeword If we have a cyclic code, then it is the case that the remainder of dividing D x(d) by D n - 1 ( also called the remainder of D x(d) modulo D n - 1 ) is also a code word To see this: Dx(D) = x n-1 D n + + x 0 D = x n-1 (D n - 1) + + x 0 D + x n-1 Define g(d) to be the lowest degree monic polynomial which is a code word in a cyclic code define m to be its degree Because of linearity, any D j g(d) is also a codeword, so in general a(d) g(d) is a codeword Moreover all code words have this form
25 Implementing systematic cyclic codes In polynomial form, a systematic binary code is of the form: x(d) = D r t(d) - d(d) = D r t(d) + d(d) The check bits are d(d) and d(d) has degree less than r Let s take d(d) to be the remainder of D r t(d) / g(d) What about the syndrome? s(d) = x(d) / g(d) = D r t(d) / g(d) + d(d) / g(d) = 2 d(d) = 0 For some q(d), we have g(d) q(d) + d(d) = D r t(d) So g(d) q(d) = D r t(d) - d(d) = D r t(d) + d(d) which is our codeword x(d)
26 Implementing cyclic codes Divide by g(d) circuit D r s(d) g g g r-1... s 1 + s s r-1 + D r s(d)... Divide by g(d) + Encoder Circuits are easily implemented using shift registers to represent polynomials in D
27 Decoding Brute force: divide y(d) by g(d), from there get syndrome, check syndrome against syndrome table, generate n-bit error pattern, then add to y(d) to obtain corrected word However, we know that we can only correct a number t of errors Let s define the Meggitt set to be the set of error patterns such that e n-1 = 1 Use Meggitt s theorem: suppose that g(d) h(d) = D n - 1 and e(d) / g(d) = s(d) then [D b e(d) mod (D n - 1)]/g(D) = [D b s(d)]/g(d) So only need to store syndromes for error patterns in the Meggitt set
28 Different types of codes: Hamming codes Hamming codes are some of the few perfect codes Sphere of radius v: set of all sequences at distances v or less from a sequence A v-error-correcting sphere-packed code has the property that the spheres of size v around the code are non-overlapping and that every sequence is at most v+1 in distance from some code word A v-error-correcting sphere-packed code for which every sequence is at most v+1 in distance from some code word is perfect Hamming codes: rows of H are all the different nonzero sequences of length k-n Thus k = 2 k-n -1 For all Hamming codes, the minimum Hamming distance is 3, so they can correct 1 error or detect 2 The rate of a Hamming code is R = (2 r - r -1 )/(2 r -1 )
29 Cyclic Redundancy Check Codes (CRCs) Normally look only to see whether the syndrome is 0 or not, which is why they are error-detecting This is not a function of the CRC code, it s more of the way it s decoded Usually used in a concatenated fashion with an error correcting code g(d) = (x+1) p(d) where p(d) is a primitive polynomial, which means that it divides D 2r but no lower order polynomial of the form D v -1 Length k = 2 r-1-1 Usually used in shortened mode: for a systematic code, stuff with 0s and those 0s need not be transmitted but must be taken into account at the decoding
30 Convolutional codes Rather than map a block to a block, we code continuously Tend to be used in lower BER channels We can define the code as a set of states: for M memory elements in the encoder, there are 2 M states The trellis structure is as follows, say M=2 Decoding 11 We use a time between known states 01 Possible ways of getting from one state 00 to another are called the adversary paths 10 t t + 1 Trellis Viterbi algorithm is a means of applying dynamic programming to path selection
31 Beyond codes based on distance What is the best performance we can get with a coded system? Let us consider a channel with bandwidth W, with noise energy N per Hz, and with a maximum E on the mean transmitted Shannon s limit is that if we have no delay constraint and no complexity constrain on the coder and decoder, the best achievable error-free rate (capacity) is given by W 2 & E log$ 1 + % NW Bits per second A modulation to achieve this limit is a one that itself looks like noise and the codes are random # "
32 Capacity Recall the BSC 0 X p 1-p Y 0 1 p 1 1-p Capacity is: I(X; Y) = H(Y) - H(Y X) H is entropy H(Z) = - Σ p Z (z) log(p Z (z) ) Let s work it out...
33 Random codes The codes are random in that the codewords are selected at random from a large number of codewords The coding theorem gives results based on the fact that an error is unlikely because the for long enough codewords, two codewords are unlikely to be the close enough to be confused Codes have emerged which work on that principle Examples: Turbo Codes and Low Density Parity Check Codes
CS6304 / Analog and Digital Communication UNIT IV - SOURCE AND ERROR CONTROL CODING PART A 1. What is the use of error control coding? The main use of error control coding is to reduce the overall probability
More informationCommunications II Lecture 9: Error Correction Coding. Professor Kin K. Leung EEE and Computing Departments Imperial College London Copyright reserved
Communications II Lecture 9: Error Correction Coding Professor Kin K. Leung EEE and Computing Departments Imperial College London Copyright reserved Outline Introduction Linear block codes Decoding Hamming
More informationB. Cyclic Codes. Primitive polynomials are the generator polynomials of cyclic codes.
B. Cyclic Codes A cyclic code is a linear block code with the further property that a shift of a codeword results in another codeword. These are based on polynomials whose elements are coefficients from
More information16.36 Communication Systems Engineering
MIT OpenCourseWare http://ocw.mit.edu 16.36 Communication Systems Engineering Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 16.36: Communication
More informationRoll No. :... Invigilator's Signature :.. CS/B.TECH(ECE)/SEM-7/EC-703/ CODING & INFORMATION THEORY. Time Allotted : 3 Hours Full Marks : 70
Name : Roll No. :.... Invigilator's Signature :.. CS/B.TECH(ECE)/SEM-7/EC-703/2011-12 2011 CODING & INFORMATION THEORY Time Allotted : 3 Hours Full Marks : 70 The figures in the margin indicate full marks
More informationCyclic codes: overview
Cyclic codes: overview EE 387, Notes 14, Handout #22 A linear block code is cyclic if the cyclic shift of a codeword is a codeword. Cyclic codes have many advantages. Elegant algebraic descriptions: c(x)
More informationLecture 12. Block Diagram
Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data
More informationChannel Coding I. Exercises SS 2017
Channel Coding I Exercises SS 2017 Lecturer: Dirk Wübben Tutor: Shayan Hassanpour NW1, Room N 2420, Tel.: 0421/218-62387 E-mail: {wuebben, hassanpour}@ant.uni-bremen.de Universität Bremen, FB1 Institut
More informationIntroduction to Wireless & Mobile Systems. Chapter 4. Channel Coding and Error Control Cengage Learning Engineering. All Rights Reserved.
Introduction to Wireless & Mobile Systems Chapter 4 Channel Coding and Error Control 1 Outline Introduction Block Codes Cyclic Codes CRC (Cyclic Redundancy Check) Convolutional Codes Interleaving Information
More informationx n k m(x) ) Codewords can be characterized by (and errors detected by): c(x) mod g(x) = 0 c(x)h(x) = 0 mod (x n 1)
Cyclic codes: review EE 387, Notes 15, Handout #26 A cyclic code is a LBC such that every cyclic shift of a codeword is a codeword. A cyclic code has generator polynomial g(x) that is a divisor of every
More informationCyclic Codes. Saravanan Vijayakumaran August 26, Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 25 Cyclic Codes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 26, 2014 2 / 25 Cyclic Codes Definition A cyclic shift
More informationELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS
EC 32 (CR) Total No. of Questions :09] [Total No. of Pages : 02 III/IV B.Tech. DEGREE EXAMINATIONS, APRIL/MAY- 207 Second Semester ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS Time: Three Hours
More informationExercise 1. = P(y a 1)P(a 1 )
Chapter 7 Channel Capacity Exercise 1 A source produces independent, equally probable symbols from an alphabet {a 1, a 2 } at a rate of one symbol every 3 seconds. These symbols are transmitted over a
More informationChannel Coding and Interleaving
Lecture 6 Channel Coding and Interleaving 1 LORA: Future by Lund www.futurebylund.se The network will be free for those who want to try their products, services and solutions in a precommercial stage.
More informationOptimum Soft Decision Decoding of Linear Block Codes
Optimum Soft Decision Decoding of Linear Block Codes {m i } Channel encoder C=(C n-1,,c 0 ) BPSK S(t) (n,k,d) linear modulator block code Optimal receiver AWGN Assume that [n,k,d] linear block code C is
More informationIntroduction to Convolutional Codes, Part 1
Introduction to Convolutional Codes, Part 1 Frans M.J. Willems, Eindhoven University of Technology September 29, 2009 Elias, Father of Coding Theory Textbook Encoder Encoder Properties Systematic Codes
More informationChannel Coding I. Exercises SS 2017
Channel Coding I Exercises SS 2017 Lecturer: Dirk Wübben Tutor: Shayan Hassanpour NW1, Room N 2420, Tel.: 0421/218-62387 E-mail: {wuebben, hassanpour}@ant.uni-bremen.de Universität Bremen, FB1 Institut
More informationChapter 7. Error Control Coding. 7.1 Historical background. Mikael Olofsson 2005
Chapter 7 Error Control Coding Mikael Olofsson 2005 We have seen in Chapters 4 through 6 how digital modulation can be used to control error probabilities. This gives us a digital channel that in each
More informationCSCI 2570 Introduction to Nanocomputing
CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication
More informationECE8771 Information Theory & Coding for Digital Communications Villanova University ECE Department Prof. Kevin M. Buckley Lecture Set 2 Block Codes
Kevin Buckley - 2010 109 ECE8771 Information Theory & Coding for Digital Communications Villanova University ECE Department Prof. Kevin M. Buckley Lecture Set 2 Block Codes m GF(2 ) adder m GF(2 ) multiplier
More informationError Correction and Trellis Coding
Advanced Signal Processing Winter Term 2001/2002 Digital Subscriber Lines (xdsl): Broadband Communication over Twisted Wire Pairs Error Correction and Trellis Coding Thomas Brandtner brandt@sbox.tugraz.at
More informationError Correction Methods
Technologies and Services on igital Broadcasting (7) Error Correction Methods "Technologies and Services of igital Broadcasting" (in Japanese, ISBN4-339-06-) is published by CORONA publishing co., Ltd.
More informationSIDDHARTH GROUP OF INSTITUTIONS :: PUTTUR Siddharth Nagar, Narayanavanam Road UNIT I
SIDDHARTH GROUP OF INSTITUTIONS :: PUTTUR Siddharth Nagar, Narayanavanam Road 517583 QUESTION BANK (DESCRIPTIVE) Subject with Code : CODING THEORY & TECHNIQUES(16EC3810) Course & Branch: M.Tech - DECS
More informationConvolutional Coding LECTURE Overview
MIT 6.02 DRAFT Lecture Notes Spring 2010 (Last update: March 6, 2010) Comments, questions or bug reports? Please contact 6.02-staff@mit.edu LECTURE 8 Convolutional Coding This lecture introduces a powerful
More informationError Detection & Correction
Error Detection & Correction Error detection & correction noisy channels techniques in networking error detection error detection capability retransmition error correction reconstruction checksums redundancy
More informationCOMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei
COMPSCI 650 Applied Information Theory Apr 5, 2016 Lecture 18 Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei 1 Correcting Errors in Linear Codes Suppose someone is to send
More informationSIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land
SIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land Ingmar Land, SIPCom8-1: Information Theory and Coding (2005 Spring) p.1 Overview Basic Concepts of Channel Coding Block Codes I:
More informationMath 512 Syllabus Spring 2017, LIU Post
Week Class Date Material Math 512 Syllabus Spring 2017, LIU Post 1 1/23 ISBN, error-detecting codes HW: Exercises 1.1, 1.3, 1.5, 1.8, 1.14, 1.15 If x, y satisfy ISBN-10 check, then so does x + y. 2 1/30
More informationA Brief Encounter with Linear Codes
Boise State University ScholarWorks Mathematics Undergraduate Theses Department of Mathematics 8-2014 A Brief Encounter with Linear Codes Brent El-Bakri Boise State University, brentelbakri@boisestate.edu
More informationOutline. EECS Components and Design Techniques for Digital Systems. Lec 18 Error Coding. In the real world. Our beautiful digital world.
Outline EECS 150 - Components and esign Techniques for igital Systems Lec 18 Error Coding Errors and error models Parity and Hamming Codes (SECE) Errors in Communications LFSRs Cyclic Redundancy Check
More informationFault Tolerance & Reliability CDA Chapter 2 Cyclic Polynomial Codes
Fault Tolerance & Reliability CDA 5140 Chapter 2 Cyclic Polynomial Codes - cylic code: special type of parity check code such that every cyclic shift of codeword is a codeword - for example, if (c n-1,
More informationCoding on a Trellis: Convolutional Codes
.... Coding on a Trellis: Convolutional Codes Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete November 6th, 2008 Telecommunications Laboratory (TUC) Coding on a Trellis:
More informationDr. Cathy Liu Dr. Michael Steinberger. A Brief Tour of FEC for Serial Link Systems
Prof. Shu Lin Dr. Cathy Liu Dr. Michael Steinberger U.C.Davis Avago SiSoft A Brief Tour of FEC for Serial Link Systems Outline Introduction Finite Fields and Vector Spaces Linear Block Codes Cyclic Codes
More informationSolutions or answers to Final exam in Error Control Coding, October 24, G eqv = ( 1+D, 1+D + D 2)
Solutions or answers to Final exam in Error Control Coding, October, Solution to Problem a) G(D) = ( +D, +D + D ) b) The rate R =/ and ν i = ν = m =. c) Yes, since gcd ( +D, +D + D ) =+D + D D j. d) An
More informationBinary Convolutional Codes
Binary Convolutional Codes A convolutional code has memory over a short block length. This memory results in encoded output symbols that depend not only on the present input, but also on past inputs. An
More informationNAME... Soc. Sec. #... Remote Location... (if on campus write campus) FINAL EXAM EE568 KUMAR. Sp ' 00
NAME... Soc. Sec. #... Remote Location... (if on campus write campus) FINAL EXAM EE568 KUMAR Sp ' 00 May 3 OPEN BOOK exam (students are permitted to bring in textbooks, handwritten notes, lecture notes
More information18.2 Continuous Alphabet (discrete-time, memoryless) Channel
0-704: Information Processing and Learning Spring 0 Lecture 8: Gaussian channel, Parallel channels and Rate-distortion theory Lecturer: Aarti Singh Scribe: Danai Koutra Disclaimer: These notes have not
More informationEE 229B ERROR CONTROL CODING Spring 2005
EE 229B ERROR CONTROL CODING Spring 2005 Solutions for Homework 1 1. Is there room? Prove or disprove : There is a (12,7) binary linear code with d min = 5. If there were a (12,7) binary linear code with
More informationMATH3302 Coding Theory Problem Set The following ISBN was received with a smudge. What is the missing digit? x9139 9
Problem Set 1 These questions are based on the material in Section 1: Introduction to coding theory. You do not need to submit your answers to any of these questions. 1. The following ISBN was received
More informationELEC 519A Selected Topics in Digital Communications: Information Theory. Hamming Codes and Bounds on Codes
ELEC 519A Selected Topics in Digital Communications: Information Theory Hamming Codes and Bounds on Codes Single Error Correcting Codes 2 Hamming Codes (7,4,3) Hamming code 1 0 0 0 0 1 1 0 1 0 0 1 0 1
More informationChapter 9 Fundamental Limits in Information Theory
Chapter 9 Fundamental Limits in Information Theory Information Theory is the fundamental theory behind information manipulation, including data compression and data transmission. 9.1 Introduction o For
More informationchannel of communication noise Each codeword has length 2, and all digits are either 0 or 1. Such codes are called Binary Codes.
5 Binary Codes You have already seen how check digits for bar codes (in Unit 3) and ISBN numbers (Unit 4) are used to detect errors. Here you will look at codes relevant for data transmission, for example,
More informationKnow the meaning of the basic concepts: ring, field, characteristic of a ring, the ring of polynomials R[x].
The second exam will be on Friday, October 28, 2. It will cover Sections.7,.8, 3., 3.2, 3.4 (except 3.4.), 4. and 4.2 plus the handout on calculation of high powers of an integer modulo n via successive
More informationCyclic Redundancy Check Codes
Cyclic Redundancy Check Codes Lectures No. 17 and 18 Dr. Aoife Moloney School of Electronics and Communications Dublin Institute of Technology Overview These lectures will look at the following: Cyclic
More informationLecture 3 : Introduction to Binary Convolutional Codes
Lecture 3 : Introduction to Binary Convolutional Codes Binary Convolutional Codes 1. Convolutional codes were first introduced by Elias in 1955 as an alternative to block codes. In contrast with a block
More informationECEN 604: Channel Coding for Communications
ECEN 604: Channel Coding for Communications Lecture: Introduction to Cyclic Codes Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 604: Channel Coding for Communications
More informationTurbo Codes are Low Density Parity Check Codes
Turbo Codes are Low Density Parity Check Codes David J. C. MacKay July 5, 00 Draft 0., not for distribution! (First draft written July 5, 998) Abstract Turbo codes and Gallager codes (also known as low
More informationError Correcting Codes: Combinatorics, Algorithms and Applications Spring Homework Due Monday March 23, 2009 in class
Error Correcting Codes: Combinatorics, Algorithms and Applications Spring 2009 Homework Due Monday March 23, 2009 in class You can collaborate in groups of up to 3. However, the write-ups must be done
More informationChannel capacity. Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5.
Channel capacity Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5. Exercices Exercise session 11 : Channel capacity 1 1. Source entropy Given X a memoryless
More informationDigital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
More informationConvolutional Codes. Telecommunications Laboratory. Alex Balatsoukas-Stimming. Technical University of Crete. November 6th, 2008
Convolutional Codes Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete November 6th, 2008 Telecommunications Laboratory (TUC) Convolutional Codes November 6th, 2008 1
More informationOptical Storage Technology. Error Correction
Optical Storage Technology Error Correction Introduction With analog audio, there is no opportunity for error correction. With digital audio, the nature of binary data lends itself to recovery in the event
More informationMATH Examination for the Module MATH-3152 (May 2009) Coding Theory. Time allowed: 2 hours. S = q
MATH-315201 This question paper consists of 6 printed pages, each of which is identified by the reference MATH-3152 Only approved basic scientific calculators may be used. c UNIVERSITY OF LEEDS Examination
More information4 An Introduction to Channel Coding and Decoding over BSC
4 An Introduction to Channel Coding and Decoding over BSC 4.1. Recall that channel coding introduces, in a controlled manner, some redundancy in the (binary information sequence that can be used at the
More informationChapter 5. Cyclic Codes
Wireless Information Transmission System Lab. Chapter 5 Cyclic Codes Institute of Communications Engineering National Sun Yat-sen University Outlines Description of Cyclic Codes Generator and Parity-Check
More information: Coding Theory. Notes by Assoc. Prof. Dr. Patanee Udomkavanich October 30, upattane
2301532 : Coding Theory Notes by Assoc. Prof. Dr. Patanee Udomkavanich October 30, 2006 http://pioneer.chula.ac.th/ upattane Chapter 1 Error detection, correction and decoding 1.1 Basic definitions and
More informationELEC 405/511 Error Control Coding. Binary Convolutional Codes
ELEC 405/511 Error Control Coding Binary Convolutional Codes Peter Elias (1923-2001) Coding for Noisy Channels, 1955 2 With block codes, the input data is divided into blocks of length k and the codewords
More informationLecture 2. Capacity of the Gaussian channel
Spring, 207 5237S, Wireless Communications II 2. Lecture 2 Capacity of the Gaussian channel Review on basic concepts in inf. theory ( Cover&Thomas: Elements of Inf. Theory, Tse&Viswanath: Appendix B) AWGN
More informationError Correction Review
Error Correction Review A single overall parity-check equation detects single errors. Hamming codes used m equations to correct one error in 2 m 1 bits. We can use nonbinary equations if we create symbols
More informationAssume that the follow string of bits constitutes one of the segments we which to transmit.
Cyclic Redundancy Checks( CRC) Cyclic Redundancy Checks fall into a class of codes called Algebraic Codes; more specifically, CRC codes are Polynomial Codes. These are error-detecting codes, not error-correcting
More informationELEC-E7240 Coding Methods L (5 cr)
Introduction ELEC-E7240 Coding Methods L (5 cr) Patric Östergård Department of Communications and Networking Aalto University School of Electrical Engineering Spring 2017 Patric Östergård (Aalto) ELEC-E7240
More informationEE5585 Data Compression May 2, Lecture 27
EE5585 Data Compression May 2, 2013 Lecture 27 Instructor: Arya Mazumdar Scribe: Fangying Zhang Distributed Data Compression/Source Coding In the previous class we used a H-W table as a simple example,
More informationLecture 5: Channel Capacity. Copyright G. Caire (Sample Lectures) 122
Lecture 5: Channel Capacity Copyright G. Caire (Sample Lectures) 122 M Definitions and Problem Setup 2 X n Y n Encoder p(y x) Decoder ˆM Message Channel Estimate Definition 11. Discrete Memoryless Channel
More informationLecture 18: Gaussian Channel
Lecture 18: Gaussian Channel Gaussian channel Gaussian channel capacity Dr. Yao Xie, ECE587, Information Theory, Duke University Mona Lisa in AWGN Mona Lisa Noisy Mona Lisa 100 100 200 200 300 300 400
More informationChannel Coding I. Exercises SS 2016
Channel Coding I Exercises SS 2016 Lecturer: Dirk Wübben, Carsten Bockelmann Tutor: Ahmed Emara, Matthias Woltering NW1, Room N 2400, Tel.: 0421/218-62392 E-mail: {wuebben, bockelmann, emara, woltering}@ant.uni-bremen.de
More information1 1 0, g Exercise 1. Generator polynomials of a convolutional code, given in binary form, are g
Exercise Generator polynomials of a convolutional code, given in binary form, are g 0, g 2 0 ja g 3. a) Sketch the encoding circuit. b) Sketch the state diagram. c) Find the transfer function TD. d) What
More informationAnd for polynomials with coefficients in F 2 = Z/2 Euclidean algorithm for gcd s Concept of equality mod M(x) Extended Euclid for inverses mod M(x)
Outline Recall: For integers Euclidean algorithm for finding gcd s Extended Euclid for finding multiplicative inverses Extended Euclid for computing Sun-Ze Test for primitive roots And for polynomials
More informationObjective: To become acquainted with the basic concepts of cyclic codes and some aspects of encoder implementations for them.
ECE 7670 Lecture 5 Cyclic codes Objective: To become acquainted with the basic concepts of cyclic codes and some aspects of encoder implementations for them. Reading: Chapter 5. 1 Cyclic codes Definition
More informationCyclic codes. Vahid Meghdadi Reference: Error Correction Coding by Todd K. Moon. February 2008
Cyclic codes Vahid Meghdadi Reference: Error Correction Coding by Todd K. Moon February 2008 1 Definitions Definition 1. A ring < R, +,. > is a set R with two binary operation + (addition) and. (multiplication)
More informationLecture 6 I. CHANNEL CODING. X n (m) P Y X
6- Introduction to Information Theory Lecture 6 Lecturer: Haim Permuter Scribe: Yoav Eisenberg and Yakov Miron I. CHANNEL CODING We consider the following channel coding problem: m = {,2,..,2 nr} Encoder
More informationExact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel
Exact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel Brian M. Kurkoski, Paul H. Siegel, and Jack K. Wolf Department of Electrical and Computer Engineering
More informationMa/CS 6b Class 25: Error Correcting Codes 2
Ma/CS 6b Class 25: Error Correcting Codes 2 By Adam Sheffer Recall: Codes V n the set of binary sequences of length n. For example, V 3 = 000,001,010,011,100,101,110,111. Codes of length n are subsets
More informationOne Lesson of Information Theory
Institut für One Lesson of Information Theory Prof. Dr.-Ing. Volker Kühn Institute of Communications Engineering University of Rostock, Germany Email: volker.kuehn@uni-rostock.de http://www.int.uni-rostock.de/
More informationHamming Codes 11/17/04
Hamming Codes 11/17/04 History In the late 1940 s Richard Hamming recognized that the further evolution of computers required greater reliability, in particular the ability to not only detect errors, but
More informationChapter 6. BCH Codes
Chapter 6 BCH Codes Description of the Codes Decoding of the BCH Codes Outline Implementation of Galois Field Arithmetic Implementation of Error Correction Nonbinary BCH Codes and Reed-Solomon Codes Weight
More informationECE Information theory Final (Fall 2008)
ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1
More informationMATH3302. Coding and Cryptography. Coding Theory
MATH3302 Coding and Cryptography Coding Theory 2010 Contents 1 Introduction to coding theory 2 1.1 Introduction.......................................... 2 1.2 Basic definitions and assumptions..............................
More informationTurbo Compression. Andrej Rikovsky, Advisor: Pavol Hanus
Turbo Compression Andrej Rikovsky, Advisor: Pavol Hanus Abstract Turbo codes which performs very close to channel capacity in channel coding can be also used to obtain very efficient source coding schemes.
More informationError Correcting Codes Questions Pool
Error Correcting Codes Questions Pool Amnon Ta-Shma and Dean Doron January 3, 018 General guidelines The questions fall into several categories: (Know). (Mandatory). (Bonus). Make sure you know how to
More informationX 1 : X Table 1: Y = X X 2
ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access
More informationChapter 3 Linear Block Codes
Wireless Information Transmission System Lab. Chapter 3 Linear Block Codes Institute of Communications Engineering National Sun Yat-sen University Outlines Introduction to linear block codes Syndrome and
More informationexercise in the previous class (1)
exercise in the previous class () Consider an odd parity check code C whose codewords are (x,, x k, p) with p = x + +x k +. Is C a linear code? No. x =, x 2 =x =...=x k = p =, and... is a codeword x 2
More informationBASICS OF DETECTION AND ESTIMATION THEORY
BASICS OF DETECTION AND ESTIMATION THEORY 83050E/158 In this chapter we discuss how the transmitted symbols are detected optimally from a noisy received signal (observation). Based on these results, optimal
More informationDigital Band-pass Modulation PROF. MICHAEL TSAI 2011/11/10
Digital Band-pass Modulation PROF. MICHAEL TSAI 211/11/1 Band-pass Signal Representation a t g t General form: 2πf c t + φ t g t = a t cos 2πf c t + φ t Envelope Phase Envelope is always non-negative,
More informationReed-Solomon codes. Chapter Linear codes over finite fields
Chapter 8 Reed-Solomon codes In the previous chapter we discussed the properties of finite fields, and showed that there exists an essentially unique finite field F q with q = p m elements for any prime
More informationCoding Theory: Linear-Error Correcting Codes Anna Dovzhik Math 420: Advanced Linear Algebra Spring 2014
Anna Dovzhik 1 Coding Theory: Linear-Error Correcting Codes Anna Dovzhik Math 420: Advanced Linear Algebra Spring 2014 Sharing data across channels, such as satellite, television, or compact disc, often
More informationShannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
More informationNotes 3: Stochastic channels and noisy coding theorem bound. 1 Model of information communication and noisy channel
Introduction to Coding Theory CMU: Spring 2010 Notes 3: Stochastic channels and noisy coding theorem bound January 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami We now turn to the basic
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Discussion 6A Solution
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Discussion 6A Solution 1. Polynomial intersections Find (and prove) an upper-bound on the number of times two distinct degree
More informationECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
More information9 THEORY OF CODES. 9.0 Introduction. 9.1 Noise
9 THEORY OF CODES Chapter 9 Theory of Codes After studying this chapter you should understand what is meant by noise, error detection and correction; be able to find and use the Hamming distance for a
More informationOn Compression Encrypted Data part 2. Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University
On Compression Encrypted Data part 2 Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University 1 Brief Summary of Information-theoretic Prescription At a functional
More information: Error Correcting Codes. November 2017 Lecture 2
03683072: Error Correcting Codes. November 2017 Lecture 2 Polynomial Codes and Cyclic Codes Amnon Ta-Shma and Dean Doron 1 Polynomial Codes Fix a finite field F q. For the purpose of constructing polynomial
More informationU Logo Use Guidelines
COMP2610/6261 - Information Theory Lecture 22: Hamming Codes U Logo Use Guidelines Mark Reid and Aditya Menon logo is a contemporary n of our heritage. presents our name, d and our motto: arn the nature
More informationEC2252 COMMUNICATION THEORY UNIT 5 INFORMATION THEORY
EC2252 COMMUNICATION THEORY UNIT 5 INFORMATION THEORY Discrete Messages and Information Content, Concept of Amount of Information, Average information, Entropy, Information rate, Source coding to increase
More informationNoisy channel communication
Information Theory http://www.inf.ed.ac.uk/teaching/courses/it/ Week 6 Communication channels and Information Some notes on the noisy channel setup: Iain Murray, 2012 School of Informatics, University
More informationLecture 4: Linear Codes. Copyright G. Caire 88
Lecture 4: Linear Codes Copyright G. Caire 88 Linear codes over F q We let X = F q for some prime power q. Most important case: q =2(binary codes). Without loss of generality, we may represent the information
More informationLecture 12: November 6, 2017
Information and Coding Theory Autumn 017 Lecturer: Madhur Tulsiani Lecture 1: November 6, 017 Recall: We were looking at codes of the form C : F k p F n p, where p is prime, k is the message length, and
More informationCapacity of a channel Shannon s second theorem. Information Theory 1/33
Capacity of a channel Shannon s second theorem Information Theory 1/33 Outline 1. Memoryless channels, examples ; 2. Capacity ; 3. Symmetric channels ; 4. Channel Coding ; 5. Shannon s second theorem,
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More information