Part III Advanced Coding Techniques
|
|
|
- Barnaby Patrick
- 5 years ago
- Views:
Transcription
1 Part III Advanced Coding Techniques José Vieira SPL Signal Processing Laboratory Departamento de Electrónica, Telecomunicações e Informática / IEETA Universidade de Aveiro, Portugal 2010 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
2 Ocean Store The Infinite disk ID ID Client DATA DATA Client ID DATA Client José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
3 Error Correction Code Unpredictable Channel Conditions Coder Erasure Channel Decoder m c e y m^ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
4 Digital Fountains Questions Dissemination of data: How to encode data files to distribute them by a huge number of disks around the world? Resilience: How can we encode data files to make any encoded data useful? Ratelessness: How to generate an infinite number of codewords? Answer: random coding! José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
5 Digital Fountains Questions Dissemination of data: How to encode data files to distribute them by a huge number of disks around the world? Resilience: How can we encode data files to make any encoded data useful? Ratelessness: How to generate an infinite number of codewords? Answer: random coding! José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
6 Digital Fountains Questions Dissemination of data: How to encode data files to distribute them by a huge number of disks around the world? Resilience: How can we encode data files to make any encoded data useful? Ratelessness: How to generate an infinite number of codewords? Answer: random coding! José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
7 Digital Fountains Questions Dissemination of data: How to encode data files to distribute them by a huge number of disks around the world? Resilience: How can we encode data files to make any encoded data useful? Ratelessness: How to generate an infinite number of codewords? Answer: random coding! José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
8 Essential Textbooks and Papers David J. C. Mackay, Information Theory, Inference and Learning Algorithms, Cambridge, 2004 Mackay, D. J. C., Fountain Codes, IEE Proceedings - Communications, Vol.152, N.6, pp , December, 2005 Luby, Michael, LT Codes, Proceedings of the 43rd Annual IEEE Symposium on Foundations of Computer Science (FOCS 02), pp , IEEE, November, 2002 Maymounkov, Petar, Online Codes, New York University, New York, November, 2002 Fragouli, Christina, Boudec, Jean-Yves Le, and Widmer, Jörg, Network Coding: Na Instant Primer, ACM SIGCOMM Computer Communication Review, Vol.36, N.1, pp.63-68, January, 2006 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
9 Suplementar Papers Shokrollahi, Amin, Raptor Codes, IEEE Transactions on Information Theory, Vol.52, N.6, pp , June, 2006 Shamai, Shlomo, Telatar, I. Emre, and Verdú, Sergio, Fountain Capacity, IEEE Transactions on Information Theory, Vol.53, N.11, pp , November, 2007 Dimakis, Alexandros G., Prabhakaran, Vinod, and Ramchandran, Kannan, Decentralized Erasure Codes for Distributed Networked Storage, IEEE Transactions on Information Theory, Vol.52, N.6, pp , June, 2006 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
10 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
11 Linear Codes Coding with Real Numbers One linear combination [ ] c1 = [ g 1 ] m K K 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
12 Linear Codes Coding with Real Numbers Two linear combinations [ ] [ ] c1 g1 = m c 2 g K K 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
13 Linear Codes Coding with Real Numbers Adding redundancy to a signal N > K c 1 c 2. c N N 1 g 1 g 2 = m. g N N K K 1 c = Gm José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
14 Linear Codes Coding with Real Numbers Adding redundancy to a signal N > K c 1 c 2. c N N 1 g 1 g 2 = m. g N N K K 1 c = Gm José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
15 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
16 Erasures Erasures Lost samples at known positions How to recover the message m from incomplete c? c 1 c 2 c 3 c 4 c 5 N 1 g 1 g 2 = g 3 m g 4 g 5 K 1 N K José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
17 Erasures If only L samples of c are received we have: c 1 g 1 Xc 2 Xg 2 c 3 = g 3 m Xc 4 Xg 4 g 5 K 1 c 5 N 1 N K Define J = {1, 3, 5} as the set of the received samples José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
18 Erasures Solve the following system of equations to obtain the original signal m c 1 c 3 c 5 L 1 g 1 = g 3 m g 5 L K K 1 c(j) = G(J)m José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
19 Erasures Depending on L (number of received samples), there are three possible situations L < K Underdetermined system of equations. In general not enough information to recover m uniquely. Additional restrictions can be imposed in order to get an unique solution. L = K Determined system of equations, one solution (max.). ˆm = G(J) 1 c(j) L > K Overdetermined system of equations. In general there is not an unique solution. In the field R we can choose the least squares solution, the one that best approximates all the equations in L 2 sense. ˆm = (G(J) T G(J)) 1 G(J) T c(j) = G(J) c(j) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
20 Erasures Depending on L (number of received samples), there are three possible situations L < K Underdetermined system of equations. In general not enough information to recover m uniquely. Additional restrictions can be imposed in order to get an unique solution. L = K Determined system of equations, one solution (max.). ˆm = G(J) 1 c(j) L > K Overdetermined system of equations. In general there is not an unique solution. In the field R we can choose the least squares solution, the one that best approximates all the equations in L 2 sense. ˆm = (G(J) T G(J)) 1 G(J) T c(j) = G(J) c(j) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
21 Erasures Depending on L (number of received samples), there are three possible situations L < K Underdetermined system of equations. In general not enough information to recover m uniquely. Additional restrictions can be imposed in order to get an unique solution. L = K Determined system of equations, one solution (max.). ˆm = G(J) 1 c(j) L > K Overdetermined system of equations. In general there is not an unique solution. In the field R we can choose the least squares solution, the one that best approximates all the equations in L 2 sense. ˆm = (G(J) T G(J)) 1 G(J) T c(j) = G(J) c(j) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
22 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
23 Correcting Errors Errors Errors: Lost samples at unknown positions m c y Coder Decoder m^ How can we find the errors positions? e José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
24 Correcting Errors Consider an N N orthogonal matrix partitioned in the following way: F = G H N K N (N K) [ G T H T ] [ G [GH] = T G G T H H T G H T H So we have H T G = 0 ] = [ I 0 0 I ] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
25 Correcting Errors Consider an N N orthogonal matrix partitioned in the following way: F = G H N K N (N K) [ G T H T ] [ G [GH] = T G G T H H T G H T H So we have H T G = 0 ] = [ I 0 0 I ] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
26 Correcting Errors Consider an N N orthogonal matrix partitioned in the following way: F = G H N K N (N K) [ G T H T ] [ G [GH] = T G G T H H T G H T H So we have H T G = 0 ] = [ I 0 0 I ] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
27 Correcting Errors Received y and given F, if there are no errors, then y = c = Gm We can use the matrix H to test the received signal y s = H T y = H T c = H}{{ T G} m = 0 =0 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
28 Correcting Errors Received y and given F, if there are no errors, then y = c = Gm We can use the matrix H to test the received signal y s = H T y = H T c = H}{{ T G} m = 0 =0 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
29 Correcting Errors The received vector y is a corrupted version of c at unknown positions, example y 1 c 1 0 y 2 y 3 y 4 = c 2 c 3 c e 3 0 y 5 c 5 0 y = c + e The matrix H is used to verify that the received signal y is a codeword s = H T y = H T c + H T e = H T e 0 The syndrome s is a linear combination of the columns of H T where the e i are the coefficients. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
30 Correcting Errors The received vector y is a corrupted version of c at unknown positions, example y 1 c 1 0 y 2 y 3 y 4 = c 2 c 3 c e 3 0 y 5 c 5 0 y = c + e The matrix H is used to verify that the received signal y is a codeword s = H T y = H T c + H T e = H T e 0 The syndrome s is a linear combination of the columns of H T where the e i are the coefficients. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
31 Correcting Errors The received vector y is a corrupted version of c at unknown positions, example y 1 c 1 0 y 2 y 3 y 4 = c 2 c 3 c e 3 0 y 5 c 5 0 y = c + e The matrix H is used to verify that the received signal y is a codeword s = H T y = H T c + H T e = H T e 0 The syndrome s is a linear combination of the columns of H T where the e i are the coefficients. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
32 Correcting Errors The received vector y is a corrupted version of c at unknown positions, example y 1 c 1 0 y 2 y 3 y 4 = c 2 c 3 c e 3 0 y 5 c 5 0 y = c + e The matrix H is used to verify that the received signal y is a codeword s = H T y = H T c + H T e = H T e 0 The syndrome s is a linear combination of the columns of H T where the e i are the coefficients. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
33 Correcting Errors Example: repetition code c 2 2 c = 2 c 1 = c 2 Repetition code: [ G H ] 1 F = Suppose we[ code 1 c = Gm = 1 ] [2] = [ 2 2 ] m = [ 2] c 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
34 Correcting Errors Example: repetition code c 2 c 1 = c 2 3 y = 2 Repetition code: [ G H ] 1 F = Suppose we[ code 1 c = Gm = 1 ] [2] = [ 2 2 If [ an] error [ occurs ] y [ = c ] + e = = ] c 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
35 g Correcting Errors Example: repetition code c 2 h y c 1 = c 2 c 1 Repetition code: [ G H ] 1 F = Suppose we[ code 1 c = Gm = 1 ] [2] = [ 2 2 If [ an] error [ occurs ] y [ = c ] + e = = We can verify that the received vector [ y has ] an error 3 H T y = [1 1] = ] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
36 Linear Codes Coding with Real Numbers c 3 C o d e w o r d s s u b s p a c e c 2 M e s s a g e s u b s p a c e c 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
37 Linear Codes Coding with Real Numbers c 3 1 G = 0 1/ / 2 1/ 2 H = 1/ 2 1 g 2 g 1 c 2 h c 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
38 Linear Codes Coding with Real Numbers c 3 1 c = 1 1 c 2 c 1 1 m = 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
39 Linear Codes Coding with Real Numbers Linear code: F = G /2 1/2 To code the message m = [ 1 1 If an error occurs y = c + e = H ] we get c = Gm = /2 To verify that the received vector y has an error 1 s = H T y = [ ] 1 1/2 = = 1 1 1/2 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
40 Linear Codes Coding with Real Numbers Linear code: F = G /2 1/2 To code the message m = [ 1 1 If an error occurs y = c + e = H ] we get c = Gm = /2 To verify that the received vector y has an error 1 s = H T y = [ ] 1 1/2 = = 1 1 1/2 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
41 Linear Codes Coding with Real Numbers Linear code: F = G /2 1/2 To code the message m = [ 1 1 If an error occurs y = c + e = H ] we get c = Gm = /2 To verify that the received vector y has an error 1 s = H T y = [ ] 1 1/2 = = 1 1 1/2 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
42 Linear Codes Coding with Real Numbers Linear code: F = G /2 1/2 To code the message m = [ 1 1 If an error occurs y = c + e = H ] we get c = Gm = /2 To verify that the received vector y has an error 1 s = H T y = [ ] 1 1/2 = = 1 1 1/2 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
43 Correcting Errors The Syndrome as a linear combination of columns of H T s 1. s N K = H T h 1 h 2 h N e 1 e 2 e 3.. e N s 1. s N K = e 1 h 1 + e 2 h e N h N José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
44 Correcting Errors The Syndrome as a linear combination of columns of H T s 1. s N K = H T h 1 h 2 h N e 1 e 2 e 3.. e N s 1. s N K = e 1 h 1 + e 2 h e N h N José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
45 Correcting Errors Brute force approach Problem Find the linear combination of vectors h i that best approximates s The error vector e is a sparse vector, so we want the sparsest solution Brute force approach: test all error patterns Equivalent to solve the following optimization problem: Problem min e 0 s.t. s = H T e José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
46 Correcting Errors Brute force approach Problem Find the linear combination of vectors h i that best approximates s The error vector e is a sparse vector, so we want the sparsest solution Brute force approach: test all error patterns Equivalent to solve the following optimization problem: Problem min e 0 s.t. s = H T e José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
47 Correcting Errors Brute force approach Problem Find the linear combination of vectors h i that best approximates s The error vector e is a sparse vector, so we want the sparsest solution Brute force approach: test all error patterns Equivalent to solve the following optimization problem: Problem min e 0 s.t. s = H T e José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
48 Correcting Errors Brute force approach Search for the best linear combination s 1. = e 1 h 1 + e 2 h e N h N s N K Find the minimum of s ŝ 2 for all error patterns and each number of errors n N o Combinations 1 ŝ = e i h i N ( 2 ŝ = e i h i + e j h N ) j 2... L ŝ = i=j e L ( N ) ih i n=1 n This is a NP hard combinatorial problem. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
49 Correcting Errors Brute force approach Search for the best linear combination s 1. = e 1 h 1 + e 2 h e N h N s N K Find the minimum of s ŝ 2 for all error patterns and each number of errors n N o Combinations 1 ŝ = e i h i N ( 2 ŝ = e i h i + e j h N ) j 2... L ŝ = i=j e L ( N ) ih i n=1 n This is a NP hard combinatorial problem. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
50 Correcting Errors Brute force approach Search for the best linear combination s 1. = e 1 h 1 + e 2 h e N h N s N K Find the minimum of s ŝ 2 for all error patterns and each number of errors n N o Combinations 1 ŝ = e i h i N ( 2 ŝ = e i h i + e j h N ) j 2... L ŝ = i=j e L ( N ) ih i n=1 n This is a NP hard combinatorial problem. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
51 Avoiding the combinatorial explosion Solutions Solution 1: Use coding matrices G and parity check matrices H with a convenient structure: Hamming DFT - (BCH cyclic codes) DCT etc. Solution 2: Use random matrices and L 1 minimization to obtain a sparse solution for the error vector Solution 3: Use sparse random matrices and fast algorithms that take advantage of sparsity (LDPC and LT codes) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
52 Avoiding the combinatorial explosion Solutions Solution 1: Use coding matrices G and parity check matrices H with a convenient structure: Hamming DFT - (BCH cyclic codes) DCT etc. Solution 2: Use random matrices and L 1 minimization to obtain a sparse solution for the error vector Solution 3: Use sparse random matrices and fast algorithms that take advantage of sparsity (LDPC and LT codes) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
53 Avoiding the combinatorial explosion Solutions Solution 1: Use coding matrices G and parity check matrices H with a convenient structure: Hamming DFT - (BCH cyclic codes) DCT etc. Solution 2: Use random matrices and L 1 minimization to obtain a sparse solution for the error vector Solution 3: Use sparse random matrices and fast algorithms that take advantage of sparsity (LDPC and LT codes) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
54 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
55 Solution 1 Coding with structured matrices Choose β i as the roots of unity in any field (finite or not). Note that the roots of unity are the solutions of a n = 1 Construct the Vandermonde matrix β0 0 β1 0 βn 1 0 β0 1 β1 1 βn with β i β j β0 N 1 β1 N 1 β N 1 N 1 These codes can correct at least N K 2 errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
56 Solution 1 Coding with the DFT In a Galois field, only certain values of N have roots of unity In the Complex field C the roots of unity of order N are β i = e j 2π N i - DFT matrix These codes are known as the BCH codes A codeword c is generated by evaluating the IDFT of a zero padded message vector m c = IDFT m 0 0 = G m José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
57 Solution 1 Coding with the DFT The syndrome s is part of the DFT of e s = H T e The complete equation will be [ ] [ ] s G T = e s If we have a way of obtaining s then we could calculate the error e by inverse transform. As e is sparse with only L values different from zero, s is a linear combination of only L components: s n = H T L a k s n k k=1 If we know 2L values of the syndrome we can correct L errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
58 Solution 1 Coding with the DFT The syndrome s is part of the DFT of e s = H T e The complete equation will be [ ] [ ] s G T = e s If we have a way of obtaining s then we could calculate the error e by inverse transform. As e is sparse with only L values different from zero, s is a linear combination of only L components: s n = H T L a k s n k k=1 If we know 2L values of the syndrome we can correct L errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
59 Solution 1 Coding with the DFT The syndrome s is part of the DFT of e s = H T e The complete equation will be [ ] [ ] s G T = e s If we have a way of obtaining s then we could calculate the error e by inverse transform. As e is sparse with only L values different from zero, s is a linear combination of only L components: s n = H T L a k s n k k=1 If we know 2L values of the syndrome we can correct L errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
60 Solution 1 Coding with the DFT The syndrome s is part of the DFT of e s = H T e The complete equation will be [ ] [ ] s G T = e s If we have a way of obtaining s then we could calculate the error e by inverse transform. As e is sparse with only L values different from zero, s is a linear combination of only L components: s n = H T L a k s n k k=1 If we know 2L values of the syndrome we can correct L errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
61 Solution 1 Coding with the DFT The syndrome s is part of the DFT of e s = H T e The complete equation will be [ ] [ ] s G T = e s If we have a way of obtaining s then we could calculate the error e by inverse transform. As e is sparse with only L values different from zero, s is a linear combination of only L components: s n = H T L a k s n k k=1 If we know 2L values of the syndrome we can correct L errors José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
62 Solution 1 Decoding example Error vector Consider the following example: N = 16 Two errors at J = {2, 14} n José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
63 Solution 1 Decoding example Syndrome In this example, due to the error vector symmetry, the syndrome is a real vector We have the following difference equation 0 s n = a 1 s n 1 + a 2 s n k We have two unknowns and with the four known elements of the syndrome we can form { s3 = a 1 s 2 + a 2 s 1 s 4 = a 1 s 3 + a 2 s 2 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
64 Solution 1 Stability Problems Due to the structure of the coding matrix G and the parity check matrix H, the syndrome reconstruction is very sensitive to burst of errors This is a direct consequence of the structure of the matrix H. Contiguous row vectors are almost colinear, leading to bad conditioned system of equations To improve the reconstruction stability, we have to modify the structure of H On real number codes we have stability problems. On finite fields those codes behave poorly for burst errors Conclusion: The coding matrix structure is fundamental for both fields: Real and Finite José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
65 Solution 1 Stability Problems Due to the structure of the coding matrix G and the parity check matrix H, the syndrome reconstruction is very sensitive to burst of errors This is a direct consequence of the structure of the matrix H. Contiguous row vectors are almost colinear, leading to bad conditioned system of equations To improve the reconstruction stability, we have to modify the structure of H On real number codes we have stability problems. On finite fields those codes behave poorly for burst errors Conclusion: The coding matrix structure is fundamental for both fields: Real and Finite José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
66 Solution 1 Stability Problems Due to the structure of the coding matrix G and the parity check matrix H, the syndrome reconstruction is very sensitive to burst of errors This is a direct consequence of the structure of the matrix H. Contiguous row vectors are almost colinear, leading to bad conditioned system of equations To improve the reconstruction stability, we have to modify the structure of H On real number codes we have stability problems. On finite fields those codes behave poorly for burst errors Conclusion: The coding matrix structure is fundamental for both fields: Real and Finite José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
67 Solution 1 Stability Problems Due to the structure of the coding matrix G and the parity check matrix H, the syndrome reconstruction is very sensitive to burst of errors This is a direct consequence of the structure of the matrix H. Contiguous row vectors are almost colinear, leading to bad conditioned system of equations To improve the reconstruction stability, we have to modify the structure of H On real number codes we have stability problems. On finite fields those codes behave poorly for burst errors Conclusion: The coding matrix structure is fundamental for both fields: Real and Finite José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
68 Solution 1 Stability Problems Due to the structure of the coding matrix G and the parity check matrix H, the syndrome reconstruction is very sensitive to burst of errors This is a direct consequence of the structure of the matrix H. Contiguous row vectors are almost colinear, leading to bad conditioned system of equations To improve the reconstruction stability, we have to modify the structure of H On real number codes we have stability problems. On finite fields those codes behave poorly for burst errors Conclusion: The coding matrix structure is fundamental for both fields: Real and Finite José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
69 Solution 1 + 1/2 Turbo Codes The first attempt to randomise the coding matrix structure was achieved with turbo codes The message is codded with two different coding matrices usually the DFT and a column permuted version Those codes perform better than the BCH codes for both fields. They come close to the Shannon limit DFT coder C 0 m P DFT coder C 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
70 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
71 Solution 2 Coding with random matrices Consider a N K random real number coding matrix G c = Gm To correct errors we need to evaluate the syndrome with a parity check matrix H which must obey the condition H T G = 0 The columns of H should be orthogonal to the columns of G. This can be obtained by applying the Gram-Schmidt orthogonalization algorithm to a N N random matrix José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
72 Solution 2 Coding with random matrices Problem How to solve the underdetermined system of equations s = H T e As H has no structure a general method must be found Additional restrictions must be applied to the vector e in order to define an unique solution, e.g.: Minimum energy - min e 2 (L 2 norm) Sparsest - min e 0 (L 0 pseudonorm) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
73 Solution 2 Coding with random matrices Problem How to solve the underdetermined system of equations s = H T e As H has no structure a general method must be found Additional restrictions must be applied to the vector e in order to define an unique solution, e.g.: Minimum energy - min e 2 (L 2 norm) Sparsest - min e 0 (L 0 pseudonorm) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
74 Solution 2 Coding with random matrices Problem How to solve the underdetermined system of equations s = H T e As H has no structure a general method must be found Additional restrictions must be applied to the vector e in order to define an unique solution, e.g.: Minimum energy - min e 2 (L 2 norm) Sparsest - min e 0 (L 0 pseudonorm) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
75 Solution 2 Coding with random matrices Problem How to solve the underdetermined system of equations s = H T e As H has no structure a general method must be found Additional restrictions must be applied to the vector e in order to define an unique solution, e.g.: Minimum energy - min e 2 (L 2 norm) Sparsest - min e 0 (L 0 pseudonorm) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
76 Solution 2 Coding with random matrices Problem How to solve the underdetermined system of equations s = H T e As H has no structure a general method must be found Additional restrictions must be applied to the vector e in order to define an unique solution, e.g.: Minimum energy - min e 2 (L 2 norm) Sparsest - min e 0 (L 0 pseudonorm) José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
77 Solution 2 L 0 to L 1 equivalence Donoho and Elad in 2001 founded empirically and theoretically that instead of solving the hard L 0 problem to find the sparsest solution Problem min e 0 s.t. s = H T e They could solve the easiest L 1 problem and under certain conditions, still obtain the same sparsest solution Problem min e 1 s.t. s = H T e This problem can be solved by Linear Programing using the Simplex algorithm or Interior Point methods José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
78 Solution 2 L 0 to L 1 equivalence Donoho and Elad in 2001 founded empirically and theoretically that instead of solving the hard L 0 problem to find the sparsest solution Problem min e 0 s.t. s = H T e They could solve the easiest L 1 problem and under certain conditions, still obtain the same sparsest solution Problem min e 1 s.t. s = H T e This problem can be solved by Linear Programing using the Simplex algorithm or Interior Point methods José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
79 Solution 2 L 0 to L 1 equivalence Donoho and Elad in 2001 founded empirically and theoretically that instead of solving the hard L 0 problem to find the sparsest solution Problem min e 0 s.t. s = H T e They could solve the easiest L 1 problem and under certain conditions, still obtain the same sparsest solution Problem min e 1 s.t. s = H T e This problem can be solved by Linear Programing using the Simplex algorithm or Interior Point methods José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
80 Solution 2 Coding with random matrices We can write the equation to solve in the following form s 1. s N K = e 1 h 1 + e 2 h e N h N The syndrome s is a linear combination of L vectors h i We want to find the linear combination of vectors h i that better explains the syndrome using the smallest number of vectors h i José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
81 Solution 2 Coding with random matrices L < 1 + 1/M(HT ) 2 = ebp ebp is the Equivalent Break Point and is an estimate of the maximum number of correctable errors M(A) is the mutual incoherence of matrix A Definition M(H T ) = max i j hi T h j, such that hk 2 = 1 How to choose H? All the sets of N K columns of H T should be linearly independent Ideally, hi T h j 0 i j José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
82 Solution 2 Coding with random matrices L < 1 + 1/M(HT ) 2 = ebp ebp is the Equivalent Break Point and is an estimate of the maximum number of correctable errors M(A) is the mutual incoherence of matrix A Definition M(H T ) = max i j hi T h j, such that hk 2 = 1 How to choose H? All the sets of N K columns of H T should be linearly independent Ideally, hi T h j 0 i j José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
83 Solution 2 Coding with random matrices L < 1 + 1/M(HT ) 2 = ebp ebp is the Equivalent Break Point and is an estimate of the maximum number of correctable errors M(A) is the mutual incoherence of matrix A Definition M(H T ) = max i j hi T h j, such that hk 2 = 1 How to choose H? All the sets of N K columns of H T should be linearly independent Ideally, hi T h j 0 i j José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
84 Solution 2 Coding with random matrices L < 1 + 1/M(HT ) 2 = ebp ebp is the Equivalent Break Point and is an estimate of the maximum number of correctable errors M(A) is the mutual incoherence of matrix A Definition M(H T ) = max i j hi T h j, such that hk 2 = 1 How to choose H? All the sets of N K columns of H T should be linearly independent Ideally, hi T h j 0 i j José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
85 Solution 2 Coding with random matrices L < 1 + 1/M(HT ) 2 = ebp ebp is the Equivalent Break Point and is an estimate of the maximum number of correctable errors M(A) is the mutual incoherence of matrix A Definition M(H T ) = max i j hi T h j, such that hk 2 = 1 How to choose H? All the sets of N K columns of H T should be linearly independent Ideally, hi T h j 0 i j José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
86 Solution 2 Coding with random matrices Random matrices are the solution José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
87 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
88 Solution 3 Coding with random sparse matrices With random sparse matrices is possible to find efficient algorithms to code and decode. This algorithms make use of the sparsity and avoid the slower L1 optimisation We will introduce two different types of codes that uses sparse matrices LDPC codes Low-Density Parity-Check codes [Gallager 1968] LT codes The first rateless erasure codes [Luby 2002] Online codes Almost the first rateless erasure code [Maymounkov 2002] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
89 Solution 3 Coding with random sparse matrices With random sparse matrices is possible to find efficient algorithms to code and decode. This algorithms make use of the sparsity and avoid the slower L1 optimisation We will introduce two different types of codes that uses sparse matrices LDPC codes Low-Density Parity-Check codes [Gallager 1968] LT codes The first rateless erasure codes [Luby 2002] Online codes Almost the first rateless erasure code [Maymounkov 2002] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
90 Solution 3 Coding with random sparse matrices With random sparse matrices is possible to find efficient algorithms to code and decode. This algorithms make use of the sparsity and avoid the slower L1 optimisation We will introduce two different types of codes that uses sparse matrices LDPC codes Low-Density Parity-Check codes [Gallager 1968] LT codes The first rateless erasure codes [Luby 2002] Online codes Almost the first rateless erasure code [Maymounkov 2002] José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
91 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
92 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
93 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
94 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
95 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
96 Solution 3 LDPC codes The parity check matrix is highly sparse: the number of nonzero elements grows linearly with N Due to sparsity low complexity algorithms exists The parity check matrix can be generated randomly but must obey certain rules Usually N is very large (1000 to or more) Usually the coding matrix is not sparse, which implies a coding complexity quadratic with N As N becomes large the LDPC codes approach the Shannon limit [MacKay1999] An optimal LDPC code can get within db of channel capacity José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
97 Solution 3 LDPC codes example The LDPC parity check matrix can be represented by a Tanner graph H T s = H T e e 1 e 2 e 3 e 4 e 5 e 6 e 7 e 8 s 1 s 2 s 3 s 4 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
98 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
99 Fountain Codes What is a Digital Fountain? Is a new paradigm for data transmission that changes the standard approach where a user must receive an ordered stream of data symbols to one where the user must receive enough symbols to reconstruct the original information. With a Digital Fountain is possible to generate an infinite data stream from a K symbol file. Once the receiver gets any K symbols from the stream it can reconstruct the original message. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
100 Fountain Codes Digital Fountain? The name Digital Fountain comes from the analogy with a water fountain filling a glass of water. The glass must be filled up, not with some specific drops of water. José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
101 Fountain Codes Concept One linear combination [ ] c1 = [ g 1 ] m K K 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
102 Fountain Codes Concept Two linear combinations [ ] [ ] c1 g1 = m c 2 g K K 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
103 Fountain Codes Concept Infinite number of linear combinations c 1 g 1 c 2 = g 2 m.. N 1 N K K 1 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
104 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
105 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
106 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
107 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
108 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
109 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
110 Fountain Codes Concept On Fountain Codes, each line g i of G is generated online Each g i has only a finite number of 1 s (degree) The number of 1 s is a random variable with distribution ρ The symbols to combine (XOR) are chosen randomly We can generate linear combinations as needed The receiver can be filled with codewords until it is sufficient to decode the original message The K original symbols can be recovered from K(1 + ɛ) coded symbols with probability 1 δ José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
111 Online Code Decoding 1 Find a codeword c with all message symbols decoded except one 2 Recover that message symbol m x = c m 1 m 2 m i 1 where m 1, m 2,..., m i 1 are the recovered symbols associated with c 3 Apply the previous steps until no more message symbols left José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
112 Online Code Decoding Example c 0 c 1 c 2 c 3 Received codewords s 0 s 1 s 2 1 s 0 = c 0 Symbols to decode José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
113 Online Code Decoding Example c 0 c 1 c 2 c 3 s 0 s 1 s s 1 = c 1 s 0 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
114 Online Code Decoding Example c 0 c 1 c 2 c 3 s 0 s 1 s s 2 = c 2 s 0 José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
115 Online Code Decoding Example c 0 c 1 c 2 c 3 s 0 s 1 s Done José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
116 Online Code Distribution Example 0.35 Distribution ρ with δ = 0.05 ε = degree José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
117 Online Code Decoding simulation 1 δ = 0.05 ε = Percent of decoded x Experimental (1 δ) Test number José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
118 Outline 1 Linear Codes in any Field Correcting Erasures Correcting Errors 2 Coding Matrices Structured Matrices Random Matrices Sparse Random Matrices 3 Coding with Sparse Random Matrices Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
119 Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
120 Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
121 Fountain Codes Applications José Vieira (IEETA, Univ. Aveiro) MAPtele / 68
Cyclic Redundancy Check Codes
Cyclic Redundancy Check Codes Lectures No. 17 and 18 Dr. Aoife Moloney School of Electronics and Communications Dublin Institute of Technology Overview These lectures will look at the following: Cyclic
Introduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
ECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
An Introduction to Low Density Parity Check (LDPC) Codes
An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,
Fountain Uncorrectable Sets and Finite-Length Analysis
Fountain Uncorrectable Sets and Finite-Length Analysis Wen Ji 1, Bo-Wei Chen 2, and Yiqiang Chen 1 1 Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology, Chinese
Quasi-cyclic Low Density Parity Check codes with high girth
Quasi-cyclic Low Density Parity Check codes with high girth, a work with Marta Rossi, Richard Bresnan, Massimilliano Sala Summer Doctoral School 2009 Groebner bases, Geometric codes and Order Domains Dept
LDPC Codes. Intracom Telecom, Peania
LDPC Codes Alexios Balatsoukas-Stimming and Athanasios P. Liavas Technical University of Crete Dept. of Electronic and Computer Engineering Telecommunications Laboratory December 16, 2011 Intracom Telecom,
Convergence analysis for a class of LDPC convolutional codes on the erasure channel
Convergence analysis for a class of LDPC convolutional codes on the erasure channel Sridharan, Arvind; Lentmaier, Michael; Costello Jr., Daniel J.; Zigangirov, Kamil Published in: [Host publication title
On the minimum distance of LDPC codes based on repetition codes and permutation matrices 1
Fifteenth International Workshop on Algebraic and Combinatorial Coding Theory June 18-24, 216, Albena, Bulgaria pp. 168 173 On the minimum distance of LDPC codes based on repetition codes and permutation
Lecture 14 October 22
EE 2: Coding for Digital Communication & Beyond Fall 203 Lecture 4 October 22 Lecturer: Prof. Anant Sahai Scribe: Jingyan Wang This lecture covers: LT Code Ideal Soliton Distribution 4. Introduction So
On Encoding Symbol Degrees of Array BP-XOR Codes
On Encoding Symbol Degrees of Array BP-XOR Codes Maura B. Paterson Dept. Economics, Math. & Statistics Birkbeck University of London Email: m.paterson@bbk.ac.uk Douglas R. Stinson David R. Cheriton School
LDPC Codes. Slides originally from I. Land p.1
Slides originally from I. Land p.1 LDPC Codes Definition of LDPC Codes Factor Graphs to use in decoding Decoding for binary erasure channels EXIT charts Soft-Output Decoding Turbo principle applied to
Windowed Erasure Codes
Windowed Erasure Codes Chris Studholme Department of Computer Science University of Toronto Email: cvs@csutorontoca Ian Blake Department of Electrical and Computer Eng University of Toronto Email: ifblake@commutorontoca
Regenerating Codes and Locally Recoverable. Codes for Distributed Storage Systems
Regenerating Codes and Locally Recoverable 1 Codes for Distributed Storage Systems Yongjune Kim and Yaoqing Yang Abstract We survey the recent results on applying error control coding to distributed storage
Iterative Encoding of Low-Density Parity-Check Codes
Iterative Encoding of Low-Density Parity-Check Codes David Haley, Alex Grant and John Buetefuer Institute for Telecommunications Research University of South Australia Mawson Lakes Blvd Mawson Lakes SA
Raptor Codes: From a Math Idea to LTE embms. BIRS, October 2015
Raptor Codes: From a Math Idea to LTE embms BIRS, October 2015 The plan is to... 1 introduce LT codes and Raptor codes 2 provide insights into their design 3 address some common misconceptions 2 / 31 The
Lecture 12. Block Diagram
Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data
Codes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Prelude Information transmission 0 0 0 0 0 0 Channel Information transmission signal 0 0 threshold
Codes on Graphs. Telecommunications Laboratory. Alex Balatsoukas-Stimming. Technical University of Crete. November 27th, 2008
Codes on Graphs Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete November 27th, 2008 Telecommunications Laboratory (TUC) Codes on Graphs November 27th, 2008 1 / 31
Fountain Codes. Amin Shokrollahi EPFL
Fountain Codes Amin Shokrollahi EPFL Content Fountain Codes LT-Codes Raptor Codes Extensions Transmission Problem Transmit information from one sender to multiple receivers where the channel between sender
Compressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
Construction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor
Construction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor Pravin Salunkhe, Prof D.P Rathod Department of Electrical Engineering, Veermata Jijabai
Section 3 Error Correcting Codes (ECC): Fundamentals
Section 3 Error Correcting Codes (ECC): Fundamentals Communication systems and channel models Definition and examples of ECCs Distance For the contents relevant to distance, Lin & Xing s book, Chapter
CS6304 / Analog and Digital Communication UNIT IV - SOURCE AND ERROR CONTROL CODING PART A 1. What is the use of error control coding? The main use of error control coding is to reduce the overall probability
Erasure Codes with a Banded Structure for Hybrid Iterative-ML Decoding
Erasure Codes with a Banded Structure for Hybrid Iterative-ML Decoding Alexandre Soro 1, Mathieu Cunche 2, Jérôme Lacan 1 and Vincent Roca 2 1 Univ of Toulouse, ISAE/DMIA, 10 avenue Edouard Belin, BP 54032-31055
Structured Low-Density Parity-Check Codes: Algebraic Constructions
Structured Low-Density Parity-Check Codes: Algebraic Constructions Shu Lin Department of Electrical and Computer Engineering University of California, Davis Davis, California 95616 Email:shulin@ece.ucdavis.edu
Graph-based codes for flash memory
1/28 Graph-based codes for flash memory Discrete Mathematics Seminar September 3, 2013 Katie Haymaker Joint work with Professor Christine Kelley University of Nebraska-Lincoln 2/28 Outline 1 Background
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Jilei Hou, Paul H. Siegel and Laurence B. Milstein Department of Electrical and Computer Engineering
Arrangements, matroids and codes
Arrangements, matroids and codes first lecture Ruud Pellikaan joint work with Relinde Jurrius ACAGM summer school Leuven Belgium, 18 July 2011 References 2/43 1. Codes, arrangements and matroids by Relinde
Constructions of Nonbinary Quasi-Cyclic LDPC Codes: A Finite Field Approach
Constructions of Nonbinary Quasi-Cyclic LDPC Codes: A Finite Field Approach Shu Lin, Shumei Song, Lan Lan, Lingqi Zeng and Ying Y Tai Department of Electrical & Computer Engineering University of California,
Low-density parity-check codes
Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,
Vector spaces. EE 387, Notes 8, Handout #12
Vector spaces EE 387, Notes 8, Handout #12 A vector space V of vectors over a field F of scalars is a set with a binary operator + on V and a scalar-vector product satisfying these axioms: 1. (V, +) is
An Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
Quasi-Cyclic Asymptotically Regular LDPC Codes
2010 IEEE Information Theory Workshop - ITW 2010 Dublin Quasi-Cyclic Asymptotically Regular LDPC Codes David G. M. Mitchell, Roxana Smarandache, Michael Lentmaier, and Daniel J. Costello, Jr. Dept. of
Optimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel
Optimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel H. Tavakoli Electrical Engineering Department K.N. Toosi University of Technology, Tehran, Iran tavakoli@ee.kntu.ac.ir
Lecture 4: Linear Codes. Copyright G. Caire 88
Lecture 4: Linear Codes Copyright G. Caire 88 Linear codes over F q We let X = F q for some prime power q. Most important case: q =2(binary codes). Without loss of generality, we may represent the information
RECURSIVE CONSTRUCTION OF (J, L) QC LDPC CODES WITH GIRTH 6. Communicated by Dianhua Wu. 1. Introduction
Transactions on Combinatorics ISSN (print: 2251-8657, ISSN (on-line: 2251-8665 Vol 5 No 2 (2016, pp 11-22 c 2016 University of Isfahan wwwcombinatoricsir wwwuiacir RECURSIVE CONSTRUCTION OF (J, L QC LDPC
1.6: Solutions 17. Solution to exercise 1.6 (p.13).
1.6: Solutions 17 A slightly more careful answer (short of explicit computation) goes as follows. Taking the approximation for ( N K) to the next order, we find: ( N N/2 ) 2 N 1 2πN/4. (1.40) This approximation
Extended Subspace Error Localization for Rate-Adaptive Distributed Source Coding
Introduction Error Correction Extended Subspace Simulation Extended Subspace Error Localization for Rate-Adaptive Distributed Source Coding Mojtaba Vaezi and Fabrice Labeau McGill University International
Spatially Coupled LDPC Codes
Spatially Coupled LDPC Codes Kenta Kasai Tokyo Institute of Technology 30 Aug, 2013 We already have very good codes. Efficiently-decodable asymptotically capacity-approaching codes Irregular LDPC Codes
Thresholds for the Recovery of Sparse Solutions via L1 Minimization
Thresholds for the Recovery of Sparse Solutions via L Minimization David L. Donoho Department of Statistics Stanford University 39 Serra Mall, Sequoia Hall Stanford, CA 9435-465 Email: donoho@stanford.edu
Variable Rate Channel Capacity. Jie Ren 2013/4/26
Variable Rate Channel Capacity Jie Ren 2013/4/26 Reference This is a introduc?on of Sergio Verdu and Shlomo Shamai s paper. Sergio Verdu and Shlomo Shamai, Variable- Rate Channel Capacity, IEEE Transac?ons
Iterative Decoding for Wireless Networks
Iterative Decoding for Wireless Networks Thesis by Ravi Palanki In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
Error Detection, Correction and Erasure Codes for Implementation in a Cluster File-system
Error Detection, Correction and Erasure Codes for Implementation in a Cluster File-system Steve Baker December 6, 2011 Abstract. The evaluation of various error detection and correction algorithms and
Physical Layer and Coding
Physical Layer and Coding Muriel Médard Professor EECS Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media:
AN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
Proofs of Retrievability via Fountain Code
Proofs of Retrievability via Fountain Code Sumanta Sarkar and Reihaneh Safavi-Naini Department of Computer Science, University of Calgary, Canada Foundations and Practice of Security October 25, 2012 Outsourcing
New Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
Block Codes :Algorithms in the Real World
Block Codes 5-853:Algorithms in the Real World Error Correcting Codes II Reed-Solomon Codes Concatenated Codes Overview of some topics in coding Low Density Parity Check Codes (aka Expander Codes) -Network
Low-Density Parity-Check Codes
Department of Computer Sciences Applied Algorithms Lab. July 24, 2011 Outline 1 Introduction 2 Algorithms for LDPC 3 Properties 4 Iterative Learning in Crowds 5 Algorithm 6 Results 7 Conclusion PART I
EE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes
EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check
Chapter 7. Error Control Coding. 7.1 Historical background. Mikael Olofsson 2005
Chapter 7 Error Control Coding Mikael Olofsson 2005 We have seen in Chapters 4 through 6 how digital modulation can be used to control error probabilities. This gives us a digital channel that in each
Lecture 4 : Introduction to Low-density Parity-check Codes
Lecture 4 : Introduction to Low-density Parity-check Codes LDPC codes are a class of linear block codes with implementable decoders, which provide near-capacity performance. History: 1. LDPC codes were
Time-invariant LDPC convolutional codes
Time-invariant LDPC convolutional codes Dimitris Achlioptas, Hamed Hassani, Wei Liu, and Rüdiger Urbanke Department of Computer Science, UC Santa Cruz, USA Email: achlioptas@csucscedu Department of Computer
Weaknesses of Margulis and Ramanujan Margulis Low-Density Parity-Check Codes
Electronic Notes in Theoretical Computer Science 74 (2003) URL: http://www.elsevier.nl/locate/entcs/volume74.html 8 pages Weaknesses of Margulis and Ramanujan Margulis Low-Density Parity-Check Codes David
Codes designed via algebraic lifts of graphs
p./40 Codes designed via algebraic lifts of graphs Clemson Mini-Conference on Discrete Mathematics Oct. 3, 2008. Christine A. Kelley Department of Mathematics University of Nebraska-Lincoln email: ckelley2@math.unl.edu
Lecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
Making Error Correcting Codes Work for Flash Memory
Making Error Correcting Codes Work for Flash Memory Part I: Primer on ECC, basics of BCH and LDPC codes Lara Dolecek Laboratory for Robust Information Systems (LORIS) Center on Development of Emerging
Compressed Sensing Using Reed- Solomon and Q-Ary LDPC Codes
Compressed Sensing Using Reed- Solomon and Q-Ary LDPC Codes Item Type text; Proceedings Authors Jagiello, Kristin M. Publisher International Foundation for Telemetering Journal International Telemetering
Low-complexity error correction in LDPC codes with constituent RS codes 1
Eleventh International Workshop on Algebraic and Combinatorial Coding Theory June 16-22, 2008, Pamporovo, Bulgaria pp. 348-353 Low-complexity error correction in LDPC codes with constituent RS codes 1
An Introduction to Sparse Approximation
An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,
Low Density Parity Check (LDPC) Codes and the Need for Stronger ECC. August 2011 Ravi Motwani, Zion Kwok, Scott Nelson
Low Density Parity Check (LDPC) Codes and the Need for Stronger ECC August 2011 Ravi Motwani, Zion Kwok, Scott Nelson Agenda NAND ECC History Soft Information What is soft information How do we obtain
A Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing via Polarization of Analog Transmission
Li and Kang: A Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing 1 A Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing via Polarization
Codes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Funded by: National Science Foundation (NSF) Seagate Technology Defense Advanced Research Projects
Low-Density Parity-Check codes An introduction
Low-Density Parity-Check codes An introduction c Tilo Strutz, 2010-2014,2016 June 9, 2016 Abstract Low-density parity-check codes (LDPC codes) are efficient channel coding codes that allow transmission
MATH32031: Coding Theory Part 15: Summary
MATH32031: Coding Theory Part 15: Summary 1 The initial problem The main goal of coding theory is to develop techniques which permit the detection of errors in the transmission of information and, if necessary,
A Key Recovery Attack on MDPC with CCA Security Using Decoding Errors
A Key Recovery Attack on MDPC with CCA Security Using Decoding Errors Qian Guo Thomas Johansson Paul Stankovski Dept. of Electrical and Information Technology, Lund University ASIACRYPT 2016 Dec 8th, 2016
Turbo Codes are Low Density Parity Check Codes
Turbo Codes are Low Density Parity Check Codes David J. C. MacKay July 5, 00 Draft 0., not for distribution! (First draft written July 5, 998) Abstract Turbo codes and Gallager codes (also known as low
A Class of Quantum LDPC Codes Derived from Latin Squares and Combinatorial Design
A Class of Quantum LDPC Codes Derived from Latin Squares and Combinatorial Design Salah A Aly Department of Computer Science, Texas A&M University, College Station, TX 77843-3112, USA Email: salah@cstamuedu
Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras
Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras e-mail: hari_jethanandani@yahoo.com Abstract Low-density parity-check (LDPC) codes are discussed
Secure RAID Schemes from EVENODD and STAR Codes
Secure RAID Schemes from EVENODD and STAR Codes Wentao Huang and Jehoshua Bruck California Institute of Technology, Pasadena, USA {whuang,bruck}@caltechedu Abstract We study secure RAID, ie, low-complexity
Universal variable-length data compression of binary sources using fountain codes
Universal variable-length data compression of binary sources using fountain codes Giuseppe Caire Shlomo Shamai Amin Shokrollahi Sergio Verdú Institut Eurecom Technion EPFL Princeton University giuseppe.caire@eurecom.fr,
Design and Analysis of LT Codes with Decreasing Ripple Size
Design and Analysis of LT Codes with Decreasing Ripple Size 1 Jesper H. Sørensen, Petar Popovski, Jan Østergaard, Aalborg University, Department of Electronic Systems, E-mail: {jhs, petarp, arxiv:1011.2078v2
Capacity of a channel Shannon s second theorem. Information Theory 1/33
Capacity of a channel Shannon s second theorem Information Theory 1/33 Outline 1. Memoryless channels, examples ; 2. Capacity ; 3. Symmetric channels ; 4. Channel Coding ; 5. Shannon s second theorem,
On Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels
2012 IEEE International Symposium on Information Theory Proceedings On Generalied EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels H Mamani 1, H Saeedi 1, A Eslami
LDPC codes based on Steiner quadruple systems and permutation matrices
Fourteenth International Workshop on Algebraic and Combinatorial Coding Theory September 7 13, 2014, Svetlogorsk (Kaliningrad region), Russia pp. 175 180 LDPC codes based on Steiner quadruple systems and
Construction and Performance Evaluation of QC-LDPC Codes over Finite Fields
MEE10:83 Construction and Performance Evaluation of QC-LDPC Codes over Finite Fields Ihsan Ullah Sohail Noor This thesis is presented as part of the Degree of Master of Sciences in Electrical Engineering
ECC for NAND Flash. Osso Vahabzadeh. TexasLDPC Inc. Flash Memory Summit 2017 Santa Clara, CA 1
ECC for NAND Flash Osso Vahabzadeh TexasLDPC Inc. 1 Overview Why Is Error Correction Needed in Flash Memories? Error Correction Codes Fundamentals Low-Density Parity-Check (LDPC) Codes LDPC Encoding and
MATH3302 Coding Theory Problem Set The following ISBN was received with a smudge. What is the missing digit? x9139 9
Problem Set 1 These questions are based on the material in Section 1: Introduction to coding theory. You do not need to submit your answers to any of these questions. 1. The following ISBN was received
Local correctability of expander codes
Local correctability of expander codes Brett Hemenway Rafail Ostrovsky Mary Wootters IAS April 4, 24 The point(s) of this talk Locally decodable codes are codes which admit sublinear time decoding of small
THE ANALYTICAL DESCRIPTION OF REGULAR LDPC CODES CORRECTING ABILITY
Transport and Telecommunication Vol. 5, no. 3, 04 Transport and Telecommunication, 04, volume 5, no. 3, 77 84 Transport and Telecommunication Institute, Lomonosova, Riga, LV-09, Latvia DOI 0.478/ttj-04-005
Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel
Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Pål Ellingsen paale@ii.uib.no Susanna Spinsante s.spinsante@univpm.it Angela Barbero angbar@wmatem.eis.uva.es May 31, 2005 Øyvind Ytrehus
Efficient design of LDPC code using circulant matrix and eira code Seul-Ki Bae
Efficient design of LDPC code using circulant matrix and eira code Seul-Ki Bae The Graduate School Yonsei University Department of Electrical and Electronic Engineering Efficient design of LDPC code using
Fully-parallel linear error block coding and decoding a Boolean approach
Fully-parallel linear error block coding and decoding a Boolean approach Hermann Meuth, Hochschule Darmstadt Katrin Tschirpke, Hochschule Aschaffenburg 8th International Workshop on Boolean Problems, 28
Coding for loss tolerant systems
Coding for loss tolerant systems Workshop APRETAF, 22 janvier 2009 Mathieu Cunche, Vincent Roca INRIA, équipe Planète INRIA Rhône-Alpes Mathieu Cunche, Vincent Roca The erasure channel Erasure codes Reed-Solomon
Practical Polar Code Construction Using Generalised Generator Matrices
Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:
Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme
Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme Hsien-Ping Lin Shu Lin and Khaled Abdel-Ghaffar Department of Electrical and Computer Engineering University of California Davis
B. Cyclic Codes. Primitive polynomials are the generator polynomials of cyclic codes.
B. Cyclic Codes A cyclic code is a linear block code with the further property that a shift of a codeword results in another codeword. These are based on polynomials whose elements are coefficients from
Solutions of Exam Coding Theory (2MMC30), 23 June (1.a) Consider the 4 4 matrices as words in F 16
Solutions of Exam Coding Theory (2MMC30), 23 June 2016 (1.a) Consider the 4 4 matrices as words in F 16 2, the binary vector space of dimension 16. C is the code of all binary 4 4 matrices such that the
MATH3302. Coding and Cryptography. Coding Theory
MATH3302 Coding and Cryptography Coding Theory 2010 Contents 1 Introduction to coding theory 2 1.1 Introduction.......................................... 2 1.2 Basic definitions and assumptions..............................
ML and Near-ML Decoding of LDPC Codes Over the BEC: Bounds and Decoding Algorithms
1 ML and Near-ML Decoding of LDPC Codes Over the BEC: Bounds and Decoding Algorithms Irina E. Bocharova, Senior Member, IEEE, Boris D. Kudryashov, Senior Member, IEEE, Vitaly Skachek, Member, IEEE, Eirik
Coding Theory and Applications. Linear Codes. Enes Pasalic University of Primorska Koper, 2013
Coding Theory and Applications Linear Codes Enes Pasalic University of Primorska Koper, 2013 2 Contents 1 Preface 5 2 Shannon theory and coding 7 3 Coding theory 31 4 Decoding of linear codes and MacWilliams
Solutions to problems from Chapter 3
Solutions to problems from Chapter 3 Manjunatha. P manjup.jnnce@gmail.com Professor Dept. of ECE J.N.N. College of Engineering, Shimoga February 28, 2016 For a systematic (7,4) linear block code, the parity
MATH 433 Applied Algebra Lecture 21: Linear codes (continued). Classification of groups.
MATH 433 Applied Algebra Lecture 21: Linear codes (continued). Classification of groups. Binary codes Let us assume that a message to be transmitted is in binary form. That is, it is a word in the alphabet
Performance Comparison of LDPC Codes Generated With Various Code-Construction Methods
Performance Comparison of LDPC Codes Generated With Various Code-Construction Methods Zsolt Polgar, Florin rdelean, Mihaly Varga, Vasile Bota bstract Finding good LDPC codes for high speed mobile transmissions
Enhancing Binary Images of Non-Binary LDPC Codes
Enhancing Binary Images of Non-Binary LDPC Codes Aman Bhatia, Aravind R Iyengar, and Paul H Siegel University of California, San Diego, La Jolla, CA 92093 0401, USA Email: {a1bhatia, aravind, psiegel}@ucsdedu
an author's https://oatao.univ-toulouse.fr/18723 http://dx.doi.org/10.1109/isit.2017.8006599 Detchart, Jonathan and Lacan, Jérôme Polynomial Ring Transforms for Efficient XOR-based Erasure Coding. (2017)
(Classical) Information Theory III: Noisy channel coding
(Classical) Information Theory III: Noisy channel coding Sibasish Ghosh The Institute of Mathematical Sciences CIT Campus, Taramani, Chennai 600 113, India. p. 1 Abstract What is the best possible way
Coding Techniques for Data Storage Systems
Coding Techniques for Data Storage Systems Thomas Mittelholzer IBM Zurich Research Laboratory /8 Göttingen Agenda. Channel Coding and Practical Coding Constraints. Linear Codes 3. Weight Enumerators and
GALLAGER S binary low-density parity-check (LDPC)
1560 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 6, JUNE 2009 Group-Theoretic Analysis of Cayley-Graph-Based Cycle GF(2 p )Codes Jie Huang, Shengli Zhou, Member, IEEE, Jinkang Zhu, Senior Member,