Linear Block Codes. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay

Size: px

Start display at page:

Download "Linear Block Codes. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay"

Hubert Sydney Griffin
6 years ago
Views:

1 1 / 26 Linear Block Codes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay July 28, 2014

2 Binary Block Codes

3 3 / 26 Let F 2 be the set {0, 1}. Binary Block Code Definition An (n, k) binary block code is a subset of F n 2 containing 2k elements Example n = 3, k = 1, C = {000, 111} Example n 2, C = Set of vectors of even Hamming weight in F n 2, k = n 1 n = 3, k = 2, C = {000, 011, 101, 110} This code is called the single parity check code

4 4 / 26 Encoding Binary Block Codes The encoder maps k-bit information blocks to codewords. Definition An encoder for an (n, k) binary block code C is an injective function from F k 2 to C Example (3-Repetition Code) 0 000, or 1 000, 0 111

5 5 / 26 Decoding Binary Block Codes The decoder maps n-bit received blocks to codewords Definition A decoder for an (n, k) binary block code is a function from F n 2 to C Example (3-Repetition Code) n = 3, C = {000, 111} Since encoding is injective, information bits can be recovered as 000 0, 111 1

6 6 / 26 Optimal Decoder for Binary Block Codes Optimality criterion: Maximum probability of correct decision Let x C be the transmitted codeword Let y F n 2 be the received vector Maximum a posteriori (MAP) decoder is optimal ˆx MAP = argmax x C Pr(x y) If all codewords are equally likely to be transmitted, then maximum likelihood (ML) decoder is optimal ˆx ML = argmax x C Pr(y x) Over a BSC with p < 1 2, the minimum distance decoder is optimal if the codewords are equally likely ˆx = argmin x C d(x, y)

7 7 / 26 Error Correction Capability of Binary Block Codes Definition The minimum distance of a block code C is defined as d min = min d(x, y) x,y C,x y Example (3-Repetition Code) C = {000, 111}, d min = 3 Example (Single Parity Check Code) C = Set of vectors of even weight in F n 2, d min = 2 Theorem For a binary block code with minimum distance d min, the minimum distance decoder can correct upto d min 1 2 errors.

8 8 / 26 Complexity of Encoding and Decoding Encoder Map from F k 2 to C Worst case storage requirement = O(n2 k ) Decoder Map from F n 2 to C ˆx ML = argmax x C Pr(y x) Worst case storage requirement = O(n2 k ) Time complexity = O(n2 k ) Need more structure to reduce complexity

9 Binary Linear Block Codes

10 10 / 26 Vector Spaces over F 2 Define the following operations on F 2 Addition = = = = 0 Multiplication 0 0 = = = = 1 F 2 is also represented as GF(2) Fact The set F n 2 is a vector space over F 2

11 11 / 26 Binary Linear Block Code Definition An (n, k) binary linear block code is a k-dimensional subspace of F n 2 Theorem Let S be a nonempty subset of F n 2. Then S is a subspace of Fn 2 if u + v S for any two u and v in S. Example (3-Repetition Code) C = {000, 111} φ = 000, = 111, = 000 Example (Single Parity Check Code) C = Set of vectors of even weight in F n 2 wt(u + v) = wt(u) + wt(v) 2 wt(u v)

12 12 / 26 Encoding Binary Linear Block Codes Definition A generator matrix for a k-dimensional binary linear block code C is a k n matrix G whose rows form a basis for C. Linear Block Code Encoder Let u be a 1 k binary vector of information bits. The corresponding codeword is v = ug Example (3-Repetition Code) G = [ ] [ ] = [ 0 ] [ ] [ ] = [ 1 ] [ ]

13 13 / 26 Encoding Binary Linear Block Codes Example (Single Parity Check Code) n = 3, k = 2, C = {000, 011, 101, 110} [ ] G = [ 0 0 ] 0 [ 0 1 ] 1 [ 1 0 ] 1 [ 1 1 ] 0 = [ 0 0 ] [ ] = [ 0 1 ] [ ] = [ 1 0 ] [ ] = [ 1 1 ] [ ]

14 14 / 26 Encoding Complexity of Binary Linear Block Codes Need to store G Storage requirement = O(nk) O(n2 k ) Time complexity = O(nk) Complexity can be reduced further by imposing more structure in addition to linearity Decoding complexity? What is the optimal decoder?

15 15 / 26 Decoding Binary Linear Block Codes Codewords are equally likely ML decoder is optimal ˆx ML = argmax x C Pr(y x) Equally likely codewords and channel is BSC Minimum distance decoder is optimal ˆx ML = argmin x C d(x, y) To exploit linear structure to reduce decoding complexity, we need to study the dual code

16 16 / 26 Inner Product of Vectors in F n 2 Definition Let u = (u 1, u 2,..., u n ) and v = (v 1, v 2,..., v n ) belong to F n 2. The inner product of u and v is given by u v = n u i v i i=1 u v = 0 u and v are orthogonal. Examples ( ) (0 1 1 ) = = 0 ( ) (0 1 1 ) = = 1 ( ) (0 1 1 ) = = 0 ( ) (0 1 1 ) = = 0 Nonzero vectors can be self-orthogonal

17 Dual Code of a Linear Block Code Definition Let C be an (n, k) binary linear block code. Let C be the set of vectors in F n 2 which are orthogonal to all the codewords in C. { } C = u F n 2 u v = 0 for all v C C is a linear block code and is called the dual code of C. Example (3-Repetition Code) C = {000, 111}, C =? = = = = = = = = 0 C = {000, 011, 101, 110} = Single Parity Check Code 17 / 26

18 18 / 26 Dimension of the Dual Code Example (3-Repetition Code and SPC Code) C = {000, 111}, dim C = 1 C = {000, 011, 101, 110}, dim C = 2 dim C + dim C = = 3 Theorem dim C + dim C = n Corollary C is an (n, k) binary linear block code C is an (n, n k) binary linear block code

19 19 / 26 Parity Check Matrix of a Code Definition Let C be an (n, k) binary linear block code and let C be its dual code. A generator matrix H for C is called a parity check matrix for C. Example (3-Repetition Code) C = {000, 111} C = {000, 011, 101, 110} A generator matrix of C is H = H is a parity check matrix of C. [ 1 0 ]

20 20 / 26 Parity Check Matrix Completely Describes a Code Theorem Let C be a linear block code with parity check matrix H. Then v C v H T = 0 Example (3-Repetition [ Code) ] C = {000, 111}, H = Forward direction: v C v H T = 0 [ ] = [ 0 0 ], [ ] = [ 0 0 ]

21 21 / 26 Parity Check Matrix Completely Describes a Code Theorem Let C be a linear block code with parity check matrix H. Then v C v H T = 0 Example (3-Repetition [ Code) ] C = {000, 111}, H = Reverse direction: v C v H T = 0 v H T = [ ] 1 0 v 1 v 2 v = [ ] v 1 + v 3 v 2 + v v H T = 0 v 1 + v 3 = 0, v 2 + v 3 = 0 v 1 = v 3, v 2 = v 3 v 1 = v 2 = v 3

22 22 / 26 Decoding Binary Linear Block Codes Let a codeword x be sent through a BSC to get y, where e is the error vector y = x + e The probability of observing y given x was transmitted is given by Pr(y x) = p d(x,y) (1 p) n d(x,y) = p wt(e) (1 p) n wt(e) = (1 p) n ( p 1 p ) wt(e) If p < 1 2, lower weight error vectors are more likely

23 Decoding Binary Linear Block Codes Optimal decoder is given by ˆx ML = argmin x C d(x, y) = y + ê ML where ê ML = Most likely error vector such that y + e C. y + e C (y + e) H T = 0 e H T = y H T If s = y H T, the most likely error vector is ê ML = argmin wt(e) e F n 2,e HT =s Time complexity = O ( p(n)2 k) where p is a polynomial For each s, the ê ML can be precomputed and stored s is 1 n k binary vector Storage required is O(n2 n k ) 23 / 26

24 Summary

25 25 / 26 Complexity Comparison General Block Codes Encoding = O(n2 k ) Decoding = O(n2 k ) Linear Block Codes Encoding = O(nk) Decoding = O(p(n)2 min(k,n k) ) Observations Linear structure in codes reduces encoding complexity Decoding complexity is still exponential Need for codes with low complexity decoders

26 Questions? Takeaways? 26 / 26

EE512: Error Control Coding

EE512: Error Control Coding Solution for Assignment on Linear Block Codes February 14, 2007 1. Code 1: n = 4, n k = 2 Parity Check Equations: x 1 + x 3 = 0, x 1 + x 2 + x 4 = 0 Parity Bits: x 3 = x 1,