Lecture Introduction. 2 Linear codes. CS CTT Current Topics in Theoretical CS Oct 4, 2012

Size: px

Start display at page:

Download "Lecture Introduction. 2 Linear codes. CS CTT Current Topics in Theoretical CS Oct 4, 2012"

Marcus Page
5 years ago
Views:

1 CS CTT Current Topics in Theoretical CS Oct 4, 01 Lecturer: Elena Grigorescu Lecture 14 Scribe: Selvakumaran Vadivelmurugan 1 Introduction We introduced error-correcting codes and linear codes in the last lecture In this lecture we will discuss in some more details properties of linear codes and we ll describe classical examples of linear codes We will also show the Hamming bound, which is a bound that relates the distance, rate, and block length parameters of codes and is tight for the Hamming codes Recall that a (n, k, d) q - code is a code of block length n, message length k and distance d = Δ(C) over an alphabet Σ with Σ = q A linear codes [n, k, d] q is a subspace of F n q of dimension k Linear codes Linear codes can be described explicitly by multiple representations as follows 1 Generator matrix representation: C = {c F n q m F k q, c = m G} Parity check matrix representation: C = {c c H = 0, H F n (n k) q } 3 Basis representation: Let v 1, v,, v k F n q be a set of linearly independent vectors in C Then C = { k i=1 α iv i α i F q and v i } Notice that these are the vectors v i could be the rows of a generator matrix G for C Notice that given a code, the generator matrix is not unique This is also the case for parity check representation or basis representation Recall the following useful fact about linear codes Lemma 1 Δ(C) = min c C wt(c) Determining the minimum distance of arbitrary codes is NP hard Knowledge of the parity check matrix of a particular code is helpful in determining its minimum distance as we will show next Proposition Let C be a linear codes Then Δ(C) = d is the minimum positive integer such that its parity check matrix H has d linearly dependent rows Proof By Lemma 1 we know that there is a codeword c C such that d = wt(c) Therefore c H = 0 Let {i 1, i,, i d } be the set of indices st c ij 0 Therefore ( 0 0 ci1 0 c id 0 ) H = 0 which implies that d i=1 c i j h ij = 0, where h i denotes the vector representing the ith row of H So there exist d linearly dependent rows of H Note that if H had fewer than d linearly dependent rows then there would exist a codeword of smaller than d weight, a contradiction 1

2 1 Systematic codes Definition 3 A (n, k, d) code in which each message is mapped to a codeword whose first k entries form the message itself So, a linear systematic code is a code whose first entries are the message itself and the remaining n k entries are parity checks of entries of the message Namely, m = (m 0, m 1,, m k ) (m, iɛs 1 m i, iɛs m i,, iɛs n k m i ), where S i < [k], i Systematic codes are often used in practical applications Recall that we have already seen an example of a systematic code in the last lecture This was the [7, 4, 3] linear code generated by G = One can indeed easily check that a code is systematic if it can be generated by a matrix G = [I k k A], where I k k {0, 1} k k is the identity matrix Indeed, note that a message m = (m 1,, m k ) gets mapped to (m 1,, m k ) G = (m 1,, m k, m A) We will next argue handwavely that any linear code is systematic, meaning given a generator matrix for a linear code it can be transformed into a new generator matrix of the code of the form mentioned above We note that a systematic code is not neccesarily a linear code Proposition 4 Given linear code C represented by a generator matrix G, G can be transformed into G = [I k k A] via row and column operations that preserve the code Indeed note that the following operations do not change the code: (Let us only focus on binary code now) 1 Adding rows (mod ) could only change the basis of the code Permuting rows does not change the basis 3 Permuting columns permutes the names of the coordinates, and results in a code that is essentially equivalent to the initial one One can show that these operations are enough to obtain a matrix whose first block is a I k k matrix In what follows we ll introduce some common linear codes 3 The Hamming Code Proposition 5 l positive integer, an [ l 1, l l 1, 3 ] In words, one can encode messages of length k = l l 1 into codewords of length n = l 1 and obtain a code of distance 3, called the Hamming code This code was proposed in the famous paper of Hamming, 1950 that introduced the area of coding theory Proof We define the code by describing its parity check matrix H H is a n (n k) binary matrix where the rows are the vectors representing the binary expansion of the integers in 1,,, n (therefore each such representation has length l and there are l 1 rows)

3 Namely, H = By Proposition 4 the above code has distance 3 Indeed, it is easy to check that the minimum number of linearly dependent rows is 3 (the first 3 rows are linearly dependent and there cannot be fewer linearly dependent rows) Therefore we have shown that we can get a code of distance 3 by appending log k parity bits to messages of length k Could we have achieved the same distance by appending fewer redundancy bits? The next bound, which was also demonstrated in Hamming s initial paper [1], shows that this tradeoff is essentially tight 31 The Hamming Bound Let y {0, 1} n and let t > 0 Define by B(y, t) a ball of radius t around y, ie (All the distances are Hamming distances) B(y, t) = {x {0, 1} n d(x, y) t} Define V ol(b(y, t)) = B(y, t) Therefore, for any y we have V ol(b(y, t) = V ol(b(0, t)) It is easy to see that the volume of a ball of radius t is t ( ) n V ol(b(0, t)) = i Theorem 6 (The Hamming bound) If (n, k, d) is a code, then i=0 Proof k V ol ( B ( 0, [ ])) d 1 n 3

4 Since d is the minimum distance of the code, if follows that balls of radius [ ] d 1 don t intersect Since the message length is k, there are k code words So if we draw balls of radius [ ] d 1 around the codewords in C then c C B ( c, [ ]) d 1 could at most cover all the n points in the ambient space and so k = c C ( B c, [ d 1 ]) n We now verify that the Hamming bound is tight for the Hamming code For d = 3 V ol(b(ˉ0, 1)) = n + 1 Indeed, plugging in the parameters of the Hamming code we have that l l 1 ( l ) = l 1 = n, and so k (n + 1) n The Hamming bound shown here for the binary alphabet can be easily generalized to alphabets of size q The volume of a ball of radius t is in this case V ol (B (0, t)) = t i=0 ( ) n (q 1) i i Theorem 7 (Hamming bound) If C is a (n, k, d) q code then ( ) d 1 q k V ol q, 0 q n 4 The Hadamard code We will next define the dual of linear codes Definition 8 The dual of linear code C is the code C = {y y, c = 0 c C} Since C is a linear subspace so is its dual Proposition 9 C is also linear code Proposition 10 C is generated by H, where H is the parity check of C, and H is its transpose So C = {c ch = 0} and C = {y z, y = zh } The dual of the Hamming code is called the Hadamard code (actually the Hadamard code is slightly modified by an extra 0 appended to each codeword) Therefore the Hadamard code is generated by the matrix H = So the columns are the binary representations of all the integers 0, 1,,, k 1 4

5 Proposition 11 The Hadamard code (denoted Had) is a [ n = k, k, k 1] code (So its distance is d = n ) Proof We only need to show its distance Namely, we show that c Had, c 0, we have wt(c) = n/ So c can be expressed as (m 0, m 1,, m k 1 ) }{{} 0 α 0 1 }{{} 1 }{{} 1 α 1 α k 1 This implies that c = ( < m, α 0 >, < m, α 1 >, < m, α k 1 > ) We will show how to pair up the set of coordinates into n/ pairs st for any pair (i, j) it is the case that c i c j Let i be such that m i 0 For α {0, 1} k pair it up with α + e i, where e i is the ith fundamental basis vector e i = 00 0 }{{} 1 00 So i which finishes the proof < m, α + e i >=< m, α > + < m, e i >=< m, α > +1, So the Had code has good distance d = n (so δ(had) = 1 ), but it has very bad rate r = k 0, as k k A good code is one where both the rate and the relative distance are constants (as the block length n Next we see a code that can achieve these parameters 5 Reed-Solomon Codes The Reed-Solomon code is defined over a large alphabets q [] Given the prime power q, and n q, and k n, the Reed-Solomon code can be constructed as follows: 1 Pick α 1 α n F q (all distinct) (We can do so since n q) The code maps messages in F k q to mesages in F n q First, associate to the message (m 0, m k 1 ) F k q the polynomial m(x) = k 1 i=0 m ix i F q [x] Finally encode the message (m 0, m k 1 ) by (m(α 1 ),, m(α n ) Proposition 1 RS q,n,k is a linear code Proof We need to verify that c 1, c C c 1 + c C and that α F q αc 1 C Let c 1 = (m (α 1 ), m (α ) m (α n )) and c = (p (α 1 ), p (α ) p (α n )) Then 5

6 c 1 + c = (m (α 1 ) + p (α 1 ), ) = ((m + p) (α 1 ), (m + p) (α ), ) RS Also, αc 1 = (αm (α 1 ), αm (α ),, αm (α n )) = ((αm) (α 1 ), ) RS The above inclusions hold since (m + p)(x) is a polynomial of degree k 1 and so is the polynomial αc 1 In the next lecture we will show that RS codes have the best possible distance for fixed n, k, and we ll introduce the Singleton Bound References [1] R W Hamming Error detecting and error correcting codes Bell System Technical Journal, 6: , 1950 [] Irving S Reed and Gustave Solomon Polynomial codes over certain finite fields Journal of the Society for Industrial and Applied Mathematics, 8: ,

Lecture B04 : Linear codes and singleton bound

IITM-CS6845: Theory Toolkit February 1, 2012 Lecture B04 : Linear codes and singleton bound Lecturer: Jayalal Sarma Scribe: T Devanathan We start by proving a generalization of Hamming Bound, which we