Coding Theory Massoud Malek Reed-Muller Codes An iportant class of linear block codes rich in algebraic and geoetric structure is the class of Reed-Muller codes, which includes the Extended Haing code. It was discovered by Muller in 1954, and the first decoding algorith was devised by Reed also in 1954. For each positive integer and each integer r with 0 r, there is an r th Reed-Muller Code RM (r, ), which is a binary linear code of paraeters [ 2, ( ) + 0 ( ) + + 1 ( ) ], 2 r. r order Note that RM (1, ) code is a sall code (n 2 ) with a large iniu distance (n 2 1 ), so a good decoding algorith is the ost eleentary: for each received word w, find a codeword in RM (1, ) closed to, w. This can be done very efficiently. In fact, RM (1, 5) was used by Mariner 9 to transit black and white pictures of Mars back to the Earth in 1972. Definition. There are any ways to define the Reed-Muller codes. First we choose an inductive definition. Later, we shall explain how to construct Reed-Muller codes using the Kronecker product. The zeroth order Reed-Muller codes RM (0, ), for 0, are defined to be the repetition codes { 0, 1 } of length 2. RM (0, 0) {0, 1} IK, RM (0, 1) {0 0, }, RM (0, 2) {0 0 0 0, },...... The first order Reed-Muller codes RM (1, ) are binary codes defined for all integers 1, recursively by: (i) RM (1, 1) { 0 0,, 1 0, } IK 2. (ii) For 1, RM (1, + 1) { (u, u) : u RM (1, ) } { (u, u + 1) : u RM (1, ) }. Exaple. RM (1, 2) { (x, y + x) : x { 0 0 0 0 }, y { 0 0 }}. 0 0 0 0 0 0 0 0, 0 0 0 0,, 0 0, 1 0, 1 0,, 0 0 0 0, RM (1, 3) 0 0 0 0, 0 0 0 0, 0 0 0 0, 0 0,. 1 0 0 0, 1 0 0 0, 0 0 0 0, 0 0 0 0. Diension. For 1, the Reed-Muller code RM (1, ) is a binary [ 2, + 1, 2 ] 1 linear code, in which every codeword except 0 and 1 has weight 2 1. RM ( 1, ) consists of all binary words of length 2 that have even weight.
Coding Theory Reed-Muller Codes Page 2 Theore 1. The codes RM (r, ) and RM ( r 1, ) are dual of each other. Proof. Let B r, and B r 1, be( orthonoral ) ( ) bases( of) RM (r, ) and and RM ( r 1, ), respectively. Then B contains + + + linearly independent vectors and ( ) ( ) 0 ( 1 ) r B r 1, contains + + + linearly independent vectors. The facts r 1 that and [( ) ( ) ( )] [( ) 2 + + + r r + 1 ) ( ) ( ) ( ) (,,..., 1 1 ( 0 ( + r + 2 ) r 1 ) ( )] + ( ). r + 1 iply that the vector space IK is generated by the union of two orthonoral bases B and B r 1,. Hence the code RM ( r 1, ) is the dual of RM (r, ) According to this theore, RM (r, 2 r+1) codes are self dual. Thus RM (1, 3) and RM (2, 5) are self-dual. Generator Matrix. Rather than using the recursive description of the code, we will give a recursive construction for the generator atrix of RM (r, ) code, which we will denote by G(r, ). For r 0, define G(0, ) [... ], and for r, define G(, ) G( 1, ). 0 0... If G(r, ) is a generator atrix for RM (r, ), then a generator atrix for RM (r, + 1) is Exaples. G(r, + 1) [ G(r, ) G(r, ) ] 0 G(r 1, ) G(0, 1) [ ], G(0, 2) [ ], G(0, 3) [ ], G(1, 1) G(1, 1) G(1, 1), G(1, 2), θ G(0, 1) 0 0 G(1, 2) G(1, 2) G(1, 2) G(2, 2), G(1, 3), 0 0 0 0 θ G(0, 2) 0 0 0 0 0 0 0 0 0 0 111111 111111 0101 0101 0101 0101 0011 0011 0011 0011 G(2, 2) G(2, 2) G(2, 3) 0001 0001, and G(2, 3) 0001 0001 G(3, 3). θ G(1, 2) 00000001 000111 000111 0000 0101 0000 0101 0000 0011 0000 0011 0000 0001
Coding Theory Reed-Muller Codes Page 3 Corollary 1.1. The parity-check atrix of G(r, ) is G( r 1, ) t. Proof. According to Theore 1, the generator G(r, ) of RM (r, ) is orthogonal to the generator G( r 1, ) of RM ( r 1, ). This clearly iplies that the parity-check atrix of G(r, ) is G( r 1, ) t. Theore 2. For r > 0, the code RM (r, 1) is contained in RM (r, ). Proof. We prove by induction. We start with G(1, + 1) G(1, ) G(1, ). 0 G(0, ) (Since the top row entries of [ G(1, ) G(1, ) ] are all ones. we conclude that RM (0, ) [ ] is contained in RM (1, ). In general, since G(r 1, 1) is a sub-atrix of G(r 1, ) and G(r 2, 1) is a sub-atrix of G(r 1, 1) we have obviously G(r 1, ) G(r 1, 1) G(r 1, 1) 0 G(r 2, 1) a sub-atrix of G(r, ). Thus RM (r, 1) is contained in RM (r, ). Corollary 2.1. For any integer 2, the code RM ( 2, ) is an extended Haing code and RM (1, ) is its dual code. Prof. Recall that any [ 2, 2 1, 3 ] code is a Haing code and any [ 2, 2 1, 4 ] code is an extended Haing code. Since RM ( 2, ) is a [ 2, 2 1, 4 ] code, we conclude that that it is an extended Haing code. The reainder of the proof follows fro Theore 1. An iediate result of this corollary is that RM (1, 3) is a self dual code. Kronecker Product. The Kronecker product also called tensor product or the direct product of two atrices A (a(ij)) and B is defined as follows: a 11 B a 12 B... a 1n B a A B 21 B a 22 B... a 2n B....... a 1 B a 2 B... a n B For exaple G(1, 1) G(1, 1) 0 0 0 0 0 0 0 0 G(1, 2). Thus by using the Kronecker product, we ay build generators for Reed-Muller codes. Here is how we define the generators: Let G 0 [ ], Z 1 Z, G(0, 0) [ 1 ], and J [ ]. Then
Coding Theory Reed-Muller Codes Page 4 (1) G(0, 1) J G(0, 0) [ ] and Z 2 Z 1 Z 1 [ 0 0 ]. (2) G(0, 2) [ ], G(1, 2), and G(22). 0 0 0 0 0 0 (3) Z 3 Z Z 2 [ 0 0 0 0 0 0 0 ], G(0, 3) J G(02) [ 1 ] J G(1, 2) G(1, 3). Z G(0, 2) 0 0 0 0 0 0 0 0 Note that the above atrix G(1, 3) G(3 2, 3) generates an extended self-dual Haing code. For exaple, G(0, 1) [ ], G(0, 2) [ ], G(0, 3) [ ], 11 11 G(1, 1) G(1, 1) G(1, 1), G(1, 2) 01 01, 01 θ G(0, 1) 01 1111 111111 G(1, 2) 0101 G(1, 2) G(1, 2) 0101 0101 G(2, 2), G(1, 3), 0001 0011 θ G(0, 2) 0011 0011 0001 000111 111111. 111111 0101 0101 0101 0101 0011 0011 0011 0011 G(2, 2) G(2, 2) G(2, 3) 0001 0001, and G(2, 3) 0001 0001 G(3, 3) θ G(1, 2) 00000001 000111 000111 0000 0101 0000 0101 0000 0011 0000 0011 0000 0001 Reed Decoding This decoding algorith is quite siple. we explain the algorith with the RM (1, 3) code which is also a self dual extended Haing code. Since the diension of RM (1, 3) is four and the generator atrix RM (1, 3) has four rows, we conclude that the rows of v 1 v G(1, 3) 2 v 3 0 0 0 0 v 4 0 0 0 0 for a basis of the one-error-correcting code RM (1, 3). Exaple 1. Suppose the word w [ 0 0 ] was received. Clearly there is at least one error involved (Why?) Step 1. We express w as a linear cobination of v 1, v 2, v 3, v 4. Thus w [ 0 0 ] λ 1 v 1 + λ 2 v 1 + λ 3 v 3 + λ 4 v 4 [ λ 1, λ 1 + λ 2, λ 1 + λ 3, λ 1 + λ 2 + λ 3, λ 1 + λ 4, λ 1 + λ 2 + λ 4, λ 1 + λ 3 + λ 4, λ 1 + λ 2 + λ 3 + λ 4 ].
Coding Theory Reed-Muller Codes Page 5 Step 2. If there is no error in the first digit, then λ 1 0, λ 2 1, λ 3 0, λ 4 1. But in the sixth colun we have λ 2 + λ 4 1 + 1 1, which is a contradiction. Thus the sixth digit of w ust be changed fro 1 to zero. Note that the word v [ 0 0 ] is a codeword. Exaple 2. Suppose the word w [ 1 0 0 0 ] was received with at ost one digit-error. Solution. If there is no error in the first digit, then λ 1 1, λ 2 1, λ 3 0, λ 4 1. But we have contradictions in the sixth, seventh, and eight coluns. Since the word v [ 0 0 0 0 ] is a codeword, we conclude that the error occurred in the first digit. Exercise. Suppose the following words were received using the RM (1, 3) code: w 1 [ 0 1 ] w 2 [ 0 0 0 0 ] w 3 [ 0 0 0 0 ] w 4 [ 0 0 ] Use Reed Decoding algorith to decode the above words, assuing that in each word, there is at ost one digit-error. Hadaard Matrices. A Hadaard atrix of order n is an n n atrix H n with eleents 1 and - 1 such that H n Hn t n I n (i.e., H n is orthogonal). For exaple, H 1 [1], H 2. An n n Hadaard atrix with n > 2 exists only 1 1 if 4 divides n. For n > 4, in order to construct a Hadaard atrix of order n using the Kronecker product, n ust be divisible by 8. H 4 H 2 H 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Since H n Hn t n I n, any two different rows of H n ust be orthogonal, and the atrix obtained fro the perutation of rows or coluns of H n is also a Hadaard atrix, but the syetry ay be lost. Clearly H n is also a Hadaard atrix. Decoding Algorith for First order Reed-Muller Codes, Using Hadaard Matrices. The RM (1, ) code ay be decoded using a Hadaard atrix H 2 and the 1 1 Kronecker product. This algorith requires that for j 1, 2,...,, the atrices H j I 2 j H I 2 j 1 be constructed first. For exaple, for RM (1, ) code, we need to construct: 0 0 H2 1 1 1 0 0 I 2 H, H2 2 H I 0 0 2 0 1 1 0 1 0 1 0 0 1
Coding Theory Reed-Muller Codes Page 6 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 H3 1 0 1 0 0 0 0 I 4 H I 1 I 4 H 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 H3 2 0 1 0 0 0 0 I 2 H I 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 H3 3 0 0 0 0 I 1 H I 4 H I 4 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 The recursive nature of the construction of RM (1, ) codes suggests that there is a recursive approach to decoding as well. Fast Decoding Algorith for RM(1,). The generator atrix G(1, ) of the RM (1, ) code, changes the essage a [a 0, a 1, a 2,..., a ] of zeros and ones into a codeword v [v 0, v 1, v 2,...,... v 2 1]. Suppose the word v was sent and the word w (w 0, w 1, w 2,......,, w 2 1) is received. To decode w, we proceed as follows: Step 1. First, we replace all the zeros by -1 in w to obtain w 0 as follows: Step 2. For j 2, 3,...,, copute: w 0 2 w e, where e [ 1, 1, 1,..., 1, 1, 1 ]. w 1 w 0 H 1, w 2 w 1 H 2, w 3 w 2 H 3,, w 1 w 2 H 1, w w 1 H. Step 3. Find the position i 0 of w (w,0, w,1,..., w,2 ), where w,i0 coponent, in absolute value. is the largest Let v(i 0 ) IK be the binary representation of i 0 0 000... 00, 1 100... 00, 2 010... 00, 3 110... 00,........., 2 1 111... 11.
Coding Theory Reed-Muller Codes Page 7 If w,i0 is positive, then, the presued essage is [1, v(i 0 )], and if it is negative, then the presued essage is [0, v(i 0 )]. Note. In the above stateent, the presued essage does not ean that it is the correct essage. To ake sure that the essage obtained fro this ethod is the sent essage, we should encode the essage with the generator atrix G(1, ) and copared it with the received word w; only if the encoded word is equal to w, then we ay assert that the procedure was successful. Exaples. Consider the Reed-Muller code RM(1, 3) generated by the atrix G(1, 3). a. Suppose the word w 1 0 is received, then: Step 1. w 0 2 w e (1, 1, 1, 1, 1, 1, 1, 1). Step 2. Copute: w 1 w 0 H 1 3 [ 0, 2, 0, 2, 0, 2, 2, 0 ] w 2 w 1 H 2 3 [ 0, 4, 0, 0, 2, 2, 2, 2 ] w 3 w 2 H 3 3 [ 2, 6, 2, 2, 2, 2, 2, 2 ] Step 3. The second coponent of w 3 is 6 occurring in position 1. Since v(1) 100 and 6 > 0, then the presued essage is a 0 0. b. Suppose w 1 0 0 0 is the received word, then Step 1. w 0 2 w e [ 1, 1, 1, 1, 1, 1, 1, 1 ]. Step 2. Copute: w 1 w 0 H 1 3 [0, 2, 2, 0, 2, 0, 2, 0 ] w 2 w 1 H 2 3 [ 2, 2, 2, 2, 4, 0, 0, 0 ] w 3 w 2 H 3 3 [2, 2, 2, 2, 6, 2, 2, 2 ] Step 3. The fifth coponent of w 3 is 6 occurring in position 4. Since v(4) 001 and 6 < 0, then the presued essage is a 0001. Matlab. % Choose, then define the Hadaard atrix H 2. >> 3 ; H hadaard ( 2 ) ; % Use H 2 to define H 1, H 2,..., H. >> H31 kron ( eye(4), H ) ; H32 kron ( kron ( eye(2), H ), eye(2) ) ; >> H33 kron( H, eye(4) ) Exaple a. Suppose the received word is: >> wa [1 0 ] wa 1 0 STEP 1. Replace all the zero coponents of wa by -1. >> xa wa ones (1, 8) ; wxa wa + xa
Coding Theory Reed-Muller Codes Page 8 wxa 1 1 STEP 2. Now we evaluate: >> wxa1 wxa H31, wxa2 wxa1 H32, wxa3 wxa2 H33 wxa1 wxa2 wxa3 0 2 0 2 0 2 2 0 0 4 0 0 2 2 2 2 2 6 2 2 2 2 2 2 STEP3. The larges coponent of wxa3 (in absolute value) is 6 occurring in position 1 (the first position is position zero). The binary position of 6 is 1 0 0. va1 [ 1 0 0 ] va1 1 0 0 Since 6 is positive, the presued essage is va [ 0 0 ] va 0 0 The first position is position zero, so the binary position of 2 is 000. Exaple b. Suppose the received word is: wb [ 1 0 0 0 ] wb 1 0 0 0 STEP 1. Replace all the zero coponents of wa by -1. >> xb wb ones (1, 8) ; wxb wb + xb wxb 1 1 1 1 STEP 2. Now we valuate >> wxb1 wxb H31, wxb2 wxb1 H32, wxb3 wxb2 H33 wxb1 wxb2 wxb3 0 2 2 0 2 0 2 0 2 2 2 2 4 0 0 0 2 2 2 2 6 2 2 2 STEP3. The larges coponent of wxb3 (in absolute value) is 6, occurring in position 4. The binary position of 6 is 1 0 0. vb4 [ 0 ] vb4
Coding Theory Reed-Muller Codes Page 9 0 Since -6 is negative, the presued essage is: vb [ 0 0 ] vb 0 0 The first position is position zero, so the binary position of 2 is 0 0 0. % Suppose the vector ŵ 1 0 0 0 is received. Then w 0 2 w e (1, 1, 1, 1, 1, 1, 1, 1). >> w [ 1 0 0 0, ] ; e ones(1, 2 ) ; w 0 2 w e w0 1 1 1 1 % Define w 1 w 0 H 1 3, w 2 w 1 H 2 3, and w 3 w 2 H 3 3 >> w1 w 0 H31, w2 w 1 H32, w3 w 2 H33 ; w1 0 2 2 0 2 0 2 0 w2 2 2 2 2 4 0 0 0 w3 2 2 2 2 6 2 2 2 % 4 v(4) 0 and 6 < 0, so a 0 0. >> a 0 0 a 0 0 Exercises. Consider the Reed-Muller code RM (1, 3), then decode the following received words: (a) w 0 0 0 1 (b) w 0 0 0 0 (c) w 0 0 1 0. Mariner 9 and Reed-Muller RM(1,5) Code. Mariner 9 was the first spaceship to be put into orbit around Mars. As it was entioned previously, the ain reason for selecting the Hadaard [Reed-Muller(32,64,16)] code RM (1, 5), for transitting black and white pictures of the planet was its fast decoding algorith. Since RM (1, 5) is a 6 diensional code, there is a basis with 6 eleents (any linear cobination of which gives a code word). The data type 6-tuple is used to provide the coefficients for the linear cobination of the basis vectors. Thus associating a unique codeword to each data type. This siple coputation can be hard wired and requires no eory.