exercise in the previous class (1)

Transcription

1 exercise in the previous class () Consider an odd parity check code C whose codewords are (x,, x k, p) with p = x + +x k +. Is C a linear code? No. x =, x 2 =x =...=x k = p =, and... is a codeword x 2 =, x =x =...=x k = p =, and... is a codeword the sum of the two codewords =..., not a codeword

2 exercise in the previous class (2) Construct a 2D code for 6-bit information (a,..., a 6 ) as follows. determine the generator and parity check matrices encode using the generator matrix correct an error in the sequence a a 2 a p (a a a 5 a 6 p,..., a 6 ) 2 (a,..., a 6, p, p 2, q, q 2, q, r) q q 2 q r parity symbols: p = x + x 2 + x p 2 = x + x 5 + x 6 q = x + x q 2 = x 2 + x 5 q = x + x 6 r = x + x 2 + x + x + x 5 + x 6 2

3 transpose exercise in the previous class () p = x + x 2 + x p 2 = q = x x + x 5 + x 6 + x q 2 = x 2 + x 5 q = x + x 6 r = x + x 2 + x + x + x 5 + x 6 G: ( )G = ( ) coefficients ( 係数 ) H: as is H( ) T = ( ) T = the -th column

4 in the previous class... linear codes: definition, encoding, decoding one-bit error at the i-th symbol position syndrome equals the i-th vector of H if several column vectors in H are the same, then we cannot correct one-bit errors in a codeword. if all column vectors in H are different, then we can correct all one-bit errors in a codeword.

5 design of error correcting codes Construct a parity check matrix with all column vectors differ, then we have a one-bit error correcting code. OK examples: NG examples: H= H= H= H= H= C = {v Hv T = mod 2}, the discussion is easier if the right-submatrix of H is an identity matrix... 5

6 construction of a code coefficients H= as is transpose G= p = x + x, p 2 = x + x 2, p = x 2 + x,,,,,. codewords 6

7 the shape of a check matrix a parity check matrix with m rows and n columns a code with... length = n # of information symbols = n m (= k) # of parity symbols = m H n = 9 m = 5 code length 9 = information symbols + 5 parity symbols vertically longer H means more parity symbols in a codeword less number of information symbols NG not efficient, not favorable... good 7

8 Hamming code To design a one-bit error correcting code with small redundancy, construct a horizontally longest check matrix (all columns differ). Hamming code determine m, # of parity check symbols list up all nonzero vectors with length m use the vectors as columns of H (any order is OK, but let the right-submatrix be an identity) 8 Richard Hamming m = :,G H length 7 = information + parity

9 Parameters of Hamming code Hamming code determine m, # of parity check symbols design H to have 2 m different column vectors H has m rows and 2 m columns length # of information symbols # of parity symbols (n, k) code: code with length n, and k information symbols n = 2 m k = 2 m m m m n k

10 comparison of codes two codes which can correct one-bit errors: (7, ) Hamming code (9, ) 2D code which is the better? Hamming code is more efficient (small redundancy) Hamming code is more reliable correct data transmission with BSC with error prob. p: Hamming code: (-p) 7 + 7p(-p) 6 2D code:(-p) 9 + 9p(-p) 8 shorter is the better =.85 =.77 if p=.

11 codes better than Hamming code? (7, ) Hamming code parity bits are added to correct possible one-bit errors Is there a one-bit error correcting (6, ) code, with only 2 parities? No. Assume that such a code exists, then... there are 2 = 6 codewords # of vectors decoded to a given codeword = +6=7 # of vectors decoded to any one of codewords = 7 6 = 2 # of vectors with length 6 = 2 6 = 6, which is < 2 {, } 6 contradiction! ( 矛盾 )

12 Hamming code is perfect (7, ) Hamming code there are 2 = 6 codewords # of vectors decoded to a given codeword = +7=8 # of vectors decoded to any one of codewords = 8 6 = 28 # of vectors with length 7 = 2 7 = 28 {, } 7 all of 28 vectors are exactly partitioned to 6 classes with 8 vectors Hamming code is a perfect one-bit error correcting code: 2 k n + n = 2 n 2

13 advanced topic: multi-bit errors? Hamming code is a perfect one-bit error correcting code. Are there codes which correct two or more errors? Yes, there are many... one-bit error: syndrome = one column vector of H two-bits error: syndrome = sum of two column vectors of H different combinations of t columns in H results in different sums, the code corrects t-bits errors.

14 advanced topic: two-bits error correcting code h i + h j h i + h j if i, j {i, j } different two-bits errors results in different syndromes received the syndrome is T = h 2 + h 6 errors at the 2nd and 6th bits should be transmitted. H

15 ability of the code Error-correcting capability of a code is determined by the relation of column vectors of a parity check matrix. It is not easy to consider all the combinations of columns. More handy and easy means is needed. For linear codes, we can use minimum distance, or minimum weight 5

16 similarity of vectors a codeword u is sent, errors occur, and v is received: In a practical channel, the distance between u and v is small. BSC with error prob.. u = v =, with probability.729 v =, with probability.8 v =, with probability.9 v =, with probability. If there is another codeword u near u, then v = u occurs with notable probability. safe not safe 6

17 Hamming distance a=(a, a 2,..., a n ), b=(b, b 2,..., b n ): binary vectors the Hamming distance between a and b, d H (a, b) = the number of symbols which are differ between a and b d H (,) = 2 d H (, ) = 2 d H (, ) = d H (, ) = d H (a, b) = d H (a + b, ) If a vector u with length n is sent over BSC with error prob. p, then a vector v with d H (u, v) = i is received with prob. ( p) n i p i. inverse correlation between the distance and the probability ( 逆相関 ) 7

18 Hamming distance between codewords code with length... vectors = vertices of a -dimensional hyper-cube codewords = subset of vertices C ={,,, } C 2 ={,,, } two or more edges between codewords good some codewords are side-by-side bad 8

19 minimum distance the minimum Hamming distance of a code C: d min min{ d H ( a, b) a, bc, a b}. C ={,,, } C 2 ={,,, } d min = 2 d min = 9

20 computation of the minimum distance consider a linear code whose generator matrix is 2 G d min = for this code Do we need to consider all of 2 k 2 k combinations?

21 minimum Hamming weight the Hamming weight of a vector u: w H (u) =d H (u, )...# of s the minimum Hamming weight of a code C: w min =min{w H (u) : uc, u } v u + v Lemma: if C is linear, then w min = d min. proof of w min d min : u let u and v be codewords with d min = d H (u, v). u + v C, and w min w H (u + v) = d H (u + v, ) = d H (u, v) = d min. proof of d min w min : let u be the codeword with w min = w H (u). d min d H (u, ) = w H (u) = w min. 2

22 examples of minimum Hamming weight (9, ) 2D code:the minimum Hamming weight is (7, ) Hamming code: the minimum Hamming weight is 7 22

23 general case of Hamming code lemma: The minimum Hamming weight of a Hamming code is. proof sketch: Let H = (h,..., h n ) be a parity check matrix: {h,..., h n } = the set of all nonzero vectors if codeword u with weight, then Hu T = h i =...contradiction if codeword u with weight 2, then Hu T = h i + h j =...this means that h i = h j, contradiction no codewords with weight or 2 2

24 proof (cnt d) lemma: The minimum Hamming weight of a Hamming code is. proof sketch: Let H = (h,..., h n ) be a parity check matrix: {h,..., h n } = the set of all nonzero vectors Choose x, y as you like, and choose z so that h x + h y = h z. Let u be a vector having at the x-th, y-th and z-th positions, then Hu T = h x + h y + h z =, meaning that uc. codewords with weight are constructible 2

25 minimum distance and error correction What does d min = mean? Any two codewords are differ at three or more symbols. At least three-bits errors are needed to change a codeword to a different codeword. u v u v u v error error error error error {u d H (u, u )=, uc} {v d H (v, v )=, vc} = We can distinguish a result of one-bit error from a codeword u, and a result of one-bit error from other codeword v. 25

26 decoding territory d min =:define territories around codewords radius =... territories do not overlap radius = 2... territories do overlap rule of the error correction: if a received vector r falls in the territory of a codeword u, then r is decoded to u. if d min =, then the maximum radius of the territory is at most. the code can correct up to one-bit errors 26

27 general discussion define t max = d min ( x is the largest integer x) 2 d min =7, t max = t max t max d min =8, t max = t max t max territories do not overlap if the radius t max C can correct up to t max bits errors in a codeword. 27

28 examples d min t max

29 about t max t max is the maximum radius that is allowed we can consider smaller territories with radius < t max t max t vectors which do not belong to any territory detect errors, but do not correct them 29

30 advantage and disadvantage sent codeword received t decoded to the correct codeword error detection only decoded to a wrong codeword radius correct detect wrong large large small large small small large small The radius should be controlled according to applications.

31 familiar image? A B A: award of, Yen B, C, D: penalty of,, Yen A B C D C D P(A) large P(B), P(C), P(D) large P(miss) small P(A) small P(B), P(C), P(D) small P(miss) large

32 summary of today s class Hamming code one-bit error correcting perfect code the minimum distance and minimum weight handy measure of the error correcting capability large minimum distance means more power 2

33 exercise Define (one of) (5, ) Hamming code: construct a parity check matrix, and determine the corresponding generator matrix Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is.. H