Ma/CS 6b Class 25: Error Correcting Codes 2

Size: px

Start display at page:

Download "Ma/CS 6b Class 25: Error Correcting Codes 2"

Tracy Mitchell
6 years ago
Views:

Ma/CS 6b Class 25: Error Correcting Codes 2 By Adam Sheffer Recall: Codes V n the set of binary sequences of length n. For example, V 3 = 000,001,010,011,100,101,110,111.

1 Ma/CS 6b Class 25: Error Correcting Codes 2 By Adam Sheffer Recall: Codes V n the set of binary sequences of length n. For example, V 3 = 000,001,010,011,100,101,110,111. Codes of length n are subsets of V n. For example, we might have 000 = up, 110 = down, 011 = left, 101 = right. This code is 000,110,011,101 V 3. Sequences that are in the code are called codewords (or just words). 1

2 Recall: Distances Given a, b V n the distance a, b is the number of bits that are different in a and b , = 2. Given a code C V n, the distance of C is a, b. min a,b C If C has distance d: We can detect any errors, as long as no more than d 1 bits were flipped. We can correct any errors, as long as fewer than d/2 bits were flipped. Recall: Linear Codes A code C V n is linear if for every a, b C, we have a + b C. For any m n matrix H, the set C = a V n : Ha = 0 m is a linear code. We say that H is the check matrix of C. Theorem. If no column in H consists entirely of zeros and no two columns of H are identical, then C can correct at least one bit flip. 2

3 Code Size Problem. We are interested in a code with a check matrix H that has r rows and does not contain zero columns and identical columns. What is the maximum number of columns in H? There are 2 r distinct columns of r entries. After ignoring the all zeros column, we remain with 2 r 1 possible columns. For example, when r = 3, we have Recall: Standard Form We consider matrices of the form H = b 1,1 b 1,n r b 2,1 b 2,n r b 3,1 b 3,n r b r,1 b r,n r This is called standard form.. 3

4 Hamming Codes Hamming codes are a family of linear codes: For every r 2, we have a code that corresponds to the check matrix H r having r rows and n = 2 r 1 distinct nonzero columns. We assume that the matrix is in standard form. A vector x = x 1, x 2,, x n is in the code if x 1 = b 1,1 x r b 1,n r x n, x r = b r,1 x r b r,n r x n. Determining x r+1,, x n uniquely determines x 1,, x r. So there are 2 n r words in the code. Properties of a Hamming Code Recall: The dimension of a linear code C V n is log 2 C. A Hamming code is defined with respect to a check matrix H r having r rows and n = 2 r 1 columns. That is, the code is a subset of V 2r 1. There are 2 n r words in the code. That is, the dimension of the code is 2 r r 1. Since H r has distinct columns and no zero column, the code can fix at least one bit flip. 4

5 Recall: Naïve Error Correction We consider words of size four. Each word is considered as a 2 2 matrix, and we add row and column parity bits. Bits of the message Parity bits A code of dimension four that corrects one bit flip at the cost of sending eight bits for every four bits that we wish to transmit. We say that this is an 8,4 -code because we send eight bit words which contain four bits of transmitted information. Hamming Code with r = 3 Hamming code with r = 3 has dimension 2 r r 1 = 4. The code is a subset V 2r 1 = V 7. Thus, this is a 7,4 -code that corrects one bit flip. Using this instead of the code from the previous slide shortens the transmission by one eighth! Can we do any better? 5

6 A Lower Bound Theorem. (proved in previous class) Let C V n be a linear code of dimension k that can correct e bit flips. Then 2 n k 1 + n 1 + n n e. In our case, e = 1 and k = 4. We know that there exists a code with n = 7. When n = 6, we have = 4 < Perfect Codes 2 n k 1 + n 1 + n n e. We proved that the Hamming code with r = 3 is the most efficient code of dimension 4 that can correct one bit flip. Such a code is said to be perfect. For general r the corresponding Hamming code is a subset of V n for n = 2 r 1 of dimension k = 2 r r 1, and can correct one bit flip. 2 n k = 2 r. 1 + n 1 = 1 + n = 2r. Thus, every Hamming code is perfect! 6

7 Correcting Errors Suppose that we are using a linear code C V n with check matrix H. We receive a transmitted message c. How do we check whether there is an error in c? And how do we correct such an error? Naïve solution. Check whether c C. If not, we compute the distance of c from every word in c and look for a C that minimizes this distance. Errors with a Hamming Code Suppose that we are using a Hamming code C V n with check matrix H. e i V n all zeros except for a single 1 in the i th position. A received message z V n with one bit flipped can be written as z = c + e i for some c C and some i. We have Hz = H c + e i = Hc + He i = He i. Notice that He i is the i th column of H. 7

8 Correcting with a Hamming Code Suppose that we are using a Hamming code C V n with check matrix H. We receive a transmitted z V n. Calculate Hz. z is a valid word if Hz = 0 r. Otherwise, find the column of H that is identical to Hz. If this is column i, the i th bit was flipped in z. Example Consider the Hamming code C with r = We have H 3 = Set v 1 = and v 2 = Are v 1, v 2 C? v 1 C since H 3 v 1 = 0 3. For v 2, we have H 3 v 2 = 111, so it is the word C with the last bit flipped. 8

Bipartite Expanders A bipartite graph G = V 1 V 2, E is an n, m, d, γ, α -expander if V 1 = n an V 2 = m. Every vertex of V 1 is of degree d. For every subset S V 1 with 1 S γn, we have N S dα S.

9 Bipartite Expanders A bipartite graph G = V 1 V 2, E is an n, m, d, γ, α -expander if V 1 = n an V 2 = m. Every vertex of V 1 is of degree d. For every subset S V 1 with 1 S γn, we have N S dα S. Expanders again?!? * This definition is slightly different from the one we previously used. Expander Codes Let G = V 1 V 2 be a n, m, d, γ, α - expander. The adjacency matrix of G is of the form 0 n n A T A 0 m m, where A is an m n matrix. We consider the code C V n with check matrix A. The dimension of the kernel of A (= the dimension of C) is at least n m, so we require n > m. 9

10 Properties The dimension of the expander code is at least n m. That is, it is an (n, n m)-code. Wikipedia: Expander codes are the only known asymptotically good codes which can be both encoded and decoded from a constant fraction of errors in polynomial time. Words as Vertex Sets A binary sequence c V n corresponds to the subset S c = v i V 1 c i = 1 V 1. We have Ac = 0 m iff for every vertex u V 2, u is adjacent to an even number of vertices of S c. In the figure, we have V 1 = v 1,, v 5, V 2 = u 1, u 2, u 3 and v A = v 2 v 3 v 4 v * This graph is not an expander. u 1 u 2 u 3 10

11 A Few Simple observations In the figure, we have V 1 = v 1,, v 5 : is a code word is not. In general: 0 n is always a word. For S V 1, if there exists u i that is adjacent to a single element of S, then S does not correspond to a word. v 1 v 2 v 3 v 4 v 5 u 1 u 2 u 3 Unique Neighbors Consider a bipartite graph G = (V 1 V 2, E) and let S V 1. We denote by U S the set of vertices of V 2 that are connected to exactly one vertex of S. Notice that U S N S. We call the elements of U S unique neighbors of S. If S corresponds to a code word, then U S =. v 1 v 2 v 3 v 4 v 5 u 1 u 2 u 3 11

12 Sizes of Sets Let G = V 1 V 2 be an n, m, d, γ, α -expander. For a subset S V 1, let U S V 2 be the set of vertices adjacent to exactly one element of S. Lemma 1. For every S V 1 with S γn, d S N S U S d 2α 1 S. N S U S is implies by U S N(S). d S N S is immediate since every vertex of S is of degree d. Sizes of Sets (cont.) Let G = V 1 V 2 be a n, m, d, γ, α -expander. For a subset S V 1, let U S V 2 the set of vertices adjacent to exactly one element of S. Lemma 1. For every S V 1 with S γn, d S N S U S d 2α 1 S. By the expansion property, N S dα S. Since σ v S deg v = d S, the number of vertices of N S that are adjacent to more than one vertex of S is at most d S dα S. Thus, U S 2α 1 d S. 12

13 Lemma 2. When 0.5 < α < 1, any S V 1 with S < 2αγn satisfies U S. Proof. By Lemma 1, if S γn then U S d 2α 1 S > 0. It remains to consider γn < S < 2αγn. Consider a subset T S such that T = γn. By Lemma 1, U T d 2α 1 γn. Since S T < γn 2α 1, we have N S T d S T < dγn 2α 1. Thus, we have U S U T N S T > 0. Distance of Expander Codes Theorem. Let 0.5 < α < 1, let the bipartite graph G = V 1 V 2 be a n, m, d, γ, α -expander and let C V m be the corresponding code. Then the distance of C is at least 2αγn. By choosing the parameters of the expander G, we can decide how many error bits can be detected and corrected. 13

Proof Recall that if for S V 1 has a unique neighbor, then S does not correspond to a code word. By Lemma 2, every subset S V 1 with fewer than 2αγn vertices has a unique neighbor.

14 Proof Recall that if for S V 1 has a unique neighbor, then S does not correspond to a code word. By Lemma 2, every subset S V 1 with fewer than 2αγn vertices has a unique neighbor. Thus, every nonzero word in the code contains at least 2αγn one bits. Recall: the distance of a code C equals the minimum weight of a nonzero word in C. Thus, the distance of the expander code is at least 2αγn. The Other Direction also Works! We can also build expanders from linear codes! Theorem. M n k check matrix of a code C V k with distance between (1/2 ε)n and (1/2 + ε)n. The graph G = (V, E) with vertex set V = V k and with x, y E iff x y is a row of M is a (2 k, n, 2ε)-expander. 14

15 More About Error Correction 15

Ma/CS 6b Class 24: Error Correcting Codes

Ma/CS 6b Class 24: Error Correcting Codes By Adam Sheffer Communicating Over a Noisy Channel Problem. We wish to transmit a message which is composed of 0 s and 1 s, but noise might accidentally flip some