: Error Correcting Codes. December 2017 Lecture 10

Transcription

1 : Error Correcting Codes. December 017 Lecture 10 The Linear-Programming Bound Amnon Ta-Shma and Dean Doron 1 The LP-bound We will prove the Linear-Programming bound due to [, 1], which gives an upper bound on the code rate of a given distance. The bound s name hints the proof technique, however we will see a different proof which doesn t rely on linear programming, due to Navon and Samorodnitsky [3]. The linear-programming bound beats the Elias-Bassalygo bound when the relative distance is not too small. Before we proceed, consider the notion of a maimal eigenvalue restricted to a specific subset of indices. Definition 1. Let A C m m and B [m]. Define λ B (A) = ma v: v =1,supp(v) B v Av. Throughout, we consider A as the binary adjacency matri of the Hamming cube of dimension n. That is, the rows and column are indeed by {0, 1} n and A[, y] = 1 iff (, y) = 1 (as n-bit strings). Make sure you understand why λ {0,1} n = n. We abbreviate λ B = λ B (A), and note that we can think of every such v R n as a function v : {0, 1} n R. The way we establish an upper bound on the code s cardinality is by first proving a lower bound on λ B where B is a Hamming ball and then arguing that if a large enough B has a large maimal eigenvalue (w.r.t. the code s distance) then it must be the case that the code s cardinality is not too large. The Fourier Transform We will only consider Fourier epansion over the Boolean cube. Let V = {f : F n that it is a vector space on F n over R of dimension n. A natural basis for V is R} and note 1 w () = { 1, = w 0, otherwise for every w F n. It is also an inner-product space under the inner product f 1, f = E [f F n 1 ()f ()] = 1 n f 1 ()f (), and it is easy to see that the basis {1 w } w F n is an orthogonal basis under this inner product. We now introduce another basis, that contains only functions that are homomorphisms. 1 G

2 Definition. A character of the finite group G is a homomorphism χ : G C, i.e., χ( + y) = χ()χ(y) for every, y G, where the addition is the group operation in G, and the multiplication is the group operation in C. In our case, G = F n and we have an eplicit representation of the characters. For S Fn, define χ S V as χ S () = ( 1) S,. Verify that every character is a homomorphism. Now, Claim 3. The set of all characters of F n is orthonormal (under the above inner product). Proof. First, χ S, χ S = 1 n χ S()χ S () = 1 n n = 1. Now, for S T, χ S, χ T = 1 n χ S ()χ T () = 1 n ( 1),S+T. As S T, S + T is nonzero, say at indices I [n]. Eactly half of the -s have odd weight restricted to I and eactly half have even weight. Thus, the above sum is 0. The Fourier transform of a function is the linear transformation from V to V that maps the natural basis to the Fourier basis (of characters). Thus, every f V can be (uniquely) written as f = S ˆf(S) χ S, and the coefficients ˆf(S) are called the Fourier coefficients. We give their basic properties: Claim 4. Let f, g V. We have: 1. ˆf(S) = f, χs.. f, g = ˆf(S)ĝ(S) S (Parseval s identity). 3. ˆf( ) = E[f]. Proof. For item (1), f, χ S = T ˆf(T )χ T, χ S = T ˆf(T ) χ S, χ T = ˆf(S) χ S, χ S = ˆf(S). For item (), f, g = f, S ĝ(s)χ S = S ĝ(s) f, χ S = S ˆf(S)ĝ(S). For item (3), ˆf( ) = f, = 1 n f() ( 1) 0 = 1 n f() = E[f].

3 We now define a convolution between two functions. Definition 5. Let f, g V. The convolution f g V is defined as (f g)() = E y f(y)g( + y). Verify that the convolution operator is commutative and associative. Also, a key property is the following one: Claim 6. f g = ˆf ĝ. Proof. Fi S F n. We have: ˆf(S) ĝ(s) = f, χ S g, χ S = 1 n f()g(y)χ S ()χ S (y) y y = 1 n f()g(y)χ S ( + y) = 1 n f()g(z + )χ S (z) = 1 n (f g)(z) χ S (z) = f g, χ S = f g(s). z z.1 Fourier transform and codes For C F n, we let 1 C be the characteristic function of C, in the sense that 1 C () = 1 if C and 0 otherwise. We record a few easy claims. Claim 7. Let C be a linear code. Then, 1 C = C n 1 C. Proof. For every S F n, 1 C (S) = 1 C, χ S = 1 1 C() ( 1),S = 1 C n ( 1),S. Now, if S C then all inner products are 0 and we get C. n n Otherwise, there eists c 0 C such that c 0, S = 1. For every C it holds that ( 1),S + ( 1) +c0,s = ( 1),S (1 + ( 1) c0,s ) = 0. Summing it over all C, we get: 0 = ( ) ( 1),S (1 + ( 1) +c0,s ) = ( 1),S + ( 1) +c0,s = C C C C( 1),S, as required. Claim 8. Let C be a linear code. Then, 1 C 1 C = C n 1 C. Proof. For every F n, (1 C 1 C )() = 1 y 1 C(y)1 C ( + y) = 1 y C 1 C( + y). Now, if C n then + y C and the sum is C. Otherwise, + y / C and the sum is 0. n n Let e i F n be the vector (a 1,..., a n ) with a j = δ i,j. Let L : F n R defined by L(e i) = n for every 1 i n, and 0 elsewhere. Claim 9. For every f V it holds that Af = L f. Consequently, (Af)() = i [n] f( + e i). 3

4 Proof. Follows easily by (f L)() = 1 n y L(y)f( + y) = i [n] f( + e i) and inspecting the neighbors of in the Hamming cube. Claim 10. For every S F n, ˆL(S) = n w(s). Proof. ˆL(S) = L, χs = 1 n L() ( 1),S = i [n] ( 1)S i = (n w(s)) w(s). Finally, we give one last eample: Claim 11. Let B = B(0, τn) and C F n a code. Then, (1 C 1 B )(z) = C B(z, τn) / n. Proof. For every z F n, (1 C 1 B )(z) = 1 n 1 C ()1 B ( + z) = 1 n 1 B(z,τn) (). C 3 The approach We say f g if f() g() for every F n. Lemma 1. Let B = B(0, r = τn) be the Hamming ball of radius r. Then there eists a function f V with the following properties: f is supported on B, f 0, Af λ r f for λ r = r(n r) o(n) = n( τ(1 τ) o(1)). Definition 13. We say C F n has dual distance d if the Fourier transform of 1 C points of Hamming weight 0 < S < d. vanishes on Claim 14. If C F n C. is a linear code with dual distance d then d is also the minimal distance of Proof. We want to show that 1 C () = 0 for with 0 < w() < d. As C is linear, 1 C and by definition 1 C () vanishes on such -s. = n C 1 C, Lemma 15. Suppose C F n is a vector space with dual distance d (i.e., it s dual code has distance at least d). Let B = B r for an integer r such that λ r n d + 1. Then, (z + B r ) n n. z C 4

5 Let δ = d/n < 1, Take τ = 1 δ(1 δ) + o(1) and r = τn. So, ( (1 λ r = n ) ( 1 δ(1 δ) + o(1) + ) ) δ(1 δ) + o(1) o(1) ) = n (δ δ + 14 ( 1 + o(1) o(1) ) = n 4δ 4δ o(1) ( ) 1 = n (1 δ) + o(1) = n d + o n (1) n d + 1, and the premise of Lemma 15 is satisfied by choosing the o(1) terms both in λ r and in τ appropriately. From now on, that is the r we should think of. Now take C = C. Then, balls of radius r centered at the points of the dual code cover an -fraction of the space. Then, 1 n and so we obtain: C B r = C n(h(τ)+o(1)) n n, Corollary 16. Let C be a [n, k, d] code and δ = d/n is the relative distance. Then: C (1 H(τ) o(1))n and therefore C = n C (H(τ)+o(1))n, where τ = 1 δ(1 δ). Asymptotically, this gives R(δ) H( 1 δ(1 δ)). We are left with proving Lemma 1 and Lemma Proving a lower bound on the Dirichlet eigenvalue of a ball in F n Lemma 17. Let B = B(0, r = τn) be the Hamming ball of radius r. Then there eists a function f V with the following properties: f is supported on B, f 0, Af, f λ r f, f for λ r = r(n r) o(n) = n( τ(1 τ) o(1)). Proof. We construct a specific eigenfunction f that achieves the bound. f will be symmetric, so it is fully defined by its values on n + 1 vectors of distinct Hamming weights. We overload notation and write f(i) for the value gives on weight i vectors. We choose f such that f gives the same weight for each level on its support. Let M = n = o(n). Define f as follows: 1 i [r M, r], f(i) = ( n i) 0 otherwise. 5

6 Now we need to compute (Af)(v) = n j=1 f(v + e j). Notice that Af is also symmetric and Af(i) = if(i 1) + (n i)f(i + 1). Also, if i [r M, r 1], ( n ) f(i) f(i + 1) = i+1 ( n = i) n i i + 1. Thus, for i [r M + 1, r 1], Af(i) = i(n i)f(i) + (n i)(i + 1)f(i). Hence, f Af = r 1 k=r M+1 r 1 k=r M+1 ( ) n ( k(n k + 1) + (n k)(k + 1))f(k) k k(n k + 1) + (n k)(k + 1). As k(n k + 1), (n k)(k + 1) (r M)(n r) r(n r) M, we get that f Af (M 1)( r(n r) M) = r(n r) o(n), whereas f f 1, completing the proof. Having that we prove Lemma 1 Proof. A is a symmetric, irreducible (i.e., the corresponding graph is connected) operator with non-negative entries. Let A be its restriction to B (one can view it as either restricting the matri A to the B B sub-rectangle, or as the operator Π B AΠ B where Π B is projection on B). A is also symmetric, irreducible and with non-negative entries. By the Perron-Frobenius theorem the greatest eigenvalue of A is obtained by a non-negative vector f 0 supported on B. Say A f = λ f. We have already seen an f supported on B such that f Af f f λ r. However, f Af = f Π B AΠ Bf = f A f, and λ is the largest singular value of A, hence we must have λ λ r. Also Af A f because A A 0 and f 0, hence we have as desired. Af A f λ f λf, 6

7 3. The covering bound Proof. Let B = B r and f be the function guaranteed by Lemma 1 for B. Define I.e., for z F n : F (z) = 1 n F n F = 1 C f. 1 C ()f( + z) = 1 n w C f(z + w). Hence, F is supported on w C (w + B). We will bound AF, F from both sides. One side: By definition, AF = F L = (1 C f) L = 1 C (f L) = 1 C Af. As 1 C 0 and Af λ B f we have AF = 1 C Af λ B 1 C f = λ B F. Thus, Other side: It holds that AF, F λ B F, F. AF, F = S ÂF (S) F (S) = S L F (S) F (S) = S L(S) F (S) F (S). But F (S) = 1 C f(s) = 1 C (S) f(s), and C has dual distance d, so we get zero for every set S of cardinality between 1 and d 1. Hence, AF, F = L( )( F ( )) + = n( F ( )) + n( F ( )) + S S:w(S) d S:w(S) d L(S)( F (S)) (n w(s))( F (S)) (n d)( F (S)). Together we get that (n d + 1) F, F λ B F, F AF, F n E[F ] + (n d) F, F Thus, F, F n E[F ]. But, F is supported on Λ = w C (w + B), and by Cauchy-Schwartz, ( ) E[F ] 1 = n 1 F () 1 Λ F () = Λ n n F, F Hence 1 n Λ n, as desired. Λ 7

8 References [1] Philippe Delsarte. An algebraic approach to the association schemes of coding theory. Philips Res. Reports Suppls., 10, [] Robert McEliece, Eugene Rodemich, Howard Rumsey, and Lloyd Welch. New upper bounds on the rate of a code via the delsarte-macwilliams inequalities. IEEE Transactions on Information Theory, 3(): , [3] Michael Navon and Ale Samorodnitsky. Linear programming bounds for codes via a covering argument. Discrete & Computational Geometry, 41(),