Lecture 7: ɛ-biased and almost k-wise independent spaces

Transcription

1 Lecture 7: ɛ-biased and almost k-wise independent spaces Topics in Complexity Theory and Pseudorandomness (pring 203) Rutgers University wastik Kopparty cribes: Ben Lund, Tim Naumovitz Today we will see ɛ-biased spaces, almost k-wise independent spaces, and some applications. Distributions Let µ be a distribution on {0, } n. Lemma. µ is uniformly distributed on {0, } n iff for each nonempty [n] (where the addition is mod 2). Proof. Let χ : {0, } n R where Pr x µ [ x i = ] = 2 Thus: χ (x) = { if x i = 0 if x i =. χ (x) = ( ) x i. We can treat µ as a real valued function µ: {0, } n R with µ(x) 0 for all x {0, } n and n µ(x) =. x {0,} For nonempty, consider the inner product: µ, χ = x {0,} n µ(x)χ (x) = x {0,} n,χ (x)= µ(x) x {0,} n,χ (x)=0 µ(x) = Pr [ x i = 0] Pr [ x i = ] = 0 (by our hypothesis) x µ x µ Aside: Characters and Fourier analysis The χ are called the characters of the group (Z n 2, +). We have the following important properties:

2 . χ (x + y) = χ (x) χ (y) 2. For any nonempty, we have x {0,} n χ (x) = 0. This follows from the following calculation. Pick any i. Then: χ (x) = χ (x + e i ) = χ (x), x {0,} n x {0,} n x {0,} n where e i is the 0- vector with only in the ith coordinate. 3. Furthermore, χ (x) χ T (x) = ( ) x i+ i T x i = ( ) T = χ T (x). ometimes we will also treat, T as their characteristic vectors in Z n 2, and in this notation, χ (x) χ T (x) = χ +T (x). 4. χ, χ T = x {0,} n χ (x)χ T (x) = { x {0,} n χ 0 if T 0 T (x) = 2 n if = T. This means that {x } are an orthogonal system, and since there are 2 n of them, they are a basis for R {0,}n. Let ˆµ() = µ, χ : the th Fourier coefficient of µ. End Aside Proof (continued) We have ˆµ() = 0 for each nonempty. Thus µ = α χ for some α, and since µ is a distribution, we have µ = 2 n χ, and this is the uniform distribution. Lemma 2. uppose that for all nonempty [n], [ ] x i = (( ɛ)/2, ( + ɛ)/2). Pr x µ Then µ is ɛ close to the uniform distribution in L 2, and thus µ is ɛ2 n/2 close the the uniform distribution in L and µ is ɛ close the the uniform distribution in L. Proof. The hypothesis gives us that ˆµ() ɛ We also have ˆµ( ) = µ, χ = Aside: Parseval s Identity Lemma 3. Take any f : {0, } n R. We have Then f = 2 n f, χ χ = ˆf() 2 n χ. x f(x) 2 = 2 n ˆf() 2 2

3 Proof. This follows from the orthogonality of the χ : f(x) 2 = f, f x = 4 n = 4 n,t = 2 n ˆf()χ, T ˆf(T )χ T ˆf() ˆf(T ) χ, χ T ˆf() 2. End Aside Proof (cont d): Let U = χ 2 be the uniform distribution. n We have Û( ) =, and Û() = 0 for all nonempty. Thus µ U (which is what we want to show is small in the L 2 norm), has the following Fourier coefficients: { 0 if = µ U() = ˆµ() otherwise Now using the Parseval identity, we get that µ U 2 2 = 2 n ˆµ() 2 2 n ɛ2 (2 n ) ɛ 2. This completes the proof, and the result for L and L follow from Cauchy-chwarz and trivially. Thus if a distribution fools linear functions really well, it is almost uniform.. Notes on the L distance There are many distances between probability distributions. But the L distance has special status when we are interested in pseudorandomness. Lemma 4 (Data processing inequality). uppose µ, ν are distributions over the same domain D. Let f be a function defined on D. Pick x µ, y ν. f(x) f(y) L µ ν L. This is closely related to the following simple characterization of L distance: 2 µ ν L ɛ if and only if there exists a distinguishing test T : D {0, } such that T (µ) T (ν) L = Pr [T (x) = ] Pr [T (x) = ] ɛ. x µ x ν ketch of Proof: Graph the distributions µ and ν over their domain (They have the same domain). 2 µ ν L is the area between the curves where µ is above ν. Choose T to output on sets where µ > ν. This gives us Pr x µ [D(x) = ] Pr x ν [D(x) = ] is the area between µ and ν on these sets, which is 2 µ ν ɛ. 3

4 Relationship to pseudorandom generators simple µ s.t. small circuits C, When we studied PRGs, the goal was to find a Pr [C(x) = ] Pr [C(x) = ] ɛ. x µ x U The only difference between this condition and the condition for small L distance is the complexity constraint that C is small. THI MAKE ALL THE DIFFERENCE IN THE WORLD! By the above discussion, we could try to show that µ is a PRG by showing the stronger condition that µ is ɛ-close to the uniform distribution in L. This strengthening ruins the approach: there cannot be a µ which is generated using a small seed that is close to the uniform distribution in L distance: µ U L ɛ support(µ) ( ɛ) 2 n. Try to show this. 2 ɛ-biased distributions and k-wise independence Definition 5. µ is ɛ-biased if for all nonempty [n], Pr[ x i = ] [( ɛ)/2, ( + ɛ)/2]. Note:. µ is 0-biased µ is uniform. 2. ɛ-biased µ is (ɛ, ɛ2 n/2 )-close to uniform in (L 2, L ). 3. ɛ-biased µ is ɛ-close to uniform in L. When is µ k-wise independent? µ is k-wise independent k, we have ˆµ() = 0. Proof: ( ) Take such an. ince µ is k-wise independent, we have Pr x µ [ x i = ] = 2. By definition of ˆµ, this yields ˆµ() = 0. ( ) Let be as stated. Look at µ. T, T, we know that Pr x µ [ x i = ] = 2, so ˆ µ (T ) = 0, meaning µ is uniform. This implies that µ is k-wise independent. When is δ-almost µ k-wise independent? uppose [n], k, we have ˆµ() ɛ, then µ is δ-almost k-wise independent in L for δ = ɛ2 k/2. Take any [n] = k. Look at ν = µ, a distribution on {0, }. T, since ˆν(T ) = ˆµ(T ), we have ˆν(T ) ɛ. This implies that ν is ɛ 2 k/2 close to uniform on {0, }. 4

5 How much randomness is needed to generate these spaces? For k-wise independence: we saw that k(log n) bits suffices. In fact, Ω(k log n) bits are needed (you will prove this in the homework). For ɛ-biased spaces: First let us show that there exist simple ɛ-biased spaces; we will later see how to get them explicitly. We try to get an ɛ-biased µ which is uniform on some K {0, } n with K small. Then log K bits suffice to generate a sample from µ. We use the probabilistic method: Choose K at random as follows: pick y,..., y m in {0, } n uniformly. We want that for all linear functions h : Z n 2 Z 2, Fix h. Then: Pr [h(y i) = ] (( ɛ)/2, ( + ɛ)/2). i [m] Pr [y,..., y m are bad for h ] e Ω(ɛ2m), y,...,y m by a Chernoff bound (This is because for a random y Z n 2, Pr[h(y) = ] = 2 ). We then union bound over all h to get Pr [ h : y,..., y m are bad for h] 2 n e Ω(ɛ2m). y,...,y m Now choose m = O( n ), so that this is less than. We thus get an ɛ-biased space which can ɛ 2 be generated using log n + 2 log ɛ + O() bits of true randomness. It turns out that this seed length is near optimal. Later this class we will explicitly construct ɛ-biased spaces with seed length O(log n + log ɛ ). For δ-almost k-wise independent spaces: We know that ɛ-biased spaces are automatically δ-almost k-wise independent for some δ. It turns out that this already gives us almost k-wise independent spaces using smaller seed length than what is needed for pure k-wise independence. Indeed, if we take ɛ = 2 k/2 δ and take an explicit ɛ biased space as mentioned above, then this is δ-almost k-wise independent, and has a seed length of O(log n + log ɛ ) = O(log n + k + log δ ). 3 Efficient construction of δ-almost k-wise independent spaces One way to get k-wise independence is to multiply a random seed y by a matrix M: y y T M. In this construction, y is a vector with O(k log n) elements chosen uniformly at random, and M has n columns. Each element of y T M is y, a i, where a i is a column of M. Claim: The output y T M will be k-wise indpendent if and only if every k rows of M are indepenent. Proof: Let [n]. uppose a i, y is not uniformly distributed. This is equivalent to y, a i is not uniformly distributed. But, this implies that a i = 0, since y, b is uniform for any fixed b 0. 5

6 Claim: If, instead of taking y from a uniformly random distribution, we take y from an ɛ-biased distribution, y T M will still be ɛ2 k/2 -almost k-wise independent. Proof: uppose not; then there exists [n],, k such that bias ( a i, y ) ɛ. ince each set of k columns of M are independent, we know that a i 0. In addition, y is chosen from an ɛ-biased space. o, we ve reached a contradiction. How many bits of randomness will we need to generate an n bit sample from a δ-almost k-wise independent space using this procedure? We need a δ2 k/2 -biased sample of length O(k log n). ince log m + 2 log( ɛ ) bits of uniform randomness are needed for m bits of ɛ-biased randomness, we need O(log k + log log n) + O(log( δ ) + k) = O(k + log log n + log( δ )) total bits of randomness. 4 Applications of δ-almost k-wise independent distribution 4. k-universal sets A k universal set {0, } n has the property that the projection of onto any k indexes contains all 2 k possible patterns. We can use δ-almost k-wise independent distribution to construct k- universal sets. If δ = 0, then any δ-almost k-wise independent distribution has k-universal 2 k support. The size of the k-universal set we get out of this is 2 O(k+log log n+log( δ )) = ( 2 k log n ) O(), and is nearly optimal. (Note the surprisingly tiny dependence on n!) 4.2 Ramsey graphs Pick the edges (x ij ) i<j {0, } (n 2) from a δ-almost k-wise independent space; we can interpret x ij = as an edge between vertex i and vertex j, and x ij = 0 as the absence of an edge. Fix [n], = k. By the data processing inequality, Taking a union bound, Pr[x ij are all 0 or all for all i, j ] 2 2 (k 2) + δ. Pr[, = k, such that is a clique or independent set] ( ) n (2 2 (k 2) + δ) k By setting δ = 2 (k 2) and n = 2 k/0 in the above inequality, we ensure that the probability that there is a clique or independent set of size k is less than. Thus, we ve described an explicit family of 2 O(log2 n) graphs on n vertexes, at least one of which is O(log n) Ramsey. As a corollary, we can construct an O(log n) Ramsey graph in 2 O(log2 n) time. 6

7 5 Construction of ɛ-biased spaces 5. Finite extension field review The construction described here will use the finite field F 2 n. This is an n-dimensional vector space over F 2. Addition is the same as for F n 2 ; multiplication is a bilinear map. A polynomial P (x) F 2 n[x] of degree d has at most d roots. 5.2 Properties needed from an ɛ-biased space uppose y... y m is an ɛ-biased space; let G be the n by m matrix with columns y... y m. Pick x {0, } n. Consider x T G. If x 0, it must have (( ɛ)/2, ( + ɛ)/2) fraction of s. Pick any x, y {0, } n, and consider the Hamming distance (x T G, y T G); this is the number of s in x T G y T G = (x y) T G, which is in the range (( ɛ)/2, ( + ɛ)/2). Thus, the image of x T G is a linear space in {0, } m of dimension n, such that any two vectors in the space have distance of 2 ± ɛ/ Construction and proof of correctness A point from the ɛ-biased space is calculated from two uniformly chosen elements α, β F 2 n. The point is calculated as (α, β) [, β, α, β,... α N, β ]. where, is the inner product over F n 2, when elements of F 2 n F 2. are represented in some basis over If N is set to ɛ2 n, then this construction needs 2 log(n/ɛ) = 2 log N + 2 log( ɛ ) bits of randomness. We need to show that the sample space described is ɛ-biased. Pick (α, β) uniformly from F 2 n, and take any [N]. We need to show that [ ] α i, β = 0 (( ɛ)/2, ( + ɛ)/2). Pr α,β We have αi β = αi, β. There are two cases to [ consider; either α is a root of P (x) = xi, or it is not. If α is not a root of P (x), then Pr β αi, β = 0 ] = 2. If α is a root of P (x),then certainly αi, β = 0. However, there are at most N roots of P (x), so the probability that α is a root is at most N 2 = ɛ. Thus, the space is ɛ-biased. n 7