Topics in Probabilistic Combinatorics and Algorithms Winter, 016 3. Basic Derandomization Techniques Definition. DTIME(t(n)) : {L : L can be decided deterministically in time O(t(n)).} EXP = { L: L can be determined in time O ( nc )} BPP : probability polynomial time algorithm A such that: x L P r[ A(x) accepts] 1 3. x / L P r[ A(x) accepts] 1 3. Determination Algorithm (I) Exponetial / Logorithm Proposition. BPP EXP. Proof. L : BPP, A: require time t(n) = n c1. For x {0, 1} n and coin tosses r {0, 1} m, m t(n), P x [ A(x, r) accepts ] = 1 n can compute this in exponential time. Then L DTIME ( t(n) t(n)) L EXP. r {0,1} m A(x, r) Remark. If t(n) = O(log n), L DTIME (t(n)n c ) P. (II) Nondeterminatic Method Lemma 1. If A(x, r) is a randomized algorithm for a language L with error probability smaller than n on accept x with length n. Here, for every x, a fixed sequence of coin tosses r n such that A(x, r n ) is correct for all x {0, 1} n. Proof. R n R {0, 1} r. P r Rn [ x {0, 1} n such that A(x, R n ) incorrect ] x P r Rn [ A(x, R n ) incorrect ] < n n = 1 Thus, there exists R n = r n such that A(x, r n ) correct for all x. 1
Definition. P: solve the problem in polynomial time. NP: verify the problem in polynomial time. Example: Hamiltonian cycle: G: n vertices, find a simple cycle of length n. Theorem. RP NP. Proof. L RP, randomized polynomial time algorithm A: x L P r[ A(x) accepts] 1 x / L P r[ A(x) accepts] = 0. the error probability n. a fixed sequence of coin tosses r n such that A(x, r n ) accept L NP. (III) The Method of Conditional Expections MAXCUT Problem: Given G = (V, E). Find a partition V = S T, S T = subject to maximize cut(s, t) = {{u, v} E, u S, v T }. Algorithm: (randomized M AXCU T approximation) Input: G : (V, E), no loop. Flip n coins r 1,, r n, put i in S if x i = 0, put i in T if x i = 1. Output: (S, T ). Remark. Expectation: E ( cut(s, T ) ) = E Proof. Conditional exceptation after choosing r 1,, r i, e(r 1, r,, r i ) = E[ cut(s, T ) R 1 = r 1,, R i = r i ] e( ) = E e(r 1, r,, r i ) = cut(s i, T i ) + 1 cut(u, [n]) where U = {i + 1,, n}. Choose r i+1 to maximize cut(s, T ), e(r 1,, r i+1 ) e(r 1,, r i ) e( ) = E Algorithm (deterministic M AXCU T approximation) Input: G : ([n], E)
(1) Set S =, T =. () For i = 1,,, n: (a) If cut({i + 1}, T ) > cut({i + 1}, S), set S S {i + 1}. (b) Else, set T T {i + 1}. Pairwise Independence Hashing Problem: S [N], Query: { Is x S? 1 if x S Simple solution: Use table T, T [x] = 0 otherwise. Determine hash function: h : [N] [N] T : a table of size M, set T [h(x)] = x if x S (So, to test y S compute h(y) and chcek if T [h(y)] = y.) Well-define if h s is 1 1. H : [N] [M]: randomize. For k N, S with S k, P r[ x y such that H(x) = H(y) ] P r[ H(x) = H(y) ] x y,x S,y S ( ) k 1 M < ɛ k if M Remark. X, Y : pairwise independent iff x X, b Y P r[x = a, Y = b] = P r[x = a]p r[y = b]. Definition. H : {h : [N] [M]} is pairwise independent if x 1 x [N], y 1 y [M], H H P r[ H(x 1 ) = y 1 H(x ) = y ) = 1 M (1) Equivalently, (1) is same as the following two conditions: (i) x [N], the random varible H(x) is uniformly distribution in [M]. (ii) x 1 x [N], the random varible H(x 1 ), H(x ) are pairwsie independent. Example 3. F is a finite field, H = {h a,b : F F}, where h a,b (x) = ax + b. Proposition. The family of functions H in Example 3 is pairwise independent. 3
Proof. Let x 1 x, y 1 y, { ax1 + b = y 1 ax + b = y solve a, b uniquely. P r H H [ H a,b (x 1 ) = y 1 H a,b (x ) = y ] = 1 M Randomness-Efficient Error Reduction and Sampling Lemma 4. (Chebyshevs Inequality) If X is a random variable with expectation µ, then P r[ X E(X) ɛ ] V ar[x] ɛ Proposition. (Pairwise Independent Tail Inequality) Let X 1,, X t be pairwise independent random variables taking values in the interval [0, 1], and let X = ( i X i)/t. Then Proof. tɛ V ar(x) = 1 t E [ t i=1 (X i E(X)) ] Then use Chebyshev inequlity, = 1 t E[ (X i E(X)) (X j E(X)) ] i j t E (X i E(X)) E (X j E(X)) i j t E (X i E(X)) i t V ar (X i ) 1 t. i tɛ Example 5 (Error Reduction). Proposition tells us that if we use t = O( k ) pairwise independent repetitions, we can reduce the error probability of a BPP algorithm from 1/3 to k. If the original BPP algorithm uses m random bits, then we can do this by choosing h : {0, 1} k+o(1) {0, 1} m at random from a 4
pairwise independent family, and running the algorithm using coin tosses h(x) for all x {0, 1} k+o(1). This requires m + max{m, k + O(1)} = O(m + k) random bits. Number of Repetitions Number of Random Bits Independent Repetitions O(k) O(km) Pairwise Independent Repetitions O( k ) O(m + k) Example 6 (Sampling). Given oracle access to a function f : {0, 1} m [0, 1], we want to approximate E(f) to within an additive error of ɛ. We use Chernoff s inequality for X = this requires t = O t i=1 Xi t and we have P r[ X E(X) > ɛ ] < e tɛ /4 = δ. ( log 1 ) δ ɛ with error probability ɛ. Number of Samples Number of Random Bits Truly Random Sample O((1/ɛ ) log(1/δ)) O(m (1/ɛ ) log(1/δ)) Pairwise Independent Repetitions O((1/ɛ ) (1/δ)) O(m + log(1/ɛ) + log(1/δ)) Proposition. X 1,, X t : k-pairwise independent random varibles, taking values in [0, 1], X = t i=1 t, t k ɛ k Example 7 (Construction k-wise independence from polynomials.). Let F be a finite filed. Define the family of functions H = {h a0,a 1,,a k 1 :F F} where each h a0,a 1,,a k 1 (x) = a 0 +a 1 x+ +a k 1 x k 1 for a 0, a 1,, a k 1 F. 5