Finite fields, randomness and complexity Swastik Kopparty Rutgers University
This talk Three great problems: Polynomial factorization Epsilon-biased sets Function uncorrelated with low-degree polynomials
Polynomial factorization
Polynomial factorization Algorithmic problem: Given a polynomial, factorize it into irreducible factors Over F q, degree d, n variables n=1: [Berlekamp] randomized poly(d, log q) time short and sweet, one of the first nontrivial randomized algorithms General n: [Grigoriev, Chistov,, Lenstra, Kaltofen] randomized poly(d n, log q) Via reduction to 1-variable case Big open question: deterministic? Essentially optimal: d n is number of monomials Or is it?
Arithmetic circuit representation Arithmetic circuit representation Polynomial presented as an arithmetic circuit of size s, degree d Can we factorize this in time poly(s,d)? Before that Can we do anything with polynomial presented as an arithmetic circuit? Polynomial identity testing Test if C(X 1,, X n ) 0 (the identically 0 polynomial) Classical randomized algorithm: Pick a random point a F n, and check if C(a) = 0. No deterministic algorithm known. Central problem in complexity theory
Factorization in the circuit representation [Kaltofen, Kaltofen-Trager] Efficient factorization is possible in the circuit representation! Randomized poly(s) time algorithm. How are the factors represented? As arithmetic circuits of size poly(s)! Can this be made deterministic? 2 open problems stand in the way Deterministic univariate factorization Deterministic polynomial identity testing [K-Saraf-Shpilka 15]: these are the only obstacles
Berlekamp s randomized factorization Let us factor 1-variable quadratic polynomial H(X) over F p, for p prime (in time polylog(p) ) Algorithm: Pick u, v F p uniformly at random Set G(X) = H(uX+v) (random affine shift) Suffices to factor G(X) Compute GCD(X (p-1)/2 1, G(X) ) If degree of GCD = 1, we found a factor of G(X) Else repeat and try again Two facts for analysis: For fixed α, β F p, the random variables uα + v, uβ + v are uniform and independent Probability that a random element of F p is a root of X (p-1)/2 1 is about ½ Thus with probability ½, the following event happens: one of the roots of G(X) will be a root of X (p-1)/2 1, and the other root of G(X) will not be
Epsilon-biased sets
Epsilon-biased sets Setting: F 2 n Linear functions l a : F 2 n F 2 la x = a, x S F 2 n is called ε-biased if: For all nonzero linear functions l a : F 2 n F 2 l a is not too biased on S: Pr l a x = 0 1/2 ε, 1/2 + ε x S Many applications is pseurodandomness, coding theory
Basic questions: existence and construction How small can epsilon-biased sets be? How to construct small epsilon-biased sets explicitly? Compute i th bit of the j th element in time poly(log n)
Small epsilon-biased sets Randomized construction: probabilistic method Random set of size O( n ε2) is epsilon-biased with high probability Deterministic constructions: Known with size: O( n2 ε 2) O( n ε 3) O( n [Alon-Goldreich-Hastad-Peralta] [Naor-Naor, based on Reed-Solomon codes] ε 2 1.25 ) [BenAroya-TaShma, based on Hermitian codes]
Lower bounds on epsilon-biased sets MRRW bound 77: epsilon biased sets must have size at least n Ω ε 2 log 1 ε Alon 04: very clever linear-algebra proof Next: less clever version of Alon s proof
Lower bounds on epsilon-biased sets Let S be epsilon biased Let f: F 2n R be given by: f(x) = 1/ S f(x) = 0 if x S otherwise Let F: F 2n R be the Fourier transform of f F(a) = x f x 1 a,x Key point of epsilon-bias: F(a) ε for all nonzero a F(0) = 1
Lower bounds on epsilon-biased sets Ingredient 1: Fourier transform is l 2 norm preserving: x f x 2 = 1 2 n a F a 2 LHS: x f x 2 = 1 S RHS: 1 2 n a F a 2 1+ 2n 1 ε 2 1 2 n 2n + ε2 So S Ω min 1 ε 2, 2n Close!
Lower bounds for epsilon-biased sets Ingredient 2: Convolution and Fourier transform Let t be an integer Define g: F 2n R by: g(x) = f(x 1 ) f(x 2 ) f(x t ) where the sum is over all x 1,, x t s.t. x 1 + + x t = x g is the convolution of f with itself t times g is the probability distribution of x 1 + x 2 + + x t, where x i chosen from S uniformly Let G:F 2n R be the Fourier transform of g Then G(a) = F(a) t.
Putting everything together x g x 2 = 1 2 n a G a 2 RHS: 1 2 n a G a 2 1+ 2n 1 ε 2t 1 2 n 2n + ε2t LHS: x g x 2 =? Cauchy-Schwarz: (support(g) ) ( x g x 2 ) x g x 2 = 1 So ( x g x 2 ) 1/support(g) support(g) S t Summarizing: S min 2 n 1, t ε 2t Optimizing value of t gives S Ω n ε 2 log 1 ε
A nice deterministic construction with size O( n2 ε 2) [AGHP 90] Pick an irreducible polynomial h(t) of degree d = O(log n ε ) Pick a bit-sequence s of length d Consider linear recurrence starting with s, with characteristic polynomial h Generate n bit sequence x out of this recurrence Analysis: Based on: FACT: Let A(T) be a nonzero degree n polynomial Then Pr[ h(t) divides A(T) ] < ε, where h(t) is random irreducible polynomial of degree d Missing piece: How to pick irreducible polynomials? Let I d be the set of all irreducible polynomials of degree d Want a bijection from {1,2,.., I d } I d computable in time poly(d) [K-Kumar-Saks 15] Can be done: indexing irreducible polynomials over finite fields
Functions uncorrelated with polynomials
Functions uncorrelated with polynomials Let f : F 2 n F 2 Define correlation of f with degree d polynomials Maximum ε s.t. there exists h(x1,, xn) of degree d s.t. Pr x F 2 n f x = h x = 1 2 + ε Denoted Corr(f, P d ) How low can this correlation be? Probabilistic method: For d < n/3, a random function f has w.h.p. Corr(f, P d ) < 2 Ω n Open question: find an explicit example of such a function
Conection to circuit complexity Circuit Class AC 0 (mod 2) Bounded depth Boolean circuits Allowed gates: AND, OR, NOT, Parity (unbounded fan-in) [Razborov 87] Such circuits can be approximated by polynomials
Application to circuit complexity [Razborov 87] Circuits can be approximated by polynomials For every poly-size AC 0 (mod 2) circuit C h(x 1,, x n ) F 2 [x 1,, x n ] with deg(h) = n 0.1 s.t. Pr x F 2 n C x = h x = 1 n ω 1 Corollary: If Corr(f, P n 0.1 ) < n ω 1 then for all poly-size AC 0 (mod 2) circuits C, i.e., f is average case hard for these circuits. Pr C x = f x < 1 x Fn 2 2 + n ω 1 Corollary: (Via Nisan-Wigderson hardness vs randomness) Explicit such f implies efficient pseudorandom generators against AC 0 (mod 2)
Functions uncorrelated with polynomials For small d, exponentially small correlation known: f(x 1,, x n ) = x 1 x 2 x 3 + x 4 x 5 x 6 + (n/3 disjoint monomials) f(x) = Tr(x 7 ) (viewed as map from F 2 n F 2 ) has 2 Ω n correlation with degree 2. Proof by (repeatedly) squaring and Cauchy-Schwarz: E x F2 n 1 f x +h x Analogous results for all d << log n. For larger d, only much weaker correlation known. f(x) = Majority(x) [ Smolensky 94] f(x) = Tr(x 1/3 ) [K 11] Have 1/n 0.4 correlation with degree n 0.1.
Majority is uncorrelated with degree n 0.1 Key Lemma: Majority is versatile For all g: F 2n F 2, there exist polynomials g, g with deg(g ), deg(g ) n/2 s.t. for all x, g(x) = g (x) Maj(x) + g (x) Proof: Consider x s.t. Maj(x) = 0 Need g (x) = g(x) for such x This uniquely defines g of degree n/2 Consider x s.t. Maj(x) = 1 Need g (x) = g(x) g (x) for such x This uniquely defines g of degree n/2 Maj = 1 Maj = 0 {0,1} n
Majority is uncorrelated with degree n 0.1 Suppose deg(h) < n 0.1, and Pr[ h(x) = Majority(x)] = ½ + ε Let S = { x: h(x) = Majority(x) } By versatility: Every function g: S F 2 can be written as: g = g Maj + g = g h + g which is degree n/2 + n 0.1. Now counting: #(functions on S) = 2 S. #(polynomials of degree at most n/2 + n 0.1 ) < 2 1 2 +n 0.4 2 n 1 So S < + 2 n 0.4 2 n So ε < n 0.4
Wrap-up Open questions: Polynomial factorization: Deterministic univariate polynomial factorization? Deterministic polynomial identity testing? Epsilon biased sets: What is the optimal size? Efficient constructions matching this? Functions uncorrelated with polynomials: Explicit functions highly uncorrelated with polynomials?