Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets

Transcription

1 Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Tokyo Institute of Technology & Simon Fraser University

2 Understanding Efficiency Better understanding of Efficient Computation Good Algorithms Strong Lower Bounds

3 Meta-Algorithms Algorithms operating on algorithms Algorithm Meta-Algorithm

4 Examples of Meta-Algorithms Computability Theory : Virus checker, Infinite-loop detector (aka Halting problem) Complexity Theory : SAT, Polynomial Identity Testing (PIT)

5 Self-Reference Computability: impossibility results (via diagonalization) Complexity: efficient ( non-trivial ) meta-algorithms ( for SAT, PIT ) lead to circuit lower bounds

6 The rest of the talk part 1: part 2: Meta-algorithms Circuit lower bounds Circuit lower bounds Meta-algorithms

7 Circuit Lower Bounds from SAT-Algorithms

8 Circuit - SAT Given: poly(n)-size circuit C on n inputs. Decide: Is C satisfiable? x 1 x 2 x 3 x n

9 If pigs can fly... If SAT in P, then E SIZE(2 n /n) P NP L in E = DTIME (2 O(n) ) of circuit size > 2 n /n constant c L c in P of circuit size > n c [ Kannan ]

10 Kannan s proof Brute-force (diagonalization) : Define a hard language in a large complexity class C (using alternation) Speed-up : Collapse C to a smaller complexity class using SAT P (removing alternation)

11 P=NP E requires size 2 n /n E 3 requires circuit size 2 n /n : For each n, use, to define the lex. first Boolean function f n : {0,1} n {0,1} of circuit size > 2 n /n P = NP PH = P E PH = E Use of Self-Reference x y R(,x,y) co-np question becomes x R (,x) contains SAT-algorithm

12 Karp-Lipton s proof: P = NP EXP not in PolySize Local Checkability of Computation EXP in PolySize EXP = 2. P = NP 2 = P. Self-Reference EXP = P. X Diagonalization

13 Local Checkability of Computation tape symbols t i m e 2x3 window tableu of computation TM s computation is correct iff all 2x3 windows are legal. (Used by Cook & Levin to show 3-SAT is NP-complete) More on local checkability in [ Arora, Impagliazzo, Vazirani ]

14 EXP in PolySize EXP = 2 Let L EXP be decided by TM M Tableau M (x, i, j) EXP polysize circuit C computing Tableau M x L C tableau windows defined by C(x,.,.) are legal & the tableau is accepting

15 Karp-Lipton s proof: P = NP EXP not in PolySize Local Checkability of Computation EXP in PolySize EXP = 2. P = NP 2 = P. Self-Reference EXP = P. X Diagonalization

16 Generalizing Karp-Lipton: co-np in NSUBEXP NEXP not in PolySize NEXP in PolySize NEXP = 2 [IKW] co-np in NSUBEXP 2 in NSUBEXP NEXP in NSUBEXP. X

17 Can we use this approach to get any actual circuit lower bounds??? [ Williams ] : Yes! For ACC 0 circuits (constant depth, with mod-gates).

18 Williams argument: ACC 0 -SAT algorithm NEXP not in ACC 0

19 Pushing Karp-Lipton further ACC 0 -UNSAT in NTime( 2 n /n c ) is enough! ACC 0 NEXP in PolySize NEXP = 2 [IKW] ACC 0 - UNSAT co-np in NSUBEXP 2 in NSUBEXP NEXP in NSUBEXP. X Relies on efficient oblivious simulation of TMs, & other ideas

20 C - Circuit Satisfiability There is k>0 such that : If C SAT for n c -size n-input circuits is in time O( 2 n /n k ) for every c, then NTime( 2 n ) is not in C - PolySize. ACC 0 SAT is solvable in the required time. Hence, NEXP not in ACC 0. [ Williams ]

21 Circuit Lower Bounds from PIT-Algorithms

22 Polynomial Identity Testing ( Arithmetic Circuit-UNSAT ) + Given: poly(n)-size arithmetic circuit C on n inputs (over rationals). 3 * Decide: Is C 0? x 1 x 2 x 3 x n Extra structure ( C is a polynomial ) makes PIT easier than UNSAT : PIT in co-rp [ Schwartz, Zippel, DeMillo-Lipton, ]

23 Boolean or Arithmetic circuit lower bound Suppose PIT in P. Then NEXP not in PolySize, OR Permanent not in Arithmetic PolySize (over integers). [ K., Impagliazzo ]

24 Important Polynomial Identities Permanent: For X = ( x i,j ) n x n Perm n (X) = x i, (i) Defining Identities (expansion by minors): Perm n (X) x 1,j Perm n-1 (X 1,j )... Perm 1 (x) x Arithmetic circuits C 1,, C n compute Perm 1,, Perm n iff C 1,,C n satisfy the identities.

25 If PIT is easy Permanent-Check: Given an arithmetic circuit C, decide if C Perm n. PIT P Permanent-Check P

26 Importance of Permanent #P- completeness : P #P = P Perm [ Valiant ] can simulate alternating, : PH P #P = P Perm [ Toda ]

27 Concluding the proof of [KI] Suppose PIT is in P. Suppose Perm is in Arithmetic PolySize. Then can guess & verify a polysize circuit for Perm n. So, By padding, PH P Perm NP. E PH NE. By diagonalization, E PH not in PolySize. QED

28 Pseudo-Random Generator (PRG) for PIT Circuit Lower Bounds PRG fools every arithmetic circuit of size s. Define a non-zero polynomial p that is zero on all outputs of PRG. Then p requires arithmetic circuit size > s. Also, p is computable in E. [ Heintz, Schnorr ]

29 Non-trivial algorithms for SAT or PIT yield circuit lower bounds.

30 Meta-Algorithms from Circuit Lower Bounds

31 Black-Box Use of Circuit Lower Bounds

32 Derandomization Computational Hardness Computational Randomness (pseudorandomness) [ Blum, Micali, Yao; Nisan & Wigderson, Babai et al., ] Hard Boolean function (truth table) PRG Hardness into Randomness Converter

33 Pseudo-Random Generator (PRG) random seed look indistinguishable to any small circuit PRG pseudorandom string random string

34 Incompressibility Argument n-bit string all n-bit strings C rejected strings Bad Each r Bad has small description relative to C : log B bits specifying the rank of r in B, plus the description of C Any string incompressible relative to C is accepted by C.

35 Incompressibility Argument n-bit string all n-bit strings C rejected strings Bad Each r Bad has small description relative to C Any string incompressible relative to C is accepted by C. High circuit complexity = incompressibility relative to small circuits

36 Hardness into Randomness Boolean circuit hardness : E requires circuit size 2 (n) PIT P [ Impagliazzo & Wigderson ] Arithmetic circuit hardness : E has a multilinear polynomial of circuit size 2 (n) PIT Quasi-P ( Time( n polylog n ) ) [ K. & Impagliazzo ]

37 Non - Black-Box Use of Circuit Lower Bounds

38 AC 0 Circuits PARITY ( x 1,, x n ) requires depth d AC 0 circuits of size > 2 n1/(d-1) [ Furst, Saxe, Sipser; Yao; Hastad ] Main Tool : Switching Lemma

39 Switching Lemma Given an AC 0 circuit C(x 1,,x n ), Choose a random subset of variables, Assign them to 0 or 1 randomly. Very likely, the circuit becomes shallow. [ Hastad ] C not too large can make it a constant function, with some variables still free. So, C can t compute PARITY.

40 AC 0 Lower Bound Strategy 1. Simplify a given small circuit (by random restriction). Useful for metaalgorithms 2. Argue that the resulting simple circuit cannot possibly compute the restriction of the original function.

41 AC 0 -functions are sparse small AC 0 -circuit sparse polynomial (Fourier) [ Linial, Mansour, Nisan ]

42 Learning AC 0 -circuits 1. Small AC 0 -circuit C is likely a constant function under a random restriction 2. C can t crucially depend on many variables 3. C is well-approximated by a linear combination of simple functions of very few variables ( most energy on low frequencies ) 4. Good approximation to C can be efficiently learned ( by learning few Fourier coefficients )

43 AC 0 into DNF x 1 x 2 x 3 x n AC 0 circuit: size cn, depth d #SAT for such DNFs is easy! DNF : 2 n (1 - ) ANDs, with = 1/(log c + d log d) d-1 ANDs have disjoint sets of satisfying assignments [ Impagliazzo, Mathews, Paturi ]

44 Flattening AC 0 -circuits Given an AC 0 circuit C(x 1,,x n ) of depth d and size c n. Using Switching Lemma, find a set of restrictions { i } that partition {0,1} n so that each C i is constant. Whp, # restrictions < 2 n (1- ), where > ( 1/(log c + d log d) d-1 ). The running time is poly(n) C (# restrictions).

45 AC 0 #SAT Algorithm [ IMP ] Convert a given small AC 0 circuit to an equivalent DNF = OR of not too many ANDs, with disjoint sets of satisfying assignment. Each AND on t variables has 2 n-t satisfying assignments. Add up over all ANDs.

46 ACC 0 #SAT Algorithm [ Williams ] [ Yao; Beigel, Tarui; Allender, Gore ] sym mod 6 x 1 x 2 x 3 x n ACC 0 circuit: size s, depth d Rather than Time 2 n poly(s) #SAT for SYM-AND : SYM-AND : S s (log s)m ANDs, with m 2 O(d) SYM : # true ANDs { 0,1 } Time ( 2 n + poly(s) ) poly(n) via Dynamic Programming ( z-transform )

47 More LB Algorithms [ Santhanam ] : Formula-SAT in time 2 n (1-1/poly(c)), for formulas of size c n. ( random restrictions ) [ Braverman ] : k-wise independence fools AC 0 circuits. ( Linial,Mansour,Nisan; Razborov Smolensky ) MORE TO COME? Yes, I think so!

48 Summary Good meta-algorithms (SAT, PIT, Learning) strong circuit lower bounds Can use this connection to get unconditional results on each side!

49 Some Challenges Non-black-box SAT-Algorithms Circuit Lower Bounds conversions? [ BPP = P circuit l.b. for NEXP, but PRG circuit l.b. for EXP ] Better circuit lower bounds for AC 0 better SAT-algorithms? [ beyond the Switching Lemma? ]