15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 16: Mootoe Formula Lower Bouds via Graph Etropy March 26, 2013 Lecturer: Mahdi Cheraghchi Scribe: Shashak Sigh 1 Recap Graph Etropy: Give G = (V, E), we defie H(G) = mi I(X; Y ) over joit distributios (X, Y ), where X V is uiformy radom ad X Y V. We showed Graph Etropy obeys the followig: Sub-additivity: H(G 1 G 2 ) H(G 1 ) + G(G 2 ). Mootoicity: If G 1 G 2, the H(G 1 ) G(G 2 ). Disjoit Uio: If G 1,..., G k are the coected compoets if G, the H(G) = V (G i ) i V (G) H(G i). Last time, we applied Graph Etropy to lower boud the size of a coverig of a graph by bipartite graphs a perfect family of hash fuctios 2 Mootoe Formula Lower Bouds via Graph Etropy Today we examie a applicatio of Graph Etropy to Circuit Complexity. 2.1 Mootoe Boolea Fuctios Defiitio 1 A boolea fuctio is oe mappig {0, 1} {0, 1}. Remark 2 We ca equivaletly cosider boolea fuctios as mappig P ([]) {0, 1}, usig the obvious bijectio betwee {0, 1} ad P ([]). Boolea fuctios are represeted by (ot ecessarily uique) boolea formulae or trees i which leaves variables ad iteral odes are logical coectives. We use these represetatios iterchageably. Defiitio 3 A boolea fuctio f : P ([N]) {0, 1} is mootoe if S T P ([]) implies f(s) f(t ). Furthermore, if f is a mootoe boolea fuctio, the the mi-terms of f of size i are (f) i = {S P ([]) : S = i, f(s) = 1, ad T S, f(t ) = 0}, (f) = (f) i. Furthermore, a boolea formula is mootoe if it cotais oly AND ad OR coectives. i=1 1
Example 4 The followig are mootoe boolea fuctios: 1. OR: x y = 0 x = y = 0. The mi-terms of OR are (OR) 1 = {{0}, {1}}, (OR) i = for i 1. 2. AND: x y = 1 x = y = 1. The mi-terms of AND are (AND) 2 = {{0, 1}}, (AND) i = for i 2. 3. MAJ 3 : MAJ 3 (x 1, x 2, x 3 ) = (x 1 x 2 ) (x 1 x 3 ) (x 2 x 3 ). Figure 1: Tree represetatios of AND, OR, ad MAJ 3 Propositio 5 A boolea fuctio is mootoe iff it ca be represeted by a mootoe boolea formula. Proof: Clearly, mootoe boolea formulae compute mootoe boolea fuctios. Let f be a mootoe boolea fuctio. The, W P ([]), F (W ) = 1 if ad oly if S (f) with S W. It follows that f is defied uiquely by (f) as follows: f(x 1,..., x ) = x j, (x 1,..., x ) {0, 1}. j S S (f) Note that this formula, called the Disjuctive Normal Form (DNF) of f, is also represeted by a biary tree, sice may-iput logic gates ca be simulated by (liearly may) two-iput gates. 2.2 Size of a Boolea Fuctio ad Threshold Fuctios Defiitio 6 The size size(φ)of a formula φ is the umber of odes i the tree represetatio of φ. 2
The size of a boolea fuctio f is size(f) = mi size(φ). φ computig f That is, size(f) is umber of odes i the smallest tree computig f. Defiitio 7 For k [], the threshold fuctio T h k : P ([]) {0, 1} is defied for S P ([]) by { T h k (S) = 1 if S k. 0 else Example 8 Threshold fuctios geeralize AND, OR, ad MAJ: AND = T h size(and) = 2 1 OR = T h 1 size(or) = 2 1 MAJ = T h /2 It ca be show that MAJ is the most complex threshold, i that it maximizes size(t h k ) over k. 2.3 Boudig the Size of Threshold Fuctios Cosider the problem of boudig size(t h k ). For geeral k, the boud size(t h k ) O(5.3 ) due to (Valiat, 1984) is kow, based o a probabilistic costructio which we do ot give here. We aalyze the case k = 2, for which the followig upper boud is easy to demostrate: Claim 9 size(t h 2 ) O(2 ). Proof: (T h 2 ) 2 = {{i, j} P ([]) : i j}. Furthermore, i 2, (T h 2 ) i =. Thus, the DNF of T h 2 is T h 2 (x 1,..., x ) = {i,j} P([]) i j x i x j, which (sice size(and), size(or) O()) idicates size(t h 2 ) O(2 ). Remark 10 Cosider the followig Divide ad Coquer costructio: Divide the iput strig x = (x 1,..., x ) {0, 1} ito y = (x 1,..., x /2 ) ad z = (x /2,..., x ). The, we have the recursive formula T h 2 (x) = T h /2 2 (y) T h /2 2 (z) (T h /2 1 (y) T h /2 1 (z)). 3
This recurrece gives a upper boud: defiig S = size(t h 2 ), the recurrece gives S 2S 1 + O(), sice clearly size(t h 1 ) O(). The solutio of this stadard recurrece (thik mergesort) is S (2 + log + 1)( log ) ad so S O( log ). Exercise: Refie this boud to size(t h 2 ) 2 log 1. The lower boud we ow give shows this is tight. We ow apply Graph Etropy to prove the lower boud size(t h 2 ) 2 log 1, followig (Newma, Ragde, ad Wigderso 1990). I order to use graph etropy we re goig defie a graph G f for a boolea fuctio f. Cosider defiig the followig: Defiitio 11 Note that is from T h 2 G f = (V, E), where V = [], ad E = (f)2. ad is ot ecessarily the umber of variables i f. Example 12 G T h 2 = K. For a sigle variable x i, G xi is the empty graph o vertices. 4
It helps ow to have a few lemmas about how graph etropy evolves with AND ad OR operatios. Lemma 13 Suppose f = g h. The, G f G g G h, ad hece H(G f ) H(G g ) + H(G h ). Proof: Suppose e = {i, j} E(G f ). The, 1 = f(e) = g(e) h(e); without loss of geerality, g(e) = 1. By costructio of G f, e (f) 2, so f({i}) = f({j}) = 0. The, g({i}) = g({j}) = 0, so e (g) 2 = E(G g ). It would be ice if we also had this property for AND, but it does t hold, as the followig example shows: Example 14 Suppose g(x 1, x 2 ) = x 1, h(x 1, x 2 ) = x 2. The, {1, 2} E(G f ), but {1, 2} / E(G g ), E(G h ). Thus, we eed a weaker statemet: G g h G g G h T g,h. Lemma 15 T g,h is the subgraph of G f iduced by edges i (g) 1 (h) 1 = ((g)1 (h) 1 ) ((h) 1 (g) 1 ). Proof: Let e = {i, j}, ad let f, g, h : {0, 1} 2 {0, 1} deote the restrictios of f, g, h, respectively, to e (sice the formulae are mootoe, we ca thik of this as settig the other coordiates to 0). The, we have f (x i, x j ) = x i x j g (x i, x j ) x i x j h { (x i, x j ) x i x j g { f = g h possible cases are = x i, h = x j g = x j, h = x i g by ispectio (x 1, x 2 ) x i x j h (x 1, x 2 ) x i x j (here, we make the simplifyig assumptio that f, g, h are o-costat fuctios; these cases ca be aalyzed separately) which i tur implies that e ((g) 1 (h) 1 ) ((h) 1 (g) 1 ) = (g) 1 (h) 1. Remark 16 Sice (g) 1 (h) 1 ad (h) 1 (g) 1 are disjoit, T g,h is bipartite. previous lecture that this implies H(T g,h ) 1. Usig subadditivity ad the facts We showed i a H(G g h ) H(G g ) + H(G h ) H(G g h ) H(G g ) + H(G h ) + 1 H(G xi ) = 0 H(G T h 2 ) = H(K ) = log, we see that ay mootoe formula for T h 2 has at least log AND gates, ad hece size(t h 2 ) log. 5
We ca get a eve tighter lower boud by tighteig the upper boud o H(T g,h ): Observe that, while V (T g,h ) = [], E(T g,h ) (g) 1 (h) 1, which implies, by the disjoit uio property, H(T g,h ) (g) 1(h) 1. Let s defie a potetial fuctio: Defiitio 17 µ(f) = H(G f ) + (f) 1. Claim 18 For both f = g h ad f = g h, µ(f) µ(g) + µ(h). Proof: Case 1: f = g h. Assumig o gate computes a costat fuctio, (f) 1 = {i : f({i}) = 1} = (g) 1 (h) 1. Thus, µ(f) = H(G f ) + (f) 1 H(G h ) + H(G h ) + (g) 1 + (h) 1 Case 2: f = g h. This time, (f) 1 = (g) 1 (h) 1. Thus, = µ(h) + µ(g). µ(f) = H(G f ) + (f) 1 H(G h ) + H(G h ) + H(T g,h ) + (g) 1 (h) 1 H(G h ) + H(G h ) + (g) 1(h) 1 + (g) 1 (h) 1 = H(G h ) + H(G h ) + (g) 1 + (h) 1 = µ(g) + µ(h). Note that each leaf has µ(x i ) = 1 ad the root has µ(t h 2 ) = log. Hece, by subadditivity ad the precedig claim, there must be at least log leaves. Sice each gate has two iputs ad oe output, it follows that there are at least log 1 iteral odes, for a total lower boud of size(t h 2 ) 2 log 1. 6