Sublinear Algorithms Lecture 3 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides.
Tentative Plan Lecture. Background. Testing properties of images and lists. Lecture 2. Testing properties of lists. Sublinear-time approximation for graph problems. Lecture 3. Testing properties of functions. Linearity testing. Lecture 4. Techniques for proving hardness. Other models for sublinear computation.
Testing Linearity
Linear Functions Over Finite Field ABoolean function :, {,}islinearif,, = + + for some,, {,} no free term Work in finite field Other accepted notation for : and Z Addition and multiplication is mod 2 =,,,=,,,that is,,, += +,, + example + Based on Ryan O Donell s lecture notes: http://www.cs.cmu.edu/~odonnell/boolean-analysis/ 4
Testing if a Boolean function is Linear Input:Boolean function :, {,} Question: Is the functionlinear or -far from linear ( 2 values need to be changed to make it linear)? Today: can answer in time 5
Motivation Linearity test is one of the most celebrated testing algorithms A special case of many important property tests Computations over finite fields are used in Cryptography Coding Theory Originally designed for program checkers and self-correctors Low-degree testing is needed in constructions of Probabilistically Checkable Proofs (PCPs) Used for proving inapproximability Main tool in the correctness proof: Fourier analysis of Boolean functions Powerful and widely used technique in understanding the structure of Boolean functions 6
Equivalent Definitions of Linear Functions Definition. islinearif,, = + + for some,,,, = for some. Definition. is linearif + = +()for all,,. [] is a shorthand for {, } Definition Definition + = + = + = +. Definition Definition Let =((,,,,,,)) Repeatedly apply Definition :,, = = =. Based on Ryan O Donell s lecture notes: http://www.cs.cmu.edu/~odonnell/boolean-analysis/ 7
Linearity Test [Blum Luby Rubinfeld 9] BLR Test (f, ε). Pick and independently and uniformly at random from,. 2. Set =+and query on,, and. Acceptiff = +. Analysis If is linear, BLR always accepts. Correctness Theorem [Bellare Coppersmith Hastad Kiwi Sudan 95] If is -far from linear then>fraction of pairs and fail BLR test. Then, by Witness Lemma (Lecture ), 2/iterations suffice. 8
Analysis Technique: Fourier Expansion
Representing Functions as Vectors Stack the 2 values of ()and treat it as a vector in {,}. = () () () () () () () ()
Linear functions There are 2 linear functions: one for each subset []. =, =,, = Parity on the positions indexed by set is,, =
Great Notational Switch Idea: Change notation, so that we work over realsinstead of a finite field. Vectors in, Vectors in R. /False /True -. Addition (mod 2) Multiplication in R. Boolean function:, {,}. Linear function, {,}is given by =. 2
Benefit of New Notation The dot product of and as vectors in, : (# s such that =()) (# s such that ()) = 2 2 (# s such that ()) disagreements between and Inner product of functions,, {,}, = dot product of and as vectors 2 = avg = E,, [ ]., = 2(fraction of disagreements between and ) 3
Benefit 2 of New Notation Claim. The functions form an orthonormal basis for R. If then and are orthogonal:, =. Let be an element on which and differ (w.l.o.g. ) Pair up all -bit strings: (, ) where is with the th bit flipped. Each such pair contributes =to,. Since all s are paired up,, =. Recall that there are 2 linear functions., = In fact,, =for all,,. (The normof,denoted, is, ) + + + + + + + + + + 4
Fourier Expansion Theorem Idea: Work in the basis, so it is easy to see how close a specific function is to each of the linear functions. Fourier Expansion Theorem Every function, Ris uniquely expressible as a linear combination (over R) of the 2 linear functions: =, where =, is the Fourier Coefficient of on set. Proof: can be written uniquely as a linear combination of basis vectors: It remains to prove that = for all. = =, = Definition of Fourier coefficients [], =, [] Linearity of, =, = if = otherwise 5
Examples: Fourier Expansion Fourier transform = = AND(, ) 2 + 2 + 2 2 MAJORITY(,, ) 2 + 2 + 2 2 6
Parseval Equality Parseval Equality Let :, R. Then, = Proof: By Fourier Expansion Theorem, =, By linearity of inner product =, By orthonormality of s = 7
Parseval Equality Parseval Equality for Boolean Functions Let :,,. Then Proof:, = = By definition of inner product, = E, [ ] = Since is Boolean 8
BLR Test in {-,} notation BLR Test (f, ε). Pick and independently and uniformly at random from,. 2. Set = and query on,, and. Acceptiff =. Vector product notation: =(,,, ) Sum-Of-Cubes Lemma. Proof: Indicator variable = if BLR accepts otherwise Pr BLR accepts =,, 2 + 2 [] = +. Pr,, BLR accepts = E,, = 2 + 2 E,, By linearity of expectation 9
Proof of Sum-Of-Cubes Lemma So far: Pr BLR accepts =,, + Next: E,, E,, By Fourier Expansion Theorem = E,, () [] () [] () [] Distributing out the product of sums = E,, () () (),, [] By linearity of expectation = E,, [ () () ()],, [] 2
Proof of Sum-Of-Cubes Lemma (Continued) Pr,, Claim. BLR accepts LetΔdenote symmetric difference of sets and a E,, [ () () ()] = 2 + 2,, [] E,, [ () () ()] E,, [ () () ()]is if ==and otherwise. a= E,, a= E,, a= E,, a= E, E, a= E, [ ] E, [ ] a= E [ ] E {,} [ ] {,} Since = Since = = Since and are independent Since and s coordinates are independent = when Δ= and Δ= == otherwise 2
Proof of Sum-Of-Cubes Lemma (Done) Pr,, BLR accepts = 2 + 2,, [] E,, [ () () ()] = 2 + 2 [] Sum-Of-Cubes Lemma. Pr BLR accepts =,, 2 + 2 [] 22
Proof of Correctness Theorem Correctness Theorem (restated) If is ε-far from linear then Pr BLR accepts. Proof: Suppose to the contrary that < Pr BLR accepts,, = 2 + 2 [] 2 + 2 max = 2 + 2 max Then max > 2. That is, > 2for some. But =, = 2(fraction of disagreements between and ) disagrees with a linear function on <fraction of values. By Sum-Of-Cubes Lemma Since Parseval Equality 23
Summary BLR tests whether a function :, {,}is linear or -far from linear ( 2 values need to be changed to make it linear) in time. 24