Matrix Rigidity of Random Toeplitz Matrices


 Victoria Simpson
 6 months ago
 Views:
Transcription
1 Matrix Rigidity of Radom Toeplitz Matrices Oded Goldreich Avishay Tal May 7, 2015 Abstract We prove that radom by Toeplitz (alteratively Hakel) matrices over F 2 have rigidity Ω( 3 r 2 log ) for rak r, with high probability. This improves, for r = o(/ log log log ), over the Ω( 2 r log( r )) boud that is kow for may explicit matrices. Our result implies that the explicit triliear [] [] [2] fuctio defied by F (x, y, z) = i,j x iy j z i+j has complexity Ω( 3/5 ) i the multiliear circuit model suggested by Goldreich ad Wigderso (ECCC, 2013), which yields a exp( 3/5 ) lower boud o the size of the socalled caoical depththree circuits for F. We also prove that F has complexity Ω( 2/3 ) if the multiliear circuits are further restricted to be of depth 2. I additio, we show that a matrix whose etries are sampled from a 2 biased distributio has complexity Ω( 2/3 ), regardless of depth restrictios, almost matchig the O( 2/3 ) upper boud for ay matrix by Goldreich ad Wigderso. We tur this radomized costructio ito a explicit 4liear costructio with similar lower bouds, usig the quadratic smallbiased costructio of Mossel et al. (RS&A, 2006). Keywords: Matrix rigidity, multiliear fuctios, multiliear circuits. Weizma Istitute of Sciece, Rehovot, Israel. Partially supported by the Mierva Foudatio with fuds from the Federal Germa Miistry for Educatio ad Research. Weizma Istitute of Sciece, Rehovot, Israel. Supported by a Adams Fellowship of the Israel Academy of Scieces ad Humaities, by a ISF grat ad by the ICORE Program of the Plaig ad Budgetig Committee.
2 Cotets 1 Itroductio Matrix Rigidity The Project of GoldreichWigderso Resolvig the Foregoig Ope Problems Overview of the Proof of Theorem Orgaizatio Prelimiaries 5 3 Mai Results 6 4 The Structure of Matrices of Small Biliear Circuits The Structure of Matrices Associated with Depth Two Biliear Circuits The Structure of Matrices Associated with Geeral Biliear Circuits Substructures Testig AN Complexity ad AN2 Complexity Lower Bouds for the ANComplexity of SmallBiased Matrices Lower Bouds for the AN2Complexity of Radom Toeplitz Matrices Explicit 4Liear Fuctios with ANComplexity Ω( 2/3 ) Digest ad Ope Problems Digest Ope Problems Refereces 21 Appedices 22 A.1 Geeralizatio to Larger Fields A.2 The Structure of Matrices Associated with Geeral Biliear Circuits
3 1 Itroductio This paper cocers the costructio of rigid matrices, a cetral ope problem posed by Valiat [Val77], ad its applicatio to lower bouds o caoical depththree Boolea circuits (where caoical is as defied by Goldreich ad Wigderso [GW13]). I particular, we improve the kow lower boud o matrix rigidity, but the improvemet is for a rage of parameters that is ot the oe motivated by Valiat s problem, but rather the oe that arises from [GW13]. Ideed, this improvemet resolves ope problems posed by Goldreich ad Wigderso [GW13]. 1.1 Matrix Rigidity The Matrix Rigidity Problem (i.e., providig explicit matrices of high rigidity) is oe of the most allurig problems i arithmetic circuits lower bouds. Itroduced i 1977 by Valiat [Val77], the problem was origially motivated by provig lower bouds for the computatio of liear trasformatios. Loosely speakig, a matrix is called rigid if it caot be writte as a sum of a low rak matrix ad a sparse matrix. Needless to say, the actual defiitio specifies both parameters. Defiitio 1.1 (Matrix Rigidity, [Val77]). A matrix A over a field F has rigidity s for rak r if every matrix of rak at most r (over F) disagrees with A o more tha s etries. Valiat showed that ay matrix with rigidity 1+δ for rak ω(/ log log ), where δ is some costat greater tha 0, caot be computed by a liear circuit of size O() ad depth O(log ). Valiat also proved that almost all by matrices, over a fiite field F (e.g., the twoelemet field F 2 ), have rigidity Ω(( r) 2 / log ) for rak r. Sice the, comig up with a explicit 1 rigid matrix has remaied a challege. The best techiques to date provide explicit by matrices of rigidity 2 r log( r ) for rak r (see [Lok09] for a survey about matrix rigidity). To the best of our kowledge, this state of affairs also holds for simple radomized costructios that use O() radom bits. The commo belief is that rigidity bouds for such radomized costructios ca be used for provig lower bouds for explicit computatioal problems that are related to the origial oes. For example, a adequate rigidity lower boud for radom Toeplitz (or Hakel) matrices 2 would yield a lower boud o the complexity of computig explicit biliear trasformatios. Ideed, this is aalogous to Adreev s proof of formula lower bouds [Ad87], where a lower boud for a radomized fuctio is trasformed ito a lower boud for a explicit fuctio (which takes the radom bits of the costructio as part of its iput). Our mai result is the followig Theorem 1.2 (o the rigidity of radom Toeplitz/Hakel matrices). Let A F2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], with probability 1 o(1), the matrix A has rigidity Ω( 3 ) for rak r. r 2 log Our bouds are asymptotically better tha Ω( 2 r log( r )) for rak r = o( log log log ), alas Valiat s origial motivatio refers to r > / log log. For rak r = 0.5+ε, where ε (0, 0.5), our boud yields a sigificat improvemet (i.e., 3 = 2 2ε 1.5 ε = 2 r 2 r ), ad this is actually the rage that is relevat for the project of Goldreich ad Wigderso [GW13]. 1 For a ifiite I N, the sequece of matrices, {A } I such that A is a matrix, is called explicit if there exists a poly()time algorithm that o iput I outputs the matrix A (ad outputs if I). 2 Recall that a Toeplitz matrix T = (T i,j) has costat diagoals (i.e., T i,j = T i+1,j+1 for every i, j). Hakel matrices are obtaied by turig Toeplitz matrices upside dow; that is, a Hakel matrix H = (H i,j) has costat skewdiagoals (i.e., H i,j = H i+1,j 1 for every i, j). Hece, ay claim regardig oe family traslates to a equivalet claim regardig the other family. 1
4 1.2 The Project of GoldreichWigderso The project started by Goldreich ad Wigderso [GW13] provides aother motivatio for the study of matrix rigidity. I fact, the problem of improvig the rigidity bouds for radom Toeplitz matrices was posed explicitly there. Specifically, provig a rigidity boud of 1.5+Ω(1) for rak 0.5+Ω(1) for radom Toeplitz matrices was proposed there as a possible ext step. Lower Bouds for Depth Three Caoical Circuits. Håstad [Hås86] showed that ay depththree Boolea circuit computig the way parity fuctio must be of size at least exp( ); to date, Håstad s result is the best lower boud for a explicit fuctio i the model of depththree Boolea circuits 3. The work of Goldreich ad Wigderso [GW13] put forward a model of depth three caoical circuits, with the uderlyig logterm goal to exhibit better lower bouds for geeral depththree Boolea circuits. Caoical circuits are restricted type of such depththree circuits, which ca be illustrated by cosiderig the smallest kow depththree circuits for way parity. The latter Õ(2 )size circuits are obtaied by combiig a CNF that computes a way parity with DNFs that compute way parities of disjoit blocks of the iput bits. The costructio, ad its optimality by [Hås86], suggests the followig scheme for obtaiig Boolea circuits that compute multiliear fuctios. First, costruct a arithmetic circuit that uses arbitrary multiliear gates of parameterized arity, ad the covert it to a Boolea circuit whose size is expoetial i the maximum betwee the arity ad the umber of gates i the arithmetic circuit. The arithmetic model is outlied ext. Lower Bouds for Multiliear Circuits. Suppose we wish to compute a tliear fuctio that depeds o t blocks of iputs, x (1),..., x (t), each of legth ; that is, the fuctio is liear i each of the x (j) s. We cosider circuits that use arbitrary multiliear gates of parameterized arity. That is, the circuits are directed acyclic graphs, where each iteral ode computes a multiliear fuctio of its iputs. We further restrict our circuit such that each iteral gate computes a multiliear formal polyomial i the iputs x (1)..., x (t). We say that such a multiliear circuit is of ANcomplexity 4 m if m equals the maximum betwee the umber of the circuit gates ad the maximal arity of the gates. For a multiliear fuctio F, we deote by C(F ) the miimal ANcomplexity of a multiliear circuit which compute the fuctio F. (We will abuse otatio ad refer to the ANcomplexity of a tesor/matrix as the ANcomplexity of the correspodig multiliear fuctio.) I the example of parity, we have a bottom layer of gates each takig iputs ad computig their parity. Above these gates, we have a gate which takes the results ad computes their parity. Overall, we got a (multi)liear circuit of ANcomplexity + 1. Goldreich ad Wigderso showed that ay multiliear circuit of ANcomplexity m yields a depththree Boolea circuit of size exp(m) computig the same fuctio (see [GW13, Prop. 2.9]). I fact, the Boolea circuits have much more structure, ad are referred to by Goldreich ad Wigderso as caoical circuits. Thus, a prelimiary step towards beatig the exp(ω( )) lower boud o the size of depththree Boolea circuits for explicit O(1)liear fuctios, 5 will be to beat the Ω( ) ANcomplexity lower boud for such fuctios i the model of multiliear circuits. Agai, as i Valiat s questio, if we just ask about the existece of hard tliear fuctios, the most tliear fuctios caot be computed by a multiliear circuit of ANcomplexity smaller tha (t) t/(t+1) : See [GW13, Thm. 4.1], which uses a coutig argumet. The more importat ad 3 That is, circuits of ubouded fai OR ad AND gates with leaves that are variables or their egatios. 4 where AN stads for Arity ad Number of gates. 5 Ideed, this suggestio presumes that there exist O(1)liear fuctios that require depththree Boolea circuits of size exp(ω( )), which is also a ope problem suggested i [GW13]. 2
5 challegig problem is to came up with a explicit tliear fuctio for which such bouds, or eve just ω( ) lower bouds, ca be proved. Reductio to (Structured) Rigidity. Goldreich ad Wigderso reduces the problem of provig lower bouds for biliear circuits to the problem of rigidity [GW13, Sec. 4.2]. They show that if a biliear circuit is of ANcomplexity m/2, the its correspodig matrix is ot m 3 rigid for rak m (i.e., it ca be expressed as a sum of a m 3 sparse matrix ad a matrix of rak at most m). Hece, ay matrix that has rigidity m 3 for rak m correspods to a biliear fuctio that caot be computed by a biliear circuit of ANcomplexity at most m/2. Furthermore, Goldreich ad Wigderso show that the sparse matrix arisig from their reductio has a additioal structure (to be specified later). This leads to a weaker otio of rigidity (see [GW13, Thm. 4.12] which establishes a separatio), called structured rigidity, for which it is potetially easier to prove lower bouds. Ope Problems i GoldreichWigderso. Oe ope problem posed by Goldreich ad Wigderso is provig that radom Toeplitz matrices have rigidity m 3 (or just structured rigidity m 3 ) for rak m = 0.5+Ω(1). This would yield a ANcomplexity lower boud of m for the correspodig biliear fuctio (via the reductio i [GW13, Thm. 4.4]) 6 as well as a similar lower boud for the followig explicit triliear fuctio (via [GW13, Prop. 4.6]): F tet (x, y, z) = i 1,i 2,i 3 []: 3 j=1 i j /2 /2 x i1 y i2 z i3. (1) 1.3 Resolvig the Foregoig Ope Problems We resolve the aforemetioed ope problem [GW13, Prob. 4.8] by provig that radom Toeplitz matrices have rigidity m 3 for rak m = Θ( 3/5 ), with high probability. This follows from our log 1/5 mai theorem (Theorem 1.2) by choosig r = m. Furthermore, we ca get rid of the logarithmic factor i the Ω otatio, by provig a slightly better lower boud for structured rigidity. Theorem 1.3 (o the structured rigidity of radom Toeplitz/Hakel matrices). Let A F 2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], the matrix A has structured rigidity Ω( 3 /r 2 ) for rak r. This implies (usig [GW13, Thm. 4.10] ad [GW13, Prop. 4.6]) that the ANcomplexity of a radom Toeplitz matrix is Ω( 3/5 ), ad ditto for the explicit triliear fuctio F tet from Eq. (1). This resolves Problems 4.7 ad 4.2 i [GW13], resp. I additio, we show that aother explicit triliear fuctio has ANcomplexity Ω( 3/5 ). Corollary 1.4 (ANcomplexity lower boud for a explicit triliear fuctio). Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 x iy j z i+j. The, C(F ) = Ω( 3/5 ). New Challeges. The most atural questio that arises from the foregoig results is to tighte the lower boud; that is, to show that radom Toeplitz matrices have ANcomplexity Ω( 2/3 ) as cojectured by [GW13]. This would be the best possible, sice ay biliear fuctio ca be computed by a biliear circuit of ANcomplexity O( 2/3 ); more geerally, by [GW13, Thm. 3.1], 6 For structured rigidity, we use [GW13, Thm. 4.10]. 3
6 for ay t 2, ay tliear fuctio ca be computed by a tliear circuit of ANcomplexity O((t) t/(t+1) ). Aother atural follow up questio is to exhibit a explicit O(1)liear fuctio havig ANcomplexity Ω( α ) for some costat α > 3/5; of course, the larger α, the better. Our progress o these ope problems is captured by the followig two results. Theorem 1.5 (depthtwo ANcomplexity lower boud for radom Toeplitz matrices). Let F be a biliear fuctio that correspods to a radom Toeplitz matrix. The, with probability 1 o(1), the fuctio F caot be computed by multiliear circuits of depth two havig ANcomplexity 2/3 /(log ) 1/3. Theorem 1.5 establishes the desired ANcomplexity lower boud for radom Toeplitz matrices, but oly for depthtwo multiliear circuits. We ote that the ANcomplexity upper boud of [GW13, Thm. 3.1] holds via depthtwo circuits, ad so Theorem 1.5 is almost optimal with respect to depthtwo multiliear circuits. Theorem 1.5 implies that the triliear fuctio F (x, y, z) = j=1 x iy j z i+j caot be computed by multiliear circuits of depth two ad ANcomplexity 2/3 /(log ) 1/3. Theorem 1.6 (improved ANcomplexity lower boud for 4liear fuctios). There exists a explicit 4liear fuctio havig ANcomplexity Ω( 2/3 /(log ) 1/3 ). Theorem 1.6 is proved by first showig that, with high probability, biliear fuctios associated with matrices that are sampled from a 2 biased sample space (over {0, 1} 2 ) have ANcomplexity Ω( 2/3 ). Note that by the aforemetioed upper boud, this lower boud is tight (up to logarithmic factors). Next, we ote that samplig such matrices ca be doe usig O() radom bits [NN93, AGHP92, MST06], which matches the amout of radomess used for samplig a radom Toeplitz matrix. Furthermore, i the explicit smallbiased costructio of Mossel et al. [MST06], each bit i the sampled strig is a biliear fuctio of the radom bits, allowig us to give a explicit 4liear fuctio with ANcomplexity Ω( 2/3 ). 1.4 Overview of the Proof of Theorem 1.2 We give a overview of the proof of Theorem 1.2 (for the case of Hakel matrices). Recall that we wish to show that a radom Hakel matrix has rigidity 3 /r 2 log for rak r, with high probability. Let A be a radom by Hakel matrix, of the form A i,j = a i+j for idepedet radom bits a 2,..., a 2. Suppose that A ca be expressed as a sum of a ssparse matrix S ad a matrix of rak at most r. Cosider a partitio of A ad S ito (/2r) (/2r) submatrices, each of size 2r 2r, such that a geeric submatrix cosists of 2r cosecutive colums ad 2r equally spaced rows (i.e., rows that are at distace /2r apart). The, there exists a pair of correspodig submatrices A ad S such that S is of sparsity s = (2r)2 s (ad, of course, A S has rak at most r). Next, 2 we ote that ay of the above submatrices of A are of the form b 1 b 2 b 3... b 2r A = b k+1 b k+2 b k+3... b k+2r b (2r 1)k+1 b (2r 1)k+2 b (2r 1)k+3... b (2r 1)k+2r where k = /2r ( 2r, by the assumptio r ), ad b 1,..., b (2r 1)k+2r is a cosecutive subsequece of a 2,..., a 2. Notice that A is a 2r 2r submatrix that depeds o (2r 1)k + 2r = Θ() radom bits. 4
7 I our mai lemma, we show that for ay fixed matrix S (eve if S is ot sparse) the submatrix matrix A S is of rak greater tha r with probability at least 1 2 Ω(), where the probability is take over the choice of A (equiv., over the choice of b 1,..., b (2r 1)k+2r ). Hece, takig a uio boud over all possible s sparse submatrices we get that, with probability at least 1 ( 2r 2r s ) 2 Ω(), the submatrix A has rigidity s for rak r. Pickig s = o( 3 r 2 log ) implies that s = o(/ log ), which completes the proof (by applyig a uio boud o all submatrices, ad iferrig that, with high probability, matrix A is of rigidity s for rak r). 1.5 Orgaizatio Our mai results (i.e., Theorems 1.2 ad 1.3 ad Corollary 1.4) are proved i Sectio 3, which follows a short prelimiary sectio (Sec. 2). Next, Theorems 1.5 ad 1.6 are proved, i two steps. I Sectio 4 we idetify structural properties of matrices that correspod to biliear fuctios of low AN (ad AN2) complexity. These properties correspod to (eve more) restricted otios of structured rigidity, ad i Sectio 5 we show that (with high probability) matrices draw from the two relevat distributios do ot satisfy these properties. We coclude, with a techical digest (Sectio 6.1) ad a list of some ope problems (Sectio 6.2). 2 Prelimiaries We deote by [] = {1,..., }. For, k N, we deote by ( k) = k ( i=0 i), ad use the boud ( ) k (2) k. For a matrix A, we deote its ith row by A i, ad its jth colum by A (j). We deote by wt(a) the umber of ozero etries i the matrix A, ad say that A is ssparse if wt(a) = s. A Hakel matrix over a field F is a square matrix with costat skewdiagoals; that is, ay matrix A F of the form A i,j = a i+j for some a 2,..., a 2 F. A Toeplitz matrix over a field F is a square matrix with costat diagoals, i.e. ay matrix A F of the form A i,j = a i j for some a ( 1),..., a 1 F. Note that a Hakel matrix is a upsidedow Toeplitz matrix. Throughout the paper, uless specified otherwise, we talk about matrices over the field F 2, ad matrix rak refers to its rak over F 2. Defiitio 2.1 (Structured Rigidity, [GW13, Def. 4.9]). We say that a matrix A has structured rigidity (m 1, m 2, m 3 ) for rak r if for every matrix R of rak at most r ad for every X 1,... X m1, Y 1,..., Y m1 [] such that X 1 = = X m1 = m 2 ad Y 1 = = Y m1 = m 3 it holds that A R m 1 k=1 (X k Y k ), where M S meas that all ozero etries of the matrix M reside i the set S [] []. We say that a matrix A has structured rigidity m 3 for rak r if A has structured rigidity (m, m, m) for rak r. Ideed, ay matrix that has rigidity s for rak r, also has structured rigidity s for rak r, but the other directio does ot hold (see [GW13, Thm. 4.12]). Defiitio 2.2. A multiliear circuit o t blocks of iputs x (1),..., x (t) {0, 1} is a directed acyclic graph whose odes are associated with arbitrary multiliear gates such that ay two gates with directed paths from the same block of iputs are ot multiplied together by aother gate. Defiitio 2.3 (the ANcomplexity of multiliear circuits with geeral gates, [GW13, Def. 2.2]). The arity of a multiliear circuit is the maximum arity of its (geeral) gates. The ANcomplexity of a multiliear circuit is the maximum betwee its arity ad its umber of gates (where we cout oly the geeral gates ad ot the leaves, i.e., variables). The ANcomplexity of a multiliear fuctio F, 5
8 deoted C(F ), is the miimum ANcomplexity of a multiliear circuit that computes F. The AN2 complexity of a multiliear fuctio F, deoted C 2 (F ), is the miimum complexity of a depthtwo multiliear circuit that computes F. Theorem 2.4 ([GW13, Thm. 4.10]). If A is a by matrix that has structured rigidity m 3 for rak m, the the correspodig biliear fuctio F satisfies C(F ) m/2. 3 Mai Results We prove our results bottomup, startig with the mai lemma, as metioed i the proof overview. Lemma 3.1 (Mai Lemma). Let m, k N, 16 k m. Let A F m m 2 be the radom matrix a 1 a 2 a 3... a m a k+1 a k+2 a k+3... a k+m a (m 1)k+1 a (m 1)k+2 a (m 1)k+3... a (m 1)k+m where a 1,..., a (m 1)k+m are uiform idepedet radom bits, ad let S F m m 2 be some fixed matrix. The, Pr A [rak(s + A) m/2] 2 km/16. Note that for k = 1 the matrix i Lemma 3.1 is a radom Hakel matrix, ad for k = m it is a totally radom matrix. The requiremet k 16 is ot essetial i the lemma; it is used to make expressios icer. For k 1 ad rak r m/2 the proof gives Pr A [rak(s+a) r] ( m r) 2 mk/8. Proof. For a fixed S ad a radom A as above, let B = S + A. If r = rak(b) m/2, the oe ca costruct a basis B i1, B i2,..., B ir of the row space of B by the followig iterative process: Let i 1 be the first ozero row of B, let i 2 > i 1 be the first row i B that is ot spaed by row i 1, let i 3 > i 2 be the first row i B that is ot spaed by rows i 1 ad i 2, etc. We get that i 1 < i 2 < < i r ad 1. For j < i 1 the jth row of B is the all zeroes row. 2. For i t 1 < j < i t the jth row of B is spaed by rows i 1,..., i t 1 of B. 3. For i r < j the jth row of B is spaed by rows i 1,..., i r of B. More cocisely, deotig by I = {i 1,..., i r }, we get j [m] \ I : B j spa{b i : i I, i < j}. (2) We boud the probability that such a sequece I = {i 1,..., i r } exists, where r m/2. We will uio boud over all possible sequeces I, ad for ay fixed sequece of legth at most m/2, we shall show that (2) holds with very low probability. Give such a sequece I, let J = [m] I be its complemet. Settig = m/k, we ca select a icreasig sequece of J / idices i J such that each two idices differ by at least. 7 Take j 1 < j 2 < < j t to be such a sequece of idices, where t J m/2 m/k k 4. For l [t], let E l be the evet that row j l is spaed by the rows idexed by I [j l 1]. The, Pr [Eq. (2) holds for I] Pr[E 1, E 2,..., E t ] = Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] (3) 7 Oe ca costruct such a set greedily: choose the miimal idex j i J, remove all idices i J [j, j + 1]. Repeat util J is empty. 6
9 Next, we show that for each l [t], we have Pr[E l E 1,..., E l 1 ] 2 m/2. However, istead of coditioig o E 1,..., E l 1, we shall coditio o a set of the radom bits, to be specified ext, that determie rows B 1,..., B jl 1 o oe had, but are idepedet from the radom row B jl o the other had. Sice j l j l 1 + m/k by our desig, we get (j l 1)k (j l 1 1)k + m. Hece, the radom bits a 1,..., a (jl 1)k, which determie B 1,..., B jl 1, leave the radom row B jl = (a (jl 1)k+1,..., a (jl 1)k+m) totally udetermied. Coditioig o the worstcase assigmet for the former radom variables (uder which E 1,..., E l 1 holds) yields a upper boud o Pr[E l E 1,..., E l 1 ]. Thus, it is eough to show that Pr[E l a 1,..., a (jl 1)k] 2 m/2 for ay possible fixed choice of values to a 1,..., a (jl 1)k. To avoid multiple subscripts, we set for the rest of the proof j j l. Let us remark that after fixig a 1,..., a (j 1)k, rows 1,..., j m/k are completely fixed, rows j m/k + 1,..., j 1 are partially fixed, ad row j is etirely udetermied. Based o that, we shall show that Pr[E l a 1,..., a (j 1)k ] 2 m/2. (4) Let I := I [j 1], ad fix a liear combiatio of the rows idexed by I, i.e., i I c ib i, amog all 2 I such liear combiatios. We show that the probability that B j = i I c i B i. (5) is 2 m. (This is similar, up to mior differeces, to the folklore result that ay fixed liear combiatio of rows i a radom Toeplitz matrix is distributed uiformly over F m 2 see [Gol08, Prop. 8.25]. We give the details for completeess.) The probability that the first bit of B j equals the first bit of the liear combiatio i (5) is exactly 1/2, sice B j,1 = S j,1 + a (j 1)k+1, ad all etries {B i,1 } i I ivolve oly bits from a 1,..., a (j 2)k+1, which were already fixed (sice (j 2)k + 1 (j 1)k). Fixig a (j 1)k+1 such that equality o the first bit holds, the secod bit B j,2 equals the resultig liear combiatio with probability 1/2 as well. This happes sice B j,2 equals S j,2 + a (j 1)k+2, where a (j 1)k+2 was t already fixed, ad all etries {B i,2 } i I ivolve oly bits from a 2,..., a (j 2)k+2, which were already fixed (sice (j 2)k + 2 (j 1)k + 1). Ad so o, every bit i the jth row of B equals the resultig liear combiatio with probability 1/2, coditioed o the fixig of the previous bits. Overall, B j = i I c ib i with probability 2 m for a fixed choice of coefficiets {c i } i I. 8 Takig a uio boud over all possible coefficiets {c i } i I gives Pr[E l E 1,..., E l 1 ] 2 I 2 m 2 m/2. Pluggig this boud ito Eq. (3) we get Pr [Eq. (2) holds for I] Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] ( 2 m/2) t 2 mk/8. where i the last iequality we used t k/4. Takig a uio boud over all possible sequeces I of legth at most m/2, whose umber is defiitely less tha 2 m, ad usig k 16, we get Pr[rak(S + A) m/2] 2 m 2 mk/8 2 mk/16. We cotiue with our mai theorem. Theorem 3.2 (radom Hakel matrices are rigid). Let A F 2 be a radom Hakel matrix A i,j = a i+j where a 2,..., a 2 are uiform idepedet radom bits. The, for every r /32, with probability 1 o(1), the matrix A has rigidity 3 for rak r. 160r 2 log 8 Alteratively, coditioed o a 1,..., a (j 1)k ad the choice of the liear combiatio, there exist exactly oe choice for a (j 1)k+1,..., a (j 1)k+m that satisfies Eq. (5). 7
10 Before provig Theorem 3.2, we state a immediate corollary of it. Corollary 3.3. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that for every ε > 0 1. With probability 1 o(1), the matrix A is c 2 2ε / log rigid for rak 1/2+ε. 2. With probability 1 o(1), the matrix A is m 3 rigid for rak m = c 3/5 log 1/5. 3. With probability 1 o(1), the matrix A is c 1+2ε / log rigid for rak 1 ε. Proof of Theorem 3.2. Suppose towards cotradictio that A ca be represeted as a sum of a matrix R of rak at most r, ad a ssparse matrix S, where s 3 /160r 2 log. Let m = 2r, ad assume for coveiece that k = /m is a iteger. Cosider the followig partitio of A s etries ito (/m) 2 submatrices, each of dimesio m m. For i [/m] ad j [/m], let I i = {i, i + k,..., i + (m 1)k}, J j = {(j 1)m + 1, (j 1)m + 2,..., jm}. (6) Deote by A i,j (R i,j, S i,j, resp.) the matrix A (R, S, resp.) restricted to rows I i ad colums J j. See Figure 1 for a example of such a submatrix. The mai observatio is that for each a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 16 Figure 1: A submatrix A 1,1 of the matrix A, for m = 4 ad k = 2. (i, j) [/m] 2, the matrix A i,j is of the form eeded by the mai lemma. Aother observatio is that sice the submatrices S i,j partitios the sparse matrix S, oe of them has sparsity at most s s m2. I additio, sice rak of a submatrix may oly decrease, for every i, j, it holds that 2 rak(r i,j ) rak(r) r. We say that A i,j is simple if it ca be represeted as a sum of a s sparse matrix ad a matrix of rak at most r. By the above discussio, A ca be represeted as S + R where S is ssparse ad R is of rak at most r, oly if there exists a submatrix A i,j that is simple. We shall show that the latter occurs with very low probability: Pr [ i, j : A i,j is simple ] i,j i,j Pr[A i,j is simple] S F m m 2 : wt(s) s Pr[rak(A i,j + S) m/2] (Uio Boud) (Uio Boud) ( ) ( ) 2 m 2 m s 2 mk/16 (Lemma 3.1) < 2 (2m 2 ) s 2 /16. (km = ) Fially, usig s 40 log, which follows from s 3 160r 2 log, we get that Pr[ i, j : Ai,j is simple] = o(1), which completes the proof. 8
11 Note that the proof works as log as the umber of possibilities for a s sparse matrix S i,j is smaller tha 2 /16 / 2. Our ext theorem exploits the fact that there is a smaller umber of possibilities for submatrices of structured sparse matrices (as i Defiitio 2.1). I fact, this is the oly property of S that the foregoig proof uses. This yields the followig improved boud. Theorem 3.4 (radom Hakel matrices are structured rigid). Let A F 2 be a radom Hakel matrix. The, for every r /32, ad s 3 /1000r 2, with probability 1 o(1), the matrix A has structured rigidity s for rak r. Before provig Theorem 3.4 we state three corollaries of it. The first corollary is immediate by choosig r = 3/5. Corollary 3.5. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that with probability 1 o(1), the matrix A has structured rigidity c 9/5 for rak 3/5. The secod corollary follows from the first corollary ad Theorem 2.4. Corollary 3.6. Let A F 2 be a radom Hakel matrix, ad let F (x, y) = j=1 A i,jx i y j. The, with probability 1 o(1), it holds that C(F ) = Ω( 3/5 ). The last corollary shows that there exists a explicit triliear form with ANcomplexity Ω( 3/5 ). This is the first improvemet over the trivial Ω( ) lower boud for explicit tesors, ad i doig so it solves Problem 4.2 from [GW13] i the affirmative. Goldreich ad Wigderso [GW13, Prop. 4.6] show that if some Toeplitz matrix have ANcomplexity Ω(m), the F tet defied i Eq. (1) has ANcomplexity Ω(m) has well. We follow their method, but preset a simpler argumet for a differet triliear fuctio. Corollary 3.7. Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 z i+jx i y j. The, C(F ) = Ω( 3/5 ). Proof. Accordig to Corollary 3.6, there exists a Hakel matrix A, defied by some diagoal values a 2,..., a 2, such that the biliear form i,j a i+jx i y j has ANcomplexity Ω( 3/5 ). Let C be a triliear circuit computig F with miimal ANcomplexity, ad deote its complexity by m. Fixig the values of the variables z i to a i, for all i {2,..., 2}, we get a biliear circuit i x ad y of ANcomplexity at most m. Thus, m = Ω( 3/5 ). We retur to prove Theorem 3.4. Proof of Theorem 3.4. The proof follows the lies of the proof of Theorem 3.2. We let m = 2r, k = /m, ad t = s 1/3. We assume towards cotradictio that A = S + R, where R is of rak at most r, ad S is a sum of t matrices S 1,..., S t F 2, such that the oes i each matrix S l are a subset of some X l Y l, where X l, Y l t. Deote by T the by matrix over F 2 with T i,j = 1 iff (i, j) is cotaied i at least oe X l Y l. It is clear from T s defiitio that the oes i S are a subset of the oes i T. As i Theorem 3.2, we partitio A, R, S, ad also T, to (/m) 2 submatrices, accordig to the partitio of row idices I 1,..., I /m ad colum idices J 1,..., J /m, defied as i the proof of Theorem 3.2 (see Eq. 6). For a radom (i, j) [/m] 2, it holds that [ E wt(t i,j ) ] t 3 m2 i,j 2, E i,j [ t ] X l I i t t m, E i,j [ t ] Y l J j t t m. (7) 9
12 We say that a submatrix T i,j is good if wt(t i,j ) 4t 3 m2 2, t X l I i 4t t m, t Y l J j 4t t m. (8) Usig Markov s iequality, each of the above three evets happe with probability at least 3/4. Usig uio boud (o the complemet evets) with probability at least 1/4 all evets occur simultaeously, makig T i,j good. Next, we cout the umber of possible good submatrices T i,j. Each such submatrix is uiquely determied by the sets X 1,..., X t ad Y 1,..., Y t, where X l = X l I i ad Y l = Y l J j. Furthermore, a collectio (X 1,..., X t) such that l X l 4t2 m correspods to a set X I i [t] of size at most 4t 2 m such that (p, l) X iff p X l (ad similarly for (Y 1,..., Y t )). Hece, the umber of possible good submatrices is at most { X I i [t] : X 4t2 m } 2 ( = mt 4t 2 m/ ) 2 ( ) 2 (2mt) 4t2 m/ 16t 2m/. We say that S i,j is good if T i,j is good, ad we say that A i,j is simple if it is the sum of a good S i,j ad a matrix of rak at most r. Next, we cout the umber of possible good submatrices S i,j. Sice the oes of S i,j are a subset of the oes i T i,j, the umber of possibilities for S i,j is at most 16t2 m/ 2 wt(t i,j) 16t2 m/ 2 4t3 m 2 / 2. Usig the boud o the umber of possible good submatrices S i,j, we may boud the probability that some A i,j is simple: Pr [ i, j : A i,j is simple ] Pr[rak(A i,j + S i,j ) m/2] (Uio Boud) i,j Recall that m = 2r ad k = /m to get which is o(1) for t 3 S i,j good ( ) 2 16t 2 m/ 2 4t3 m 2 / 2 2 mk/16 (Lemma 3.1) m Pr [ i, j : A i,j is simple ] 2 2 log + 32 log t2 r/ + 16t 3 r 2 / 2 /16, 3 ad r. 1000r 2 Geeralizatio to Larger Fields. The choice of field F 2 was ot crucial i the proofs of Lemma 3.1, Theorem 3.2 ad Theorem 3.4. Oe ca sytactically replace the field size 2 by ay prime power q, keepig the proofs itact. Furthermore, i Theorem 3.2, we slightly beefit from takig a larger field. For details see Appedix A.1. 4 The Structure of Matrices of Small Biliear Circuits I this sectio we shall further refie the structure of matrices associated with small biliear circuits, beyod the structure captured by Defiitio 2.1 ad Theorem 2.4. We begi with the structure that arises from depth2 biliear circuits, ad the cotiue to the structure arisig from geeral biliear circuits. Our aalysis follows the proof of [GW13, Thm. 4.4], ad it ca be viewed as relatig to fier otios of structured rigidity (tha the oe of Defiitio 2.1). We the follow the first part of the proof of Theorem 3.4, ad fid submatrices with correspodig (rigiditylike) parameters. 10
13 4.1 The Structure of Matrices Associated with Depth Two Biliear Circuits We say a row/colum i a matrix is msparse if it cotais at most m ozero etries. Likewise, a liear fuctio l(x) (resp. l (y)) is msparse if it depeds o at most m etries i x (resp. y). Lastly, recall that by Defiitio 2.3, C 2 (F ) is the miimal ANcomplexity of a depthtwo biliear circuit computig F. Propositio 4.1 (Structure of fuctios computed by depth two biliear circuits). If C 2 (F ) = m, the F ca be expressed as L i(x)l i (y) + Q l (x, y) where L 1,..., L m are msparse liear fuctios, L 1,..., L m are geeral liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = CL row + S l (9) where C is a geeral m matrix, L row is a m matrix with msparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. Propositio 4.1 is proved explicitly i the warmup part of the proof of [GW13, Thm. 4.4]. 4.2 The Structure of Matrices Associated with Geeral Biliear Circuits Propositio 4.2 (Structure of fuctios computed by geeral biliear circuits). If C(F ) = m, the F ca be expressed as L i (x)l i(y) + M i(x)m i (y) + Q l (x, y) where L 1,..., L m ad M 1,..., M m are msparse liear fuctios, L 1,..., L m ad M 1,..., M m are geeral liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = L col B + CL row + S l (10) where L col is a m matrix with msparse colums, B is a geeral m matrix, C is a geeral m matrix, L row is a m matrix with msparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. Propositio 4.2 is oly implicit i the proof of [GW13, Thm. 4.4], ad we iclude its proof i Appedix A Substructures I this subsectio, similarly to the first part of the proof of Theorem 3.4, we fid a submatrix (of a matrix associated with a biliear circuit) that has average rigiditylike parameters. Startig with Propositio 4.2, for C(F ) = m, we write the matrix A associated with F as A = L col B + CL row + m S l such that the ozero etries of S l are a subset of X l Y l, where X l, Y l m. Deote 11
14 by T = m X l Y l, ad ote that T m 3. Let I 1,..., I /2m ad J 1,..., J /2m be some fixed equipartitio of the row idices ad colum idices of A, respectively, where each I i ad J j is of size 2m. This partitio aturally defies (/2m) 2 submatrices as follows. For ay (i, j) we deote by A i,j (resp. S i,j l ) the matrix A (resp. S l) restricted to rows I i ad colums J j. For ay i (resp. j) we deote by L i col ad C i (resp. B j ad L j row) the matrices L col ad C (resp,. B ad L row ) restricted to I i (resp. J j ). The, oe ca write A i,j = L i col Bj + C i L j row + S i,j l, (11) where S i,j l T (I i J j ). Next, we show that there exists a choice of (i, j) with favorable properties (to be exploited i the ext sectio) of the submatrices of L i col, Lj row ad o the subsets {X l I i } l, {Y l J j } l, ad T (I i J j ). Propositio 4.3 (Structure of submatrix of matrices associated with small biliear circuits). There exists a (i, j) such that: (1) T (I i J j ) 24m5, (2) m X l I i 12m3, (3) m Y l J j 12m 3, (4) wt(li col ) 12m3 2, ad (5) wt(lj row) 12m3. If C 2 (F ) = m, the the same statemet holds, except that we ca replace L col ad B with the 0 m ad 0 m matrices, respectively. Proof. For a uiformly radom (i, j) [/2m] 2, it holds that E [ T (I i J j ) ] m 3 (2m)2 i,j [ 2 m ] E X l I i m m 2m i,j [ m ] E Y l J j m m 2m i,j E i,j [wt(li col )] m m 2m E i,j [wt(lj row)] m m 2m Usig Markov s iequality, each of the followig bad evets occur with probability at most 1/6 T (I i J j ) 6m 3 (2m)2 2 X l I i 6m m 2m Y l J j 6m m 2m wt(l i col ) 6m m 2m wt(l j row) 6m m 2m By uio boud, with probability at least 1 5/6 over the choice of (i, j), oe of the bad evets occur, which completes the proof. 12
15 We wish to express the structure captured by Eq. (11) i terms of liear equatios o the etries of the matrix A i,j l Si,j l. To do so we eed the followig defiitio. Defiitio 4.4 (Orthogoal Complemet of a Matrix). Let m. If A is a m matrix, ad B is a ( m) matrix of rak m such that BA = 0 the we say that B is a left orthogoal complemet of A. If A is a m matrix, ad B is a ( m) matrix of rak m such that AB = 0 the we say that B is a right orthogoal complemet of A. It is well kow that ay matrix over a field has a orthogoal complemet. Now, suppose that A i,j l S l i,j = L i col Bj + C i L j row (as i Eq. (11)). Let D be a m 2m matrix which is a left orthogoal complemet of L i col, ad let E be a 2m m matrix which is a right orthogoal complemet of L j row. The, D (A i,j l S l i,j ) E = 0 m m. (12) I the case of depth2 circuits we have A i,j l S l i,j = C i L j row. Usig E, the right orthogoal complemet of L j row as above, we ca write (A i,j l S l i,j ) E = 0 2m m. (13) I the ext sectio, we shall desig tests based o Equatios (12) ad (13). Remark: Note that there are may possible choices of a orthogoal complemet of a give matrix. Therefore, we shall refer to the left (right, resp.) orthogoal complemet of A as some caoical choice of a left (right, resp.) orthogoal complemet of A, say the first such matrix accordig to lexicographical order (over a fiite field F). 5 Testig AN Complexity ad AN2 Complexity We would like to desig a test such that matrices associated with (AN or AN2) complexity at most m will surely pass the test, whereas the matrices we are iterested i will fail it. (Sice the test is merely a metal experimet, i.e., we do ot ited to actually ru it, the test could be iefficiet.) The poit is that ay matrix o which the test fails must have complexity greater tha m. We will show that a radom Toeplitz matrix, as well as a matrix whose etries are sampled from a 2  biased distributio, will fail the test with overwhelmig probability, thus provig complexity lower bouds for such matrices. We will preset two tests: Oe for ANcomplexity failig most matrices take from a smallbiased space, ad oe for AN2complexity failig most Toeplitz matrices. 5.1 Lower Bouds for the ANComplexity of SmallBiased Matrices For i [/2m] ad j [/2m], let 9 I i = {i, i + (/2m),..., i + (2m 1) (/2m)}, J j = {(j 1) (2m) + 1, (j 1) (2m) + 2,..., j (2m)}. (14) 9 The specific choice for I i ad J j is ot crucial for our argumet i this subsectio, however it will be importat i the ext subsectio. Hece, sice we eed to pick some partitio, we might as well choose this oe. 13
16 ad deote by A i,j the 2mby2m submatrix of A obtaied by restrictig A to rows I i ad colums J j. Cosider the followig test, where A i,j is viewed as idexed by [2m] [2m] rather tha by I i J j. Test 1 ANComplexity Test Iput: Matrix A F 2 ad parameter m [] 1: for i = 1,..., /2m ad j = 1,..., /2m do 2: for all subsets {X i l }m of [2m] such that l Xi l 12m3 3: for all subsets {Y j l }m of [2m] such that l Y j l 12m3 4: Let T := m Xi l Y j 5: if T 24m5 the 2 6: for all matrices L i col l. do do 12m3 of dimesio 2m m ad sparsity at most do 7: Let D be the left orthogoal complemet of L i col. 8: for all matrices L j row of dimesio m 2m ad sparsity at most 12m3 do 9: Let E be the right orthogoal complemet of L j row. 10: if there exists N F 2m 2m 2 such that N T, ad D(A i,j N)E = 0 m m the 11: retur Pass. 12: retur Fail. The followig is a immediate corollary of Propositio 4.3 ad Eq. (12). Corollary 5.1. Every matrix associated with a biliear circuit of ANcomplexity at most m passes Test 1 with parameter m. I this subsectio we cosider a distributio of matrices whose etries are chose from a small biased sample space. Specifically, we shall use a sample space over strigs of legth N = 2 i order to defie by matrices. We shall show that almost all such matrices fail Test 1 with parameter m. But we eed a few prelimiaries first. Prelimiaries. Recall the defiitio of a εbiased distributio from [NN93]. Defiitio 5.2 (smallbiased distributio). A distributio X over {0, 1} N is said to be εbiased if for every oempty set S [N], it holds that E x X [( 1) i S x i ] ε. We shall use the followig property of εbiased distributios (implicit i [NN93]). Lemma 5.3 ([AGHP92, Lem. 1]). Let X be a εbiased distributio over {0, 1} N. Let l 1,..., l t be liearly idepedet liear fuctios o x 1,..., x N. The, the probability that all liear fuctios equal 0 simultaeously is at most ε + 2 t. We shall also use the followig simple fact from liear algebra. Fact 5.4. Let t,, m N such that t m. Let l 1,..., l t be a sequece of liearly idepedet liear fuctios (over F) o x 1,..., x. The, l 1,..., l t spa at least t m liearly idepedet fuctios that ivolve oly the variables x m+1,..., x. Proof. Thik of the liear fuctios as vectors i F, ad let V = spa{l 1,..., l t }. Cosider the subspace U = spa{e m+1,..., e }, where e i F is the uit vector with 1 i the ith coordiate ad 0 elsewhere. The, dim(u V ) dim(u) + dim(v ) = ( m) + t = t m, whereas U V is the spa of l 1,..., l t that is supported oly o the last m coordiates. 14
17 Actual Results. We are ow ready to aalyze the probability that a matrix sampled from a small biased space passes Test 1. The core of the aalysis refers to a sigle applicatio of Step 10, which refers to a specific choice of i, j, {Xl i}m, {Y j l }m as well as Li col, Lj row (which i tur, fixes D ad E as well). Lemma 5.5 (core of the aalysis of Test 1). Fix i, j, {Xl i}m j ad {Yl }m that pass the check of Step 5, ad fix L i col ad L j row (which i tur, fixes D ad E as well). The, a matrix A whose etries are sampled from a εbiased distributio satisfies the coditio i Step 10 with probability at most ε + 2 m2 +24m 5 / 2. Proof. For a fixed choice of i, j, {X i l }m, {Y j l }m, Li col ad L j row as above, we cosider a specific submatrix of dimesio 2m 2m of A, deoted A i,j. Note that the correspodig left (resp. right) orthogoal complemet of L i col (resp. L j row) is a mby2m (resp. 2mbym) matrix of rak m, deoted by D (resp. E). Recall that A i,j is a submatrix whose etries are sampled accordig to a εbiased distributio. Our goal is to show that the equatio D(A i,j N)E = 0 (checked i Step 10) implies a lot of liearly idepedet liear equatios o the etries of A i,j. Let Z be a 2m 2m matrix of (2m) 2 Boolea variables, where we will later take Z to be A i,j N. Iterpret the equatios DZE = 0 m m as m 2 liear equatios o the (2m) 2 variables i Z. For i [m] ad j [m], we have a equatio of the form D i ZE (j) = 0, where D i is the ith row of D ad E (j) is the jth colum of E. We ca write D i ZE (j) = 2 2 k=1 D i,k Z k,l E l,j = k,l (D i E (j) ) k,l Z k,l ; that is, the coefficiets of the equatio are the tesor product of the vector D i with the vector E (j). Thikig of these m 2 liear equatios o (2m) 2 variables as a big matrix of dimesio m 2 (2m) 2, we ote that this matrix of liear equatios is the tesor product of D ad E, sice the (i, j) row equals to D i E (j) (viewed as a (2m) 2 bit log vector). It is a kow fact that the rak of the tesor product of ay two matrices is the product of their rak; hece, we get rak(d E ) = rak(d) rak(e ) = m 2. I other words, we have a liearly idepedet set of m 2 liear equatios o the variables Z. However, we wat to get liear equatios over the variables of A, where Z = A N. Say that Z k,l is a oisy variable if (k, l) T. It will be eough to show that there are may idepedet liear equatios which ivolve oly ooisy variables of the matrix. Sice the umber of oisy variables is T, by Fact 5.4 we ca fid at least m 2 T idepedet liear equatios that do ot ivolve oisy variables. Overall, we got m 2 T idepedet liear equatios o A i,j. By Lemma 5.3, a submatrix A i,j whose etries are sampled accordig to a εbiased distributio satisfies all m 2 T equatios with probability at most ε + 2 m2 + T. Lastly, the fact that {Xl i}m j ad {Yl }m passed the check of Step 5 meas that T 24m 5 / 2, which fiishes the proof. Theorem 5.6 (Almost all εbiased matrices have high ANcomplexity). A matrix A whose etries are sampled from a ε biased distributio fails Test 1 with parameter m (which implies that the correspodig biliear fuctio has ANcomplexity greater tha m), with probability at least ( ) ( ) 2 2m 2 4 ( 1 2m 12m 3 ε + 2 m2 +24m 5 / 2) /. 15
18 I particular, for ε = 2 ad m = 2/3 10(log ) 1/3, this probability is at least 1 2 /2, for sufficietly large. Proof. We use a uio boud over all possible i, j, {Xl i}m, {Y j l }m, Li col ad L j row that ca be selected by the test, ad employ Lemma 5.5 for each possibility. The umber of optios for choosig (i, j) is (/2m) 2 ; the umber of optios for choosig {Xl i}m j (resp., {Yl }m ) is at most ( 2m 2 12m /) ; 3 the umber of optios for choosig L i col (resp., Lj row) is at most ( 2m 2 12m /) Lower Bouds for the AN2Complexity of Radom Toeplitz Matrices The followig is a degeerate versio of Test 1. Recall the defiitio of I i ad J j from Eq. (14), ad the defiitio of A i,j. Test 2 AN2Complexity Test Iput: Matrix A F 2 ad parameter m [] 1: for i = 1,..., /2m ad j = 1,..., /2m do 2: for all subsets {X i l }m of [2m] such that l Xi l 12m3 3: for all subsets {Y j l }m of [2m] such that l Y j l 12m3 4: Let T := m Xi l Y j l. do do 5: if T 24m5 2 the 6: for all matrices L j row of dimesio m 2m ad sparsity at most 12m3 do 7: Let E be the right orthogoal complemet of L j row. 8: if there exists N F2 2m 2m such that N T, ad (A i,j N)E = 0 2m m the 9: retur Pass. 10: retur Fail. The followig is a immediate corollary of Propositio 4.3 ad Eq. (13). Corollary 5.7. Every matrix associated with a biliear circuit of AN2complexity at most m passes Test 2 with parameter m. Lemma 5.8 (core of the aalysis of Test 2). Fix i, j, {Xl i}m j ad {Yl }m that pass the check of Step 5, ad fix L j row (which i tur, fixes E as well). The, a radom Hakel matrix A satisfies the coditio i Step 8 with probability at most 2 /2+6m3 / Proof. For a fixed choice of i, j, {Xl i}m, {Y j l }m ad Lj row, we cosider a specific submatrix of dimesio 2m 2m of A, deoted A i,j. Note that the correspodig right orthogoal complemet of L j row is a 2mbym matrix of rak m, deoted by E. By the defiitio of I i ad J j i Eq. (14), A i,j is of the form a 1 a 2 a 3... a 2m a k+1 a k+2 a k+3... a k+2m a (2m 1)k+1 a (2m 1)k+2 a (2m 1)k+3... a (2m 1)k+2m where k = /(2m) ad a 1,..., a (2m 1)k+2m are uiform idepedet radom bits. Our goal will be to show that the equatio (A i,j N) E = 0 2m m implies a lot of liearly idepedet liear equatios o the radom variables a 1,..., a (2m 1)k+2m. 16
19 First thik of a geeric 2m 2m matrix Z as a matrix of (2m) 2 variables, ad iterpret the equatios ZE = 0 2m m as liear equatios o Z. For each row l [2m], we have m equatios correspodig to Z l E = 0 1 m, which are liearly idepedet. Deote by T l the itersectio of T with the idices correspodig to the lth row of the submatrix, i.e. T l = T ({l} [2m]). Say that Z l,l is a oisy variable if (l, l ) T. By Fact 5.4, we ca get at least m T l idepedet liear equatios o the lth row of Z that do ot ivolve oisy variables. Summig over all l s we have at least 2m (m T l ) = 2m 2 T idepedet liear equatios that do ot ivolve the oisy etries of the matrix, ad such that each equatio ivolves oly variables from oe row of Z. Take Z = A i,j N; sice we got equatios o Z that do ot ivolve oisy etries, these are actually equatios o A i,j as well. The mai difficulty is that we wat to exhibit liearly idepedet liear equatios o the variables a 1,... a (2m 1)k+2m, but the equatios we got may ot be liearly idepedet oce we idetify multiple etries i the matrix A i,j with the same variable. 10 To solve this issue, we shall look for a set of equatios which remais liearly idepedet after this idetificatio. Let l = m T l be umber of liearly idepedet equatios we got o the lth row. Let s = (2m) 2 /, ad cosider all rows startig from some idex r [s], ad takig jumps of s. The, by the pigeohole priciple there exists a r [s] such that l (2m 2 T )/s. l:l r mod s A key poit is that by our choice of s, the lth row ad the (l + s)th row of A i,j deped o disjoit sets of radom variables, sice s k (2m)2 2m = 2m. Thus, the sets of variables out of a 1,..., a (2m 1)k+2m that participate i rows with idex i {l : l r mod s} are pairwise disjoit, ad the equatios we got o these rows are liearly idepedet as equatios over the variables a 1,..., a (2m 1)k+2m. Sice we got at least (2m 2 T )/s idepedet liear equatios o completely radom bits, all equatios hold simultaeously with probability at most 2 ( 2m2 + T )/s. The fact that {Xl i}m j ad {Yl }m passed the check i Step 5 meas that T 24m5 / 2, ad usig s = 2m 2 /, we get a probability boud of 2 ( 2m2 +24m 5 / 2 ) 4m 2, which completes the proof. Theorem 5.9 (Almost all radom Hakel matrices have high AN2complexity). A radom Hakel matrix A fails Test 2 with parameter m (which implies it has direct complexity at least m) with probability at least ( ) ( ) 2 2m m 12m 3 2 /2+6m3 /. / I particular, for m = 2/3, this probability is at least 1 2 /4, for large eough. 10(log ) 1/3 Proof. We use a uio boud over all the ( ) 2 ( 2m 2 )3 2m 12m 3 / possible ways to pick i, j, {X i l } m, {Y j l }m ad L j row, ad employ Lemma 5.8 to boud each possibility. Explicit 3Liear Fuctios with C 2 = Ω( 2/3 ). The followig is a corollary of Theorem 5.9. Corollary Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 z i+jx i y j. The, C 2 (F ) = Ω( 2/3 / log 1/3 ). We omit the proof, sice it is idetical to that of Corollary I fact, we caot expect this set of equatios to be liearly idepedet simply because there are too may equatios (i.e., more equatios tha variables). 17