Matrix Rigidity of Random Toeplitz Matrices

Transcription

1 Matrix Rigidity of Radom Toeplitz Matrices Oded Goldreich Avishay Tal May 7, 2015 Abstract We prove that radom -by- Toeplitz (alteratively Hakel) matrices over F 2 have rigidity Ω( 3 r 2 log ) for rak r, with high probability. This improves, for r = o(/ log log log ), over the Ω( 2 r log( r )) boud that is kow for may explicit matrices. Our result implies that the explicit triliear [] [] [2] fuctio defied by F (x, y, z) = i,j x iy j z i+j has complexity Ω( 3/5 ) i the multiliear circuit model suggested by Goldreich ad Wigderso (ECCC, 2013), which yields a exp( 3/5 ) lower boud o the size of the socalled caoical depth-three circuits for F. We also prove that F has complexity Ω( 2/3 ) if the multiliear circuits are further restricted to be of depth 2. I additio, we show that a matrix whose etries are sampled from a 2 -biased distributio has complexity Ω( 2/3 ), regardless of depth restrictios, almost matchig the O( 2/3 ) upper boud for ay matrix by Goldreich ad Wigderso. We tur this radomized costructio ito a explicit 4-liear costructio with similar lower bouds, usig the quadratic small-biased costructio of Mossel et al. (RS&A, 2006). Keywords: Matrix rigidity, multi-liear fuctios, multi-liear circuits. Weizma Istitute of Sciece, Rehovot, Israel. oded.goldreich@weizma.ac.il. Partially supported by the Mierva Foudatio with fuds from the Federal Germa Miistry for Educatio ad Research. Weizma Istitute of Sciece, Rehovot, Israel. avishay.tal@weizma.ac.il. Supported by a Adams Fellowship of the Israel Academy of Scieces ad Humaities, by a ISF grat ad by the I-CORE Program of the Plaig ad Budgetig Committee.

2 Cotets 1 Itroductio Matrix Rigidity The Project of Goldreich-Wigderso Resolvig the Foregoig Ope Problems Overview of the Proof of Theorem Orgaizatio Prelimiaries 5 3 Mai Results 6 4 The Structure of Matrices of Small Biliear Circuits The Structure of Matrices Associated with Depth Two Biliear Circuits The Structure of Matrices Associated with Geeral Biliear Circuits Substructures Testig AN Complexity ad AN2 Complexity Lower Bouds for the AN-Complexity of Small-Biased Matrices Lower Bouds for the AN2-Complexity of Radom Toeplitz Matrices Explicit 4-Liear Fuctios with AN-Complexity Ω( 2/3 ) Digest ad Ope Problems Digest Ope Problems Refereces 21 Appedices 22 A.1 Geeralizatio to Larger Fields A.2 The Structure of Matrices Associated with Geeral Biliear Circuits

3 1 Itroductio This paper cocers the costructio of rigid matrices, a cetral ope problem posed by Valiat [Val77], ad its applicatio to lower bouds o caoical depth-three Boolea circuits (where caoical is as defied by Goldreich ad Wigderso [GW13]). I particular, we improve the kow lower boud o matrix rigidity, but the improvemet is for a rage of parameters that is ot the oe motivated by Valiat s problem, but rather the oe that arises from [GW13]. Ideed, this improvemet resolves ope problems posed by Goldreich ad Wigderso [GW13]. 1.1 Matrix Rigidity The Matrix Rigidity Problem (i.e., providig explicit matrices of high rigidity) is oe of the most allurig problems i arithmetic circuits lower bouds. Itroduced i 1977 by Valiat [Val77], the problem was origially motivated by provig lower bouds for the computatio of liear trasformatios. Loosely speakig, a matrix is called rigid if it caot be writte as a sum of a low rak matrix ad a sparse matrix. Needless to say, the actual defiitio specifies both parameters. Defiitio 1.1 (Matrix Rigidity, [Val77]). A matrix A over a field F has rigidity s for rak r if every matrix of rak at most r (over F) disagrees with A o more tha s etries. Valiat showed that ay matrix with rigidity 1+δ for rak ω(/ log log ), where δ is some costat greater tha 0, caot be computed by a liear circuit of size O() ad depth O(log ). Valiat also proved that almost all -by- matrices, over a fiite field F (e.g., the two-elemet field F 2 ), have rigidity Ω(( r) 2 / log ) for rak r. Sice the, comig up with a explicit 1 rigid matrix has remaied a challege. The best techiques to date provide explicit -by- matrices of rigidity 2 r log( r ) for rak r (see [Lok09] for a survey about matrix rigidity). To the best of our kowledge, this state of affairs also holds for simple radomized costructios that use O() radom bits. The commo belief is that rigidity bouds for such radomized costructios ca be used for provig lower bouds for explicit computatioal problems that are related to the origial oes. For example, a adequate rigidity lower boud for radom Toeplitz (or Hakel) matrices 2 would yield a lower boud o the complexity of computig explicit biliear trasformatios. Ideed, this is aalogous to Adreev s proof of formula lower bouds [Ad87], where a lower boud for a radomized fuctio is trasformed ito a lower boud for a explicit fuctio (which takes the radom bits of the costructio as part of its iput). Our mai result is the followig Theorem 1.2 (o the rigidity of radom Toeplitz/Hakel matrices). Let A F2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], with probability 1 o(1), the matrix A has rigidity Ω( 3 ) for rak r. r 2 log Our bouds are asymptotically better tha Ω( 2 r log( r )) for rak r = o( log log log ), alas Valiat s origial motivatio refers to r > / log log. For rak r = 0.5+ε, where ε (0, 0.5), our boud yields a sigificat improvemet (i.e., 3 = 2 2ε 1.5 ε = 2 r 2 r ), ad this is actually the rage that is relevat for the project of Goldreich ad Wigderso [GW13]. 1 For a ifiite I N, the sequece of matrices, {A } I such that A is a matrix, is called explicit if there exists a poly()-time algorithm that o iput I outputs the matrix A (ad outputs if I). 2 Recall that a Toeplitz matrix T = (T i,j) has costat diagoals (i.e., T i,j = T i+1,j+1 for every i, j). Hakel matrices are obtaied by turig Toeplitz matrices upside dow; that is, a Hakel matrix H = (H i,j) has costat skew-diagoals (i.e., H i,j = H i+1,j 1 for every i, j). Hece, ay claim regardig oe family traslates to a equivalet claim regardig the other family. 1

4 1.2 The Project of Goldreich-Wigderso The project started by Goldreich ad Wigderso [GW13] provides aother motivatio for the study of matrix rigidity. I fact, the problem of improvig the rigidity bouds for radom Toeplitz matrices was posed explicitly there. Specifically, provig a rigidity boud of 1.5+Ω(1) for rak 0.5+Ω(1) for radom Toeplitz matrices was proposed there as a possible ext step. Lower Bouds for Depth Three Caoical Circuits. Håstad [Hås86] showed that ay depththree Boolea circuit computig the -way parity fuctio must be of size at least exp( ); to date, Håstad s result is the best lower boud for a explicit fuctio i the model of depth-three Boolea circuits 3. The work of Goldreich ad Wigderso [GW13] put forward a model of depth three caoical circuits, with the uderlyig log-term goal to exhibit better lower bouds for geeral depth-three Boolea circuits. Caoical circuits are restricted type of such depth-three circuits, which ca be illustrated by cosiderig the smallest kow depth-three circuits for -way parity. The latter Õ(2 )-size circuits are obtaied by combiig a CNF that computes a -way parity with DNFs that compute -way parities of disjoit blocks of the iput bits. The costructio, ad its optimality by [Hås86], suggests the followig scheme for obtaiig Boolea circuits that compute multiliear fuctios. First, costruct a arithmetic circuit that uses arbitrary multiliear gates of parameterized arity, ad the covert it to a Boolea circuit whose size is expoetial i the maximum betwee the arity ad the umber of gates i the arithmetic circuit. The arithmetic model is outlied ext. Lower Bouds for Multiliear Circuits. Suppose we wish to compute a t-liear fuctio that depeds o t blocks of iputs, x (1),..., x (t), each of legth ; that is, the fuctio is liear i each of the x (j) s. We cosider circuits that use arbitrary multiliear gates of parameterized arity. That is, the circuits are directed acyclic graphs, where each iteral ode computes a multiliear fuctio of its iputs. We further restrict our circuit such that each iteral gate computes a multiliear formal polyomial i the iputs x (1)..., x (t). We say that such a multiliear circuit is of AN-complexity 4 m if m equals the maximum betwee the umber of the circuit gates ad the maximal arity of the gates. For a multiliear fuctio F, we deote by C(F ) the miimal AN-complexity of a multiliear circuit which compute the fuctio F. (We will abuse otatio ad refer to the AN-complexity of a tesor/matrix as the AN-complexity of the correspodig multiliear fuctio.) I the example of parity, we have a bottom layer of gates each takig iputs ad computig their parity. Above these gates, we have a gate which takes the results ad computes their parity. Overall, we got a (multi)-liear circuit of AN-complexity + 1. Goldreich ad Wigderso showed that ay multiliear circuit of AN-complexity m yields a depth-three Boolea circuit of size exp(m) computig the same fuctio (see [GW13, Prop. 2.9]). I fact, the Boolea circuits have much more structure, ad are referred to by Goldreich ad Wigderso as caoical circuits. Thus, a prelimiary step towards beatig the exp(ω( )) lower boud o the size of depth-three Boolea circuits for explicit O(1)-liear fuctios, 5 will be to beat the Ω( ) AN-complexity lower boud for such fuctios i the model of multiliear circuits. Agai, as i Valiat s questio, if we just ask about the existece of hard t-liear fuctios, the most t-liear fuctios caot be computed by a multiliear circuit of AN-complexity smaller tha (t) t/(t+1) : See [GW13, Thm. 4.1], which uses a coutig argumet. The more importat ad 3 That is, circuits of ubouded fa-i OR ad AND gates with leaves that are variables or their egatios. 4 where AN stads for Arity ad Number of gates. 5 Ideed, this suggestio presumes that there exist O(1)-liear fuctios that require depth-three Boolea circuits of size exp(ω( )), which is also a ope problem suggested i [GW13]. 2

5 challegig problem is to came up with a explicit t-liear fuctio for which such bouds, or eve just ω( ) lower bouds, ca be proved. Reductio to (Structured) Rigidity. Goldreich ad Wigderso reduces the problem of provig lower bouds for biliear circuits to the problem of rigidity [GW13, Sec. 4.2]. They show that if a biliear circuit is of AN-complexity m/2, the its correspodig matrix is ot m 3 rigid for rak m (i.e., it ca be expressed as a sum of a m 3 -sparse matrix ad a matrix of rak at most m). Hece, ay matrix that has rigidity m 3 for rak m correspods to a biliear fuctio that caot be computed by a biliear circuit of AN-complexity at most m/2. Furthermore, Goldreich ad Wigderso show that the sparse matrix arisig from their reductio has a additioal structure (to be specified later). This leads to a weaker otio of rigidity (see [GW13, Thm. 4.12] which establishes a separatio), called structured rigidity, for which it is potetially easier to prove lower bouds. Ope Problems i Goldreich-Wigderso. Oe ope problem posed by Goldreich ad Wigderso is provig that radom Toeplitz matrices have rigidity m 3 (or just structured rigidity m 3 ) for rak m = 0.5+Ω(1). This would yield a AN-complexity lower boud of m for the correspodig biliear fuctio (via the reductio i [GW13, Thm. 4.4]) 6 as well as a similar lower boud for the followig explicit triliear fuctio (via [GW13, Prop. 4.6]): F tet (x, y, z) = i 1,i 2,i 3 []: 3 j=1 i j /2 /2 x i1 y i2 z i3. (1) 1.3 Resolvig the Foregoig Ope Problems We resolve the aforemetioed ope problem [GW13, Prob. 4.8] by provig that radom Toeplitz matrices have rigidity m 3 for rak m = Θ( 3/5 ), with high probability. This follows from our log 1/5 mai theorem (Theorem 1.2) by choosig r = m. Furthermore, we ca get rid of the logarithmic factor i the Ω otatio, by provig a slightly better lower boud for structured rigidity. Theorem 1.3 (o the structured rigidity of radom Toeplitz/Hakel matrices). Let A F 2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], the matrix A has structured rigidity Ω( 3 /r 2 ) for rak r. This implies (usig [GW13, Thm. 4.10] ad [GW13, Prop. 4.6]) that the AN-complexity of a radom Toeplitz matrix is Ω( 3/5 ), ad ditto for the explicit triliear fuctio F tet from Eq. (1). This resolves Problems 4.7 ad 4.2 i [GW13], resp. I additio, we show that aother explicit triliear fuctio has AN-complexity Ω( 3/5 ). Corollary 1.4 (AN-complexity lower boud for a explicit triliear fuctio). Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 x iy j z i+j. The, C(F ) = Ω( 3/5 ). New Challeges. The most atural questio that arises from the foregoig results is to tighte the lower boud; that is, to show that radom Toeplitz matrices have AN-complexity Ω( 2/3 ) as cojectured by [GW13]. This would be the best possible, sice ay biliear fuctio ca be computed by a biliear circuit of AN-complexity O( 2/3 ); more geerally, by [GW13, Thm. 3.1], 6 For structured rigidity, we use [GW13, Thm. 4.10]. 3

6 for ay t 2, ay t-liear fuctio ca be computed by a t-liear circuit of AN-complexity O((t) t/(t+1) ). Aother atural follow up questio is to exhibit a explicit O(1)-liear fuctio havig AN-complexity Ω( α ) for some costat α > 3/5; of course, the larger α, the better. Our progress o these ope problems is captured by the followig two results. Theorem 1.5 (depth-two AN-complexity lower boud for radom Toeplitz matrices). Let F be a biliear fuctio that correspods to a radom Toeplitz matrix. The, with probability 1 o(1), the fuctio F caot be computed by multiliear circuits of depth two havig AN-complexity 2/3 /(log ) 1/3. Theorem 1.5 establishes the desired AN-complexity lower boud for radom Toeplitz matrices, but oly for depth-two multiliear circuits. We ote that the AN-complexity upper boud of [GW13, Thm. 3.1] holds via depth-two circuits, ad so Theorem 1.5 is almost optimal with respect to depth-two multiliear circuits. Theorem 1.5 implies that the triliear fuctio F (x, y, z) = j=1 x iy j z i+j caot be computed by multiliear circuits of depth two ad AN-complexity 2/3 /(log ) 1/3. Theorem 1.6 (improved AN-complexity lower boud for 4-liear fuctios). There exists a explicit 4-liear fuctio havig AN-complexity Ω( 2/3 /(log ) 1/3 ). Theorem 1.6 is proved by first showig that, with high probability, biliear fuctios associated with matrices that are sampled from a 2 -biased sample space (over {0, 1} 2 ) have AN-complexity Ω( 2/3 ). Note that by the aforemetioed upper boud, this lower boud is tight (up to logarithmic factors). Next, we ote that samplig such matrices ca be doe usig O() radom bits [NN93, AGHP92, MST06], which matches the amout of radomess used for samplig a radom Toeplitz matrix. Furthermore, i the explicit small-biased costructio of Mossel et al. [MST06], each bit i the sampled strig is a biliear fuctio of the radom bits, allowig us to give a explicit 4-liear fuctio with AN-complexity Ω( 2/3 ). 1.4 Overview of the Proof of Theorem 1.2 We give a overview of the proof of Theorem 1.2 (for the case of Hakel matrices). Recall that we wish to show that a radom Hakel matrix has rigidity 3 /r 2 log for rak r, with high probability. Let A be a radom -by- Hakel matrix, of the form A i,j = a i+j for idepedet radom bits a 2,..., a 2. Suppose that A ca be expressed as a sum of a s-sparse matrix S ad a matrix of rak at most r. Cosider a partitio of A ad S ito (/2r) (/2r) submatrices, each of size 2r 2r, such that a geeric submatrix cosists of 2r cosecutive colums ad 2r equally spaced rows (i.e., rows that are at distace /2r apart). The, there exists a pair of correspodig submatrices A ad S such that S is of sparsity s = (2r)2 s (ad, of course, A S has rak at most r). Next, 2 we ote that ay of the above submatrices of A are of the form b 1 b 2 b 3... b 2r A = b k+1 b k+2 b k+3... b k+2r b (2r 1)k+1 b (2r 1)k+2 b (2r 1)k+3... b (2r 1)k+2r where k = /2r ( 2r, by the assumptio r ), ad b 1,..., b (2r 1)k+2r is a cosecutive subsequece of a 2,..., a 2. Notice that A is a 2r 2r submatrix that depeds o (2r 1)k + 2r = Θ() radom bits. 4

7 I our mai lemma, we show that for ay fixed matrix S (eve if S is ot sparse) the submatrix matrix A S is of rak greater tha r with probability at least 1 2 Ω(), where the probability is take over the choice of A (equiv., over the choice of b 1,..., b (2r 1)k+2r ). Hece, takig a uio boud over all possible s -sparse submatrices we get that, with probability at least 1 ( 2r 2r s ) 2 Ω(), the submatrix A has rigidity s for rak r. Pickig s = o( 3 r 2 log ) implies that s = o(/ log ), which completes the proof (by applyig a uio boud o all submatrices, ad iferrig that, with high probability, matrix A is of rigidity s for rak r). 1.5 Orgaizatio Our mai results (i.e., Theorems 1.2 ad 1.3 ad Corollary 1.4) are proved i Sectio 3, which follows a short prelimiary sectio (Sec. 2). Next, Theorems 1.5 ad 1.6 are proved, i two steps. I Sectio 4 we idetify structural properties of matrices that correspod to biliear fuctios of low AN (ad AN2) complexity. These properties correspod to (eve more) restricted otios of structured rigidity, ad i Sectio 5 we show that (with high probability) matrices draw from the two relevat distributios do ot satisfy these properties. We coclude, with a techical digest (Sectio 6.1) ad a list of some ope problems (Sectio 6.2). 2 Prelimiaries We deote by [] = {1,..., }. For, k N, we deote by ( k) = k ( i=0 i), ad use the boud ( ) k (2) k. For a matrix A, we deote its i-th row by A i, ad its j-th colum by A (j). We deote by wt(a) the umber of o-zero etries i the matrix A, ad say that A is s-sparse if wt(a) = s. A Hakel matrix over a field F is a square matrix with costat skew-diagoals; that is, ay matrix A F of the form A i,j = a i+j for some a 2,..., a 2 F. A Toeplitz matrix over a field F is a square matrix with costat diagoals, i.e. ay matrix A F of the form A i,j = a i j for some a ( 1),..., a 1 F. Note that a Hakel matrix is a upside-dow Toeplitz matrix. Throughout the paper, uless specified otherwise, we talk about matrices over the field F 2, ad matrix rak refers to its rak over F 2. Defiitio 2.1 (Structured Rigidity, [GW13, Def. 4.9]). We say that a matrix A has structured rigidity (m 1, m 2, m 3 ) for rak r if for every matrix R of rak at most r ad for every X 1,... X m1, Y 1,..., Y m1 [] such that X 1 = = X m1 = m 2 ad Y 1 = = Y m1 = m 3 it holds that A R m 1 k=1 (X k Y k ), where M S meas that all o-zero etries of the matrix M reside i the set S [] []. We say that a matrix A has structured rigidity m 3 for rak r if A has structured rigidity (m, m, m) for rak r. Ideed, ay matrix that has rigidity s for rak r, also has structured rigidity s for rak r, but the other directio does ot hold (see [GW13, Thm. 4.12]). Defiitio 2.2. A multiliear circuit o t blocks of iputs x (1),..., x (t) {0, 1} is a directed acyclic graph whose odes are associated with arbitrary multiliear gates such that ay two gates with directed paths from the same block of iputs are ot multiplied together by aother gate. Defiitio 2.3 (the AN-complexity of multiliear circuits with geeral gates, [GW13, Def. 2.2]). The arity of a multiliear circuit is the maximum arity of its (geeral) gates. The AN-complexity of a multiliear circuit is the maximum betwee its arity ad its umber of gates (where we cout oly the geeral gates ad ot the leaves, i.e., variables). The AN-complexity of a multiliear fuctio F, 5

8 deoted C(F ), is the miimum AN-complexity of a multiliear circuit that computes F. The AN2- complexity of a multiliear fuctio F, deoted C 2 (F ), is the miimum complexity of a depth-two multiliear circuit that computes F. Theorem 2.4 ([GW13, Thm. 4.10]). If A is a -by- matrix that has structured rigidity m 3 for rak m, the the correspodig biliear fuctio F satisfies C(F ) m/2. 3 Mai Results We prove our results bottom-up, startig with the mai lemma, as metioed i the proof overview. Lemma 3.1 (Mai Lemma). Let m, k N, 16 k m. Let A F m m 2 be the radom matrix a 1 a 2 a 3... a m a k+1 a k+2 a k+3... a k+m a (m 1)k+1 a (m 1)k+2 a (m 1)k+3... a (m 1)k+m where a 1,..., a (m 1)k+m are uiform idepedet radom bits, ad let S F m m 2 be some fixed matrix. The, Pr A [rak(s + A) m/2] 2 km/16. Note that for k = 1 the matrix i Lemma 3.1 is a radom Hakel matrix, ad for k = m it is a totally radom matrix. The requiremet k 16 is ot essetial i the lemma; it is used to make expressios icer. For k 1 ad rak r m/2 the proof gives Pr A [rak(s+a) r] ( m r) 2 mk/8. Proof. For a fixed S ad a radom A as above, let B = S + A. If r = rak(b) m/2, the oe ca costruct a basis B i1, B i2,..., B ir of the row space of B by the followig iterative process: Let i 1 be the first ozero row of B, let i 2 > i 1 be the first row i B that is ot spaed by row i 1, let i 3 > i 2 be the first row i B that is ot spaed by rows i 1 ad i 2, etc. We get that i 1 < i 2 < < i r ad 1. For j < i 1 the j-th row of B is the all zeroes row. 2. For i t 1 < j < i t the j-th row of B is spaed by rows i 1,..., i t 1 of B. 3. For i r < j the j-th row of B is spaed by rows i 1,..., i r of B. More cocisely, deotig by I = {i 1,..., i r }, we get j [m] \ I : B j spa{b i : i I, i < j}. (2) We boud the probability that such a sequece I = {i 1,..., i r } exists, where r m/2. We will uio boud over all possible sequeces I, ad for ay fixed sequece of legth at most m/2, we shall show that (2) holds with very low probability. Give such a sequece I, let J = [m] I be its complemet. Settig = m/k, we ca select a icreasig sequece of J / idices i J such that each two idices differ by at least. 7 Take j 1 < j 2 < < j t to be such a sequece of idices, where t J m/2 m/k k 4. For l [t], let E l be the evet that row j l is spaed by the rows idexed by I [j l 1]. The, Pr [Eq. (2) holds for I] Pr[E 1, E 2,..., E t ] = Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] (3) 7 Oe ca costruct such a set greedily: choose the miimal idex j i J, remove all idices i J [j, j + 1]. Repeat util J is empty. 6

9 Next, we show that for each l [t], we have Pr[E l E 1,..., E l 1 ] 2 m/2. However, istead of coditioig o E 1,..., E l 1, we shall coditio o a set of the radom bits, to be specified ext, that determie rows B 1,..., B jl 1 o oe had, but are idepedet from the radom row B jl o the other had. Sice j l j l 1 + m/k by our desig, we get (j l 1)k (j l 1 1)k + m. Hece, the radom bits a 1,..., a (jl 1)k, which determie B 1,..., B jl 1, leave the radom row B jl = (a (jl 1)k+1,..., a (jl 1)k+m) totally udetermied. Coditioig o the worst-case assigmet for the former radom variables (uder which E 1,..., E l 1 holds) yields a upper boud o Pr[E l E 1,..., E l 1 ]. Thus, it is eough to show that Pr[E l a 1,..., a (jl 1)k] 2 m/2 for ay possible fixed choice of values to a 1,..., a (jl 1)k. To avoid multiple subscripts, we set for the rest of the proof j j l. Let us remark that after fixig a 1,..., a (j 1)k, rows 1,..., j m/k are completely fixed, rows j m/k + 1,..., j 1 are partially fixed, ad row j is etirely udetermied. Based o that, we shall show that Pr[E l a 1,..., a (j 1)k ] 2 m/2. (4) Let I := I [j 1], ad fix a liear combiatio of the rows idexed by I, i.e., i I c ib i, amog all 2 I such liear combiatios. We show that the probability that B j = i I c i B i. (5) is 2 m. (This is similar, up to mior differeces, to the folklore result that ay fixed liear combiatio of rows i a radom Toeplitz matrix is distributed uiformly over F m 2 see [Gol08, Prop. 8.25]. We give the details for completeess.) The probability that the first bit of B j equals the first bit of the liear combiatio i (5) is exactly 1/2, sice B j,1 = S j,1 + a (j 1)k+1, ad all etries {B i,1 } i I ivolve oly bits from a 1,..., a (j 2)k+1, which were already fixed (sice (j 2)k + 1 (j 1)k). Fixig a (j 1)k+1 such that equality o the first bit holds, the secod bit B j,2 equals the resultig liear combiatio with probability 1/2 as well. This happes sice B j,2 equals S j,2 + a (j 1)k+2, where a (j 1)k+2 was t already fixed, ad all etries {B i,2 } i I ivolve oly bits from a 2,..., a (j 2)k+2, which were already fixed (sice (j 2)k + 2 (j 1)k + 1). Ad so o, every bit i the j-th row of B equals the resultig liear combiatio with probability 1/2, coditioed o the fixig of the previous bits. Overall, B j = i I c ib i with probability 2 m for a fixed choice of coefficiets {c i } i I. 8 Takig a uio boud over all possible coefficiets {c i } i I gives Pr[E l E 1,..., E l 1 ] 2 I 2 m 2 m/2. Pluggig this boud ito Eq. (3) we get Pr [Eq. (2) holds for I] Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] ( 2 m/2) t 2 mk/8. where i the last iequality we used t k/4. Takig a uio boud over all possible sequeces I of legth at most m/2, whose umber is defiitely less tha 2 m, ad usig k 16, we get Pr[rak(S + A) m/2] 2 m 2 mk/8 2 mk/16. We cotiue with our mai theorem. Theorem 3.2 (radom Hakel matrices are rigid). Let A F 2 be a radom Hakel matrix A i,j = a i+j where a 2,..., a 2 are uiform idepedet radom bits. The, for every r /32, with probability 1 o(1), the matrix A has rigidity 3 for rak r. 160r 2 log 8 Alteratively, coditioed o a 1,..., a (j 1)k ad the choice of the liear combiatio, there exist exactly oe choice for a (j 1)k+1,..., a (j 1)k+m that satisfies Eq. (5). 7

10 Before provig Theorem 3.2, we state a immediate corollary of it. Corollary 3.3. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that for every ε > 0 1. With probability 1 o(1), the matrix A is c 2 2ε / log rigid for rak 1/2+ε. 2. With probability 1 o(1), the matrix A is m 3 rigid for rak m = c 3/5 log 1/5. 3. With probability 1 o(1), the matrix A is c 1+2ε / log rigid for rak 1 ε. Proof of Theorem 3.2. Suppose towards cotradictio that A ca be represeted as a sum of a matrix R of rak at most r, ad a s-sparse matrix S, where s 3 /160r 2 log. Let m = 2r, ad assume for coveiece that k = /m is a iteger. Cosider the followig partitio of A s etries ito (/m) 2 submatrices, each of dimesio m m. For i [/m] ad j [/m], let I i = {i, i + k,..., i + (m 1)k}, J j = {(j 1)m + 1, (j 1)m + 2,..., jm}. (6) Deote by A i,j (R i,j, S i,j, resp.) the matrix A (R, S, resp.) restricted to rows I i ad colums J j. See Figure 1 for a example of such a submatrix. The mai observatio is that for each a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 16 Figure 1: A submatrix A 1,1 of the matrix A, for m = 4 ad k = 2. (i, j) [/m] 2, the matrix A i,j is of the form eeded by the mai lemma. Aother observatio is that sice the submatrices S i,j partitios the sparse matrix S, oe of them has sparsity at most s s m2. I additio, sice rak of a submatrix may oly decrease, for every i, j, it holds that 2 rak(r i,j ) rak(r) r. We say that A i,j is simple if it ca be represeted as a sum of a s -sparse matrix ad a matrix of rak at most r. By the above discussio, A ca be represeted as S + R where S is s-sparse ad R is of rak at most r, oly if there exists a submatrix A i,j that is simple. We shall show that the latter occurs with very low probability: Pr [ i, j : A i,j is simple ] i,j i,j Pr[A i,j is simple] S F m m 2 : wt(s) s Pr[rak(A i,j + S) m/2] (Uio Boud) (Uio Boud) ( ) ( ) 2 m 2 m s 2 mk/16 (Lemma 3.1) < 2 (2m 2 ) s 2 /16. (km = ) Fially, usig s 40 log, which follows from s 3 160r 2 log, we get that Pr[ i, j : Ai,j is simple] = o(1), which completes the proof. 8

11 Note that the proof works as log as the umber of possibilities for a s -sparse matrix S i,j is smaller tha 2 /16 / 2. Our ext theorem exploits the fact that there is a smaller umber of possibilities for submatrices of structured sparse matrices (as i Defiitio 2.1). I fact, this is the oly property of S that the foregoig proof uses. This yields the followig improved boud. Theorem 3.4 (radom Hakel matrices are structured rigid). Let A F 2 be a radom Hakel matrix. The, for every r /32, ad s 3 /1000r 2, with probability 1 o(1), the matrix A has structured rigidity s for rak r. Before provig Theorem 3.4 we state three corollaries of it. The first corollary is immediate by choosig r = 3/5. Corollary 3.5. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that with probability 1 o(1), the matrix A has structured rigidity c 9/5 for rak 3/5. The secod corollary follows from the first corollary ad Theorem 2.4. Corollary 3.6. Let A F 2 be a radom Hakel matrix, ad let F (x, y) = j=1 A i,jx i y j. The, with probability 1 o(1), it holds that C(F ) = Ω( 3/5 ). The last corollary shows that there exists a explicit triliear form with AN-complexity Ω( 3/5 ). This is the first improvemet over the trivial Ω( ) lower boud for explicit tesors, ad i doig so it solves Problem 4.2 from [GW13] i the affirmative. Goldreich ad Wigderso [GW13, Prop. 4.6] show that if some Toeplitz matrix have AN-complexity Ω(m), the F tet defied i Eq. (1) has ANcomplexity Ω(m) has well. We follow their method, but preset a simpler argumet for a differet triliear fuctio. Corollary 3.7. Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 z i+jx i y j. The, C(F ) = Ω( 3/5 ). Proof. Accordig to Corollary 3.6, there exists a Hakel matrix A, defied by some diagoal values a 2,..., a 2, such that the biliear form i,j a i+jx i y j has AN-complexity Ω( 3/5 ). Let C be a triliear circuit computig F with miimal AN-complexity, ad deote its complexity by m. Fixig the values of the variables z i to a i, for all i {2,..., 2}, we get a biliear circuit i x ad y of AN-complexity at most m. Thus, m = Ω( 3/5 ). We retur to prove Theorem 3.4. Proof of Theorem 3.4. The proof follows the lies of the proof of Theorem 3.2. We let m = 2r, k = /m, ad t = s 1/3. We assume towards cotradictio that A = S + R, where R is of rak at most r, ad S is a sum of t matrices S 1,..., S t F 2, such that the oes i each matrix S l are a subset of some X l Y l, where X l, Y l t. Deote by T the -by- matrix over F 2 with T i,j = 1 iff (i, j) is cotaied i at least oe X l Y l. It is clear from T s defiitio that the oes i S are a subset of the oes i T. As i Theorem 3.2, we partitio A, R, S, ad also T, to (/m) 2 submatrices, accordig to the partitio of row idices I 1,..., I /m ad colum idices J 1,..., J /m, defied as i the proof of Theorem 3.2 (see Eq. 6). For a radom (i, j) [/m] 2, it holds that [ E wt(t i,j ) ] t 3 m2 i,j 2, E i,j [ t ] X l I i t t m, E i,j [ t ] Y l J j t t m. (7) 9

12 We say that a submatrix T i,j is good if wt(t i,j ) 4t 3 m2 2, t X l I i 4t t m, t Y l J j 4t t m. (8) Usig Markov s iequality, each of the above three evets happe with probability at least 3/4. Usig uio boud (o the complemet evets) with probability at least 1/4 all evets occur simultaeously, makig T i,j good. Next, we cout the umber of possible good submatrices T i,j. Each such submatrix is uiquely determied by the sets X 1,..., X t ad Y 1,..., Y t, where X l = X l I i ad Y l = Y l J j. Furthermore, a collectio (X 1,..., X t) such that l X l 4t2 m correspods to a set X I i [t] of size at most 4t 2 m such that (p, l) X iff p X l (ad similarly for (Y 1,..., Y t )). Hece, the umber of possible good submatrices is at most { X I i [t] : X 4t2 m } 2 ( = mt 4t 2 m/ ) 2 ( ) 2 (2mt) 4t2 m/ 16t 2m/. We say that S i,j is good if T i,j is good, ad we say that A i,j is simple if it is the sum of a good S i,j ad a matrix of rak at most r. Next, we cout the umber of possible good submatrices S i,j. Sice the oes of S i,j are a subset of the oes i T i,j, the umber of possibilities for S i,j is at most 16t2 m/ 2 wt(t i,j) 16t2 m/ 2 4t3 m 2 / 2. Usig the boud o the umber of possible good submatrices S i,j, we may boud the probability that some A i,j is simple: Pr [ i, j : A i,j is simple ] Pr[rak(A i,j + S i,j ) m/2] (Uio Boud) i,j Recall that m = 2r ad k = /m to get which is o(1) for t 3 S i,j good ( ) 2 16t 2 m/ 2 4t3 m 2 / 2 2 mk/16 (Lemma 3.1) m Pr [ i, j : A i,j is simple ] 2 2 log + 32 log t2 r/ + 16t 3 r 2 / 2 /16, 3 ad r. 1000r 2 Geeralizatio to Larger Fields. The choice of field F 2 was ot crucial i the proofs of Lemma 3.1, Theorem 3.2 ad Theorem 3.4. Oe ca sytactically replace the field size 2 by ay prime power q, keepig the proofs itact. Furthermore, i Theorem 3.2, we slightly beefit from takig a larger field. For details see Appedix A.1. 4 The Structure of Matrices of Small Biliear Circuits I this sectio we shall further refie the structure of matrices associated with small biliear circuits, beyod the structure captured by Defiitio 2.1 ad Theorem 2.4. We begi with the structure that arises from depth-2 biliear circuits, ad the cotiue to the structure arisig from geeral biliear circuits. Our aalysis follows the proof of [GW13, Thm. 4.4], ad it ca be viewed as relatig to fier otios of structured rigidity (tha the oe of Defiitio 2.1). We the follow the first part of the proof of Theorem 3.4, ad fid submatrices with correspodig (rigidity-like) parameters. 10

13 4.1 The Structure of Matrices Associated with Depth Two Biliear Circuits We say a row/colum i a matrix is m-sparse if it cotais at most m o-zero etries. Likewise, a liear fuctio l(x) (resp. l (y)) is m-sparse if it depeds o at most m etries i x (resp. y). Lastly, recall that by Defiitio 2.3, C 2 (F ) is the miimal AN-complexity of a depth-two biliear circuit computig F. Propositio 4.1 (Structure of fuctios computed by depth two biliear circuits). If C 2 (F ) = m, the F ca be expressed as L i(x)l i (y) + Q l (x, y) where L 1,..., L m are m-sparse liear fuctios, L 1,..., L m are geeral liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = CL row + S l (9) where C is a geeral m matrix, L row is a m matrix with m-sparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. Propositio 4.1 is proved explicitly i the warm-up part of the proof of [GW13, Thm. 4.4]. 4.2 The Structure of Matrices Associated with Geeral Biliear Circuits Propositio 4.2 (Structure of fuctios computed by geeral biliear circuits). If C(F ) = m, the F ca be expressed as L i (x)l i(y) + M i(x)m i (y) + Q l (x, y) where L 1,..., L m ad M 1,..., M m are m-sparse liear fuctios, L 1,..., L m ad M 1,..., M m are geeral liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = L col B + CL row + S l (10) where L col is a m matrix with m-sparse colums, B is a geeral m matrix, C is a geeral m matrix, L row is a m matrix with m-sparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. Propositio 4.2 is oly implicit i the proof of [GW13, Thm. 4.4], ad we iclude its proof i Appedix A Substructures I this subsectio, similarly to the first part of the proof of Theorem 3.4, we fid a submatrix (of a matrix associated with a biliear circuit) that has average rigidity-like parameters. Startig with Propositio 4.2, for C(F ) = m, we write the matrix A associated with F as A = L col B + CL row + m S l such that the o-zero etries of S l are a subset of X l Y l, where X l, Y l m. Deote 11

14 by T = m X l Y l, ad ote that T m 3. Let I 1,..., I /2m ad J 1,..., J /2m be some fixed equipartitio of the row idices ad colum idices of A, respectively, where each I i ad J j is of size 2m. This partitio aturally defies (/2m) 2 submatrices as follows. For ay (i, j) we deote by A i,j (resp. S i,j l ) the matrix A (resp. S l) restricted to rows I i ad colums J j. For ay i (resp. j) we deote by L i col ad C i (resp. B j ad L j row) the matrices L col ad C (resp,. B ad L row ) restricted to I i (resp. J j ). The, oe ca write A i,j = L i col Bj + C i L j row + S i,j l, (11) where S i,j l T (I i J j ). Next, we show that there exists a choice of (i, j) with favorable properties (to be exploited i the ext sectio) of the submatrices of L i col, Lj row ad o the subsets {X l I i } l, {Y l J j } l, ad T (I i J j ). Propositio 4.3 (Structure of submatrix of matrices associated with small biliear circuits). There exists a (i, j) such that: (1) T (I i J j ) 24m5, (2) m X l I i 12m3, (3) m Y l J j 12m 3, (4) wt(li col ) 12m3 2, ad (5) wt(lj row) 12m3. If C 2 (F ) = m, the the same statemet holds, except that we ca replace L col ad B with the 0 m ad 0 m matrices, respectively. Proof. For a uiformly radom (i, j) [/2m] 2, it holds that E [ T (I i J j ) ] m 3 (2m)2 i,j [ 2 m ] E X l I i m m 2m i,j [ m ] E Y l J j m m 2m i,j E i,j [wt(li col )] m m 2m E i,j [wt(lj row)] m m 2m Usig Markov s iequality, each of the followig bad evets occur with probability at most 1/6 T (I i J j ) 6m 3 (2m)2 2 X l I i 6m m 2m Y l J j 6m m 2m wt(l i col ) 6m m 2m wt(l j row) 6m m 2m By uio boud, with probability at least 1 5/6 over the choice of (i, j), oe of the bad evets occur, which completes the proof. 12

15 We wish to express the structure captured by Eq. (11) i terms of liear equatios o the etries of the matrix A i,j l Si,j l. To do so we eed the followig defiitio. Defiitio 4.4 (Orthogoal Complemet of a Matrix). Let m. If A is a m matrix, ad B is a ( m) matrix of rak m such that BA = 0 the we say that B is a left orthogoal complemet of A. If A is a m matrix, ad B is a ( m) matrix of rak m such that AB = 0 the we say that B is a right orthogoal complemet of A. It is well kow that ay matrix over a field has a orthogoal complemet. Now, suppose that A i,j l S l i,j = L i col Bj + C i L j row (as i Eq. (11)). Let D be a m 2m matrix which is a left orthogoal complemet of L i col, ad let E be a 2m m matrix which is a right orthogoal complemet of L j row. The, D (A i,j l S l i,j ) E = 0 m m. (12) I the case of depth-2 circuits we have A i,j l S l i,j = C i L j row. Usig E, the right orthogoal complemet of L j row as above, we ca write (A i,j l S l i,j ) E = 0 2m m. (13) I the ext sectio, we shall desig tests based o Equatios (12) ad (13). Remark: Note that there are may possible choices of a orthogoal complemet of a give matrix. Therefore, we shall refer to the left (right, resp.) orthogoal complemet of A as some caoical choice of a left (right, resp.) orthogoal complemet of A, say the first such matrix accordig to lexicographical order (over a fiite field F). 5 Testig AN Complexity ad AN2 Complexity We would like to desig a test such that matrices associated with (AN or AN2) complexity at most m will surely pass the test, whereas the matrices we are iterested i will fail it. (Sice the test is merely a metal experimet, i.e., we do ot ited to actually ru it, the test could be iefficiet.) The poit is that ay matrix o which the test fails must have complexity greater tha m. We will show that a radom Toeplitz matrix, as well as a matrix whose etries are sampled from a 2 - biased distributio, will fail the test with overwhelmig probability, thus provig complexity lower bouds for such matrices. We will preset two tests: Oe for AN-complexity failig most matrices take from a small-biased space, ad oe for AN2-complexity failig most Toeplitz matrices. 5.1 Lower Bouds for the AN-Complexity of Small-Biased Matrices For i [/2m] ad j [/2m], let 9 I i = {i, i + (/2m),..., i + (2m 1) (/2m)}, J j = {(j 1) (2m) + 1, (j 1) (2m) + 2,..., j (2m)}. (14) 9 The specific choice for I i ad J j is ot crucial for our argumet i this subsectio, however it will be importat i the ext subsectio. Hece, sice we eed to pick some partitio, we might as well choose this oe. 13

16 ad deote by A i,j the 2m-by-2m sub-matrix of A obtaied by restrictig A to rows I i ad colums J j. Cosider the followig test, where A i,j is viewed as idexed by [2m] [2m] rather tha by I i J j. Test 1 AN-Complexity Test Iput: Matrix A F 2 ad parameter m [] 1: for i = 1,..., /2m ad j = 1,..., /2m do 2: for all subsets {X i l }m of [2m] such that l Xi l 12m3 3: for all subsets {Y j l }m of [2m] such that l Y j l 12m3 4: Let T := m Xi l Y j 5: if T 24m5 the 2 6: for all matrices L i col l. do do 12m3 of dimesio 2m m ad sparsity at most do 7: Let D be the left orthogoal complemet of L i col. 8: for all matrices L j row of dimesio m 2m ad sparsity at most 12m3 do 9: Let E be the right orthogoal complemet of L j row. 10: if there exists N F 2m 2m 2 such that N T, ad D(A i,j N)E = 0 m m the 11: retur Pass. 12: retur Fail. The followig is a immediate corollary of Propositio 4.3 ad Eq. (12). Corollary 5.1. Every matrix associated with a biliear circuit of AN-complexity at most m passes Test 1 with parameter m. I this subsectio we cosider a distributio of matrices whose etries are chose from a small biased sample space. Specifically, we shall use a sample space over strigs of legth N = 2 i order to defie -by- matrices. We shall show that almost all such matrices fail Test 1 with parameter m. But we eed a few prelimiaries first. Prelimiaries. Recall the defiitio of a ε-biased distributio from [NN93]. Defiitio 5.2 (small-biased distributio). A distributio X over {0, 1} N is said to be ε-biased if for every o-empty set S [N], it holds that E x X [( 1) i S x i ] ε. We shall use the followig property of ε-biased distributios (implicit i [NN93]). Lemma 5.3 ([AGHP92, Lem. 1]). Let X be a ε-biased distributio over {0, 1} N. Let l 1,..., l t be liearly idepedet liear fuctios o x 1,..., x N. The, the probability that all liear fuctios equal 0 simultaeously is at most ε + 2 t. We shall also use the followig simple fact from liear algebra. Fact 5.4. Let t,, m N such that t m. Let l 1,..., l t be a sequece of liearly idepedet liear fuctios (over F) o x 1,..., x. The, l 1,..., l t spa at least t m liearly idepedet fuctios that ivolve oly the variables x m+1,..., x. Proof. Thik of the liear fuctios as vectors i F, ad let V = spa{l 1,..., l t }. Cosider the subspace U = spa{e m+1,..., e }, where e i F is the uit vector with 1 i the i-th coordiate ad 0 elsewhere. The, dim(u V ) dim(u) + dim(v ) = ( m) + t = t m, whereas U V is the spa of l 1,..., l t that is supported oly o the last m coordiates. 14

17 Actual Results. We are ow ready to aalyze the probability that a matrix sampled from a small biased space passes Test 1. The core of the aalysis refers to a sigle applicatio of Step 10, which refers to a specific choice of i, j, {Xl i}m, {Y j l }m as well as Li col, Lj row (which i tur, fixes D ad E as well). Lemma 5.5 (core of the aalysis of Test 1). Fix i, j, {Xl i}m j ad {Yl }m that pass the check of Step 5, ad fix L i col ad L j row (which i tur, fixes D ad E as well). The, a matrix A whose etries are sampled from a ε-biased distributio satisfies the coditio i Step 10 with probability at most ε + 2 m2 +24m 5 / 2. Proof. For a fixed choice of i, j, {X i l }m, {Y j l }m, Li col ad L j row as above, we cosider a specific submatrix of dimesio 2m 2m of A, deoted A i,j. Note that the correspodig left (resp. right) orthogoal complemet of L i col (resp. L j row) is a m-by-2m (resp. 2m-by-m) matrix of rak m, deoted by D (resp. E). Recall that A i,j is a submatrix whose etries are sampled accordig to a ε-biased distributio. Our goal is to show that the equatio D(A i,j N)E = 0 (checked i Step 10) implies a lot of liearly idepedet liear equatios o the etries of A i,j. Let Z be a 2m 2m matrix of (2m) 2 Boolea variables, where we will later take Z to be A i,j N. Iterpret the equatios DZE = 0 m m as m 2 liear equatios o the (2m) 2 variables i Z. For i [m] ad j [m], we have a equatio of the form D i ZE (j) = 0, where D i is the i-th row of D ad E (j) is the j-th colum of E. We ca write D i ZE (j) = 2 2 k=1 D i,k Z k,l E l,j = k,l (D i E (j) ) k,l Z k,l ; that is, the coefficiets of the equatio are the tesor product of the vector D i with the vector E (j). Thikig of these m 2 liear equatios o (2m) 2 variables as a big matrix of dimesio m 2 (2m) 2, we ote that this matrix of liear equatios is the tesor product of D ad E, sice the (i, j) row equals to D i E (j) (viewed as a (2m) 2 -bit log vector). It is a kow fact that the rak of the tesor product of ay two matrices is the product of their rak; hece, we get rak(d E ) = rak(d) rak(e ) = m 2. I other words, we have a liearly idepedet set of m 2 liear equatios o the variables Z. However, we wat to get liear equatios over the variables of A, where Z = A N. Say that Z k,l is a oisy variable if (k, l) T. It will be eough to show that there are may idepedet liear equatios which ivolve oly o-oisy variables of the matrix. Sice the umber of oisy variables is T, by Fact 5.4 we ca fid at least m 2 T idepedet liear equatios that do ot ivolve oisy variables. Overall, we got m 2 T idepedet liear equatios o A i,j. By Lemma 5.3, a submatrix A i,j whose etries are sampled accordig to a ε-biased distributio satisfies all m 2 T equatios with probability at most ε + 2 m2 + T. Lastly, the fact that {Xl i}m j ad {Yl }m passed the check of Step 5 meas that T 24m 5 / 2, which fiishes the proof. Theorem 5.6 (Almost all ε-biased matrices have high AN-complexity). A matrix A whose etries are sampled from a ε biased distributio fails Test 1 with parameter m (which implies that the correspodig biliear fuctio has AN-complexity greater tha m), with probability at least ( ) ( ) 2 2m 2 4 ( 1 2m 12m 3 ε + 2 m2 +24m 5 / 2) /. 15

18 I particular, for ε = 2 ad m = 2/3 10(log ) 1/3, this probability is at least 1 2 /2, for sufficietly large. Proof. We use a uio boud over all possible i, j, {Xl i}m, {Y j l }m, Li col ad L j row that ca be selected by the test, ad employ Lemma 5.5 for each possibility. The umber of optios for choosig (i, j) is (/2m) 2 ; the umber of optios for choosig {Xl i}m j (resp., {Yl }m ) is at most ( 2m 2 12m /) ; 3 the umber of optios for choosig L i col (resp., Lj row) is at most ( 2m 2 12m /) Lower Bouds for the AN2-Complexity of Radom Toeplitz Matrices The followig is a degeerate versio of Test 1. Recall the defiitio of I i ad J j from Eq. (14), ad the defiitio of A i,j. Test 2 AN-2-Complexity Test Iput: Matrix A F 2 ad parameter m [] 1: for i = 1,..., /2m ad j = 1,..., /2m do 2: for all subsets {X i l }m of [2m] such that l Xi l 12m3 3: for all subsets {Y j l }m of [2m] such that l Y j l 12m3 4: Let T := m Xi l Y j l. do do 5: if T 24m5 2 the 6: for all matrices L j row of dimesio m 2m ad sparsity at most 12m3 do 7: Let E be the right orthogoal complemet of L j row. 8: if there exists N F2 2m 2m such that N T, ad (A i,j N)E = 0 2m m the 9: retur Pass. 10: retur Fail. The followig is a immediate corollary of Propositio 4.3 ad Eq. (13). Corollary 5.7. Every matrix associated with a biliear circuit of AN2-complexity at most m passes Test 2 with parameter m. Lemma 5.8 (core of the aalysis of Test 2). Fix i, j, {Xl i}m j ad {Yl }m that pass the check of Step 5, ad fix L j row (which i tur, fixes E as well). The, a radom Hakel matrix A satisfies the coditio i Step 8 with probability at most 2 /2+6m3 / Proof. For a fixed choice of i, j, {Xl i}m, {Y j l }m ad Lj row, we cosider a specific submatrix of dimesio 2m 2m of A, deoted A i,j. Note that the correspodig right orthogoal complemet of L j row is a 2m-by-m matrix of rak m, deoted by E. By the defiitio of I i ad J j i Eq. (14), A i,j is of the form a 1 a 2 a 3... a 2m a k+1 a k+2 a k+3... a k+2m a (2m 1)k+1 a (2m 1)k+2 a (2m 1)k+3... a (2m 1)k+2m where k = /(2m) ad a 1,..., a (2m 1)k+2m are uiform idepedet radom bits. Our goal will be to show that the equatio (A i,j N) E = 0 2m m implies a lot of liearly idepedet liear equatios o the radom variables a 1,..., a (2m 1)k+2m. 16

19 First thik of a geeric 2m 2m matrix Z as a matrix of (2m) 2 variables, ad iterpret the equatios ZE = 0 2m m as liear equatios o Z. For each row l [2m], we have m equatios correspodig to Z l E = 0 1 m, which are liearly idepedet. Deote by T l the itersectio of T with the idices correspodig to the l-th row of the submatrix, i.e. T l = T ({l} [2m]). Say that Z l,l is a oisy variable if (l, l ) T. By Fact 5.4, we ca get at least m T l idepedet liear equatios o the l-th row of Z that do ot ivolve oisy variables. Summig over all l s we have at least 2m (m T l ) = 2m 2 T idepedet liear equatios that do ot ivolve the oisy etries of the matrix, ad such that each equatio ivolves oly variables from oe row of Z. Take Z = A i,j N; sice we got equatios o Z that do ot ivolve oisy etries, these are actually equatios o A i,j as well. The mai difficulty is that we wat to exhibit liearly idepedet liear equatios o the variables a 1,... a (2m 1)k+2m, but the equatios we got may ot be liearly idepedet oce we idetify multiple etries i the matrix A i,j with the same variable. 10 To solve this issue, we shall look for a set of equatios which remais liearly idepedet after this idetificatio. Let l = m T l be umber of liearly idepedet equatios we got o the l-th row. Let s = (2m) 2 /, ad cosider all rows startig from some idex r [s], ad takig jumps of s. The, by the pigeo-hole priciple there exists a r [s] such that l (2m 2 T )/s. l:l r mod s A key poit is that by our choice of s, the l-th row ad the (l + s)-th row of A i,j deped o disjoit sets of radom variables, sice s k (2m)2 2m = 2m. Thus, the sets of variables out of a 1,..., a (2m 1)k+2m that participate i rows with idex i {l : l r mod s} are pairwise disjoit, ad the equatios we got o these rows are liearly idepedet as equatios over the variables a 1,..., a (2m 1)k+2m. Sice we got at least (2m 2 T )/s idepedet liear equatios o completely radom bits, all equatios hold simultaeously with probability at most 2 ( 2m2 + T )/s. The fact that {Xl i}m j ad {Yl }m passed the check i Step 5 meas that T 24m5 / 2, ad usig s = 2m 2 /, we get a probability boud of 2 ( 2m2 +24m 5 / 2 ) 4m 2, which completes the proof. Theorem 5.9 (Almost all radom Hakel matrices have high AN2-complexity). A radom Hakel matrix A fails Test 2 with parameter m (which implies it has direct complexity at least m) with probability at least ( ) ( ) 2 2m m 12m 3 2 /2+6m3 /. / I particular, for m = 2/3, this probability is at least 1 2 /4, for large eough. 10(log ) 1/3 Proof. We use a uio boud over all the ( ) 2 ( 2m 2 )3 2m 12m 3 / possible ways to pick i, j, {X i l } m, {Y j l }m ad L j row, ad employ Lemma 5.8 to boud each possibility. Explicit 3-Liear Fuctios with C 2 = Ω( 2/3 ). The followig is a corollary of Theorem 5.9. Corollary Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 z i+jx i y j. The, C 2 (F ) = Ω( 2/3 / log 1/3 ). We omit the proof, sice it is idetical to that of Corollary I fact, we caot expect this set of equatios to be liearly idepedet simply because there are too may equatios (i.e., more equatios tha variables). 17