Matrix Rigidity of Radom Toeplitz Matrices Oded Goldreich Avishay Tal Jue 12, 2016 Abstract A matrix A is said to have rigidity s for rak r if A differs from ay matrix of rak r o more tha s etries. We prove that radom -by- Toeplitz matrices over F 2 (i.e., matrices of the form A i,j = a i j for radom bits a ( 1),..., a 1 ) have rigidity Ω( 3 r 2 log ) for rak r, with high probability. This improves, for r = o(/ log log log ), over the Ω( 2 r log( r )) boud that is kow for may explicit matrices. Our result implies that the explicit triliear [] [] [2] fuctio defied by F (x, y, z) = i,j x iy j z i+j has complexity Ω( 3/5 ) i the multiliear circuit model suggested by Goldreich ad Wigderso (ECCC, 2013), which yields a exp( 3/5 ) lower boud o the size of the socalled caoical depth-three circuits for F. We also prove that F has complexity Ω( 2/3 ) if the multiliear circuits are further restricted to be of depth 2. I additio, we show that a matrix whose etries are sampled from a 2 -biased distributio has complexity Ω( 2/3 ), regardless of depth restrictios, almost matchig the kow O( 2/3 ) upper boud for ay matrix. We tur this radomized costructio ito a explicit 4-liear costructio with similar lower bouds, usig the quadratic small-biased costructio of Mossel et al. (RS&A, 2006). Keywords: Matrix rigidity, multi-liear fuctios, multi-liear circuits. Weizma Istitute of Sciece, Rehovot, Israel. oded.goldreich@weizma.ac.il. Partially supported by the Mierva Foudatio with fuds from the Federal Germa Miistry for Educatio ad Research. Istitute for Advaced Study, Priceto, NJ. avishay.tal@gmail.com. Research was doe while beig a studet at Weizma Istitute of Sciece, Rehovot, Israel. Research was supported by a Adams Fellowship of the Israel Academy of Scieces ad Humaities, by a ISF grat ad by the I-CORE Program of the Plaig ad Budgetig Committee.
Cotets 1 Itroductio 1 1.1 Matrix Rigidity....................................... 1 1.2 Goldreich-Wigderso s Project.............................. 2 1.3 Resolvig the Foregoig Ope Problems......................... 3 1.4 Overview of the Proof of Theorem 1.2.......................... 4 1.5 Orgaizatio........................................ 5 2 Prelimiaries 5 3 Mai Results 6 4 The Structure of Matrices of Small Biliear Circuits 11 4.1 The Structure of Matrices Associated with Depth Two Biliear Circuits....... 11 4.2 The Structure of Matrices Associated with Geeral Biliear Circuits......... 12 4.3 Substructures........................................ 12 5 Tests for AN Complexity ad AN2 Complexity 14 5.1 Lower Bouds for the AN-Complexity of Small-Biased Matrices............ 15 5.2 Explicit 4-Liear Fuctios with High AN-Complexity................. 17 5.3 Lower Bouds for the AN2-Complexity of Radom Toeplitz Matrices......... 18 6 Digest ad Ope Problems 20 6.1 Digest............................................ 20 6.2 Radomess-Rigidity Tradeoff............................... 20 6.3 Ope Problems....................................... 21 Refereces 23 Appedices 24 A.1 Geeralizatio to Larger Fields.............................. 24 A.2 The Structure of Matrices Associated with Geeral Biliear Circuits......... 25 A.3 Characterizatio of AN-complexity for Biliear Forms................. 26
1 Itroductio This paper cocers the costructio of rigid matrices, a cetral ope problem posed by Valiat [Val77], ad its applicatio to lower bouds o caoical depth-three Boolea circuits (a restricted model of depth-three circuits defied by Goldreich ad Wigderso [GW13]). I particular, we improve the kow lower boud o matrix rigidity, but the improvemet is for a rage of parameters that is ot the oe motivated by Valiat s problem, but rather the oe that arises from [GW13]. Ideed, this improvemet resolves ope problems posed by Goldreich ad Wigderso [GW13]. 1.1 Matrix Rigidity The Matrix Rigidity Problem (i.e., providig explicit matrices of high rigidity) is oe of the most allurig problems i arithmetic circuits lower bouds. Itroduced i 1977 by Valiat [Val77], the problem was origially motivated by provig lower bouds for the computatio of liear trasformatios. Loosely speakig, a matrix is called rigid if it caot be writte as a sum of a low rak matrix ad a sparse matrix. Needless to say, the actual defiitio specifies both parameters. Defiitio 1.1 (Matrix rigidity, [Val77]). A matrix A over a field F has rigidity s for rak r if every matrix of rak at most r (over F) differs from A o more tha s etries. Valiat showed that ay -by- matrix with rigidity 1+δ for rak ω(/ log log ), where δ is some costat greater tha 0, caot be computed by a liear circuit of size O() ad depth O(log ). Valiat also proved that almost all -by- matrices, over a fiite field F (e.g., the twoelemet field F 2 ), have rigidity Ω(( r) 2 / log ) for rak r. Sice the, comig up with a explicit 1 rigid matrix has remaied a challege. The best techiques to date provide explicit -by- matrices of rigidity 2 r log( r ) for rak r (see Friedma [Fri93] ad Shokrollahi et al. [SSS97]). See [Lok09] for a survey o the subject. To the best of our kowledge, this state of affairs also holds for simple radomized costructios that use O() radom bits. The commo belief is that rigidity bouds for such radomized costructios ca be used for provig lower bouds for explicit computatioal problems that are related to the origial oes. For example, a adequate rigidity lower boud for radom Toeplitz matrices would yield a lower boud o the complexity of computig explicit biliear trasformatios. Ideed, this is aalogous to Adreev s proof of formula lower bouds [Ad87], where a lower boud for a radomized fuctio is trasformed ito a lower boud for a explicit fuctio (which takes the O() radom bits of the costructio as part of its iput, icreasig the iput size oly by a costat factor). 2 Our mai result shows that radom Toeplitz/Hakel matrices are rigid with high probability. Recall that a Toeplitz matrix T = (T i,j ) has costat diagoals (i.e., T i,j = T i+1,j+1 for every i, j). Hakel matrices are obtaied by turig Toeplitz matrices upside dow; that is, a Hakel matrix H = (H i,j ) has costat skew-diagoals (i.e., H i,j = H i+1,j 1 for every i, j). Hece, ay claim regardig oe family traslates to a equivalet claim regardig the other family. Theorem 1.2 (O the rigidity of radom Toeplitz/Hakel matrices). Let A F2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], with probability 1 o(1), the matrix A has rigidity Ω( 3 ) for rak r. r 2 log 1 For a ifiite I N, the sequece of matrices, {A } I such that A is a matrix, is called explicit if there exists a poly()-time algorithm that o iput I outputs the matrix A (ad outputs if I). 2 Lower bouds for matrix multiplicatio ad polyomial multiplicatio, i the model of arithmetic circuits over the reals with bouded costats, were previously achieved usig this approach [Raz03, BL04]. 1
Our bouds are asymptotically better tha Ω( 2 r log( r )) for rak r = o( log log log ), alas Valiat s origial motivatio refers to r > / log log. For rak r = 0.5+ε, where ε (0, 0.5), our boud yields a sigificat improvemet (i.e., 3 = 2 2ε 1.5 ε = 2 r 2 r ), ad this is actually the rage that is relevat for the project of Goldreich ad Wigderso [GW13]. 1.2 Goldreich-Wigderso s Project The work of Goldreich ad Wigderso [GW13] provides aother motivatio for the study of matrix rigidity. I fact, the problem of improvig the rigidity bouds for radom Toeplitz matrices was posed explicitly there. Specifically, provig a rigidity boud of 1.5+Ω(1) for rak 0.5+Ω(1) for radom Toeplitz matrices was proposed there as a possible ext step. Lower Bouds for Depth Three Caoical Circuits. Håstad [Hås89] showed that ay depththree Boolea circuit 3 computig the -way parity fuctio must be of size at least exp( ). Though Håstad s boud was refied durig the years [PPZ99, PPSZ05], to date, exp(ω( )) is the best lower boud for a explicit fuctio i the model of depth-three Boolea circuits. The work of Goldreich ad Wigderso [GW13] put forward a model of depth three caoical circuits, with the uderlyig log-term goal to exhibit better lower bouds for geeral depth-three Boolea circuits computig explicit multi-liear fuctios. Caoical circuits are restricted type of Boolea depth-three circuits, which ca be illustrated by cosiderig the smallest kow depth-three circuits for -way parity. The latter Õ(2 )-size circuits are obtaied by combiig a CNF that computes a -way parity with DNFs that compute -way parities of disjoit blocks of the iput bits. The costructio suggests the followig scheme for obtaiig Boolea circuits that compute multiliear fuctios. First, costruct a arithmetic circuit that uses arbitrary multiliear gates of parameterized arity, ad the covert it to a Boolea circuit whose size is expoetial i the maximum betwee the arity ad the umber of gates i the arithmetic circuit. The arithmetic model is outlied ext. Lower Bouds for Multiliear Circuits. Suppose we wish to compute a t-liear fuctio that depeds o t blocks of iputs, x (1),..., x (t), each of legth ; that is, the fuctio is liear i each of the x (j) s. We cosider circuits that use arbitrary t-liear gates of parameterized arity. That is, the circuits are directed acyclic graphs, where each iteral ode computes a t-liear fuctio of its iputs. We further restrict our circuit such that each iteral gate computes a multiliear formal polyomial i the iputs x (1)..., x (t). We say that such a multiliear circuit is of AN-complexity 4 m if m equals the maximum betwee the umber of the circuit gates ad the maximal arity of the gates. For a t-liear fuctio F, we deote by C(F ) the miimal AN-complexity of a multiliear circuit which compute the fuctio F. (We will abuse otatio ad refer to the AN-complexity of a tesor/matrix as the AN-complexity of the correspodig t-liear fuctio.) I the example of parity, we have a bottom layer of gates each takig iputs ad computig their parity. Above these gates, we have a gate which takes the results ad computes their parity. Overall, we got a (multi)-liear circuit of AN-complexity + 1. Goldreich ad Wigderso showed that ay multiliear circuit of AN-complexity m yields a depth-three Boolea circuit of size exp(m) computig the same fuctio (see [GW13, Prop. 2.9]). I fact, these Boolea circuits have much more structure, ad are referred to by Goldreich ad Wigderso as caoical circuits. Thus, a prelimiary step towards beatig the exp(ω( )) lower 3 That is, a circuit of ubouded fa-i OR ad AND gates with leaves that are variables or their egatios. 4 where AN stads for Arity ad Number of gates. 2
boud o the size of depth-three Boolea circuits for explicit O(1)-liear fuctios, 5 will be to beat the Ω( ) AN-complexity lower boud for such fuctios i the model of multiliear circuits. Agai, as i Valiat s questio, if we just ask about the existece of hard t-liear fuctios, the most t-liear fuctios caot be computed by a multiliear circuit of AN-complexity smaller tha (t) t/(t+1) : See [GW13, Thm. 4.1], which uses a coutig argumet. The more importat ad challegig problem is to came up with a explicit t-liear fuctio for which such bouds, or eve just ω( ) lower bouds, ca be proved. Reductio to (Structured) Rigidity. Goldreich ad Wigderso reduce the problem of provig lower bouds for biliear circuits to the problem of rigidity [GW13, Sec. 4.2]. They show that if a biliear circuit is of AN-complexity m/2, the its correspodig matrix is ot m 3 rigid for rak m (i.e., it ca be expressed as a sum of a m 3 -sparse matrix ad a matrix of rak at most m). Hece, ay matrix that has rigidity m 3 for rak m correspods to a biliear fuctio that caot be computed by a biliear circuit of AN-complexity at most m/2. Furthermore, Goldreich ad Wigderso show that the sparse matrix arisig from their reductio has a additioal structure (to be specified later). This leads to a weaker otio of rigidity (see [GW13, Thm. 4.12] which establishes a separatio), called structured rigidity, for which it is potetially easier to prove lower bouds. Ope Problems i Goldreich-Wigderso. Oe ope problem posed by Goldreich ad Wigderso is provig that radom Toeplitz matrices have rigidity m 3 (or just structured rigidity m 3 ) for rak m = 0.5+Ω(1). This would yield a AN-complexity lower boud of m for the correspodig biliear fuctio (via the reductio i [GW13, Thm. 4.4]) 6 as well as a similar lower boud for the followig explicit triliear fuctio (via [GW13, Prop. 4.6]): F tet (x, y, z) = x i1 y i2 z i3. (1) i 1,i 2,i 3 []: 3 j=1 i j /2 /2 1.3 Resolvig the Foregoig Ope Problems We resolve the aforemetioed ope problem [GW13, Prob. 4.8] by provig that radom Toeplitz matrices have rigidity m 3 for rak m = Θ( 3/5 ), with high probability. This follows from our log 1/5 mai theorem (Theorem 1.2) by choosig r = m. Furthermore, we ca remove the logarithmic factor i the Ω otatio, by provig a slightly better lower boud for structured rigidity. Theorem 1.3 (O the structured rigidity of radom Toeplitz/Hakel matrices). Let A F 2 be a radom Toeplitz/Hakel matrix. The, for every r [, /32], the matrix A has structured rigidity Ω( 3 /r 2 ) for rak r. This implies (usig [GW13, Thm. 4.10] ad [GW13, Prop. 4.6]) that the AN-complexity of a radom Toeplitz matrix is Ω( 3/5 ), ad ditto for the explicit triliear fuctio F tet from Eq. (1). This resolves Problems 4.7 ad 4.2 i [GW13], resp. I additio, we show that aother explicit triliear fuctio has AN-complexity Ω( 3/5 ). Corollary 1.4 (AN-complexity lower boud for a explicit triliear fuctio). Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 x iy j z i+j. The, C(F ) = Ω( 3/5 ). 5 Ideed, this suggestio presumes that there exist O(1)-liear fuctios that require depth-three Boolea circuits of size exp(ω( )), which is also a ope problem suggested i [GW13]. 6 For structured rigidity, we use [GW13, Thm. 4.10]. 3
New Challeges. The most atural questio that arises from the foregoig results is to tighte the lower boud; that is, to show that radom Toeplitz matrices have AN-complexity Ω( 2/3 ) as cojectured by [GW13]. This would be the best possible, sice ay biliear fuctio ca be computed by a biliear circuit of AN-complexity O( 2/3 ); more geerally, by [GW13, Thm. 3.1], for ay t 2, ay t-liear fuctio ca be computed by a t-liear circuit of AN-complexity O((t) t/(t+1) ). Aother atural follow up questio is to exhibit a explicit O(1)-liear fuctio havig AN-complexity Ω( α ) for some costat α > 3/5; of course, the larger α, the better. Our progress o these ope problems is captured by the followig two results. Theorem 1.5 (Depth-two AN-complexity lower boud for radom Toeplitz matrices). Let F be a biliear fuctio that correspods to a radom Toeplitz matrix. The, with probability 1 o(1), the fuctio F caot be computed by multiliear circuits of depth two havig AN-complexity 2/3 /(log ) 1/3. Theorem 1.5 establishes the desired AN-complexity lower boud for radom Toeplitz matrices, but oly for depth-two multiliear circuits. We ote that the AN-complexity upper boud of [GW13, Thm. 3.1] holds via depth-two circuits, ad so Theorem 1.5 is almost optimal with respect to depth-two multiliear circuits. Theorem 1.5 implies that the triliear fuctio F (x, y, z) = j=1 x iy j z i+j caot be computed by multiliear circuits of depth two ad AN-complexity 2/3 /(log ) 1/3. Theorem 1.6 (Improved AN-complexity lower boud for explicit 4-liear fuctios). There exists a explicit 4-liear fuctio havig AN-complexity Ω( 2/3 /(log ) 1/3 ). Theorem 1.6 is proved by first showig that, with high probability, biliear fuctios associated with matrices that are sampled from a 2 -biased sample space (over {0, 1} 2 ) have AN-complexity Ω( 2/3 ). Note that by the aforemetioed upper boud, this lower boud is tight (up to logarithmic factors). Next, we ote that samplig such matrices ca be doe usig O() radom bits [NN93, AGHP92, MST06], which matches the amout of radomess used for samplig a radom Toeplitz matrix. Furthermore, i the explicit small-biased costructio of Mossel et al. [MST06], each bit i the sampled strig is a biliear fuctio of the radom bits, allowig us to give a explicit 4-liear fuctio with AN-complexity Ω( 2/3 ). 1.4 Overview of the Proof of Theorem 1.2 We give a overview of the proof of Theorem 1.2 (for the case of Hakel matrices). Recall that we wish to show that a radom Hakel matrix has rigidity Ω( 3 /(r 2 log )) for rak r, with high probability. Let A be a radom -by- Hakel matrix, of the form A i,j = a i+j for idepedet radom bits a 2,..., a 2. We partitio A ito (/2r) 2 submatrices each of size 2r 2r ad show that with high probability each submatrix A has rigidity Ω(/ log ) for rak r. This easily implies that A has rigidity Ω((/ log ) (/2r) 2 ) = Ω( 3 ) for rak r, which will complete the proof. r 2 log Cosider the partitio of A ito (/2r) (/2r) submatrices, each of size 2r 2r, such that a geeric submatrix cosists of 2r cosecutive colums ad 2r equally spaced rows (i.e., rows that are at distace /2r apart). Next, we ote that ay of the above submatrices of A are of the form A = a i+1 a i+2 a i+3... a i+2r a i+k+1 a i+k+2 a i+k+3... a i+k+2r............... a i+(2r 1)k+1 a i+(2r 1)k+2 a i+(2r 1)k+3... a i+(2r 1)k+2r 4
where k = /2r ( 2r, by the assumptio r ), ad i is determied by the locatio of A i A (i.e., if A is the (i, j)-th submatrix, the i = i 1 + (j 1) 2k). Notice that A is a 2r 2r submatrix that depeds o (2r 1)k + 2r = Θ() radom bits. This allows us to hadle up to exp() bad evets whe applyig a uio boud. I our mai lemma, we show that for ay fixed matrix S (eve if S is ot sparse) the submatrix matrix A S is of rak greater tha r with probability at least 1 2 Ω(), where the probability is take over the choice of A (equiv., over the choice of a i+1,..., a i+(2r 1)k+2r ). As the umber of o(/ log )-sparse matrices is 2 o(), we may apply a uio boud over all possible sparse submatrices ad get that with high probability the submatrix A has rigidity Ω(/ log ) for rak r. 1.5 Orgaizatio Our mai results (i.e., Theorems 1.2 ad 1.3 ad Corollary 1.4) are proved i Sectio 3, which follows a short prelimiary sectio (Sectio 2). Next, Theorems 1.5 ad 1.6 are proved, i two steps. I Sectio 4 we idetify structural properties of matrices that correspod to biliear fuctios of low AN (ad AN2) complexity. These properties correspod to (eve more) restricted otios of structured rigidity, ad i Sectio 5 we show that (with high probability) matrices draw from the two relevat distributios do ot satisfy these properties. We coclude, with a techical digest (Sectio 6.1), a remark o the radomess-rigidity tradeoff (Sectio 6.2), ad a list of some ope problems (Sectio 6.3). I the appedices, we geeralize Theorems 1.2 ad 1.3 to geeral fiite fields (Sectio A.1) ad prove that AN-complexity ad AN2-complexity are equivalet to restricted otios of structured rigidity (Sectio A.3). 2 Prelimiaries We deote by [] = {1,..., }. For, k N, we deote by ( k) = k ( i=0 i), ad use the followig (crude) boud which suffices for our argumet ( ) ( ) 2 k mi{(2) k, (6/k) k }. (2) k k For a matrix A, we deote its i-th row by A i, ad its j-th colum by A (j). We deote by wt(a) the umber of o-zero etries i the matrix A, ad say that A is s-sparse if wt(a) s. A Hakel matrix over a field F is a square matrix with costat skew-diagoals; that is, ay matrix A F of the form A i,j = a i+j for some a 2,..., a 2 F. A Toeplitz matrix over a field F is a square matrix with costat diagoals, i.e. ay matrix A F of the form A i,j = a i j for some a ( 1),..., a 1 F. Note that a Hakel matrix is a upside-dow Toeplitz matrix. Throughout the paper, uless specified otherwise, we talk about matrices over the field F 2, ad matrix rak refers to the rak over F 2. Defiitio 2.1 (Structured rigidity, [GW13, Def. 4.9]). We say that a matrix A has structured rigidity (m 1, m 2, m 3 ) for rak r if for every matrix R of rak at most r ad for every X 1,... X m1, Y 1,..., Y m1 [] such that X 1 = = X m1 = m 2 ad Y 1 = = Y m1 = m 3 it holds that A R m 1 k=1 (X k Y k ), where M S meas that all o-zero etries of the matrix M reside i the set S [] []. We say that a matrix A has structured rigidity m 3 for rak r if A has structured rigidity (m, m, m) for rak r. Ideed, ay matrix that has rigidity s for rak r, also has structured rigidity s for rak r, but the other directio does ot hold (see [GW13, Thm. 4.12]). 5
Defiitio 2.2 (Multiliear circuits). A multiliear circuit o t blocks of iputs x (1),..., x (t) {0, 1} is a directed acyclic graph whose odes are associated with arbitrary multiliear gates, such that if two gates have directed paths to them from the same block of iputs, the the results of these two gates are ot multiplied together by aother gate. Defiitio 2.3 (The AN-complexity of multiliear circuits with geeral gates, [GW13, Def. 2.2]). The arity of a multiliear circuit is the maximum arity of its (geeral) gates. The AN-complexity of a multiliear circuit is the maximum betwee its arity ad its umber of gates (where we cout oly the geeral gates ad ot the leaves, i.e., variables). The AN-complexity of a multiliear fuctio F, deoted C(F ), is the miimum AN-complexity of a multiliear circuit that computes F. The AN2- complexity of a multiliear fuctio F, deoted C 2 (F ), is the miimum complexity of a depth-two multiliear circuit that computes F. Theorem 2.4 ([GW13, Thm. 4.10]). If A is a -by- matrix that has structured rigidity m 3 for rak m, the the correspodig biliear fuctio F satisfies C(F ) m/2. 3 Mai Results We prove our results bottom-up, startig with the mai lemma, as metioed i the proof overview. Lemma 3.1 (Mai Lemma). Let m, k N, 16 k m. Let A F m m 2 be the radom matrix a 1 a 2 a 3... a m a k+1 a k+2 a k+3... a k+m............... a (m 1)k+1 a (m 1)k+2 a (m 1)k+3... a (m 1)k+m where a 1,..., a (m 1)k+m are uiform idepedet radom bits, ad let S F m m 2 be some fixed matrix. The, Pr A [rak(s + A) m/2] 2 km/16. Note that for k = 1 the matrix i Lemma 3.1 is a radom Hakel matrix, ad for k = m it is a totally radom matrix. The requiremet k 16 is ot essetial i the lemma; it is used to make expressios icer. For k 1 ad rak r m/2 the proof gives Pr A [rak(s+a) r] ( m r) 2 mk/8. Proof. For a fixed S ad a radom A as above, let B = S + A. If r = rak(b) m/2, the oe ca costruct a basis B i1, B i2,..., B ir of the row space of B by the followig iterative process: Let i 1 be the first ozero row of B, let i 2 > i 1 be the first row i B that is ot spaed by row i 1, let i 3 > i 2 be the first row i B that is ot spaed by rows i 1 ad i 2, etc. We get that i 1 < i 2 < < i r ad 1. For j < i 1 the j-th row of B is the all zeroes row. 2. For i t 1 < j < i t the j-th row of B is spaed by rows i 1,..., i t 1 of B. 3. For i r < j the j-th row of B is spaed by rows i 1,..., i r of B. More cocisely, deotig by I = {i 1,..., i r }, we get j [m] \ I : B j spa{b i : i I, i < j}. (3) We boud the probability that such a sequece I = {i 1,..., i r } exists, where r m/2. We apply a uio boud over all possible sequeces I, ad for ay fixed sequece of legth at most m/2, we shall show that (3) holds with very low probability. Give such a sequece I, let J = [m] I be 6
its complemet. Settig = m/k, we ca select a icreasig sequece of J / idices i J such that each two idices differ by at least. 7 Take j 1 < j 2 < < j t to be such a sequece of idices, where t J m/2 m/k k 4. For l [t], let E l be the evet that row j l is spaed by the rows idexed by I [j l 1]. The, Pr [Eq. (3) holds for I] Pr[E 1, E 2,..., E t ] = Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] (4) Next, we show that for each l [t], we have Pr[E l E 1,..., E l 1 ] 2 m/2. However, istead of coditioig o E 1,..., E l 1, we shall coditio o a set of the radom bits, to be specified ext, that determie rows B 1,..., B jl 1 o oe had, but are idepedet from the radom row B jl o the other had. Sice j l j l 1 + m/k by our desig, we get (j l 1)k (j l 1 1)k + m. Hece, the radom bits a 1,..., a (jl 1)k determie B 1,..., B jl 1, ad leave the radom row B jl = (a (jl 1)k+1,..., a (jl 1)k+m) totally udetermied. Coditioig o the worstcase assigmet for the former radom variables (uder which E 1,..., E l 1 holds) yields a upper boud o Pr[E l E 1,..., E l 1 ]. Thus, it is eough to show that Pr[E l a 1,..., a (jl 1)k] 2 m/2 for ay possible fixed choice of values to a 1,..., a (jl 1)k. To avoid multiple subscripts, we set for the rest of the proof j j l. Let us remark that after fixig a 1,..., a (j 1)k, rows 1,..., j m/k are completely fixed, rows j m/k + 1,..., j 1 are partially fixed, ad row j is etirely udetermied. Based o that, we shall show that Pr[E l a 1,..., a (j 1)k ] 2 m/2. (5) Let I := I [j 1], ad fix a liear combiatio of the rows idexed by I, i.e., i I c ib i, amog all 2 I such liear combiatios. We show that the probability that B j = i I c i B i (6) is 2 m. (This is similar, up to mior differeces, to the folklore result that ay fixed liear combiatio of rows i a radom Toeplitz matrix is distributed uiformly over F m 2 see [Gol08, Prop. 8.25]. We give the details for completeess.) The probability that the first bit of B j equals the first bit of the liear combiatio i (6) is exactly 1/2, sice B j,1 = S j,1 + a (j 1)k+1, ad all etries {B i,1 } i I ivolve oly bits from a 1,..., a (j 2)k+1, which were already fixed (sice (j 2)k + 1 (j 1)k). Fixig a (j 1)k+1 such that equality o the first bit holds, the secod bit B j,2 equals the resultig liear combiatio with probability 1/2 as well. This happes sice B j,2 equals S j,2 + a (j 1)k+2, where a (j 1)k+2 was t already fixed, ad all etries {B i,2 } i I ivolve oly bits from a 2,..., a (j 2)k+2, which were already fixed (sice (j 2)k + 2 (j 1)k + 1). Ad so o, every bit i the j-th row of B equals the resultig liear combiatio with probability 1/2, coditioed o the fixig of the previous bits. Overall, B j = i I c ib i with probability 2 m for a fixed choice of coefficiets {c i } i I. 8 Takig a uio boud over all possible coefficiets {c i } i I gives Pr[E l E 1,..., E l 1 ] 2 I 2 m 2 m/2. Pluggig this boud ito Eq. (4) we get ( Pr [Eq. (3) holds for I] Pr[E 1 ] Pr[E 2 E 1 ] Pr[E t E 1,..., E t 1 ] 2 m/2) t 2 mk/8. where i the last iequality we used t k/4. Takig a uio boud over all possible sequeces I of legth at most m/2, whose umber is defiitely less tha 2 m, ad usig k 16, we get Pr[rak(S + A) m/2] 2 m 2 mk/8 2 mk/16. 7 Oe ca costruct such a set greedily: choose the miimal idex j i J, remove all idices i J [j, j + 1]. Repeat util J is empty. 8 Alteratively, coditioed o a 1,..., a (j 1)k ad the choice of the liear combiatio, there exist exactly oe choice for a (j 1)k+1,..., a (j 1)k+m that satisfies Eq. (6). 7
We cotiue with our mai theorem. Theorem 3.2 (Radom Hakel matrices are rigid). Let A F2 be a radom Hakel matrix A i,j = a i+j where a 2,..., a 2 are uiform idepedet radom bits. The, for every r /32, with probability 1 o(1), the matrix A has rigidity 3 for rak r. 160r 2 log(960r 2 /) Before provig Theorem 3.2, we state a immediate corollary of it. Corollary 3.3. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that for every ε > 0 1. With probability 1 o(1), the matrix A has rigidity c 2 for rak. 2. With probability 1 o(1), the matrix A has rigidity c 2 2ε / log for rak 1/2+ε. 3. With probability 1 o(1), the matrix A has rigidity m 3 for rak m = c 3/5 log 1/5. 4. With probability 1 o(1), the matrix A has rigidity c 1+2ε / log for rak 1 ε. Proof of Theorem 3.2. Suppose towards cotradictio that A ca be represeted as a sum of a matrix R of rak at most r, ad a s-sparse matrix S, where s 3 /(160r 2 log(960r 2 /)). Let m = 2r, ad assume for coveiece that k = /m is a iteger. Cosider the followig partitio of A s etries ito (/m) 2 submatrices, each of dimesio m m. For i [/m] ad j [/m], let I i = {i, i + k,..., i + (m 1)k}, J j = {(j 1)m + 1, (j 1)m + 2,..., jm}. (7) Deote by A i,j (R i,j, S i,j, resp.) the matrix A (R, S, resp.) restricted to rows I i ad colums J j. See Figure 1 for a example of such a submatrix. The mai observatio is that for each a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 16 Figure 1: A submatrix A 1,1 of the matrix A, for m = 4 ad k = 2. (i, j) [/m] 2, the matrix A i,j is of the form eeded by the mai lemma. Aother observatio is that sice the submatrices S i,j partitios the sparse matrix S, oe of them has sparsity at most s s m2. I additio, sice rak of a submatrix may oly decrease, for every i, j, it holds that 2 rak(r i,j ) rak(r) r. We say that A i,j is simple if it ca be represeted as a sum of a s -sparse matrix ad a matrix of rak at most r. By the above discussio, A ca be represeted as S + R where S is s-sparse ad R is of rak at most r, oly if there exists a submatrix A i,j that is simple. We shall show that the 8
latter occurs with very low probability: Usig s Pr [ i, j : A i,j is simple ] i,j i,j Pr[A i,j is simple] S F m m 2 : wt(s) s Pr[rak(A i,j + S) m/2] (Uio Boud) (Uio Boud) ( ) ( ) 2 m 2 m s 2 mk/16 (Lemma 3.1) < 2 (6m 2 /s ) s 2 /16. ( = km ad (2)) 40 log(240m 2 /), which follows from s 3, we get that 160r 2 log(960r 2 /) Pr [ i, j : A i,j is simple ] ( < 2 6m 2 40 ) /40 log(240m log(240m2 /) 2 /) 2 /16 = 2 ((240m 2 /) log(240m 2 /) ) /40 log(240m 2 /) 2 /16 2 ((240m 2 /) 2) /40 log(240m 2 /) 2 /16 = 2 2 /20 2 /16 = o(1). Note that the proof works as log as the umber of possibilities for a s -sparse matrix S i,j is smaller tha 2 /16 / 2. Our ext theorem exploits the fact that there is a smaller umber of possibilities for submatrices of structured sparse matrices (as i Defiitio 2.1). I fact, this is the oly property of S that the foregoig proof uses. This yields the followig improved boud. Theorem 3.4 (Radom Hakel matrices are structured rigid). Let A F 2 be a radom Hakel matrix. The, for every r /32, ad s 3 /1000r 2, with probability 1 o(1), the matrix A has structured rigidity s for rak r. Before provig Theorem 3.4 we state three corollaries of it. The first corollary is immediate by choosig r = 3/5. Corollary 3.5. Let A F 2 be a radom Hakel matrix. The, there exists a uiversal costat c > 0 such that with probability 1 o(1), the matrix A has structured rigidity c 9/5 for rak 3/5. The secod corollary follows from the first corollary ad Theorem 2.4. Corollary 3.6. Let A F 2 be a radom Hakel matrix, ad let F (x, y) = j=1 A i,jx i y j. The, with probability 1 o(1), it holds that C(F ) = Ω( 3/5 ). The last corollary shows that there exists a explicit triliear form with AN-complexity Ω( 3/5 ). This is the first improvemet over the trivial Ω( ) lower boud for explicit tesors, ad i doig so it solves Problem 4.2 from [GW13] i the affirmative. Goldreich ad Wigderso [GW13, Prop. 4.6] show that if some Toeplitz matrix has AN-complexity Ω(m), the F tet defied i Eq. (1) has ANcomplexity Ω(m) has well. We follow their method, but preset a simpler argumet for a differet triliear fuctio. Corollary 3.7. Let F : {0, 1} {0, 1} {0, 1} 2 {0, 1} be the triliear fuctio defied by F (x, y, z) = j=1 z i+jx i y j. The, C(F ) = Ω( 3/5 ). 9
Proof. Accordig to Corollary 3.6, there exists a Hakel matrix A, defied by some diagoal values a 2,..., a 2, such that the biliear form i,j a i+jx i y j has AN-complexity Ω( 3/5 ). Let C be a triliear circuit computig F with miimal AN-complexity, ad deote its complexity by m. Fixig the values of the variables z i to a i, for all i {2,..., 2}, we get a biliear circuit i x ad y of AN-complexity at most m. Thus, m = Ω( 3/5 ). We retur to prove Theorem 3.4. Proof of Theorem 3.4. The proof follows the lies of the proof of Theorem 3.2. We let m = 2r, k = /m, ad t = s 1/3. We assume towards cotradictio that A = S + R, where R is of rak at most r, ad S is a sum of t matrices S 1,..., S t F 2, such that the oes i each matrix S l are a subset of some X l Y l, where X l, Y l t. Deote by T the -by- matrix over F 2 with T i,j = 1 iff (i, j) is cotaied i at least oe X l Y l. It is clear from T s defiitio that the oes i S are a subset of the oes i T. As i Theorem 3.2, we partitio A, R, S, ad also T, to (/m) 2 submatrices, accordig to the partitio of row idices I 1,..., I /m ad colum idices J 1,..., J /m, defied as i the proof of Theorem 3.2 (see Eq. 7). For a radom (i, j) [/m] 2, it holds that [ E wt(t i,j ) ] t 3 m2 i,j 2, E i,j [ t ] X l I i We say that a submatrix T i,j is good if wt(t i,j ) 4t 3 m2 2, t t t m, X l I i 4t t m, E i,j [ t ] Y l J j t t t m. (8) Y l J j 4t t m. (9) Usig Markov s iequality, each of the above three evets happe with probability at least 3/4. Usig uio boud (o the complemet evets) with probability at least 1/4 all evets occur simultaeously, makig T i,j good. Next, we cout the umber of possible good submatrices T i,j. Each such submatrix is uiquely determied by the sets X 1,..., X t ad Y 1,..., Y t, where X l = X l I i ad Y l = Y l J j. Furthermore, a collectio (X 1,..., X t) such that l X l 4t2 m correspods to a set X I i [t] of size at most 4t 2 m such that (p, l) X iff p X l (ad similarly for (Y 1,..., Y t )). Hece, the umber of possible good submatrices is at most { } X I i [t] : X 4t2 m 2 ( ) mt 2 ( ) 2 = 4t 2 (2mt) 4t2 m/ 16t 2m/. m/ We say that S i,j is good if T i,j is good, ad we say that A i,j is simple if it is the sum of a good S i,j ad a matrix of rak at most r. Next, we cout the umber of possible good submatrices S i,j. Sice the oes of S i,j are a subset of the oes i T i,j, the umber of possibilities for S i,j is at most 16t2 m/ 2 wt(t i,j) 16t2 m/ 2 4t3 m 2 / 2. Usig the boud o the umber of possible good submatrices S i,j, we may boud the probability that some A i,j is simple: Pr [ i, j : A i,j is simple ] Pr[rak(A i,j + S i,j ) m/2] (Uio Boud) i,j S i,j good ( ) 2 16t 2 m/ 2 4t3 m 2 / 2 2 mk/16 (Lemma 3.1) m 10
Recall that m = 2r ad k = /m to get which is o(1) for t 3 Pr [ i, j : A i,j is simple ] 2 2 log + 32 log t2 r/ + 16t 3 r 2 / 2 /16, 3 ad r. 1000r 2 Geeralizatio to Larger Fields. The choice of field F 2 was ot crucial i the proofs of Lemma 3.1, Theorem 3.2 ad Theorem 3.4. Oe ca sytactically replace the field size 2 by ay prime power q, keepig the proofs itact. Furthermore, i Theorem 3.2, we slightly beefit from takig a larger field. For details see Appedix A.1. 4 The Structure of Matrices of Small Biliear Circuits I this sectio we shall further refie the structure of matrices associated with small biliear circuits, beyod the structure captured by Defiitio 2.1 ad Theorem 2.4. We begi by explicitly statig structural results that are implicit i the proof of [GW13, Thm. 4.4]: Sectio 4.1 refers to the structure of biliear fuctios that are computed by depth-2 biliear circuits of small ANcomplexity, whereas Sectio 4.2 refers to geeral biliear circuits. These statemets ca be viewed as relatig AN-complexity to fier otios of structured rigidity (tha the oe of Defiitio 2.1). I Sectio 4.3 we go beyod [GW13], ad aalyze the structure of the submatrices of matrices associated with small biliear circuits, by startig with the foregoig structural results (of [GW13]) ad proceedig aalogously to the first part of the proof of Theorem 3.4. The results of Sectio 4.3 will play a pivotal role i the improved lower bouds proved i Sectio 5. 4.1 The Structure of Matrices Associated with Depth Two Biliear Circuits We say a row/colum i a matrix is m-sparse if it cotais at most m o-zero etries. Likewise, a liear fuctio l(x) (resp. l (y)) is m-sparse if it depeds o at most m etries i x (resp. y). Lastly, recall that by Defiitio 2.3, C 2 (F ) is the miimal AN-complexity of a depth-two biliear circuit computig F. Propositio 4.1 (Structure of fuctios computed by depth two biliear circuits [GW13, Thm. 4.4]). Let F be a biliear fuctio over x, y {0, 1} with C 2 (F ) m. The, F ca be expressed as (i,j) P L i (x)l j(y) + Q l (x, y) (10) where P is a subset of [m] [m], L 1,..., L m ad L 1,..., L m are m-sparse liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = L col P L row + S l (11) where L col is a m matrix with m-sparse colums, P is a geeral m m matrix, L row is a m matrix with m-sparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. 11
Propositio 4.1 is proved explicitly i the warm-up part of the proof of [GW13, Thm. 4.4]. The followig propositio asserts that the coverse holds as well. This implies that the characterizatio of Prop. 4.1 captures C 2 completely. Propositio 4.2. Ay biliear form F that ca be writte as i Eq. (10), has C 2 (F ) = O(m). We defer the proof of this propositio to Appedix A.3. 4.2 The Structure of Matrices Associated with Geeral Biliear Circuits Propositio 4.3 (Structure of fuctios computed by geeral biliear circuits). Let F be a biliear fuctio over x, y {0, 1} with C(F ) m. The, F ca be expressed as L i (x)l i(y) + M i(x)m i (y) + Q l (x, y) (12) where L 1,..., L m ad M 1,..., M m are m-sparse liear fuctios, L 1,..., L m ad M 1,..., M m are geeral liear fuctios, ad each Q l is a biliear fuctio of at most m variables from x ad at most m variables from y. The matrix associated with F has the form A = L col B + CL row + S l (13) where L col is a m matrix with m-sparse colums, B is a geeral m matrix, C is a geeral m matrix, L row is a m matrix with m-sparse rows, ad each S l is a matrix whose oes reside i a m m rectagle. Propositio 4.3 is oly implicit i the proof of [GW13, Thm. 4.4], ad we iclude its proof i Appedix A.2. The followig propositio asserts that the coverse holds as well for m. This implies that the characterizatio of Prop. 4.3 captures C. Propositio 4.4. Ay biliear form F that ca be writte as i Eq. (12), has C(F ) = O(m + ). We defer the proof of this propositio to Appedix A.3. 4.3 Substructures I this subsectio, similarly to the first part of the proof of Theorem 3.4, we fid a structured submatrix of the matrix associated with ay biliear fuctio with low AN-complexity. I sectio 5, we prove that radom Toeplitz matrices ad small-biased matrices do ot have these structured submatrices, with high probability. This ultimately proves AN-complexity lower bouds for such radom matrices. Let F be a biliear fuctio over x, y {0, 1} with C(F ) m. Startig with Propositio 4.3, we write the matrix A associated with F as A = L col B + CL row + S l such that the o-zero etries of S l are a subset of X l Y l, where X l, Y l m. T = m X l Y l, ad ote that T m 3. Deote by 12
Let I 1,..., I /2m ad J 1,..., J /2m be some fixed equipartitio of the row idices ad colum idices of A, respectively, where each I i ad J j is of size 2m. This partitio aturally defies (/2m) 2 submatrices as follows. For ay (i, j) we deote by A i,j (resp. S i,j l ) the matrix A (resp. S l ) restricted to rows I i ad colums J j. For ay i (resp. j) we deote by L i col ad Ci (resp. B j ad L j row) the matrices L col ad C (resp,. B ad L row ) restricted to I i (resp. J j ). The, oe ca write A i,j = L i col Bj + C i L j row + S i,j l, (14) where S i,j l T (I i J j ). Next, we show that there exists a choice of (i, j) with favorable properties of the submatrices of L i col, Lj row ad of the subsets {X l I i } l, {Y l J j } l, ad T (I i J j ). Propositio 4.5 (Structure of submatrix of matrices associated with small biliear circuits). For every l [m], let S l X l Y l, where X l, Y l m, ad T = m X l Y l. Let I 1,..., I /2m [] ad J 1,..., J /2m [] be two partitios of [] where each I i ad J j is of size 2m. Let A i,j, L i col ad L j row be as i (14). The, there exists a (i, j) [/2m] 2 such that: (1) T (I i J j ) 24m5, (2) 2 m X l I i 12m3, (3) m Y l J j 12m3, (4) wt(li col ) 12m3, ad (5) wt(lj row) 12m3. If C 2 (F ) m, the startig with Propositio 4.1, we ca express A i,j as L i col P Lj row + m ad Propositio 4.5 holds as well i this case. Proof. For a uiformly radom (i, j) [/2m] 2, it holds that E [ T (I i J j ) ] m 3 (2m)2 i,j [ 2 m ] E X l I i m m 2m i,j [ m ] E Y l J j m m 2m i,j E i,j [wt(li col )] m m 2m E i,j [wt(lj row)] m m 2m Si,j l Usig Markov s iequality, each of the followig bad evets occur with probability at most 1/6 T (I i J j ) 6m 3 (2m)2 2 X l I i 6m m 2m Y l J j 6m m 2m wt(l i col ) 6m m 2m wt(l j row) 6m m 2m 13
By uio boud, with probability at least 1 5/6 over the choice of (i, j), oe of the bad evets occur, which completes the proof. We wish to express the structure captured by Eq. (14) i terms of liear equatios o the etries of the matrix A i,j l Si,j l. To do so we eed the followig defiitio. Defiitio 4.6 (Orthogoal complemet of a matrix). Let m. If A is a m matrix, ad B is a ( m) matrix of rak m such that BA = 0 the we say that B is a left orthogoal complemet of A. If A is a m matrix, ad B is a ( m) matrix of rak m such that AB = 0 the we say that B is a right orthogoal complemet of A. Remark 4.7. Note that there are may possible choices of a orthogoal complemet of a give matrix. Therefore, we shall refer to the left (right, resp.) orthogoal complemet of A as some caoical choice of a left (right, resp.) orthogoal complemet of A, say the first such matrix accordig to lexicographical order (over a fiite field F). It is well kow that ay matrix over a field has a orthogoal complemet. Now, suppose that A i,j l S l i,j = L i col Bj + C i L j row (as i Eq. (14)). Let D be a m 2m matrix which is the left orthogoal complemet of L i col, ad let E be a 2m m matrix which the right orthogoal complemet of L j row. The, D (A i,j l S l i,j ) E = 0 m m. (15) I the case of depth-2 circuits we have A i,j l S l i,j = L i col P Lj row. Usig E, the right orthogoal complemet of L j row as above, we ca write (A i,j l S l i,j ) E = 0 2m m. (16) I the ext sectio, we shall desig tests based o Equatios (15) ad (16). 5 Tests for AN Complexity ad AN2 Complexity Havig idetified (i Sectio 4.3) structural properties that are satisfied by ay matrix associated with ay biliear fuctio with low AN-complexity, we prove lower bouds o the AN-complexity of explicit distributios of matrices by showig that (with high probability) these distributios do ot satisfy these properties. We do so by desigig a test that always accepts matrices that have these properties, but rejects (with high probability) matrices draw from certai explicit distributios. (Sice the test is merely a metal experimet, i.e., we do ot ited to actually ru it, the test could be iefficiet.) Specifically, for a complexity boud m, ay matrix rejected by the correspodig test must have complexity greater tha m. We will show that a radom Toeplitz matrix, as well as a matrix whose etries are sampled from a 2 -biased distributio, are rejected by the correspodig test with overwhelmig probability, thus provig complexity lower bouds for such matrices. Actually, we will preset two tests: Oe for AN-complexity, rejectig most matrices take from a small-biased space, ad oe for AN2-complexity, rejectig most Toeplitz matrices, see Sectios 5.1 ad 5.3, respectively. I Sectio 5.2 we show that the lower boud for matrices take from a small-biased space yields a similar lower boud for a explicit 4-liear fuctio. 14
5.1 Lower Bouds for the AN-Complexity of Small-Biased Matrices For i [/2m] ad j [/2m], let 9 I i = {i, i + (/2m),..., i + (2m 1) (/2m)}, J j = {(j 1) (2m) + 1, (j 1) (2m) + 2,..., j (2m)}. (17) ad deote by A i,j the 2m-by-2m sub-matrix of A obtaied by restrictig A to rows I i ad colums J j. Cosider the followig test, where A i,j is viewed as idexed by [2m] [2m] rather tha by I i J j. Test 1 AN-Complexity Test Iput: Matrix A F 2 ad parameter m [] 1: for i = 1,..., /2m ad j = 1,..., /2m do 2: for all subsets {X i l }m of [2m] such that l Xi l 12m3 3: for all subsets {Y j l }m of [2m] such that l Y j l 12m3 4: Let T := m Xi l Y j 5: if T 24m5 the 2 6: for all matrices L i col l. do do 12m3 of dimesio 2m m ad sparsity at most do 7: Let D be the left orthogoal complemet of L i col (recall Remark 4.7). 8: for all matrices L j row of dimesio m 2m ad sparsity at most 12m3 do 9: Let E be the right orthogoal complemet of L j row. 10: if there exists N F 2m 2m 2 such that N T, ad D(A i,j N)E = 0 m m the 11: retur Pass. 12: retur Fail. The followig is a immediate corollary of Propositio 4.5 ad Eq. (15). Corollary 5.1. Every matrix associated with a biliear circuit of AN-complexity at most m passes Test 1 with parameter m. We cosider a distributio of matrices whose etries are chose from a small biased sample space. Specifically, we shall use a sample space over strigs of legth N = 2 i order to defie -by- matrices. We shall show that almost all such matrices are rejected by Test 1 with parameter m. But we eed a few prelimiaries first. Prelimiaries. Recall the defiitio of a ε-biased distributio from [NN93]. Defiitio 5.2 (Small-biased distributio). A distributio X over {0, 1} N is said to be ε-biased if for every o-empty set S [N], it holds that E x X [( 1) i S x i ] ε. We shall use the followig property of ε-biased distributios (implicit i [NN93]). Lemma 5.3 ([AGHP92, Lem. 1]). Let X be a ε-biased distributio over {0, 1} N. Let l 1,..., l t be liearly idepedet liear fuctios o x 1,..., x N. The, the probability that all liear fuctios evaluate to 0 o x X is at most ε + 2 t. The, the probability that all liear fuctios equal 0 simultaeously is at most ε + 2 t. 9 The specific choice for I i ad J j is ot crucial for our argumet i this subsectio, however it will be importat i the ext subsectio. Hece, sice we eed to pick some partitio, we might as well choose this oe. 15
We shall also use the followig simple fact from liear algebra. Fact 5.4. Let t,, m N such that t m. Let l 1,..., l t be a sequece of liearly idepedet liear fuctios (over F) o x 1,..., x. The, l 1,..., l t spa at least t m liearly idepedet fuctios that ivolve oly the variables x m+1,..., x. Proof. Thik of the liear fuctios as vectors i F, ad let V = spa{l 1,..., l t }. Cosider the subspace U = spa{e m+1,..., e }, where e i F is the uit vector with 1 i the i-th coordiate ad 0 elsewhere. The, dim(u V ) dim(u) + dim(v ) = ( m) + t = t m, whereas U V is the spa of l 1,..., l t that is supported oly o the last m coordiates. Actual Results. We are ow ready to aalyze the probability that a matrix sampled from a small biased space passes Test 1. The core of the aalysis refers to a sigle applicatio of Step 10, which refers to a specific choice of i, j, {Xl i}m, {Y j l }m as well as Li col, Lj row (which i tur, fixes D ad E as well). Lemma 5.5 (Core of the aalysis of Test 1). Fix i, j, {Xl i}m j ad {Yl }m that pass the check of Step 5, ad fix L i col ad L j row (which i tur, fixes D ad E as well). The, a matrix A whose etries are sampled from a ε-biased distributio satisfies the coditio i Step 10 with probability at most ε + 2 m2 +24m 5 / 2. Proof. For a fixed choice of i, j, {X i l }m, {Y j l }m, Li col ad L j row as above, we cosider a specific submatrix of dimesio 2m 2m of A, deoted A i,j. Note that the correspodig left (resp. right) orthogoal complemet of L i col (resp. L j row) is a m-by-2m (resp. 2m-by-m) matrix of rak m, deoted by D (resp. E). Recall that A i,j is a submatrix whose etries are sampled accordig to a ε-biased distributio. Our goal is to show that the equatio D(A i,j N)E = 0 (checked i Step 10) implies a lot of liearly idepedet liear equatios o the etries of A i,j. Let Z be a 2m 2m matrix of (2m) 2 Boolea variables, where we will later take Z to be A i,j N. Iterpret the equatios DZE = 0 m m as m 2 liear equatios o the (2m) 2 variables i Z. For i [m] ad j [m], we have a equatio of the form D i ZE (j) = 0, where D i is the i-th row of D ad E (j) is the j-th colum of E. We ca write D i ZE (j) = 2 2 k=1 D i,k Z k,l E l,j = k,l (D i E (j) ) k,l Z k,l ; that is, the coefficiets of the equatio are the tesor product of the vector D i with the vector E (j). Thikig of these m 2 liear equatios o (2m) 2 variables as a big matrix of dimesio m 2 (2m) 2, we ote that this matrix of liear equatios is the tesor product of D ad E, sice the (i, j) row equals to D i E (j) (viewed as a (2m) 2 -bit log vector). It is a kow fact that the rak of the tesor product of ay two matrices is the product of their rak; hece, we get rak(d E ) = rak(d) rak(e ) = m 2. I other words, we have a liearly idepedet set of m 2 liear equatios o the variables Z. However, we wat to get liear equatios over the variables of A, where Z = A N. Say that Z k,l is a oisy variable if (k, l) T. It will be eough to show that there are may idepedet liear equatios which ivolve oly o-oisy variables of the matrix. Sice the umber of oisy variables is T, by Fact 5.4 we ca fid at least m 2 T idepedet liear equatios that do ot ivolve oisy variables. Overall, we got m 2 T idepedet liear equatios o A i,j. By Lemma 5.3, a submatrix A i,j whose etries are sampled accordig to a ε-biased distributio satisfies all m 2 T equatios 16
with probability at most ε + 2 m2 + T. Lastly, the fact that {Xl i}m j ad {Yl }m passed the check of Step 5 meas that T 24m 5 / 2, which fiishes the proof. Theorem 5.6 (Almost all ε-biased matrices have high AN-complexity). A matrix A whose etries are sampled from a ε biased distributio is rejected by Test 1 with parameter m (which implies that the correspodig biliear fuctio has AN-complexity greater tha m), with probability at least ( ) ( ) 2 2m 2 4 ( 1 2m 12m 3 ε + 2 m2 +24m 5 / 2) /. I particular, for ε = 2 ad m = 2/3 10(log ) 1/3, this probability is at least 1 2 /2, for sufficietly large. Proof. We use a uio boud over all possible i, j, {Xl i}m, {Y j l }m, Li col ad L j row that ca be selected by the test, ad employ Lemma 5.5 for each possibility. The umber of optios for choosig (i, j) is (/2m) 2 ; the umber of optios for choosig {Xl i}m j (resp., {Yl }m ) is at most ( 2m 2 12m /) ; 3 the umber of optios for choosig L i col (resp., Lj row) is at most ( 2m 2 12m /). 3 5.2 Explicit 4-Liear Fuctios with AN-Complexity Ω( 2/3 ) We show that based o the ε-biased geerator of Mossel, Shpilka ad Trevisa [MST06] (described ext), the AN-complexity lower boud for the radomized biliear fuctio i Theorem 5.6 yields a similar lower boud o a explicit 4-liear fuctio. To describe Mossel et al. s costructio, we begi with some prelimiaries. Let N be a atural umber, deote by F = GF (2 N ), ad suppose we have a explicit represetatio of F as the quotiet F 2 [x]/(p(x)) where p(x) is a irreducible polyomial over F 2 of degree N. We remark that for N = 2 3 k, the polyomial p(x) may be chose to be x 2 3k + x 3k + 1 (cf. [LN97, Ex. 3.96]). 10 The, 1, x, x 2,..., x N 1 is a basis for GF (2 N ) over F 2. The map φ : F F defied by φ : z z x is a liear trasformatio over F 2, thus may be represeted by a matrix A F N N 2. The Frobeius trasformatio ϕ : F F defied by ϕ : z z 2 is also a liear trasformatio over F 2, thus may be represeted by a matrix B F N N 2. Give the polyomial p(x), the matrix A (resp. B) ca be computed i poly(n) time by writig the images of the basis elemets φ(1), φ(x),..., φ(x N 1 ) (resp. ϕ(1), ϕ(x),..., ϕ(x N 1 )) as polyomials modulo p(x). The geerator of Mossel et al. is give 2N iput bits c 1,..., c N ad d 1,..., d N, ad outputs N bits where 1 N, such that each output bit is a biliear fuctio i c = (c 1,..., c N ) ad d = (d 1,..., d N ). The output of the geerator o vectors c, d F N 2 is the N bits g i,j = c (A i B j )d for i {0,..., N 1} ad j {0,..., 1}. Mossel et al. [MST06] proved that this is a ε-bias geerator. Theorem 5.7 ([MST06, Thm. 6]). The bias of ay o-trivial liear combiatio of the g i,j s is at most 2 N 2. Corollary 5.8. Let N = 2 3 k ad let A ad B be the explicit matrices as above for the field GF (2 N ). Let = N/3 ad let F : {0, 1} {0, 1} {0, 1} N {0, 1} N be the 4-liear fuctio defied by F (a, b, c, d) = 1 1 i=0 j=0 a ib j (c T A i B j d). The, C(F ) = Ω( 2/3 / log 1/3 ). 10 For geeral N, it is ot kow how to fid such a polyomial p(x) without advice or radomess [KKS14]. 17