Matrix Multiplication II

Transcription

1 Matrx Multplcaton II Yuval Flmu January 14, 201 Thee note tarted ther lfe a a lecture gven at the Toronto Student Semnar on February 9, The materal taken motly from the clac paper by Coppermth and Wnograd [CW]. Other ource are 15.7 of Algebrac Complexty Theory [ACT], Stother the [Sto], V. Wllam recent paper [Wl], and the paper by Cohn at al. [CKSU]. Starred ecton are the one we ddn t have tme to cover. We preent three dfferent algorthm, all taken from [CW], n rapd ucceon. All thee algorthm are baed on Straen groundbreakng laer method. Straen orgnal dea are decrbed n the appendx. 1 Algorthm 1 The tartng pont the dentty ɛ ɛ ( ( ) x [0] 0 y[1] z [1] + x [1] y [0] 0 z[1] + x [1] y [1] z [0] 0 + O(ɛ 4 ) = (x [0] 0 + ɛx[1] )(y [0] 0 + ɛy[1] )(z [0] 0 + ɛz[1] ) x [0] 0 + ɛ2 x [1] (1 qɛ)x [0] 0 y[0] 0 z[0] 0. ) ( y [0] 0 + ɛ2 y [1] ) ( z [0] 0 + ɛ2 On the left-hand de, we have a um of three matrx product 1, 1, q, q, 1, 1, 1, q, 1 harng ome varable. We can ndcate thee hared varable ung upercrpt: z [1] ɛ ( 1, 1, q 0,1,1 + q, 1, 1 1,0,1 + 1, q, 1 1,1,0 ) + O(ɛ 4 ). The notaton n, m, p,j,k ndcate a tenor of type n, m, p whoe x-varable have upercrpt, whoe y-varable have upercrpt j, and whoe z-varable have upercrpt k. The pont that dfferent upercrpt of x correpond to djont et of varable. The dentty a t tand n t very ueful, nce the three (rather trval) matrx product nvolved all hare varable. The dea of the laer method to compute the Nth tenor power of the dentty, and then omehow eparate the varable. The N th tenor power of the dentty ha (q + 2) N term on the rght-hand de. What about the left-hand de? It of the form ) + 1

2 ɛ N F + O(ɛ N+1 ), where F now the um of N dfferent matrx product tenor, each of volume q N (the volume of a tenor n, m, p nmp). The plan now to take a large ubet of the varable o that f we zero out all the other varable, F eparate nto a djont um. More pecfcally, we can thnk of F a a et of trple of ndce. We wll ngle out large et of ndce X, Y, Z uch that among the trple F X Y Z, no varable repeat. In other word, for each x X, there at mot one par (y, z) Y Z uch that (x, y, z) F. How many trple can we expect to obtan n th way? Let P be the ze of the projecton of F nto one of the coordnate (they are all the ame by ymmetry). We defntely cannot expect to get more than P trple. In our cae, P = 2 N. However, th bound a tad nave, a the followng argument from [CKSU] how. Clafy the trple accordng to ther ource dtrbuton, that, how many trple of each type 110, 101, 011 generated them. If there are ( a, b, c of thoe, repectvely (a + b + c = N), then the projecton on the coordnate have ze N ) ( c, N ) ( b, N ) a. Hence the number of trple of type a, b, c at mot the mnmum of thee, whch at mot ( ) N N/. There are O(N 2 ) type, hence there can be at mot O(N 2 ) ( N N/) 2 h(1/)n trple, where h(p) = p log 2 p (1 p) log 2 (1 p) the bnary entropy functon. What th argument teache u that t enough to conder one type of trple, n th cae N/, N/, N/. Techncally, th wll manfet telf n the followng way. For a ubet G of F, contruct a conflct graph by takng the et G a vertce, and connectng any two vertce whch hare a coordnate. The ubet G contng of all et of trple of type N/, N/, N/ ha the property that all vertce have the ame degree. Converely, the degree n the graph correpondng to F depend on the ource dtrbuton: a trple of type a, b, c ha roughly 2 N a + 2 N b + 2 N c neghbor. To ee th, conder for example the frt coordnate, whch cont of c zeroe and N c one. Each ndex equal to one could come from ether 110 or 101, whle each ndex equal to zero certanly came from 011. Surprngly, th method of reducng F to G by pecfyng a ource dtrbuton optmal: we can zero varable o that G eparate to a djont um of P 1 o(1) term, where P the projecton of G to each of the coordnate. We wll tate th hortly, but frt, a defnton. Defnton 1. Let T Z be a fnte collecton of trple, and let φ: T R + be arbtrary. We defne below the numercal quantty cap(t, φ), the capacty of T wth repect to φ. Let N 1. We extend φ to T N multplcatvely, and to ubet of T N addtvely. For {1, 2, }, let α : T N Z N denote the projecton to the th coordnate. Say that ubet X α (T N ) ( {1, 2, }) are good f α njectve on T N X 1 X 2 X for {1, 2, }. The N-capacty cap N (T, φ) the maxmum of φ(t N X 1 X 2 X ) over all good ubet X 1, X 2, X. The capacty cap(t, φ) defned a cap(t, φ) = lm up N N cap N (T, φ). For example, n our cae T = {(1, 1, 0), (1, 0, 1), (0, 1, 1)}, and we can take φ to be contant. In the next ecton, we wll ee an example n whch φ ha to be non-contant. We defned the capacty a a lmt uperor, but a tenor product contructon how that cap N1 +N 2 (T, φ) N1 (T, φ) N2 (T, φ), and th mple that the lmt actually ext. Lemma 1. Let T, φ be a n Defnton 1. Suppoe that furthermore, each trple n T um to a contant, that T cloed under permutaton, and that φ contant over permutaton. Then 2

3 cap(t, φ) atfe the followng nequalte: log 2 cap(t, φ) max H(π 1) + π(t) log π 2 φ(t), t T log 2 cap(t, φ) max H(π) max H(τ) + H(π 1 ) + π(t) log π τ : τ 1 =π 2 φ(t), 1 t T where π range over all permutaton-nvarant probablty dtrbuton over T, π 1 the dtrbuton of the frt coordnate under π, and H the entropy functon gven by H(π) = t T π(t) log 2 π(t). Note that the maxmum ext nce the range of π compact. In our preent cae, T permutaton-nvarant, and o the lemma how that cap(t, 1) = 2 H(1/,2/) = 2 h(1/). We are now n poton to apply Schönhage aymptotc um nequalty. Takng the N th tenor power of the dentty and zerong out varable approprately, we get on the left-hand ze a djont um of roughly 2 h(1/)n term of volume q N. Hence 2 h(1/)n q τn (q + 2) N. Rearrangng varable, we deduce q + 2 ω log q 2 h(1/). Pluggng q = 8, we get ω The contructon n Lemma 1 randomzed. In other word, the fnal algorthm n t explct. Th make no dfference n our model, nce t non-unform. The method of condtonal expectaton can determnze the lemma, n cae one really care about unformty, though fndng the algorthm, whle t take polynomal tme, mght take more tme than executng t. Tghtne of Lemma 1 When the range of T mall, τ = π. In partcular, th the cae (a can be checked ung lnear algebra) f all trple n T are non-negatve and um to a contant whch at mot 5. For a um of 6, we have the followng counterexample: X = S(0, 2, 4), S(1, 2, ) 2, Y = S(0,, ) 2, S(1, 1, 4) 2, S(2, 2, 2) 6. Here S(a, b, c) a horthand for all dtnct permutaton of (a, b, c). Both X and Y have the ame projecton nto each of the coordnate, but the conttuent are dfferent. We have H(X) = H( 1 18 [6], 1 9 [6] ).50, H(Y ) = H( 1 [6] 9, 1 ) Coppermth and Wnograd [CW], followed by Wllam [Wl], avod th ue by retrctng themelve to dtrbuton π for whch τ = π. That, ther dtrbuton are pecfed only by the margnal. A we remarked, n the former cae, th doen t matter. However, n the latter cae, t could matter. Indeed, the N th power of the algorthm contan all non-negatve trple (a, b, c) wth a + b + c = 2N. Th property follow from the fact that t hold for the bac algorthm.

4 1.1 Proof of Lemma 1 Before delvng nto the proof telf, we need ome auxlary reult. Lemma 2. Let π be a partton of n 2 nto k part. Then there a contant C k uch that ( n π) n C k 2 H(π/n)n nc k. Proof. Wthout lo of generalty, all part n π are non-empty. Accordng to Wkpeda, there ext contant K 1, K 2 uch that n! K 1 n n+1/2 e n K 2. Therefore ( n K 1 K2 k π) A K 2 K1 k, where A = nn+1/2 e n p π pp+1/2 e p n = p π p (n/p) p p π n =. p π p2h(π/n)n We can etmate Therefore K 1 K2 k n 1 k n (1 k)/2 A 2 H(π/n)n n. ( n π) 2 H(π/n)n K 2 K1 k n 1/2. Lemma. Let π be a fnte probablty dtrbuton. There ext a contant C uch that for all n, there a fnte probablty dtrbuton σ uch that nσ ntegral, σ(t) π(t) 1/n and H(π) H(σ) C/n. Proof. Let S be the upport of π. Conder σ L and σ H defned by σ L (t) = nπ(t) /n and σ H (t) = nπ(t) /n. Clearly σ L 1 whle σ H 1. It not hard to conclude that there a probablty dtrbuton σ uch that nσ ntegral and σ(t) π(t) 1/n. Now H(π)/ π(t) = log e(π(t)+1). In partcular, H(π) 1 = C contant. It follow that H(π) H(σ) C/n. We dvde the proof nto two part: upper bound and lower bound. 4

5 Upper bound Fx N 1, let X 1, X 2, X be good ubet of T N, and let F = T N X 1 X 2 X. Let D denote the et of all ource dtrbuton, that the et of all functon d: T N that um to N. Note D = poly(n). For each d D, let F d F cont of thoe element conformng to the dtrbuton d. Alo, let d denote the projecton dtrbuton on the th component. Snce the projecton are njectve on F, ung Lemma 2 we get ( ) F d mn N d mn 2H(d /N)N poly(n). Therefore φ(f ) = d D φ(t) d(t) F d t T poly(n) max d D t T φ(t) d(t) mn {1,2,} 2H(d /N)N. Now let be the et of all probablty dtrbuton on T. For any d D, d/n, and o Takng Nth root, we deduce that cap N (T, φ) poly(n) max π cap(t, φ) max π t T t T φ(t) π(t) φ(t) π(t)n mn {1,2,} 2H(p )N. mn {1,2,} 2H(π )N. Gven a dtrbuton π, let S(π) denote t ymmetrzaton. Note that S(π) = (π 1 +π 2 +π )/. Snce the entropy functon concave, H(S(π) 1 ) H(π 1) + H(π 2 ) + H(π ) On the other hand, nce φ permutaton-nvarant, φ(t) S(π)(t). t T φ(t) π(t) = t T mn(h(π 1 ), H(π 2 ), H(π )). Therefore the maxmum obtaned at ome ymmetrc dtrbuton. Lower bound Th the dffcult part of the proof. We wll ue Salem-Spencer et, whch are ubet of the nteger wthout three-term arthmetc progreon. Salem and Spencer [SS] howed contructvely that for all M, there ext uch ubet of {1,..., M} of ze M 1 o(1). Ther contructon wa mproved by Behrend [Beh], Moer [Mo] and Elkn [Elk]. Behrend and Elkn proof are randomzed, whle Moer proof contructve. For u, the orgnal contructon uffce. In fact, we wll employ Salem-Spencer et over the group Z M, for M odd. We can contruct uch et by takng Salem-Spencer ubet of {0,..., (M 1)/2}. Such et tll have ze M 1 o(1). Wthout lo of generalty, aume that all trple n T um to zero. Let π be any dtrbuton on T. Fx N 1, and let F be the ubet of T N correpondng to ource dtrbuton Nσ, where σ gven by Lemma. 5

6 Let M be an odd prme to be determned later. Let h: Z n M Z M be a random lnear functon, and chooe x 1, x 2, x Z M randomly under the contrant x 1 + x 2 + x = 0. Fnally, for x T N defne h (x) = h(α (x)) + x. Our contructon guarantee that f α (x) α (y) and j then (h (x), h (y), h j (x)) a unformly random element n Z p (nce M prme). Fnally, defne h 1 (x) = 2h 1 (x), h 2 (x) = 2h 2(x) and h (x) = h (x); the reaon behnd th werd defnton wll become apparent later on. Snce M odd, (h (x), h (y), h j (x)) alo a unformly random element. Let A be ome Salem-Spencer et for Z M of ze M 1 o(1). Defne et Y 1, Y 2, Y a follow: By contructon, for every x T N, Y = {x α (F ) : h (x) A}. h 1 (x) + h 2 (x) + h (x) = h(α 1 (x) + α 2 (x) + α (x)) = 0. Therefore h 1 (x) + h 2 (x) = 2h (x). Snce A a Salem-Spencer et, we deduce that h 1 (x) = h 2 (x) = h (x). Converely, f h 1 (x) = h 2 (x) then 2h 1 (x) = 2h (x) and o h 1 (x) = h (x). Th a property atfed by every element of T n Y 1 Y 2 Y. Th et therefore partton nto et (V a ) a A, where V a contan thoe member uch that h 1 (x) = h 2 (x) = h (x) = a. We contruct the et X out of the et Y eparately for each a. Gven a A, form a graph whoe vertex et V a, and two vertce are connected by an edge f they conflct, that, they hare one of the three coordnate. Denotng the et of edge by U a, there are at leat V a U a connected component. Chooe a repreentatve from each connected component. The et X contan all th coordnate of repreentatve. The graph wa contructed to guarantee that X 1, X 2, X good, and the number of trple a V a U a. Th contructon work for mple example, n whch F = T N α 1 (F ) α 2 (F ) α (F ); however, th not alway the cae. To refne th contructon, let V a = V a F, and let U a U a cont of thoe conflct touchng V a. When choong repreentatve, chooe a repreentatve from F poble. There are at leat V a U a connected component contanng trple from F (proof: frt add the edge n U a \U a), and o the reultng number of trple from F at leat a V a U a. Let u etmate now the ze of the et V a and U a. We need to defne a et G related to F : G = {(x 1, x 2, x ) : x α (F )}. We alo defne P = α 1 (F ). Each element n x F belong to V a wth probablty 1/M 2, nce h 1 (x), h 2 (x) unform over Z2 M, and h 1 (x) = h 2 (x) mple h 1 (x) = h 2 (x) = h (x). Next, we count the number of potental conflct nvolvng F. For each x α (F ), there are G /P trple havng x a ther th coordnate. Hence the number of conflct nvolvng F bounded by F G /P. Each conflct (x, y) belong to U a wth probablty 1/M, nce (aumng t a 1-conflct) h 1 (x), h 2 (x), h 2 (y) unform over Z M (nce α 1(x) = α 1 (y) and x y mple α 2 (x) α 2 (y)), and o h 1 (x) = h 2 (x) = h 2 (y) = a happen wth probablty 1/M. Hence the expected value of V a U a F F G M 2 P M. Choong M to be a prme of ze roughly 6 G /P, the reultng number of trple at leat M 1 o(1) F o(1) F = M 2M 2 2 G P. 6

7 Ung Lemma 2 and argument mlar to Lemma, we etmate P 2 H(σ 1)N 2 H(π 1)N. Smlarly, F 2 H(π)N and G 2 H(τ)N, where τ maxmze H(τ) under the contrant τ = π for {1, 2, }; the optmum permutaton-nvarant by the concavty of H. Note we etmate G by the larget contrbuton to t nce there are only polynomally many ource dtrbuton. Puttng everythng together, we get the tatement of the lemma. Remark An alternatve contructon, ung Straen orgnal approach and avodng Salem- Spencer et, appear n 15.7 of [ACT]. We dd take Straen connected component argument, whch replace a le elegant one due to Coppermth and Wnograd. Open problem Whch bound correct, the lower bound or the upper bound? 2 Algorthm 2 The dentty we tarted from can be lghtly tweaked to yeld even more: [ ] ( ) ɛ x [0] 0 y[1] z [1] + x [1] y [0] 0 z[1] + x [1] y [1] z [0] 0 + x [0] 0 y[0] 0 z[2] q+1 + x[0] 0 y[2] q+1 z[0] 0 + x[2] q+1 y[0] 0 z[0] 0 ɛ ( (x [0] 0 + ɛx[1] )(y [0] 0 + ɛy[1] )(z [0] 0 + ɛz[1] ) x [0] 0 + ɛ2 x [1] ) ( y [0] 0 + ɛ2 y [1] ) ( z [0] 0 + ɛ2 (1 qɛ)(x [0] 0 + ɛ x [2] q+1 )(y[0] 0 + ɛ y [2] q+1 )(z[0] 0 + ɛ z [2] q+1 ). z [1] ) + + O(ɛ 4 ) = The new dentty ha three more varable x [2] q+1, y[2] q+1, z[2] q+1 and three more term n the left-hand de 1, 1, 1 0,0,2 + 1, 1, 1 0,2,0 + 1, 1, 1 2,0,0. All th at the cot of no new term n the rght-hand de! We follow the analy of Algorthm 1, only now we have one more degree of freedom. Suppoe p gve weght α/ to each of the term of volume q, and weght β/ to each of the term of volume 1 (o α+β = 1). The projecton r gve probablte (α+2β)/, 2α/, β/ to 0, 1, 2, correpondngly. Followng our earler reaonng, Lemma 1 together wth the aymptotc um nequalty yeld ω log q α q H((α+2β)/,2α/,β/). Th tme the optmzaton more dffcult, but can be done numercally (or wth Lagrange multpler), gvng q = 6, α 0.952, β and ω Remark The tranton from Algorthm 1 to Algorthm 2 nvolve addng three new varable. In exactly the ame way, we could add even more varable, addng for example the term x [0] 0 y[0] 0 z[] q+2. However, th doe not ncreae the capacty. Indeed, uppoe the et X 1 contaned two member a, b dfferng only n z q+1 v. z q+2. Every trple n X 1 X 2 X n whch the frt coordnate a ha a doppelgänger n whch the frt coordnate b, o X 1, X 2, X n t good. 7

8 Algorthm Algorthm ue exactly the ame dentty a Algorthm 2, only quared. There an added complcaton, nce not all term are matrx product tenor. We wll ee how to handle th oon, but frt let ee how the dentty quared look lke. If we follow the recpe gven o far, the ndce wll be vector of length 2. Intead, we wll take the um of the two component (Coppermth and Wnograd call th couplng). Th enure that the x-, y- and z-ndce have a contant um..1 Squared dentty 1. The dentty quared ha (q + 2) 2 term on the rght-hand de, and 15 term on the left-hand de, whch break up a follow: (a) term mlar to 1, 1, 1 0,0,4, comng from (b) 6 term mlar to 1, 1, 2q 0,1,, comng from 1, 1, 1 0,0,2 1, 1, 1 0,0,2. 1, 1, q 0,1,1 1, 1, 1 0,0,2 1, 1, 1 0,0,2 1, 1, q 0,1,1. (c) term mlar to 1, 1, q ,2,2, comng from 1, 1, 1 0,2,0 1, 1, 1 0,0,2 1, 1, 1 0,0,2 1, 1, 1 0,2,0 1, 1, q 0,1,1 1, 1, q 0,1,1. (d) term mlar to T 1,1,2 4, comng from 1, q, 1 1,1,0 1, 1, 1 0,0,2 + 1, 1, 1 0,0,2 1, q, 1 1,1,0 + q, 1, 1 1,0,1 1, 1, q 0,1,1 + 1, 1, q 0,1,1 q, 1, 1 1,0,1. The fourth tenor, T 4, unfortunately not a matrx product tenor n telf, but t can be ued to produce matrx product tenor, n ome ene that we wll make prece. We wll have to generalze the aymptotc um nequalty from (n m p ) τ R ( n, m, p ) ω τ (1) to a more general form V τ (T ) R ( ) T ω τ. (2).2 The value of a tenor We frt preent the defnton of V τ, and then gve ome ntuton. The proof of (2) gven n the ubecton below. The operator ρ operate on tenor by rotatng the varable x, y, z to y, z, x. For example, f T = n, m, p = n m p x j y jk z k j=1 k=1 8

9 then ρ(t ), whch obtaned by replacng x y, y z, z x, ρ(t ) = n m p p n m x k y j z jk = x k y jz kj = p, n, m. j=1 k=1 k=1 j=1 Here x k = x k and z kj = z jk denote tranpoton. Next, we defne ymmetrzaton: S(T ) = T ρ(t ) ρ 2 (T ). So S( n, m, p ) = nmp, nmp, nmp. Symmetrzaton what allowed u to prove ω log nmp R( n, m, p ). The Nth tenor power of a tenor T T N = T T (N factor). We ay that a tenor T break up nto T = n, m, p f we can zero ome varable to get T. Th the ame proce we ued n Algorthm 1 and 2. We defne { V τ,n (T ) = max (n m p ) τ : T break up nto } n, m, p. For example, V τ,n ( n, m, p ) = (nmp) nτ. Fnally, V τ (T ) = up V τ,n (T ) 1/N. N So V τ ( n, m, p ) = (nmp) τ. The dea behnd th defnton that t mmedately mple that Indeed, uppoe that V τ,n (T ) 1/N R(T ). So for ome n, m, p, V τ (T ) = R(T ) ω τ. () V τ,n (T ) = (n m p ) τ, and S(T ) N reduce to T = n, m, p. The latter mple that R(T ) R(S(T ) N ) R(T ) N. The aymptotc um nequalty (1) tate that (n m p ) τ R(T ) ω τ. Snce V τ,n (T ) R(T ) N R(T ), the condton roughly hold, hence ω τ. 4 Calculatng the value It tme for u to take a cloer look at T 4. Wth the orgnal ndce, t look lke th: T 4 = 1, q, 1 10,10,02 + 1, q, 1 01,01,20 + q, 1, q 10,01,11 + q, 1, q 01,10,11. 9

10 Let rename the ndce to make thng clearer: T 4 = 1, q, 1,, + 1, q, 1 4,4,4 + q, 1, q,4,5 + q, 1, q 4,,5. The tenor T 4 not ymmetrc. Before ymmetrzng t, we take the Nth power, and only conder term wth (α/2)n, (α/2)n, (β/2)n, (β/2)n factor of each of the bac type, where α + β = 1. Each uch term ha volume q (α+2β)n. The projecton nto the x-ndex or the y-ndex ha N/2 coordnate equal to and N/2 coordnate equal to 4. The projecton nto the z-ndex ha (α/2)n coordnate equal to, (α/2)n coordnate equal to 4, and βn coordnate equal to 5. If we now ymmetrze, the volume of each bac term q (α+2β)n, and the projecton nto each coordnate ha ze ( ) N 2 ( N/2 α 2 N, α 2 N N, βn Ung Lemma 1, we get that the value at leat roughly the Nth root of ( ) N 2 ( N/2 Takng the Nth root, we get α 2 N, α 2 N N, βn ). ) q (α+2β)nτ 2 2N 2 H(α/2,α/2,β)N q (α+2β)nτ. V τ (T 4 ) 4 2 H(α/2,α/2,β) q (α+6β)τ = 4(α/2) α β β q (+β)τ. We can fnd the optmal value of β by ubttutng α = 1 β, dfferentatng the rght-hand de wth repect to β, and equatng to zero. We fnd that the optmal value of α, β are α = 2 q τ + 2, β = qτ q τ + 2. Subttutng th back, we get V τ (T 4 ) 4 1/ q τ (2 + q τ ) 1/. 4.1 Analyzng the algorthm Havng calculated the value of T 4, t tme to go back to the man plan. Prevouly we have optmzed the exact fracton to take of each of the bac term. However we have 15 of thee. In the end, we re only really ntereted n the ze of the projecton nto each coordnate (aumng the projecton ha the ame ze for all coordnate). Snce there are only 5 value of each coordnate, th reduce the number of degree of freedom to 4. In fact, nce n the dentty, the three ndce alway um to 4, the average of all component of the x-ndex hould be 4/. Th take away one more degree of freedom. Suppoe α + β + γ + δ = 1. Take term n whch each term of the frt knd appear (α/)n tme, each term of the econd type appear (β/6)n tme, each term of the thrd type appear (γ/)n tme, and each term of the fourth type appear (δ/)n tme. The normalzed htogram of each coordnate wll be 2α + β + γ : β + 2δ : 2γ + δ : β : α. 10

11 The aymptotc um nequalty equate (q + 2) 2 wth the entropy of th dtrbuton multpled by the value, whch V τ (T 1 ) α V τ (T 2 ) β V τ (T ) γ V τ (T 4 ) δ. We don t need to explctly compute the value of T 1, T 2, T nce they are matrx product tenor. If we optmze over q, α, β, γ, δ, we get that ω Proof* The generalzed form of the aymptotc um nequalty (2) mplctly ued n Coppermth and Wnograd orgnal paper [CW]. It tated explctly n Wllam work [Wl], but not proved there. The treatment below taken from Stother the [Sto], who explctly prove t (we lghtly modfy h notaton). Lemma 4. For every two tenor T 1, T 2, A a conequence, for every tenor T, Proof. Exerce. Lemma 5. For every tenor T, V τ,n (T 1 T 2 ) V τ,n (T 1 )V τ,n (T 2 ). V τ,cn (T ) V τ,n (T ) c. V τ (T ) = lm N V τ,n(t ) 1/N. Proof. We defned V τ (T ) a the upremum of V τ,n (T ) 1/N. Pck any ɛ > 0. For ome N, V τ,n (T ) 1/N V τ (T ) ɛ. Let M = cn + r, where c 1 and 0 r < N. Snce Lemma 4 how that c M = 1 N r MN > 1 N 1 M, V τ,m (T ) 1/M V τ,cn (T ) 1/M V τ,n (T ) c/m (V τ (T ) ɛ)v τ (T ) 1/M. If M large enough, V τ,m (T ) 1/M V τ (T ) 2ɛ. Corollary 6. For every two tenor T 1, T 2, V τ (T 1 T 2 ) V τ (T 1 )V τ (T 2 ). Lemma 7. For every two tenor T 1, T 2, V τ (T 1 T 2 ) V τ (T 1 ) + V τ (T 2 ). 11

12 Proof. Lemma 5 mple that for large N, V τ,n (T 1 ) V τ (T 1 ) N, V τ,n (T 2 ) V τ (T 2 ) N. The Nth tenor power S(T 1 T 2 ) N a djont um of varou term, ncludng S(T 1 ) N 1 S(T 2 ) N 2 for all N 1, N 2 atfyng N 1 + N 2 = N. Hence the defnton of value mple that If N 1 and N 2 are both large, then V τ,n (T 1 T 2 ) V τ,n1 (T 1 )V τ,n2 (T 2 ). V τ,n1 (T 1 )V τ,n2 (T 2 ) V τ (T 1 ) N 1 V τ (T 2 ) N 2. The rght-hand de maxmzed when N 1 /N 2 V τ (T 1 )/V τ (T 2 ), n whch cae V τ (T 1 ) N 1 V τ (T 2 ) N 2 (V τ (T 1 ) + V τ (T 2 )) N N + 1 When N large, both of N 1, N 2 are large, and o V 1/N τ,n (T 1 T 2 ) V τ (T 1 ) + V τ (T 2 ). Corollary 8. The generalzed aymptotc um nequalty (2) hold. 5 General formulaton* The etmate of ω obtaned ung Algorthm a two-tered proce. The frt tep to calculate everal value, and the econd tep to combne thee value and get an etmate of ω. Each tep requre combnng Lemma 1 wth the aymptotc um nequalty. We can phrae the method n the followng lemma. Lemma 9. Let T be a tenor whch a um of tenor {T : I}, where T ha ndce. Gven a probablty dtrbuton p on I, let r 1, r 2, r be the reultng projecton on the three coordnate. Then log 2 V τ (T ) H(r 1) + H(r 2 ) + H(r ) + E p [log 2 V τ (T )]. Proof ketch. Suppoe frt that the tenor T are matrx multplcaton tenor. Ung the method of proof of Lemma 1, we can demontrate the extence of ubet T N S(S) N of ze very roughly 1 N log 2 T N H(r 1) + H(r 2 ) + H(r ). Each ubet T N correpond to a djont um of matrx multplcaton tenor n, m, p uch that. (nmp) τ = V τ (T ) p()n. 12

13 The etmate of V τ (T ) now follow from the formula for V τ,n (T ). For the general cae, the reader can how that the defnton of V τ reman vald f we replace the orgnal defnton of V τ,n wth { V τ,n (T ) = max V τ (T ) : T break up nto } T. S In order to apply the lemma, we need to ue the bae cae tatng that V τ ( n, m, p ) = (nmp) τ. In order to ue the lemma to obtan a bound on ω, we ue the aymptotc um nequalty V ω (T ) R(T ). A an example, conder the tenor T 4 encountered n Algorthm. The ndex et and known value are gven by V τ (T,, ) = V τ (T 4,4,4 ) = q τ, V τ (T,4,5 ) = V τ (T 4,,5 ) = q 2τ. Convexty mple that n order to maxmze the entropy, we need the probablty p to depend only on the value (n th cae, only on the volume of the correpondng matrx multplcaton tenor), hence we gve each of the frt two trple probablty α/2, and each of the latter two trple probablty β/2 (where α + β = 1). We have H(r 1 ) = H(r 2 ) = H(1/2, 1/2) = 1, H(r ) = H(α/2, α/2, β). Regardng volume, we have E p [log 2 V τ (T )] = τ(α + 2β) log 2 q. The lemma now mple log 2 V τ (T 4 ) 2 + H(α/2, α/2, β) + τ(α + 2β) log 2 q. 6 Straen veron* Straen tart wth the dentty ɛ (x [1] y [0] 0 z[1] + x [0] 0 y[1] z [1] ) + O(ɛ 2 ) = (x [0] 0 + ɛx[1] )(y [0] 0 + ɛy[1] )z x [0] 0 y[0] 0 z. On the left, we have T = q, 1, 1 1,0,1 + 1, 1, q 0,1,1. Straen n effect calculate V τ (T ). Rae the dentty to the Nth power. The projecton have ze 2 N, 2 N, 1, o after ymmetrzng all the projecton have ze 4 N. All the ndvdual term have volume q N. Lemma 1 (whch Straen prove wth an explct contructon) how that V τ (T ) 4 1/ q τ. 1

14 Therefore, the aymptotc um nequalty how that ω τ where 4 1/ q τ = q + 1. For q = 5, th gve ω One dfference between Straen account and the later one the technque ued to eparate a tenor nto djont component. Coppermth and Wnograd do th by zerong out ome of the varable. Straen gve a dfferent ɛ-weght to each nput varable, and then obtan an approxmate equaton (an equaton of the type ued to defne border rank). H method obvate the need for Salem-Spencer et. Queton: The Coppermth-Wnograd contructon can be ealy emulated ung a Straen contructon. I the other drecton alo true? Reference [ACT] Peter Bürger, Mchael Clauen and M. Amn Shokrollah, Algebrac Complexty Theory, Sprnger, [Beh] Felx Behrend, On et of nteger whch contan no three term n arthmetc progreon, Proc. Nat. Acad. Sc. 2:1 2, [Elk] Mchael Elkn, An mproved contructon of progreon-free et, Irael J. Math. 184:9 128, [CKSU] Henry Cohn, Robert Klenberg, Baláz Szegedy and Chr Uman, Group-theoretc algorthm for matrx multplcaton, FOCS [CW] Dan Coppermth and Shmuel Wnograd, Matrx multplcaton va arthmetc progreon, J. Symb.Comput. 9(): , [Mo] Leo Moer, On non-averagng et of nteger, Canad. J. Math. 5:245 25, 195. [SS] Raphaël Salem and Donald Spencer, On et of nteger whch contan no three n arthmetc progreon, Proc. Nat. Acad. Sc. 28:561 56, [Sto] Andrew Jame Stother, On the complexty of matrx multplcaton, Ph.D. the (U. of Ednburgh), [Wl] Vrgna Valevka-Wllam, Breakng the Coppermth-Wnograd barrer, STOC