Inner Rank and Lower Bounds for Matrix Multiplication

Transcription

1 Inner Rank and Lower Bounds for Matrix Multiplication Joel Friedman University of British Columbia jf Jerusalem June 19, 2017 Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

2 Outline We assume only linear algebra and tensor product. Intro to Strassen s classical [Str69] algorithm. Intro to rank of matrix multiplication tensors. Intro to border rank (infinitesimal, Zariski, norm). Inner rank and the optimality of Strassen s classical algorithm wrt rank and border rank over an arbitrary field. Easy generalization to n n case. No new bounds over C, but very short proofs over any field, and promising future directions. Refs: Markus Blaser survey: [Blä13]; [BCS97] Chs ; many interesting recent developments cited in [Fri17] 1, including [Blä99, Blä03, Lan14, MR14, LO15, CZ16, LM17]. 1 The current bibliographical remarks need updating; check my website in the near future and/or the arxiv posting... Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

3 Strassen s (Classical) Algorithm A, B F n n, n n matrices, entries in F, a field (sometimes a ring). Can compute AB with n 3 operations: (AB) ik = j A ijb jk. Strassen [Str69] gives an algorithm that takes O(n ω ) operations, where ω = log 2 7 < 3. For n = 2, Strassen [Str69] writes all four entries of AB as Z-linear combination of some 7 bilinear forms over Z: ( a11 + a 22 )( b11 + b 22 ), ( a21 + a 22 )( b11 ),..., ( a12 a 22 )( b21 + b 22 ). By using block matrices and recursion, one gets O(n log 2 7 ) as above. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

4 Strassen s (Classical) Algorithm Improve O(n log 2 7 ), theoretically and/or in practice? The O(n 3 ) algorithm better for small n (e.g., n = 2 k + 1, k small). We use tensor rank complexity, i.e., the rank of associated 3-tensors (see next slide). There are other complexity measures, and practical considerations (simplicity of algorithm, memory management, etc.). Often assume F = C. Strassen s algorithm works over Z. Main questions: Complexity for large n. Complexity for very small n, for use in practical algorithms. Similar questions for non-square matrices, and all sizes. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

5 Tensor Rank F denotes a field, arbitrary unless otherwise specified. means tensor product F with F understood. Vector spaces are finite dimensional. F n 1 n 2 denotes the set of n 1 n 2 matrices over F; e ij is standard basis vector there; n 1, n 2, n 3 def = e ij e jk e ki F n 1 n 2 F n 2 n 3 F n 3 n 1. i [n 1 ], j [n 2 ], k [n 3 ] For τ A B C, R(τ) is smallest integer r for which τ = r ρ=1 α ρ β ρ γ ρ for some α ρ A, β ρ B, γ ρ C. Strassen s classical algorithm implies that R( n, n, n ) O(n log 2 7 ). Presently one knows that for large n and ω = , 3n 2 + o(n 2 ) R( n, n, n ) O(n ω ). Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

6 Reduction n 1, n 2, n 3 def = ijk e ij e jk e ki over i [n 1 ], j [n 2 ], k [n 3 ]. For τ A B C, R(τ) is smallest integer r for which τ = r ρ=1 α ρ β ρ γ ρ for some α ρ A, β ρ B, γ ρ C. Say that τ reduces to τ, τ τ if τ = (M N L)τ for some (arbitrary) linear maps M: A A, N : B B, L: C C. τ τ implies R(τ ) R(τ). Much like reduction in complexity theory. E.g., 0 τ for all τ. If M, N, L are each injections, then they each have a left inverse, and then M N L has a left inverse, and hence M N L does not change the rank. In particular, if M, N, L are each isomorphisms, then M N L does not change the rank. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

7 Border Rank: Infinitesimal Strassen: R( 2, 2, 2 ) 7 implies n, n, n O(n log 2 7 ). For any m: R( m, m, m ) m ω implies R( n, n, n ) O(n ω ). Also R(τ h ) m ω implies the same for any deformation τ ɛ = m, m, m + O(ɛ) (think of R or C). Therefore: R inf (τ) def = smallest integer r for which R(ɛ h τ + O(ɛ h+1 )) r for some h and family (via the base extension functor F F[ɛ]). To upper bound the exponent of matrix multiplication, R inf ( n, n, n ) [infinitesimal border rank] works just as well as R( n, n, n ). This idea has been applied to matrix multiplication to improve upper bounds, such as [BiniCapoRomaLott79] and many others. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

8 Border Rank: Zariski Zariski topology: the coarsest topology s.t. all polynomials are continuous; assume F algebraically closed or sufficiently large. Warning: the Zariski closure of {(n, 2 n, 2 2n )} n=1,2,... in C 3 is the entire space. R(τ) def = R Zar (τ) def = smallest integer r for which τ lies in the Zariski closure of the set of rank-r tensors. Easy to see R inf (τ) R Zariski (τ); [I believe] known to be equal over an algebrically closed field[adl83] (see Appendix A of [Fri17], [Str83]). Easy to see R inf R norm R Zar over R, C or, more genearally, any local field. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

9 Border Rank: More Remarks The rank of a 2-tensor, i.e., element of A B Hom(A, B) Hom(B, A) etc. is expressed by (non-)vanishing of polynomials; hence the rank is Zariski semicontinuous for 2-tensors. The rank of a 3-tensor (or 4-tensor, etc.) is not Zariski semicontinuous (see [Blä13]) and is NP-complete [Hs90]. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

10 Inner Rank: Summary Short proof of Hopcroft-Kerr [HK71] and Winograd [Win71]: R( 2, 2, 2 ) F 7. Short proof of Landsberg [Lan06]: R( 2, 2, 2 ) C 7; our proof valid for any field. Immediate generalization: R( n, n, n ) F 2n 2 n + 1. Curiously this is the exact bound of Landsberg-Michalek [LM17] over C. Coincidence? Equivalent proofs? Duality? More recently, R( n, n, n ) C 2n 2 log 2 n + 1 [LM16]. Currently best: R( n, n, n ) C 3n 2 + o(n 2 ) [Lan14, MR14]. We argue that inner rank has the potential to aid in improved bounds, both in R and R. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

11 Inner Rank and R( 2, 2, 2 ) > 6 If n, n, n = r ρ=1 α ρ β ρ γ ρ then n 3 r ρ=1 Rank n n(γ ρ ). Not well-known but known to (independently observed by) a number of experts [Mic, CZ16] and also me; perhaps others. Tensor rank invariant under isomorphism(s) applied to any factor. Any five nonzero vectors that span F 2 2 can be taken by an isomorphism to rank 1 matrices. [Easy] Proof: Combine the above. Intuitively, this amounts to twisting the rank function, or defining a variant of the usual rank function. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

12 More Details on R( 2, 2, 2 ) Let π be the canonical map ( F n n ) 3 ( F n n ) 2 F n F n Hom( (F n n F n), F n n F n) π n, n, n is the map e ij e k e jk e i. Hence its rank is n 3. Assume 2, 2, 2 = 6 ρ=1 α ρ β ρ γ ρ. We may assume γ 1,..., γ 5 are nonzero and span all of F 2 2. After scaling and rearranging, can assume that γ 1,..., γ 4 span, and γ 5 = γ γ s for 1 s 4. If s = 1, 2, 4, define L as taking γ 1, γ 2, γ 3, γ 4 to e 11, e 12, e 21, e 22 resp.; if s = 3, same L except Lγ 3 = e 21 + e 22. Hence there exists L such that 5 ρ=1 Rank 2 2 Lγ ρ is 5. Hence sum to ρ = 6 at most 7. But 8 = 2 3 = R( 2, 2, 2 ) r ρ=1 Rank 2 2 Lγ ρ. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

13 Inner Rank and R( 2, 2, 2 ) > 6 Assume F is algebraically closed. Cover all tensors 6 ρ=1 α ρ β ρ γ ρ ( F 2 2) 3 by two sets: C 4 : those for which the {γ ρ } span all of F 2 2 ; C 3 : their span is of dimension at most 3. Also, let B 3 be similarly defined wrt {β ρ }. τ C 4 implies πτ = M M, where M = π(i I L) 1 π 1 and M = π(i I L)τ is of rank 7. Hence πτ Rank 7. τ B 3 implies dim Image(πτ) 6 so πτ Rank 6. By symmetry, same for τ C 3, with slightly different π. Hence 2, 2, 2 is outside of the Zariski closures of C 4 and C 3 (π and π are Zariski continuous). Immediate generalization: R( n, n, n ) 2n 2 n + 1. Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

14 Hope for Improvements and Further Study A number of suggestions in [Fri17]. Example: consider n 1, n 2, n 3 = r ρ=1 α ρ β ρ γ ρ, and τ = (M N L) n1, n 2, n 3 for any M, N, L. One can play with: Choosing M to have some αρ in its kernel; choosing N to have some β ρ in its kernel; hence drop any such ρ from the inner rank bound: Rank(τ) Rank Inner Lγ ρ ; ρ supp(m, N ) must not decrease Rank(τ) too much when M, N have kernels; similar games have been played prior to inner rank. One is looking for additional various ways to twist standard maps to get better lower bounds. Consider the next slide... Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

15 Strassen s Equations, Following Landsberg-Ottoviani Any M and map φ: U V trivially gives φ Id M : U M V M. Landsberg-Ottoviani [LO15]: let A = B = C = F n n ; n, n, n gives map B A C, and hence B M A C M for any M. Now set M = Λ p A, and twist via A C Λ p A canonical Λ p+1 A C. p = 1: Strassen s equations [Str83]; [LO15] use p > 1. A generalized form of inner rank takes n, n, n in A B C to B C 1 A C 2 using a map C C 1 C 2. What about tensoring and twisting here? Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

16 References I A. Adler, Dissertation, 1983, Universität Zürich. Peter Bürgisser, Michael Clausen, and M. Amin Shokrollahi, Algebraic complexity theory, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 315, Springer-Verlag, Berlin, 1997, With the collaboration of Thomas Lickteig. MR Markus Bläser, A 5 2 n2 -lower bound for the rank of n n-matrix multiplication over arbitrary fields, 40th Annual Symposium on Foundations of Computer Science (New York, 1999), IEEE Computer Soc., Los Alamitos, CA, 1999, pp MR , On the complexity of the multiplication of matrices of small formats, J. Complexity 19 (2003), no. 1, MR , Fast matrix multiplication, Theory of Computing, Graduate Surveys 5 (2013), Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

17 References II Matthias Christandl and Jeroen Zuiddam, Tensor surgery and tensor rank, Available at: Joel Friedman, Inner rank and lower bounds for matrix multiplication, Available at: J. E. Hopcroft and L. R. Kerr, On minimizing the number of multiplications necessary for matrix multiplication, SIAM J. Appl. Math. 20 (1971), MR Johan Hå stad, Tensor rank is NP-complete, J. Algorithms 11 (1990), no. 4, MR J. M. Landsberg, The border rank of the multiplication of 2 2 matrices is seven, J. Amer. Math. Soc. 19 (2006), no. 2, MR , New lower bounds for the rank of matrix multiplication, SIAM J. Comput. 43 (2014), no. 1, MR Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

18 References III J. M. Landsberg and Mateusz Micha lek, A $2nˆ2-log(n)-1$ lower bound for the border rank of matrix multiplication, CoRR abs/ (2016)., On the Geometry of Border Rank Decompositions for Matrix Multiplication and Other Tensors with Symmetry, SIAM J. Appl. Algebra Geom. 1 (2017), no. 1, MR Joseph M. Landsberg and Giorgio Ottaviani, New lower bounds for the border rank of matrix multiplication, Theory Comput. 11 (2015), MR Mateusz Michalek, Personal Communication. Alex Massarenti and Emanuele Raviolo, Corrigendum to The rank of n n matrix multiplication is at least 3n 2 2 2n 3 2 3n [Linear Algebra Appl. 438 (11) (2013) ] [mr ], Linear Algebra Appl. 445 (2014), MR Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19

19 References IV Volker Strassen, Gaussian elimination is not optimal, Numer. Math. 13 (1969), MR , Rank and optimal computation of generic tensors, Linear Algebra Appl. 52/53 (1983), MR S. Winograd, On multiplication of 2 2 matrices, Linear Algebra Appl. 4 (1971), Joel Friedman (UBC) Inner Rank and Lower Bounds June 19, / 19