I. Approaches to bounding the exponent of matrix multiplication

Size: px

Start display at page:

Download "I. Approaches to bounding the exponent of matrix multiplication"

Alexander Barnett
6 years ago
Views:

1 I. Approaches to bounding the exponent of matrix multiplication Chris Umans Caltech Based on joint work with Noga Alon, Henry Cohn, Bobby Kleinberg, Amir Shpilka, Balazs Szegedy Modern Applications of Representation Theory, IMA, Chicago July 204

2 Introduction A X B = C Standard method: O(n 3 ) operations Strassen (969): O(n 2.8 ) operations July 3, 204 2

3 Strassen s Algorithm a, a, 2 a 2, a 2, 2 b, b, 2 X = b 2, b 2, 2 c, c, 2 c 2, c 2, 2 linear combos of A, B entries 7 mults linear combos of results P = (a, - a,2 ) (b 2, + (bb 2, 2,2 + ) b 2,2 ) P 2 = (a, + a 2,2 ) (b, (b +, b 2,2 + ) b 2,2 ) c P, = P 3 = (a + P, - a 2 - P 2, ) 4 + P (b 6, + (bb,,2 ) + b,2 ) c P,2 = P 4 = (a 4 + P, + a 5,2 ) (b 2,2 )(b 2,2 ) c P 2, = P 5 = (a 6 + P, ) 7 (b,2 + b 2,2 )(b,2 + b 2,2 ) c P 2,2 = P 6 = (a 2 - P 2,2 ) 3 + P (b 5 - P 2, - b 7, )(b 2, - b, ) P 7 = (a 2, - a 2,2 ) (b, )(b, ) July 3, 204 3

4 Strassen s Algorithm a, a, 2 a 2, a 2, 2 b, b, 2 X = b 2, b 2, 2 c, c, 2 c 2, c 2, 2 linear combos of A, B entries 7 mults linear combos of results T(n) = # operations to multiply n x n matrices: T(n) 7T(n/2) + O(n 2 ) T(n) O(n log 2 7 ) July 3, 204 4

5 Introduction A X B = C Standard method: O(n 3 ) operations Strassen (969): O(n 2.8 ) operations The exponent of matrix multiplication: smallest number such that for all >0 O(n + ) operations suffice July 3, 204 5

6 History Standard algorithm 3 Strassen (969) < 2.8 Pan (978) < 2.79 Bini; Bini et al. (979) < 2.78 Schönhage (98) < 2.55 Pan; Romani; Coppersmith + Winograd (98-982) < 2.50 Strassen (987) < 2.48 Coppersmith + Winograd (987) < Stothers (200) < Williams (20) < Le Gall (204) < July 3, 204 6

7 Introduction Other important problems have same algorithmic complexity as matrix multiplication (via reductions) determinants LUP decompositions matrix inversion solving systems of linear equations basic graph problems July 3, 204 7

8 Outline. crash course on main ideas from Strassen 969 through Le Gall conjectures implying! = 2 Two more lectures: II. group-theoretic approach III. extending to coherent configurations July 3, 204 8

9 Bilinear algorithms & tensor rank Bilinear computation of complexity m: m products of form (L.C. of A) x (L.C. of B) result matrix entries = L.C. s of these products equivalent: rank of matrix multiplication tensor <n,n,n> is at most m Strassen: bilinear w.l.o.g.! = inf { : n x n matmult in O(n ) size} = inf { : rank(<n,n,n>) O(n )} July 3, 204 9

10 tensor: 3-D array of complex numbers Tensors rank tensor: (i,j,k) entry = x i y j z k rank r tensor: sum of r rank tensors X x 2 x 3 x 4 y y 2 y 3 y 4 equivalent: slices all L.C. s of r rank matrices July 3, 204 0

11 Tensors tensor: 3-D array of complex numbers tensor T with entries T i,j,k equivalent: trilinear form i,j,k T i,j,k X i Y j Z k July 3, 204

12 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a July 3, 204 2

13 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a July 3, 204 3

14 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a July 3, 204 4

15 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a July 3, 204 5

16 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a July 3, 204 6

17 The matrix multiplication tensor <n,m,p> is a nm mp pn tensor described by trilinear form i,j,k X i,j Y j,k Z k,i Each of np slices of <n,m,p>: n m A X m B = n C p p July 3, 204 7

18 Strategies for upper bounding the rank of the matrix multiplication tensor

19 Upper bounds on rank Observation: <n,n,n> i = <n i, n i, n i > ) R(<n i, n i, n i >) R(<n,n,n>) i Strategy I: bound rank for small n by hand R(<2,2,2>) = 7! < 2.8 R(<3,3,3>) 2 [9..23] (worse bound) even computer search infeasible July 3, 204 9

20 Strassen s example July 3,

21 Upper bounds on rank Border rank = rank of sequence of tensors approaching target tensor entrywise ² rank = 3 border rank = 2: ² - ² Strategy II: bound border rank for small n Lemma: R(<n,n,n>) < r )! < log n r Idea: t-th tensor power is degree O(t) polynomial in ²; interpolate to recover coefficient on ² 0 R(<2,2,3>) 0! < 2.79 July 3, 204 2

22 Upper bounds on rank Direct sum of tensors <n,n,n> <m,m,m> <n,n,n> <m,m,m> Asymptotic Sum Inequality and example (Schönhage 98) (multiple matrix multiplications in parallel) Strategy III: bound (border) rank of direct sums of small matrix multiplication tensors R(<n,n,n > <n k,n k,n k >) < r ) i n i! < r R(<4,,3> <,6,>) 3! < 2.55 July 3,

23 Upper bounds on rank Strategy IV: Strassen laser method tensor with course structure of MM and fine structure components isomorphic to MM (many independent MMs in high tensor powers) coarse structure fine = scalar x row vector <,2,> col vector x scalar July 3, q

24 Upper bounds on rank Strategy IV: Strassen laser method tensor with course structure of MM and fine structure components isomorphic to MM (many independent MMs in high tensor powers) q border rank = q + ; q = 5 yields! < 2.48 July 3,

25 Upper bounds on rank Strategy V : (C-W) border rank of this tensor is q+2: i= q X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 zero-out variables leaving many independent MMs in high tensor power (sophisticated proof) q = 6 yields! < 2.4 July 3,

26 Upper bounds on rank Strategy VI: (C-W) border rank of this tensor is q+2: 6 pieces : target proportions in high tensor power affect # and size of independent MMs optimize by hand i= q X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 + q = 6 yields! < X 0 Y 0 Z q+ + X 0 Y q+ Z 0 + X q+ Y 0 Z 0 July 3,

27 Upper bounds on rank Strategy VII: border rank of tensor powers of T: T = i= q X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 + X 0 Y 0 Z q+ + X 0 Y q+ Z 0 + X q+ Y 0 Z 0 Tensor power # pieces bound reference C-W Stothers Williams x 0^ Le Gall x 0^ Le Gall July 3,

28 Upper bounds on rank Strategy VII: border rank of tensor powers Tensor power # pieces bound reference C-W Stothers Williams x 0^ Le Gall x 0^ Le Gall Ambainis-Filmus 204: N-th tensor power cannot beat bound of July 3,

29 Conjectures implying! = 2

30 Asymptotic Rank conjecture [CW90] T = i=,2 X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 T slices border rank = 4 asymptotic rank of T = lim n! R(T n )/n conjecture: asymptotic rank of T = 3 July 3,

31 no 3 disjoint equivoluminous subsets conjecture [CW90] one way to achieve asymptotic rank 3: m, m 2,, m n 2 H (abelian group) for all disjoint S, T, U 2 [n] the sums i 2 S m i i 2 U m i i 2 T m i are not all equal conjecture: can take H 2 o(n) July 3, 204 3

32 Strong Uniquely Solvable Puzzle Conjecture [CKSU05] T rank = 4 T rank = 3 Strong Uniquely Solvable Puzzle: gives a way to zero-out variables leaving many independent MMs in high tensor power of T (instead of T) July 3,

33 Strong Uniquely Solvable Puzzle Conjecture [CKSU05] Uniquely Solvable Puzzle: every unintended way of assembling pieces has overlap of 2 or 3 in some cell Strong Uniquely Solvable Puzzle: every unintended way of assembling pieces has overlap of exactly 2 in some cell Ã w! N rows conjecture: Strong USPs exist with N = (w choose w/3) - o() rows

34 Two Families conjecture [CKSU05] subsets A, A 2,, A n, B, B 2,, B n of Abelian group H A A 2 A n. A i + B i = A i B i B all distinct 2. (A i + B i ) Å (A j + B k ) = ; if j k B 2 all distinct conjecture: can achieve n = H /2 o() A i = B i = H /2 o() B n all distinct July 3,

35 Status of conj s implying! = 2 T [CW90] slices asymptotic rank(t) = R(T n )/n! 3? no 3 disjoint equivoluminous subsets [CW90] strong uniquely solvable puzzle [CKSU05] ) two families [CKSU05] ) sunflower conj. # false sunflower conj. #2 false July 3,

36 Sunflower conjecture # Erdos-Rado conjecture: every collection of const s s-subsets contains a 3-sunflower s!2 s known conjecture widely believed 3-sunflower July 3,

37 Status of conj s implying! = 2 T [CW90] slices asymptotic rank(t) = R(T n )/n! 3? no 3 disjoint equivoluminous subsets [CW90] strong uniquely solvable puzzle [CKSU05] ) two families [CKSU05] ) sunflower conj. # false sunflower conj. #2 false July 3,

38 Sunflower conjecture #2 Every collection of 3 n(- const) vectors in Z 3 n contains u,v,w such that u + v + w = 0 50% chance of being false? Every 3-colored collection * of 3n(- const) vectors in Z n 3 contains u,v,w s.t. u+v+w=0 60% chance of being false? July 3,

39 Status of conj s implying! = 2 T [CW90] slices asymptotic rank(t) = R(T n )/n! 3? no 3 disjoint equivoluminous subsets [CW90] strong uniquely solvable puzzle [CKSU05] ) two families [CKSU05] ) sunflower conj. # false sunflower conj. #2 false July 3,

40 A different approach So far... bound border rank of small tensor (by hand) asymptotic bound from high tensor powers Disadvantages limited universe of starting tensors high tensor powers hard to analyze Next: matrix multiplication via groups July 3,

Approaches to bounding the exponent of matrix multiplication

Approaches to bounding the exponent of matrix multiplication Chris Umans Caltech Based on joint work with Noga Alon, Henry Cohn, Bobby Kleinberg, Amir Shpilka, Balazs Szegedy Simons Institute Sept. 7,