Approaches to bounding the exponent of matrix multiplication

Transcription

1 Approaches to bounding the exponent of matrix multiplication Chris Umans Caltech Based on joint work with Noga Alon, Henry Cohn, Bobby Kleinberg, Amir Shpilka, Balazs Szegedy Simons Institute Sept. 7, 204

2 Introduction A X B = C Standard method: O(n 3 ) operations Strassen (969): O(n 2.8 ) operations Sept. 7, 204 2

3 Introduction A X B = C Standard method: O(n 3 ) operations Strassen (969): O(n 2.8 ) operations The exponent of matrix multiplication: smallest number ω such that for all ε>0 O(n ω + ε ) operations suffice Sept. 7, 204 3

4 History Standard algorithm ω 3 Strassen (969) ω < 2.8 Pan (978) ω < 2.79 Bini; Bini et al. (979) ω < 2.78 Schönhage (98) ω < 2.55 Pan; Romani; Coppersmith + Winograd (98-982) ω < 2.50 Strassen (987) ω < 2.48 Coppersmith + Winograd (987) ω < Stothers (200) ω < Williams (20) ω < Le Gall (204) ω < Sept. 7, 204 4

5 Outline. main ideas from Strassen 969 through Le Gall approach via embedding into semisimple algebra multiplication groups coherent configurations/association schemes Sept. 7, 204 5

6 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a Sept. 7, 204 6

7 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a Sept. 7, 204 7

8 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a Sept. 7, 204 8

9 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a Sept. 7, 204 9

10 The matrix multiplication tensor <n,n,n> is a n 2 x n 2 x n 2 tensor described by trilinear form i,j,k X i,j Y j,k Z k,i a a 2 a 2 a 22 b b 2 x = b 2 b 22 c c 2 c 2 c 22 b b 2 b 2 b 22 a 22 a 2 a 2 a Sept. 7, 204 0

11 The matrix multiplication tensor <n,m,p> is a nm mp pn tensor described by trilinear form i,j,k X i,j Y j,k Z k,i Each of np slices of <n,m,p>: n m A X m B = n C p p Sept. 7, 204

12 Strategies for upper bounding the rank of the matrix multiplication tensor

13 Upper bounds on rank Observation: <n,n,n> -i = <n i, n i, n i > ) R(<n i, n i, n i >) R(<n,n,n>) i Strategy I: bound rank for small n by hand R(<2,2,2>) = 7! < 2.8 R(<3,3,3>) 2 [9..23] (worse bound) even computer search infeasible Sept. 7, 204 3

14 Upper bounds on rank Border rank = rank of sequence of tensors approaching target tensor entrywise ² rank = 3 border rank = 2: ² - ² Strategy II: bound border rank for small n Lemma: R(<n,n,n>) < r )! < log n r R(<2,2,3>) 0! < 2.79 Sept. 7, 204 4

15 Upper bounds on rank Direct sum of tensors <n,n,n> <m,m,m> <n,n,n> <m,m,m> Asymptotic Sum Inequality and example (Schönhage 98) (multiple matrix multiplications in parallel) Strategy III: bound (border) rank of direct sums of small matrix multiplication tensors R(<n,n,n > <n k,n k,n k >) < r ) i n i! < r R(<4,,3> <,6,>) 3! < 2.55 Sept. 7, 204 5

16 !!! Upper bounds on rank Strategy IV: Strassen laser method tensor with coarse structure of MM and fine structure components isomorphic to MM!! (many independent MMs in high tensor powers)!! coarse structure fine = scalar x row vector <,2,> col vector x scalar Sept. 7, 204 6!! q

17 !!! Upper bounds on rank Strategy IV: Strassen laser method tensor with coarse structure of MM and fine structure components isomorphic to MM (many independent MMs in high tensor powers)!!!!!! q border rank = q + ; q = 5 yields! < 2.48 Sept. 7, 204 7

18 Upper bounds on rank Coppersmith-Winograd and beyond: border rank of this tensor is q+2: i= q X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 + X 0 Y 0 Z q+ + X 0 Y q+ Z 0 + X q+ Y 0 Z 0 6 pieces : target proportions in high tensor power affect # and size of independent MMs q = 6 yields! < Sept. 7, 204 8

19 Upper bounds on rank Coppersmith-Winograd and beyond: analyze tensor powers of this tensor T q = i= q X 0 Y i Z i + X i Y 0 Z i + X i Y i Z 0 + X 0 Y 0 Z q+ + X 0 Y q+ Z 0 + X q+ Y 0 Z 0 Tensor power # pieces bound reference C-W Stothers Williams x 0^ Le Gall x 0^ Le Gall Sept. 7, 204 9

20 Upper bounds on rank Coppersmith-Winograd and beyond Tensor power # pieces bound reference C-W Stothers Williams x 0^ Le Gall x 0^ Le Gall Ambainis-Filmus 204: N-th tensor power cannot beat bound of Sept. 7,

21 A different approach So far... bound border rank of small tensor (by hand) asymptotic bound from high tensor powers Disadvantages limited universe of starting tensors high tensor powers hard to analyze Sept. 7, 204 2

22 matrix multiplication via groups and coherent configurations / association schemes

23 The general approach Cohn-Umans 2003, 202: embed n x n matrix multiplication into semisimple algebra multiplication semi-simple: isomorphic to block-diagonal MM = commutative, diagonal key hope: nice basis w/ combinatorial structure reduce n x n MM to smaller MMs; recurse Sept. 7,

24 The Group Algebra given finite group G, group algebra C[G] has elements Σ g a g g with multiplication (Σ g a g g)(σ h b h h) = Σ f (Σ gh = f a g b h )f structure: C[G] ' (C d d) (C d k dk) group elements are nice basis Sept. 7,

25 Nice basis embedding: Subgroups X, Y, Z of G satisfy the triple product property if for all x X, y Y, z Z : xyz = iff x = y = z =. Sept. 7,

26 The embedding: Q(S) = {s - t: s, t S} Subsets X, Y, Z of G satisfy the triple product property if for all x Q(X), y Q(Y), z Q(Z): xyz = iff x = y = z =. A = Σa x,y (x y - ) B = Σb y2,z 2 (y 2 z - 2 ) Claim: (AB) x3,z 3 = coeff. on (x 3 z - 3 ) in A*B. Sept. 7,

27 The embedding: Q(S) = {s - t: s, t S} Subsets X, Y, Z of G satisfy the triple product property if for all x Q(X), y Q(Y), z Q(Z): xyz = iff x = y = z =. A = Σa x,y (x y - ) B = Σb y2,z 2 (y 2 z - 2 ) Claim: (AB) x3,z 3 = coeff. on (x 3 z - 3 ) in A*B. (x y - )(y 2 z - 2 ) = x 3 z - 3 ) x - 3 x y - y 2 z - 2 z 3 = Sept. 7,

28 How many multiplications? Embedding + structure of C[G] yields bound on rank ( # multiplications): we use m Σd i 3 mults really m = Σd i! mults at least m Σd i 2 = G mults = First Challenge: embed k k matrix multiplication in group of size ¼ k 2 Sept. 7,

29 The embedding First Challenge: embed k k matrix multiplication in group of size ¼ k 2 simple pigeonhole argument: embedding in an abelian group requires group to have size k 3 Sept. 7,

30 The triangle construction Theorem: can embed k k matrix multiplication in symmetric group of size k 2 + o() n objects subgroup X subgroup Y subgroup Z need X, Y, Z in S n all with size S n /2 Sept. 7,

31 The triangle construction X moves points within rows Y moves points within columns Z moves points within diagonals want: xyz = x = y = z = Sept. 7, 204 3

32 The triangle construction Theorem: can embed k k matrix multiplication in symmetric group of size k 2 + o() n objects subgroup X subgroup Y subgroup Z unfortunately, d max > X (= Y = Z ) Sept. 7,

33 What should we be aiming for? Theorem: in group G supporting k x k matrix multiplication with character degrees d, d 2, d 3,, we obtain: k ω i d i ω If X, Y, Z µ G satisfy T.P.P. and ( X Y Z ) /3 = k G /2 o() d max G /2 ² then! = 2 i d i! d max! 2 G Sept. 7,

34 Constructions in linear groups Good candidate family: SL(n, q) for fixed dimension n In SL(n, R) these three subgroups satisfy the triple product property: upper-triangular with ones on the diagonal lower-triangular with ones on the diagonal the special orthogonal group SO(n, R) and dim. of each is ½ dim. of G as n! Sept. 7,

35 Group algebra approach [CKSU 2005] wreath product groups yield :! < 2.48,! < 2.4 key part of construction is combinatorial two conjectures implying! = 2 Main disadvantage: non-trivial results require non-abelian groups most ideas foiled by too-large char. degrees Sept. 7,

36 General semi-simple algebras (finite dimensional, complex) algebra specified by nice basis e, e 2,, e r structure constants i,j,k satisfying e i e j = k i,j,k e k realizes MM if contains*: structural tensor of algebra mult. MM tensor <n,n,n> i i,j,k Sept. 7, j k

37 Weighted vs. unweighted MM Technical problem: MM tensor <n,n,n> given by i,j,k X i,j Y j,k Z k,i embedding into algebra bounds rank of tensor given by i,j,k i,j,k X i,j Y j,k Z k,i (with i,j,k 0) group algebra: i,j,k always 0 or Sept. 7,

38 Weighted vs. unweighted MM s-rank of tensor T: minimum rank of tensor with same support as T Does upper bound on s-rank of MM tensor imply upper bound on ordinary rank? Example: a a 2 a 2 a 22 b b 2 x = b 2 b 22 a b + a b 2 + a 2 b 2 a 2 b 22 a 2 b + a 2 b 2 + a 22 b 2 a 22 b 22 Sept. 7,

39 Weighted vs. unweighted MM s-rank of tensor T: minimum rank of tensor with same support as T Does upper bound on s-rank of MM tensor imply upper bound on ordinary rank? Example: a a 2 a 2 a 22 x b b 2 b 2 b 22 does it help if can compute this in 6 multiplications?! a b + a b 2 + a 2 b 2 a 2 b 22 a 2 b + a 2 b 2 + a 22 b 2 2 a 22 b 22 39

40 Weighted vs. unweighted MM s-rank can be much smaller than rank: rank n same support: = n-th root of unity rank rank maybe it s easy to show s-rank of n n matrix multiplication is n 2 (!!) Sept. 7,

41 Weighted vs. unweighted MM! = inf { : rank(<n,n,n>) O(n )}! s = inf{ : s-rank(<n,n,n>) O(n )} Theorem:! (3! s 2)/2 in particular,! s 2 + ² )! 2 + (3/2)² Proof idea: find ¼ n 2 copies of <n,n,n> in 3 rd tensor power when broken up this way, can rescale Sept. 7, 204 4

42 A promising family of semisimple algebras

43 Coherent configurations group theory without groups points X, partition R, R 2,, R r of X 2 diagonal {(x,x) : x 2 X} is the union of some classes for each i, there is i * such that R i * = {(y,x) : (x,y) 2 R i } exist integers p i,j k such that for all (x,y) 2 R k : #{z: (x,z) 2 R i and (z,y) 2 R j } = p i,j k if one class: association scheme p i,j k = p j,i k : commutative x i z k j y Sept. 7,

44 Coherent configs: examples Hamming scheme: points 0/ vectors classes determined by hamming distance distance-regular graph: points = vertices classes determined by distance in graph metric Sept. 7,

45 Coherent configs: examples scheme based on finite group G set X = finite group G z classes R g = {(x,xg) : x 2 X} f g p f,g h = if fg=h, 0 otherwise x h y Schurian : group G acts on set X classes = orbits of (diagonal) G-action on X 2 Sept. 7,

46 Coherent configs: examples Schurian : group G acts on set X classes = orbits of (diagonal) G-action on X 2 one Schurian scheme: group scheme group G x G acts on G via (g,h) x = gxh - orbits all of the form {(x,y): xy - 2 C i } for conjugacy class C i always commutative! Sept. 7,

47 Adjacency algebra CC: points X, partition R, R 2,, R r of X 2 for each class R i, matrix A i with A i [x,y] = iff (x,y) 2 R i 3 CC axioms ) {A i } generate a semisimple algebra e.g., 3 rd axiom implies A i A j = k p ij k A k if the CC based on group G, algebra is C[G] Sept. 7,

48 Nice basis conditions group algebra C[G]: nice basis yields triple product property adjacency algebras of CCs: nice basis yields triangle condition: (i,j ) (j,k ) can look like class names (k,i ) iff i = i, j = j, k = k Sept. 7,

49 Nice basis conditions Schurian CCs: nice basis yields group G acts on set X subsets A,B,C of X realize < A, B, C > if: a a f b g c h b c A B C fgh = implies a = a, b = b, c = c Sept. 7,

50 Coherent configs vs. groups Generalization for generalization s sake? recall group framework: non-commutative necessary Theorem: in group G realizing n n matrix multiplication, with character degrees d, d 2, d 3,, we obtain: R(<n,n,n>) i d i ω d max ω-2 G goals: G ¼ n 2 and small d max Sept. 7,

51 Coherent configs vs. groups Generalization for generalization s sake? coherent configuration framework: commutative suffices! combinatorial constructions from old setting yield! s < 2.48,! s < 2.4 conjectures from old setting (if true) would imply! s = 2 Sept. 7, 204 in commutative Schurian CC s even group schemes even symmetric

52 Proof idea we prove a general transformation: if can realize several independent matrix multiplications in CC can do this in abelian groups conjectures: can pack optimally then high symmetric power of CC realizes single matrix multiplication reproves Schönhage s Asymptotic Sum Inequality preserves commutativity Sept. 7,

53 Commutative CCs suffice Main point embedding n x n matrix multiplication into a commutative coherent configuration of rank ¼ n 2 is a viable route to! = 2 (no representation theory needed) Sept. 7,

54 Open problems find a construction in new framework that proves non-trivial bound on! s is not based on constructions from old setting is the (border) s-rank of <2,2,2> = 6? embed n n MM into commutative coherent configuration of rank ¼ n 2 Sept. 7,