BOOLEAN TENSOR FACTORIZATIONS. Pauli Miettinen 14 December 2011

Transcription

1 BOOLEAN TENSOR FACTORIZATIONS Pauli Miettinen 14 December 2011

2 BACKGROUND: TENSORS AND TENSOR FACTORIZATIONS X

3 BACKGROUND: TENSORS AND TENSOR FACTORIZATIONS C X A B

4 BACKGROUND: BOOLEAN MATRIX FACTORIZATIONS Given a binary matrix X and a positive integer R, find two binary matrices A and B such that A has R columns and B has R rows and X A o B. A o B is the Boolean matrix product of A and B, R (A B) ij = b il c lj r=1

5 BOOLEAN TENSOR FACTORIZATIONS: THE IDEA 1. Take existing (normal) tensor factorization 2. Make everything binary and define summation as = 1 3. Try to understand what you just did. Research problem. What can we say about Boolean tensor factorizations and how do they relate to normal tensor factorizations and Boolean matrix factorizations?

6 RANK-1 (BOOLEAN) TENSORS b X = a X = a b

7 RANK-1 (BOOLEAN) TENSORS c X = b a X = a 1 b 2 c

8 THE CP TENSOR DECOMPOSITION c 1 c 2 c R X b 1 b b R a 1 a 2 a R R x ijk a ir b jr c kr r=1

9 THE CP TENSOR DECOMPOSITION C X A B R x ijk a ir b jr c kr r=1

10 THE BOOLEAN CP TENSOR DECOMPOSITION c 1 c 2 c R X b 1 b 2 b R a 1 a 2 a R R x ijk a ir b jr c kr r=1

11 THE BOOLEAN CP TENSOR DECOMPOSITION C X B A R x ijk a ir b jr c kr r=1

12 DIGRESSION: FREQUENT TRI- ITEMSET MINING Rank-1 N-way binary tensors define an N-way itemset Particularly, rank-1 binary matrices define an itemset In itemset mining the induced sub-tensor must be full of 1s Here, the items can have holes Boolean CP decomposition = lossy N-way tiling

13 TENSOR RANK The rank of a tensor is the minimum number of rank-1 tensors needed to represent the tensor exactly. c 1 c 2 c R X = b 1 b b R a 1 a 2 a R

14 BOOLEAN TENSOR RANK The Boolean rank of a binary tensor is the minimum number of binary rank-1 tensors needed to represent the tensor exactly using Boolean arithmetic. c 1 c 2 c R X = b 1 b 2 b R a 1 a 2 a R

15 SOME RESULTS ON RANKS Normal tensor rank is NPhard to compute Normal tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} But no more than min{nm, nk, mk}

16 SOME RESULTS ON RANKS Normal tensor rank is NPhard to compute Boolean tensor rank is NP-hard to compute Normal tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} But no more than min{nm, nk, mk}

17 SOME RESULTS ON RANKS Normal tensor rank is NPhard to compute Normal tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} Boolean tensor rank is NP-hard to compute Boolean tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} But no more than min{nm, nk, mk}

18 SOME RESULTS ON RANKS Normal tensor rank is NPhard to compute Normal tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} But no more than min{nm, nk, mk} Boolean tensor rank is NP-hard to compute Boolean tensor rank of n-by-m-by-k tensor can be more than min{n, m, k} But no more than min{nm, nk, mk}

19 SPARSITY Binary matrix X of Boolean rank R and X 1s has Boolean rank-r decomposition A o B such that A + B 2 X [M., ICDM 10] Binary N-way tensor X of Boolean tensor rank R has Boolean rank-r CP-decomposition with factor matrices A1, A2,, AN such that i Ai N X Both results are existential only and extend to approximate decompositions

20 THE TUCKER TENSOR DECOMPOSITION C B X A G P Q R x ijk g pqr a ip b jq c kr p=1 q=1 r=1

21 THE BOOLEAN TUCKER TENSOR DECOMPOSITION C B X A G P Q R x ijk g pqr a ip b jq c kr p=1 q=1 r=1

22 THE ALGORITHMS The normal CP-decomposition can be solved using matricization and ALS is the Khatri Rao matrix product (C B) T is R-by-mk For normal matrices, we can use standard leastsquares projections X (1) = A(C B) T X (2) = B(C A) T X (3) = C(B A) T One projection per mode Similar algorithms for the Tucker decomposition

23 THE ALGORITHMS For Boolean case, matrix product must be changed Khatri Rao stays as it Finding the optimal projection is NP-hard even to approximate Good initial values are needed due to multiple local minima X (1) = A (C B) T X (2) = B (C A) T X (3) = C (B A) T Obtained using Boolean matrix factorization to matricizations

24 THE TUCKER CASE The core tensor has global effects C Updates are hard X A G B Core tensor is usually small We can afford more time per element x ijk P Q R g pqr a ip b jq c kr In Boolean case many changes make no difference p=1 q=1 r=1

25 SYNTHETIC EXPERIMENTS Reconstruction error 12 x BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error 9 x BTucker_ALS Tucker_ALS B Tucker_ALS F r 0 (4, 4, 4) (8, 4, 4) (8, 8, 8) (16, 8, 4) Ranks

26 SYNTHETIC EXPERIMENTS Reconstruction error 12 x BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error 9 x BTucker_ALS Tucker_ALS B Tucker_ALS F r 0 (4, 4, 4) (8, 4, 4) (8, 8, 8) (16, 8, 4) Ranks

27 SYNTHETIC EXPERIMENTS Reconstruction error 12 x BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error 9 x BTucker_ALS Tucker_ALS B Tucker_ALS F r 0 (4, 4, 4) (8, 4, 4) (8, 8, 8) (16, 8, 4) Ranks

28 REAL-WORLD EXPERIMENTS Reconstruction error BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error BTucker_ALS Tucker_ALS B Tucker_ALS F r 1000 (5, 5, 5) (10, 10, 10) (15, 15, 15) (30, 30, 15) Ranks

29 REAL-WORLD EXPERIMENTS Reconstruction error BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error BTucker_ALS Tucker_ALS B Tucker_ALS F r 1000 (5, 5, 5) (10, 10, 10) (15, 15, 15) (30, 30, 15) Ranks

30 REAL-WORLD EXPERIMENTS Reconstruction error BCP_ALS CP_ALS B CP_ALS F CP_OPT B CP_OPT F Reconstruction error BTucker_ALS Tucker_ALS B Tucker_ALS F r 1000 (5, 5, 5) (10, 10, 10) (15, 15, 15) (30, 30, 15) Ranks

31 CONCLUSIONS Boolean tensor decompositions are a bit like normal tensor decompositions And a bit like Boolean matrix factorizations They generalize other data mining techniques in many ways The playing field between Boolean and normal tensor factorizations is more level

32 CONCLUSIONS Boolean tensor decompositions are a bit like normal tensor decompositions And a bit like Boolean matrix factorizations They generalize other data mining techniques in many ways The playing field between Boolean and normal tensor factorizations is more level!ank Y#!