Jointly Clustering Rows and Columns of Binary Matrices: Algorithms and Trade-offs

Transcription

1 Jointly Clustering Rows and Columns of Binary Matrices: Algorithms and Trade-offs Jiaming Xu Joint work with Rui Wu, Kai Zhu, Bruce Hajek, R. Srikant, and Lei Ying University of Illinois, Urbana-Champaign June 7, 204

2 2 / 3 Motivation Data matrices with both row and column cluster structure arise in many applications User rating matrix Gene expression matrix Goal: Cluster rows and columns based on a noisy, partially observed matrix

3 3 / 3. like: +; dislike: Simple model 2. n users (movies) form r clusters of equal size K 3. users in the same cluster give the same rating to movies in the same cluster 4. block rating is + or with equal prob. Ground truth Y : binary block-constant matrix

4 3 / 3. like: +; dislike: Simple model 2. n users (movies) form r clusters of equal size K 3. users in the same cluster give the same rating to movies in the same cluster 4. block rating is + or with equal prob Ground truth Y : binary block-constant matrix Partial and noisy observation R: erasure prob. ɛ flipping prob. p

5 4 / 3 When cluster recovery is possible (impossible)? Assume that 0 p < /2 is a constant. ur results apply to the general setting allowing any K, ɛ. large cluster K = n small cluster low erasure ɛ = n α high erasure α

6 4 / 3 When cluster recovery is possible (impossible)? Assume that 0 p < /2 is a constant. ur results apply to the general setting allowing any K, ɛ. large cluster easy K = n small cluster low erasure ɛ = n α hard α high erasure

7 5 / 3 utline of the remainder. Impossible regime 2. Nearest-neighbor clustering 3. Spectral method 4. Convex method 5. Maximum likelihood estimation (MLE)

8 6 / 3 Impossible regime Genie-aided with the set of flipped entries revealed

9 6 / 3 Impossible regime Genie-aided with the set of flipped entries revealed Construct a new user clustering by swapping two rows in two different row clusters

10 6 / 3 Impossible regime Genie-aided with the set of flipped entries revealed Construct a new user clustering by swapping two rows in two different row clusters K = n ɛ = n α α

11 6 / 3 Impossible regime Genie-aided with the set of flipped entries revealed Construct a new user clustering by swapping two rows in two different row clusters K = n /2 /2 impossible ɛ = n α α

12 6 / 3 Impossible regime Genie-aided with the set of flipped entries revealed Construct a new user clustering by swapping two rows in two different row clusters K = n? /2 /2 impossible ɛ = n α α

13 7 / 3 Nearest-neighbor clustering Similarity between two users: The number of movies with the same observed rating [Dabeer et al. 2] Algorithm: Each user finds the K most similar users

14 7 / 3 Nearest-neighbor clustering Similarity between two users: The number of movies with the same observed rating [Dabeer et al. 2] Algorithm: Each user finds the K most similar users K = n /2 /2 ɛ = n α α

15 7 / 3 Nearest-neighbor clustering Similarity between two users: The number of movies with the same observed rating [Dabeer et al. 2] Algorithm: Each user finds the K most similar users K = n B NN /2 A /2 ɛ = n α α

16 7 / 3 Nearest-neighbor clustering Similarity between two users: The number of movies with the same observed rating [Dabeer et al. 2] Algorithm: Each user finds the K most similar users K = n B NN? /2 A /2 ɛ = n α α

17 Spectral method. Approximately clustering rows and columns of the best rank r approximation P r (R) 2. Majority voting within each block of R 3. Reclustering by assigning rows and columns to nearest centers 8 / 3

18 8 / 3 Spectral method. Approximately clustering rows and columns of the best rank r approximation P r (R) 2. Majority voting within each block of R 3. Reclustering by assigning rows and columns to nearest centers K = n B NN /2 A /2 ɛ = n α α

19 8 / 3 Spectral method. Approximately clustering rows and columns of the best rank r approximation P r (R) 2. Majority voting within each block of R 3. Reclustering by assigning rows and columns to nearest centers K = n B spectral NN /2 A /2 ɛ = n α α

20 8 / 3 Spectral method. Approximately clustering rows and columns of the best rank r approximation P r (R) 2. Majority voting within each block of R 3. Reclustering by assigning rows and columns to nearest centers K = n B spectral NN? /2 A /2 ɛ = n α α

21 9 / 3 Convex method Clustering by first recovering ground truth Y : Y R Y

22 9 / 3 Convex method Clustering by first recovering ground truth Y : Y R Y MLE is to find a block-constant binary matrix Y matching R as much as possible

23 9 / 3 Convex method Clustering by first recovering ground truth Y : Y R Y MLE is to find a block-constant binary matrix Y matching R as much as possible A convex relaxation of MLE: max Y R ij Y ij λ Y i,j s.t. Y ij [, ], λ = C ( ɛ)n, C 3

24 0 / 3 Performance of convex method Assume a technical conjecture (come back later) holds K = n B spectral NN /2 A /2 ɛ = n α α

25 0 / 3 Performance of convex method Assume a technical conjecture (come back later) holds K = n NN B spectral convex /2 A /2 ɛ = n α α

26 0 / 3 Performance of convex method Assume a technical conjecture (come back later) holds K = n NN B spectral convex? /2 A /2 ɛ = n α α

27 / 3 Performance of MLE K = n B spectral convex NN /2 A /2 ɛ = n α α

28 / 3 Performance of MLE K = n B spectral convex NN MLE (p = 0) C /2 A /2 ɛ = n α α

29 / 3 Performance of MLE K = n B spectral convex NN MLE (p = 0) C /2 A /2 ɛ = n α α Conjecture: MLE succeeds all the way up to the gray region

30 2 / 3 Conjecture on convex method Conjecture: For a r r random sign matrix B with SVD B = UΣV, UV log r scales as r.

31 2 / 3 Conjecture on convex method Conjecture: For a r r random sign matrix B with SVD B = UΣV, UV log r scales as r UB V B r log r log 2 r

32 3 / 3 Please check our paper for details Thank you! Questions?