Applications. Nonnegative Matrices: Ranking

Transcription

1 Applications of Nonnegative Matrices: Ranking and Clustering Amy Langville Mathematics Department College of Charleston Hamilton Institute 8/7/2008

2 Collaborators Carl Meyer, N. C. State University David Gleich, Stanford University Michael Berry, University of Tennessee-Knoxville College of Charleston (current and former students) Luke Ingram Ibai Basabe Kathryn Pedings Neil Goodson Colin Stephenson Emmie Douglas

3 Nonnegative Matrices Ranking webpages Ranking sports teams Recommendation systems Meta-algorithms Rank and rating aggregation Cluster aggregation

4 Ranking webpages

5

6 1998: enter Link Analysis uses hyperlink structure to focus the relevant set combine traditional IR score with popularity score Page and Brin 1998 Kleinberg

7 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR

8 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections?

9 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections? It s huge. over 10 billion pages, average page size of 500KB 20 times size of Library of Congress print collection Deep Web X bigger than Surface Web

10 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections? It s huge. over 10 billion pages, average page size of 500KB 20 times size of Library of Congress print collection Deep Web X bigger than Surface Web It s dynamic. content changes: 40% of pages change in a week, 23% of.com change daily size changes: billions of pages added each year

11 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections? It s huge. over 10 billion pages, average page size of 500KB 20 times size of Library of Congress print collection Deep Web X bigger than Surface Web It s dynamic. content changes: 40% of pages change in a week, 23% of.com change daily size changes: billions of pages added each year It s self-organized. no standards, review process, formats errors, falsehoods, link rot, and spammers!

12 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections? It s huge. over 10 billion pages, average page size of 500KB 20 times size of Library of Congress print collection Deep Web X bigger than Surface Web It s dynamic. content changes: 40% of pages change in a week, 23% of.com change daily size changes: billions of pages added each year It s self-organized. no standards, review process, formats errors, falsehoods, link rot, and spammers! A Herculean Task!

13 Web Information Retrieval IR before the Web = traditional IR IR on the Web = web IR How is the Web different from other document collections? It s huge. over 10 billion pages, each about 500KB 20 times size of Library of Congress print collection Deep Web X bigger than Surface Web It s dynamic. content changes: 40% of pages change in a week, 23% of.com change daily size changes: billions of pages added each year It s self-organized. no standards, review process, formats errors, falsehoods, link rot, and spammers! Ah, but it s hyperlinked! Vannevar Bush s 1945 memex

14 Elements of a Web Search Engine WWW Crawler Module Page Repository User Indexing Module Results Queries query-independent Indexes Ranking Module Query Module Special-purpose indexes Content Index Structure Index

15 The Ranking Module (generates popularity scores) Measure the importance of each page

16 The Ranking Module (generates popularity scores) Measure the importance of each page The measure should be Independent of any query Primarily determined by the link structure of the Web Tempered by some content considerations

17 The Ranking Module (generates popularity scores) Measure the importance of each page The measure should be Independent of any query Primarily determined by the link structure of the Web Tempered by some content considerations Compute these measures off-line long before any queries are processed

18 The Ranking Module (generates popularity scores) Measure the importance of each page The measure should be Independent of any query Primarily determined by the link structure of the Web Tempered by some content considerations Compute these measures off-line long before any queries are processed Google s PageRank c technology distinguishes it from all competitors

19 The Ranking Module (generates popularity scores) Measure the importance of each page The measure should be Independent of any query Primarily determined by the link structure of the Web Tempered by some content considerations Compute these measures off-line long before any queries are processed Google s PageRank c technology distinguishes it from all competitors Google s PageRank = Google s $$$$$

20

21

22

23

24

25 Google s PageRank (Lawrence Page & Sergey Brin 1998) The Google Goals Create a PageRank r(p ) that is not query dependent Off-line calculations No query time computation Let the Web vote with in-links But not by simple link counts One link to P from Yahoo! is important Many links to P from me is not Share The Vote Yahoo! casts many votes value of vote from Y ahoo! is diluted If Yahoo! votes for n pages Then P receives only r(y )/n credit from Y

26 Google s PageRank (Lawrence Page & Sergey Brin 1998) The Google Goals Create a PageRank r(p ) that is not query dependent Off-line calculations No query time computation Let the Web vote with in-links But not by simple link counts One link to P from Yahoo! is important Many links to P from me is not Share The Vote Yahoo! casts many votes value of vote from Y ahoo! is diluted If Yahoo! votes for n pages Then P receives only r(y )/n credit from Y

27 Google s PageRank (Lawrence Page & Sergey Brin 1998) The Google Goals Create a PageRank r(p ) that is not query dependent Off-line calculations No query time computation Let the Web vote with in-links But not by simple link counts One link to P from Yahoo! is important Many links to P from me is not Share The Vote Yahoo! casts many votes value of vote from Y ahoo! is diluted If Yahoo! votes for n pages Then P receives only r(y )/n credit from Y

28 Google s PageRank (Lawrence Page & Sergey Brin 1998) The Google Goals Create a PageRank r(p ) that is not query dependent Off-line calculations No query time computation Let the Web vote with in-links But not by simple link counts One link to P from Yahoo! is important Many links to P from me is not Share The Vote Yahoo! casts many votes value of vote from Y ahoo! is diluted If Yahoo! votes for n pages Then P receives only r(y )/n credit from Y

29 The Definition r(p )= r(p ) P P B P PageRank B P = {all pages pointing to P } P = number of out links from P

30 The Definition r(p )= r(p ) P P B P PageRank B P = {all pages pointing to P } P = number of out links from P Successive Refinement Start with r 0 (P i )=1/n for all pages P 1,P 2,...,P n

31 The Definition r(p )= r(p ) P P B P Successive Refinement Start with r 0 (P i )=1/n PageRank B P = {all pages pointing to P } P = number of out links from P for all pages P 1,P 2,...,P n Iteratively refine rankings for each page r 1 (P i )= r 0 (P ) P P B Pi

32 The Definition r(p )= r(p ) P P B P Successive Refinement Start with r 0 (P i )=1/n PageRank B P = {all pages pointing to P } P = number of out links from P for all pages P 1,P 2,...,P n Iteratively refine rankings for each page r 1 (P i )= r 0 (P ) P P B Pi r 2 (P i )= r 1 (P ) P P B Pi

33 The Definition r(p )= r(p ) P P B P Successive Refinement Start with r 0 (P i )=1/n PageRank B P = {all pages pointing to P } P = number of out links from P for all pages P 1,P 2,...,P n Iteratively refine rankings for each page r 1 (P i )= r 0 (P ) P P B Pi r 2 (P i )= r 1 (P ) P P B Pi... r j+1 (P i )= r j (P ) P P B Pi

34 In Matrix Notation After Step k π T k = [r k(p 1 ),r k (P 2 ),...,r k (P n )]

35 In Matrix Notation After Step k π T k = [r k(p 1 ),r k (P 2 ),...,r k (P n )] π T k+1 = πt k H where h ij = { 1/ Pi if i j 0 otherwise

36 In Matrix Notation After Step k π T k = [r k(p 1 ),r k (P 2 ),...,r k (P n )] π T k+1 = πt k H where h ij = { 1/ Pi if i j 0 otherwise PageRank vector = π T = lim k π T k = eigenvector for H Provided that the limit exists

37 Tiny Web 1 2 P 1 P 2 P 3 P 4 P 5 P H = P 1 P 2 P 3 P 4 P 5 P 6 4

38 H = Tiny Web P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 P 3 P 4 P 5 P 6 4

39 H = Tiny Web P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 P 4 P 5 P 6 4

40 H = Tiny Web P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P 4 P 5 P 6 4

41 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P 5 P 6 4

42 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P 6 4

43 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P

44 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P A random walk on the Web Graph

45 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P A random walk on the Web Graph PageRank = π i = amount of time spent at P i

46 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote 2 3 Markov chain 6 5 4

47 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

48 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

49 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

50 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

51 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

52 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

53 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

54 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

55 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote

56 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote 2 page 2 is a dangling node

57 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P A random walk on the Web Graph PageRank = π i = amount of time spent at P i Dead end page (nothing to click on) a dangling node

58 Tiny Web H = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P A random walk on the Web Graph PageRank = π i = amount of time spent at P i Dead end page (nothing to click on) a dangling node π T =(0, 1, 0, 0, 0, 0) = e-vector = Page P 2 is a rank sink

59 The Fix Allow Web Surfers To Make Random Jumps

60 Ranking with a Random Surfer Rank each page corresponding to a search term by number and quality of votes cast for that page. Hyperlink as vote 2 surfer teleports

61 The Fix Allow Web Surfers To Make Random Jumps ( 1 Replace zero rows with et n = n, 1 n,..., 1 ) n S = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 1/6 1/6 1/6 1/6 1/6 1/6 P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P

62 The Fix Allow Web Surfers To Make Random Jumps ( 1 Replace zero rows with et n = n, 1 n,..., 1 ) n S = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 1/6 1/6 1/6 1/6 1/6 1/6 P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P S = H + aet 6 is now row stochastic = ρ(s) = 1

63 The Fix Allow Web Surfers To Make Random Jumps ( 1 Replace zero rows with et n = n, 1 n,..., 1 ) n S = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 1/6 1/6 1/6 1/6 1/6 1/6 P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P S = H + aet 6 is now row stochastic = ρ(s) = 1 Perron says π T 0 s.t. π T = π T S with i π i = 1

64 Nasty Problem The Web Is Not Strongly Connected

65 Nasty Problem The Web Is Not Strongly Connected S is reducible S = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 1/6 1/6 1/6 1/6 1/6 1/6 P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P

66 Nasty Problem The Web Is Not Strongly Connected S is reducible S = P 1 P 2 P 3 P 4 P 5 P 6 P 1 0 1/2 1/ P 2 1/6 1/6 1/6 1/6 1/6 1/6 P 3 1/3 1/ /3 0 P /2 1/2 P /2 0 1/2 P Reducible = PageRank vector is not well defined Frobenius says S needs to be irreducible to ensure a unique π T > 0 s.t. π T = π T S with i π i = 1

67 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i )

68 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i ) The powers S k fail to converge

69 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i ) The powers S k fail to converge π T k+1 = πt k S fails to convergence

70 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i ) The powers S k fail to converge π T k+1 = πt k S fails to convergence Convergence Requirement Perron Frobenius requires S to be primitive

71 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i ) The powers S k fail to converge π T k+1 = πt k S fails to convergence Convergence Requirement Perron Frobenius requires S to be primitive No eigenvalues other than λ = 1 on unit circle

72 Irreducibility Is Not Enough Could Get Trapped Into A Cycle (P i P j P i ) The powers S k fail to converge π T k+1 = πt k S fails to convergence Convergence Requirement Perron Frobenius requires S to be primitive No eigenvalues other than λ = 1 on unit circle Frobenius proved S is primitive S k > 0 for some k

73 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1

74 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n

75 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n PageRank vector π T = left-hand Perron vector of G

76 Ranking with a Random Surfer If a page is important, it gets lots of votes from other important pages, which means the random surfer visits it often. Simply count the number of times, or proportion of time, the surfer spends on each page to create ranking of webpages.

77 Ranking with a Random Surfer If a page is important, it gets lots of votes from other important pages, which means the random surfer visits it often. Simply count the number of times, or proportion of time, the surfer spends on each page to create ranking of webpages. Proportion of Time Page 1 =.04 Page 2 =.05 Page 3 =.04 Page 4 =.38 Page 5 =.20 Page 6 = Ranked List of Pages Page 4 Page 6 Page 5 Page 2 Page 1 Page 3

78 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n PageRank vector π T = left-hand Perron vector of G Some Happy Accidents x T G = αx T H + βv T Sparse computations with the original link structure

79 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n PageRank vector π T = left-hand Perron vector of G Some Happy Accidents x T G = αx T H + βv T Sparse computations with the original link structure λ 2 (G) =α Convergence rate controllable by Google engineers

80 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n PageRank vector π T = left-hand Perron vector of G Some Happy Accidents x T G = αx T H + βv T Sparse computations with the original link structure λ 2 (G) =α Convergence rate controllable by Google engineers v T can be any positive probability vector in G = αh + uv T

81 The Google Fix Allow A Random Jump From Any Page G = αs +(1 α)e > 0, E = ee T /n, 0 <α<1 G = αh + uv T > 0 u = αa +(1 α)e, v T = e T /n PageRank vector π T = left-hand Perron vector of G Some Happy Accidents x T G = αx T H + βv T Sparse computations with the original link structure λ 2 (G) =α Convergence rate controllable by Google engineers v T can be any positive probability vector in G = αh + uv T The choice of v T allows for personalization

82 Google matrix G > 0 Perron-Frobenius guarantees existence of π uniqueness of π convergence of algorithm rate of convergence: 2 (G) = α

83 Ranking Sports Teams

84 PageRank applied to Sports (joint work with Carl Meyer) webpages vote with hyperlinks losing teams vote with point differentials 45 Duke 45 Miami 20 VT UNC 2 UVA not as successful at ranking and predicting winners as mhits

85 mhits each team i gets both offensive rating o i and defensive rating d i mhits Thesis: A team is a good defensive team (i.e., deserves a high defensive rating d j ) when it holds its opponents (particularly strong offensive teams) to low scores. A team is a good offensive team (i.e., deserves a high offensive rating o i ) when it scores many points against its opponents (particularly opponents with high defensive ratings). Miami 17 UNC Duke VT UVA P= Duke Miami UNC UVA VT Duke Miami UNC UVA VT Graph Point Matrix P 0

86 Summation Notation mhits Equations d j = X i I j p ij 1 o i and o i = X j L i p ij 1 d j Matrix Notation: iterative procedure d (k) = P T 1 o (k) and o (k) = P 1 d (k 1) related to the Sinkhorn-Knopp algorithm for matrix balancing (uses successive row and column scaling to transform P 0 into doubly stochastic matrix S) P. Knight uses Sinkhorn-Knopp algorithm to rank webpages

87 mhits Results: tiny NCAA (data from Luke Ingram) Duke Miami 38 VT UNC 7 UVA 5

88 Weighted mhits Weighted Point Matrix P 0 p ij = w ij p ij (w ij = weight of matchup between teams i and j) possible weightings w ij w Linear w Logarithmic t 0 tf t t 0 tf t w Exponential w Step Function t 0 tf t t 0 t s tf t

89 mhits Results: full NCAA (image from Neil Goodson and Colin Stephenson)

90 mhits Results: full NCAA weightings can easily be applied to all ranking methods interesting possibilities for weighted webpage ranking

91 mhits Point Matrix P 0 Perron-Frobenius guarantees (for irreducible P with total support) existence of o and d uniqueness of o and d convergence of mhits algorithm rate of convergence of mhits algorithm σ2 2 (S), where S = D (1/o) P D (1/d)

92 Recommendation Systems

93 Recommendation Systems A 0 purchase history matrix A = Item Item Item m User 1 User 2... User n Find similar users Find similar items Cluster users into classes EX: use NMF so that A m n W m k H k n

94 NMF Algorithm: Lee and Seung 2000 Mean Squared Error objective function min ka WHk 2 F s.t. W, H 0 W = abs(randn(m,k)); H = abs(randn(k,n)); for i = 1 : maxiter H = H.* (W T A)./ (W T WH ); W = W.* (AH T )./ (WHH T ); end Many parameters affect performance (k, obj. function, sparsity constraints, algorithm, etc.). NMF is not unique! (proof of convergence to fixed point based on E-M convergence proof)

95 Interpretation with NMF columns of W are the underlying basis vectors, i.e., each of the n columns of A m n can be built from k columns of W m k. columns of H give the weights associated with each basis vector.... A 2 = A 1 WH 1 = item 1 5 item 2 0 item 3 0 item 4 11 item 5 1 w 1. WH 1 = h w 2. h w k. h k W, H 0 immediate interpretation (additive parts-based rep.)

96 user rating matrix A = Netflix movies,.5 million users movie movie movie m User 1 User 2... User n Find similar users Find similar movies Cluster users and movies into classes Rank users Rank movies

97 (vismatrix tool - David Gleich) Clustering Netflix (movie-movie matrix; x 17770)

98 user rating matrix A = Netflix movies,.5 million users movie movie movie m User 1 User 2... User n Find similar users Find similar movies Cluster users and movies into classes Rank users Rank movies

99 mhits on Netflix each movie i gets a rating m i and each user gets a rating u j mhits Thesis: A movie is a good (i.e., deserves a high rating m i ) if it gets high ratings from good (i.e., discriminating) users. A user is good (i.e., deserves a high rating u j ) when his or her ratings match the true rating of a movie. mhits Netflix Algorithm u = e; for i = 1 : maxiter m = A u; m = 5(m min(m)) max(m) min(m) ; u = 1 ((R (R>0). (em T )). 2 )e ; end

100 Netflix mhits results subset: 1500 super users 1 st Raiders of the Lost Ark 2 nd Silence of the Lambs 3 rd The Sixth Sense 4 th Shawshank Redemption 5 th LOR: Fellowship of the Ring 6 th The Matrix 7 th LOR: The Two Towers 8 th Pulp Fiction 9 th LOR: The Return of the King 10 th Forrest Gump 11 th The Usual Suspects 12 th American Beauty (rate 1000 movies) 13 th Pirates of the Carribbean: Black Pearl 14 th The Godfather 15 th Braveheart

101 The Enron Dataset (data from Mike Berry) PRIVATE collection of 150 Enron employees during ,000 terms and 65,000 messages Term-by-Message Matrix f astow1 f astow2 skilling subpoena dynegy

102 Text Mining Applications Data compression Find similar terms Find similar documents Cluster documents Cluster terms Topic detection and tracking

103 Unclustered Enron A

104 NMF-clustered Enron A

105 Clustering the Enron Dataset (image from Mike Berry)

106 Tracking Enron clusters over time (image from Mike Berry)

107 Meta-Algorithms

108 Rank Aggregation

109 Rank Aggregation average rank Borda count simulated data graph theory

110 Average Rank

111 Borda Count for each ranked list, each item receives a score equal to the number of items it outranks.

112 Simulated Data 3 ranked lists mhits VT beats Miami by 1 point, UVA by 2 points,... Miami beats UVA by 1 point, UNC by 2 points,... UVA beats UNC by 1 point, Duke by 2 points UNC beats Duke by 1 point repeat for each ranked list generates game scores for teams Simulated Game Data

113 Simulated Data

114 Graph Theory more voting ranked lists are used to form weighted graph possible weights * w ij = # of ranked lists having i below j * w ij = sum of rank differences of lists having i below j run algorithm (e.g., Markov, PageRank, HITS) to determine most important nodes

115 Aggregation Rating Aggregation

116 Rating Aggregation rating vectors form rating differential matrix R for each rating vector

117 Rating Aggregation rating differential matrices R 0 differing scales normalize

118 Rating Aggregation rating differential matrices

119 Rating Aggregation rating differential matrices combine into one matrix average

120 Rating Aggregation rating differential matrices combine into one matrix average

121 Rating Aggregation average rating differential matrix R average 0 run ranking method * Markov method on R T average * row sums of R average / col sums of R average * Perron vector of R average

122 Rating Aggregation average rating differential matrix R average 0 run ranking method * Markov method on R T average * row sums of R average / col sums of R average * Perron vector of R average

123 Cluster Aggregation many clustering algorithms = many clustering results Can we combine many results to make one super result? Cluster Aggregation Algorithm 1. Create aggregation matrix F 0 f ij = # of methods having items i and j in the same cluster 2. Run favorite clustering method on F

124 Cluster Aggregation Example Method 1 Method 2 Method 3 item cluster assignment item cluster assignment item cluster assignment Cluster Aggregated Graph Cluster Aggregated Results Fiedler using just one eigenvector Fiedler using two eigenvectors

125 Conclusions Applied problems often have nonnegative data. This nonnegativity is exploitable structure. proofs for convergence and rates of convergence proofs for existence and uniqueness Nonnegativity is easily interpretable. Often it pays to maintain nonnegativity throughout the modeling process. nonnegative matrix factorizations nonnegative rating vectors