Combinatorial Laplacian and Rank Aggregation Yuan Yao Stanford University ICIAM, Zürich, July 16 20, 2007 Joint work with Lek-Heng Lim
Outline 1 Two Motivating Examples 2 Reflections on Ranking Ordinal vs. Cardinal Global, Local, vs. Pairwise 3 Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus Combinatorial Laplacian Operator 4 Hodge Theory Cyclicity of Pairwise Rankings Consistency of Pairwise Rankings 5 Conclusions and Future Work
Two Motivating Examples Example I: Customer-Product Rating Example (Customer-Product Rating) m-by-n customer-product rating matrix X R m n X typically contains lots of missing values (say 90%). The first-order statistics, mean score for each product, might suffer from most customers just rate a very small portion of the products different products might have different raters, whence mean scores involve noise due to arbitrary individual rating scales
Two Motivating Examples From 1 st Order to 2 nd Order: Pairwise Rankings The arithmetic mean of score difference between product i and j over all customers who have rated both of them, k g ij = (X kj X ki ) #{k : X ki, X kj exist}, is translation invariant. If all the scores are positive, the geometric mean of score ratio over all customers who have rated both i and j, g ij = is scale invariant. ( k ( Xkj X ki ) ) 1/#{k:X ki,x kj exist},
Two Motivating Examples More invariant Define the pairwise ranking g ij as the probability that product j is preferred to i in excess of a purely random choice, g ij = Pr{k : X kj > X ki } 1 2. This is invariant up to a monotone transformation.
Two Motivating Examples Example II: Purely Exchange Economics Example (Pairwise ranking in exchange market) n goods V = {1,..., n} in an exchange market, with an exchange rate matrix A, such that 1 unit i = a ij unit j, a ij > 0. which is a reciprocal matrix, i.e. a ij = 1/a ji Ideally, a product triple (i, j, k) is called triangular arbitrage-free, if a ij a jk = a ik Money (universal equivalent): does there exist a universal equivalent with pricing function p : V R +, such that a ij = p j /p i?
Two Motivating Examples From Pairwise to Global Under the logarithmic map, g ij = log a ij, we have an equivalent theory: the triangular arbitrage-free is equivalent to g ij + g jk + g ki = 0 universal equivalent is a global ranking f : V R (f i = log p i ) such that g ij = f j f i =: (δ 0 f )(i, j) Here Global ranking universal equivalent (price) Pairwise ranking exchange rates
Two Motivating Examples Observations In both examples, contain cardinal information involve pairwise comparisons How important are they?
Reflections on Ranking Ordinal vs. Cardinal Ordinal Rank Aggregation Problem: given a set of partial/total order { i : i = 1,..., n} on a common set V, find ( 1,..., n ), as a partial order on V, satisfying certain optimal condition. Examples: voting Social Choice Theory Notes: Impossibility Theorems (Arrow et al.) Hardness in solving (NP-hard for Kemeny optimality etc.)
Reflections on Ranking Ordinal vs. Cardinal Cardinal Rank Aggregation Problem: given a set of functions f i : V R (i = 1,..., n), find (f 1,..., f n ) f as a function on V, satisfying certain optimal condition. Examples: customer-product rating, e.g. Amazon, Netflix stochastic choice with f as probability distributions on V, e.g. Google search, cardinal utility in Economics Notes relaxations leave rooms for possibility ordinal rankings induced from cardinal rankings, but with information loss
Reflections on Ranking Global, Local, vs. Pairwise Global, Local, and Pairwise Rankings Global ranking is a function on V, f : V R Local (partial) ranking: restriction of global ranking on a subset U, f : U R Pairwise ranking: g : V V R (with g ij = g ji ) Note: pairwise rankings are simply skew-symmetric matrices sl(n) or certain equivalence classes in sl(n). Also we may view pairwise rankings as weighted digraphs.
Reflections on Ranking Global, Local, vs. Pairwise Why Pairwise Ranking? Human mind can t make preference judgements on moderately large sets (e.g. no more than 7 ± 2 in psychology study) But human can do pairwise comparison more easily and accurately Pairwise ranking naturally arises in tournaments, exchange Economics, etc. Pairwise ranking may reduce the bias caused by the arbitrariness of rating scale Pairwise ranking may contain more information than global ranking (to be seen soon)!
Discrete Exterior Calculus and Combinatorial Laplacian Our Main Theme Below we ll outline an approach to analyze cardinal, and pairwise rankings, in a perspective from discrete exterior calculus. Briefly, we ll reach an orthogonal decomposition of pairwise rankings, by Hodge Theory, Pairwise = Global + Consistent Cyclic + Inconsistent Cyclic
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus Simplicial Complex of Products Let V = {1,..., n} be the set of products or alternatives to be ranked. Construct a simplicial complex K: 0-simplices K 0 : V 1-simplices K 1 : edges {i, j} such that comparison (i.e. pairwise ranking) between i and j exists 2-simplices K 2 : triangles {i, j, k} such that every edge exists in K 1 more considerations on consistency, like triangular arbitrage-free Note: it suffices here to construct K up to dimension 2!
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus Cochains k-cochains C k (K, R): vector space of k + 1-alternating tensors associated with K k+1 {u : V k+1 R, u iσ(0),...,i σ(k) = sign(σ)u i0,...,i k } for (i 0,..., i k ) K k+2, where σ S k+1 is a permutation on (0,..., k). Inner product in C k (K, R): standard Euclidean In particular, global ranking: 0-cochains f C 0 (K, R) = R n pairwise ranking: 1-cochains g C 1 (K, R), g ij = g ji
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus Coboundary Maps k-dimensional coboundary maps δ k : C k (V, R) C k+1 (V, R) are defined as the alternating difference operator k+1 (δ k u)(i 0,..., i k+1 ) = ( 1) j+1 u(i 0,..., i j 1, i j+1,..., i k+1 ) j=0 δ k plays the role of differentiation δ k+1 δ k = 0 In particular, (δ 0 f )(i, j) = f j f i is gradient of global ranking f (δ 1 g)(i, j, k) = g ij + g jk + g ki is curl of pairwise ranking g
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus A View from Discrete Exterior Calculus We have the following cochain complex in other words, C 0 (K, R) δ 0 C 1 (K, R) δ 1 C 2 (K, R), and Global grad Pairwise curl Triplewise curl grad(global Rankings) = 0 Pairwise rankings = alternating 2-tensors = skew-symmetric matrices = log of Saaty s reciprocal matrices Triplewise rankings = alternating 3-tensors See also: Douglas Arnold s talk on Tuesday
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus What does it tell us? Global grad Pairwise curl Triplewise grad(global) (i.e. im(δ 0 )): a proper subset of pairwise rankings induced from global curl(pairwise) (i.e. im(δ 1 )): measures the consistency/triangular arbitrage on triangle {i, j, k} (δ 1 g)(i, j, k) = g ij + g jk + g ki ker(curl) (i.e. ker(δ 1 )): consistent, curl-free, triangular arbitrage-free, in particular curl grad(global) = 0 (i.e. δ 1 δ 0 = 0) says global rankings are consistent/curl-free
Discrete Exterior Calculus and Combinatorial Laplacian Discrete Exterior Calculus Reverse direction: conjugate operators Gradient grad (= div) Pairwise curl Triplewise grad : δ0 T under Euclidean inner product, gives the total inflow-outflow difference at each vertex (negative divergence) (δ0 T g)(i) = g i g i ker(δ0 T ), as divergence-free, is cyclic (interior/boundary) curl : δ1 T, gives interior cyclic pairwise rankings along triangles in K 2, which are inconsistent
Discrete Exterior Calculus and Combinatorial Laplacian Combinatorial Laplacian Operator Combinatorial Laplacian Define the k-dimensional combinatorial Laplacian, k : C k C k by k = δ k 1 δ T k 1 + δt k δ k, k > 0 k = 0, 0 = δ T 0 δ 0 is the well-known graph Laplacian k = 1, 1 = curl curl div grad Important Properties: k positive semi-definite ker( k ) = ker(δ T k 1 ) ker(δ k): k-harmonics, dimension equals to k-th Betti number Hodge Decomposition Theorem
Hodge Theory Hodge Decomposition Theorem Theorem The space of pairwise rankings, C 1 (V, R), admits an orthogonal decomposition into three C 1 (V, R) = im(δ 0 ) H 1 im(δ T 1 ) where H 1 = ker(δ 1 ) ker(δ T 0 ) = ker( 1 ).
Hodge Theory Hodge Decomposition Illustration Figure: Hodge Decomposition for Pairwise Rankings
Hodge Theory An Example from Jester Dataset Figure: Hodge Decomposition for a pairwise ranking on four Jester jokes (No.1-4): ĝ 1 gives a global ranking (order: 1 > 2 > 3 > 4) which accounts for 90% of the total norm; ĝ 2 is the consistent cyclic part on triangles {{123}, {124}} with 7% norm; and ĝ 3 is the inconsistent cyclic part.
Hodge Theory Cyclicity of Pairwise Rankings Acyclic-Cyclic Decomposition Corollary Every pairwise ranking admits a unique orthogonal decomposition, g = proj im(δ0 ) g + proj ker(δ T 0 ) g i.e. Pairwise = grad(global) + Cyclic Note: Pairwise rankings induced from global are exactly acyclic component, as the orthogonal complement of cyclic pairwise rankings.
Hodge Theory Consistency of Pairwise Rankings Consistency Definition A pairwise ranking g is consistent on a triangle (2-simplex) (i, j, k) if g ij + g jk + g ki = 0, in other words, (δ 1 g)(i, j, k) = 0. Note: Consistency depends on the triangles (2-simplices), so for a pairwise ranking g, curl(g)(i, j, k) measures the curl distribution over triangles (2-simplices) in K 2 x Curl distribution of Jester dataset 104 9 8 7 6 5 4 3 2 1 0 0.2 0.15 0.1 0.05 0 0.05 0.1 0.15
Hodge Theory Consistency of Pairwise Rankings Consistent Decomposition Corollary 1 A consistent pairwise ranking g associated with K, has a unique orthogonal decomposition g = proj im(δ0 ) g + proj H1 g = grad(global) + Harmonic i.e. where harmonic is cyclic on the holes of the complex K. 2 Every consistent pairwise ranking on a contractible K, is induced from a global ranking. Note: (2) rephrases the famous theorem in exchange Economics: triangular arbitrage-free implies arbitrage-free and the existence of universal equivalent.
Conclusions and Future Work Conclusions and Future Work Conclusions Future Hodge Theory provides an orthogonal decomposition for pairwise rankings Such decomposition is helpful to characterize the cyclicity and (triangular) consistency of pairwise rankings Comparisons with other spectral methods Fourier Analysis on symmetry groups (Diaconis) Markov Chain based methods (PageRank, etc.) as graph Laplacians Design new algorithms Applications on large scale data sets, e.g. Netflix dataset.
Acknowledgements Gunnar Carlsson (Stanford) Persi Diaconis (Stanford) Nick Eriksson (Stanford) Fei Han (UCB) Susan Holmes (Stanford) Xiaoye Jiang (Stanford) Ming Ma (UCB and Beijing Institute of Technology) Michael Mahoney (Yahoo! Research) Steve Smale (TTI-U Chicago and UCB) Shmuel Weinberger (U Chicago)