Part I: Preliminary Results. Pak K. Chan, Martine Schlag and Jason Zien. Computer Engineering Board of Studies. University of California, Santa Cruz

Transcription

1 Spectral K-Way Ratio-Cut Partitioning Part I: Preliminary Results Pak K. Chan, Martine Schlag and Jason Zien Computer Engineering Board of Studies University of California, Santa Cruz May, 99 Abstract Recent research on partitioning has focussed on the ratio-cut cost metric which maintains a balance between the sizes of the edges cut and the sizes of the partitions without xing the size of the partitions a priori. Iterative approaches and spectral approaches to two-way ratio-cut partitioning have yielded higher quality partitioning results. In this paper we develop a spectral approach to multi-way ratio-cut partitioning which provides a generalization of the ratio-cut cost metric to k-way partitioning and a lower bound on this cost metric. Our approach uses Lanczos algorithm to nd the k smallest eigenvalue/eigenvector pairs of the Laplacian of the graph. The eigenvectors are used to construct an orthogonal projection to map a vertex (of the graph) in an n-dimensional space into a k-dimensional subspace. We exploit the (near) orthogonality of the projected points to eect high quality clustering of points in a k-dimensional subspace. An ecient algorithm is presented for coercing the points in the k-dimensional subspace into k-partitions. Advancement over the current work is evidenced by the results of experiments on the standard MCNC benchmarks.

2 Chan, Schlag, Zien/UCSC May, 99 Introduction We present a method for k-way partitioning based on spectral techniques by extending the techniques of Hagen and Kahng []. The k-way partition problem can be formulated as the search for a projection of n-dimensional space onto a k-dimensional subspace, mapping the n unit-vector basis (the n nodes of a graph) into k distinct points (the partitions) to minimize the weighted quadratic displacement. This is essentially the formulation given by Barnes [, 3]. However, unlike Barnes' formulation we do not assume any pre-determined partition sizes. By Hall's result [4], using the k eigenvectors of the graph's Laplacian, corresponding to the smallest k eigenvalues provides a projection which minimizes the weighted quadratic displacement under the orthonormality constraint. In the case of a partition, this amounts to the number of edges cut. We show that a k projection provided by a partition can be reformulated as an orthonormal projection. This reformulation no longer minimizes the number of edges cut, but a new cost metric which incorporates the size of the partitions of a k-way partition P = fp; P; : : : P k g: kx E h cost(p) = h= jp h j : Here, E h denotes the number of edges between nodes in partition P h and nodes outside the partition. Interestingly, in the case of k =, this is the ratio-cut metric dened by Cheng and Wei scaled by the number of nodes in the graph [5]. The sum of the smallest k eigenvalues provides a lower bound on this cost metric. A geometric interpretation of the eigenvectors provides a method for transforming the eigenvector solution into a partition. More signicantly, this formulation provides a heuristic for identifying the natural number of partitions, k, a priori. Scaling the cost function above by k(k?) to oset the inuence of k on this cost metric (fewer nodes per partition and higher expected degrees) provides a means of comparing partitions across dierent k's. Denitions Given an undirected graph G with n nodes, v; v; : : : ; v n, the adjacency matrix of G is the n n matrix A(G) = [a ij ] dened by, a ij = the number of edges between v i and v j : If G is simple (no loops or parallel edges), then all of the entries in A(G) are 's or 0's and there are 0's along the diagonal. The degree matrix of G is the n n matrix D(G) = [d ij ]

3 Chan, Schlag, Zien/UCSC May, 99 3 dened by, d ij = ( degree(vi ) if i = j 0 if i 6= j The Laplacian of G is the n n matrix Q(G) = D(G)? A(G). A k-partition of the nodes of G, P = fp; P; : : : P k g is represented by an n k assignment matrix Y (P) = [y ih ] where y ih = ( if vi P h 0 if v i 6 P h The rows of Y sum to and column h sums to jp h j. A k-partition of G, P = fp; P; : : : P k g can also be represented by the n n partition matrix P = [p ij ] where ( if vi and v p ij = j are in the same partition 0 otherwise Given a simple graph G and a partition of its nodes, P, the partition graph G P is the graph with k vertices u; u; : : : ; u k where the number of edges between u g and u h for g 6= h is X v i P g X a ij = a ij y ig y jh ; v j P h i= j= where Y is the assignment matrix of P. This is the graph whose nodes are the partitions and whose interconnections are inherited from the edges of G. 3 Graph partitioning and some basic properties of the Laplacian Observation From Mohar \The Laplacian Spectrum of Graphs" [6] X i = je(g)j = degree(v i ) i= i= This follows from the Handshaking Lemma (the sum of the degrees of all nodes in an undirected graph is twice the number of edges) and the trace of a symmetric matrix (the sum of its diagonal entries) is the sum of its eigenvalues. aka Kircho matrix aka admittance matrix This was dened in Barnes' SIAM paper [].

4 Chan, Schlag, Zien/UCSC May, 99 4 Observation If Y is an n k matrix then Y T Q(G)Y is a k k matrix whose gh th component is a ij (y ig? y jg )(y ih? y jh ): i= j= Proof: Since Y T Q(G)Y = Y T (D(G)? A(G))Y = Y T D(G)Y? Y T A(G))Y the gh th component of Y T Q(G)Y is y ig d ii y ih? y ig ( a ij y jh ) = i= i= j= = y ig y ih ( a ij )? y ig ( a ij y jh ) i= j= i= j= a ij (y ig y ih )? a ij (y ig y jh ) i= j= i= j= = a ij (y ig y ih )? a ij (y ig y jh ) + a ij (y ig y ih ) i= j= i= j= i= j= X = a ij (y ig y ih )? na ij (y ig y jh ) + a ji (y jg y jh ) i= j= i= j= j= i= = a ij (y ig y ih )? a ij (y ig y jh ) + a ij (y jg y jh ) i= j= i= j= j= i= = a ij (y ig y ih? y ig y jh + y jg y jh ) i= j= = a ij (y ig? y jg )(y ih? y jh ) i= j= Observation 3 If Y is the assignment matrix for P then Y T Q(G)Y = Q(G P ), the Laplacian of G P. Proof: First consider the g th diagonal entry of Y T Q(G)Y : a ij (y ig? y jg ) : i= j= Since Y is an assignment matrix, a ij (y ig? y jg ) will be only when exactly one of v i and v j is in P g and a ij =. Summed over all i and j this gives twice the total number of edges from nodes in P g to nodes not in P g. Hence the g th diagonal entry is the degree of u g in G P. Now consider an o-diagonal entry (h 6= g). The gh th component of Y T Q(G)Y is: a ij (y ig? y jg )(y ih? y jh ): i= j=

5 Chan, Schlag, Zien/UCSC May, 99 5 Note that y ig? y jg will be non-zero if and only if exactly one of v i or v j is in P g. The same holds for y ih? y jh with respect to P h. Since a node can not be simultaneously in two partitions, (y ig? y jg )(y ih? y jh ) is non-zero exactly when one of v i or v j is in P g and the other is in P h. If this is the case and a ij must be non-zero, then the ij th term of the summation is?. Hence summing over all i and j gives minus twice the number of edges between nodes in partition P g and nodes in partition P h. Observation 4 If P is a partition matrix, then its eigenvalues (P ) = fm; m; :::; m k g, where m i is the number of nodes in the i th partition (E. Barnes). Observation 5 If Y is the assignment matrix for P then Y Y T is the partition matrix. Proof: The ij th element of Y Y T is kx y ih y jh : h= The term y ih y jh will be if and only if both v i and v j are in P h, and hence the sum is exactly when v i and v j are in the same partition and 0 otherwise. Observation 6 If Y is the assignment matrix for P then Y T Y is a diagonal matrix with jp g j in the g th entry. Proof: The gh th element of Y T Y is y ig y ih : i= The term y ig y ih can only be 0 when h 6= g since node v i can be in at most one of the two partitions. Hence the o-diagonal entries will be 0's. On the diagonal we have y ig i= which sums to the number of nodes in P g. The partition problem considered by Barnes and Donath,Homan was to solve for Y given k and jpj; jpj; : : : ; jp k j, the sizes of the partitions [7]. 4 Ratio-cut graph partitioning Now consider modifying the denition of the partition matrix to take into account the size of the partitions. Specically, a k-partition of the nodes of G, P = fp; P; : : : P k g is represented

6 Chan, Schlag, Zien/UCSC May, 99 6 by an n k ratioed assignment matrix R(P) = [r ih ] where r ih = 8 < : p if v i P h jp h j 0 if v i 6 P h The rows no longer necessarily sum to and column h now sums to q jp h j. Observation 7 If R is the ratioed assignment matrix for P then the g th diagonal entry of R T Q(G)R is the degree of u g in G P divided by jp g j, and the gh th o-diagonal entry is minus the number of edges between P g and P h divided by q jp g j jp h j. Proof: By Observation the g th diagonal entry of R T Q(G)R is: a ij (r ig? r jg ) : i= j= Since R is a ratioed assignment matrix, a ij (r ig? r jg ) will be non-zero only when exactly one of v i and v j is in P g and a ij =. Summed over all i and j this gives twice the total number of edges from nodes in P g to nodes not in P g. Each non-zero (r ig? r jg ) is q jp h j A = jp h j : Hence the g th diagonal entry is the degree of u g in G P divided by jp g j. Now consider an o-diagonal entry (h 6= g). The gh th component of R T Q(G)R is: a ij (r ig? r jg )(r ih? r jh ): i= j= Note that r ig? r jg will be non-zero if and only if exactly one of v i or v j is in P g. The same holds for r ih? r jh with respect to P h. Since a node can not be simultaneously in two partitions, (r ig? r jg )(r ih? r jh ) is non-zero exactly when one of v i or v j is in P g and the other is in P h. If this is the case and a ij is non-zero, then the ij th term of the summation is? q q : jp g j jp h j Hence summing over all i and j gives minus twice the number of edges between nodes in partition P g and nodes in partition P h divided by q jp g j jp h j. Given a k-partition of G, the n n ratioed partition matrix P R = [rp ij ] where ( jp rp ij = g if v j i and v j both belong to P g 0 otherwise

7 Chan, Schlag, Zien/UCSC May, 99 7 Observation 8 If R is a ratioed assignment matrix for P then RR T partition matrix. = P R, the ratioed Proof: The ij th element of RR T is kx r ih r jh : h= The term r ih r jh will be non-zero if and only if both v i and v j are in P h, hence the sum is jp h j when v i and v j are in the same partition and 0 otherwise. Observation 9 If R is the ratioed assignment matrix for P then R T R is I k, an identity matrix. Proof: The gh th element of R T R is r ig r ih : i= The term r ig r ih can only be 0 when h 6= g since node v i can be in at most one of the two partitions. Hence the o-diagonal entries will be 0's. On the diagonal we have r ig i= which sums to since there are jp g j non-zero terms which are all Hence the ratioed partition matrix meets the constraint R T R = I k. jp g j. 5 Relation between the eigenvalues and ratio cuts Dene the k-way ratio cut cost metric of a k-partition P of G to be kx degree(u h ) cost(p) = : h= jp h j (Recall that u h is the node in the partition graph corresponding to P h so degree(u h ) is the number of edges in G with exactly one endpoint in P h.) Observation 0 If k =, cost(p) is the ratio-cut cost metric dened by Wei and Cheng scaled by n, the number of nodes in the graph.

8 Chan, Schlag, Zien/UCSC May, 99 8 Proof: If k =, and E c is the number of edges cut in P = fp; Pg, then we have cost(p) = degree(u ) jpj = E c + E c jpj jpj = E c jpj +! jpj = E c jpj + jpj jpj jpj = E c n jpj jpj E c = n jpj jpj + degree(u ) jpj Observation If R is the ratioed assignment matrix associated with P, then cost(p) = trace(r T Q(G)R). This follows from Observation 7. Minimizing cost(p), amounts to nding a ratioed assignment matrix which minimizes the sum of the diagonal entries of R T QR. Observation The n k matrix which minimizes the trace(y T Q(G)Y ) subject to the constraint Y T Y = I k, is the n k matrix whose k columns consist of the k eigenvectors of Q corresponding to the k smallest 3 eigenvalues of Q. Proof: If we take the partials with respect to the variables in Y of the g th diagonal entry the matrix Lagrangian of Y T Q(G)Y?(Y T Y?I k ), and equate them to zero, this constrains column g of Y to be an eigenvector; no other columns are constrained. Thus the minimal solution will consist of a k eigenvectors of Q. These k eigenvectors must be distinct to satisfy Y T Y = I k. If V is formed with any k eigenvectors of Q then QV = V k where k is the diagonal matrix formed with the eigenvalues corresponding to the k eigenvectors in V. This means that V T QV = V T V k = I k k = k and the sum of the diagonal entries of V T QV is the sum of the eigenvalues corresponding to the k eigenvectors in V. It follows that this sum is minimized by selecting the eigenvectors corresponding to the k smallest eigenvalues of Q. 3 Eigenvalues may be repeated but eigenvectors may not.

9 Chan, Schlag, Zien/UCSC May, 99 9 Observation 3 The sum of the smallest k eigenvalues is a lower bound on the minimum cost(p) of any k-partition of G. In the case k =, since + = 0 + = we obtain E c cost(p) = n jpj jpj which amounts to the result of Hagen and Kahng [] n E c jpj jpj : obviously not a tight upper bound for a graph with large number of nodes. Observation 4 By Fiedler's inequality [8], C(? cos(=n))) where C is a cut set of the graph G, an upper bound on the optimal ratio-cut partitioning is (n? )(? cos(=n)) 6 A new k-way spectral ratio cut method The problem of nding an assignment is solved as follows:. Calculate all or enough(?) eigenvalues of Q(G).. Select k. See Section 7 below. 3. Find the eigenvectors corresponding to these k smallest eigenvalues. 4. Construct V, an n k matrix whose columns are the k eigenvectors. 5. Compute Z = V V T. 6. Construct a matrix P = [p ij ] from Z where p ij = ( if zij >= n 0 otherwise This is a heuristic of nding a partition using Z as \close" to P as possible. This method of coercing Z into a partition matrix doesn't always work. See Section 8 for alternatives.

10 Chan, Schlag, Zien/UCSC May, Selecting the number of partitions k In order to compare costs of partitions with dierent sizes, we scale our cost metric by scost(p) = n cost(p) k(k? ) : n k(k?), The quantity n is the average number of nodes in a balanced k-way partition and k? is k the degree of a node in the complete graph with k nodes. Observation 5 For a random graph with edge probability f, and any partition P of size k, scost(p) = fn n k which is a generalization of results of Cheng and Wei for bipartition [5]. (We are not sure yet what this means, may be this is the average degree of the partition graph per partition. But it will result in k = n being selected, if the heuristics by eigenvalues estimation below is accurate). We would like to select the k that produces the smallest scost() kx n degree(u h ) k(k? ) h= jp h j but since we do not know the optimal ratio-cut partition for any k we use the lower bound as an indicator. That is, we choose the k which minimizes n kx k(k? ) h= where : : : n is the non-decreasing order of the eigenvalues of Q. Note that this is a HEURISTIC. h 8 Geometrical interpretation The n n matrix in R k, RR T P R = P R. = P R, the ratioed partition matrix is a projector, since The n n matrix in R k Z = V V T (which approximates the ratioed assignment matrix P R ) also forms an orthogonal projection of R n spanned by the column vectors of Q onto a subspace of R k spanned by the k smallest eigenvectors.

11 Chan, Schlag, Zien/UCSC May, 99 Observation 6 Z has exactly k nonzero eigenvalues which are all, i.e., (V V T ) = Proof: Let the columns of the matrix X be the eigenvectors of Z, and be their corresponding matrix of eigenvalues V V T X = X V T V V T X = V T X I k = Observation 7 The nonzero eigenvalues of the ratioed partition matrix RR T are all s. Proof: Same as above. Observation 8 The \distance" between the subspaces formed by ratioed assignment matrix R and V is given by Homan-Wielandt inequality and is bounded from below: jj(rr T )? (V V T )jj 0 Observation 9 The distance between two equidimensional subspaces spanned by R and V is q? min(r T V ) where min (R T V ) is the smallest singular value of matrix R T V (By CS Decomposition, Golub and van Loan, pg. 77). This is the generalized sine angle between the two subspaces. The P R matrix projects R n onto the R k subspace dened by the orthonormal R. If the n nodes of the graph G are associated with the n unit-vectors of R n, then nodes belonging to the same partition are mapped to the same point by R. Since P R is \close" to the subspace dened by V T V, the nodes belonging to the same partition should have \close" projections under V. This suggests a heuristic for constructing the R which is \closest" to V : use proximity of the images of the unit-vectors under V. In cases of ambiguity the distance can be veried. 9 Summary We have presented some preliminary but new and basic results on spectral k-way ratio-cut partitioning in this report. The next report will demonstrate the eectiveness of this method and some experimental results.

12 Chan, Schlag, Zien/UCSC May, 99 References [] L. Hagen and A. Kahng, \New spectral methods for ratio cut partitioning and clustering," To appear in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 99. [] E. R. Barnes, \An algorithm for partitioning the nodes of a graph," SIAM Journal on Algorithm and Discrete Method, vol. 3, pp. 54{550, Dec. 98. [3] E. R. Barnes, \Partitioning the nodes of a graph," in Proceedings of Graph Theory with Applications to Algorithms and Computer Science, (Wiley), pp. 57{7, 985. [4] K. M. Hall, \An r-dimensional quadratic placement algorithm," Management Science, vol. 7, pp. 9{9, Nov [5] C.-K. Cheng and Y.-C. A. Wei, \An improved two-way partitioning algorithm with stable performance," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 0, pp. 50{5, Dec. 99. [6] B. Mohar, \The Laplacian spectrum of graphs," in Proceedings of the 6th Quadrennial International Conference on the Theory and Applications of Graphs: Graph Theory, Combinatorics and Applications, Vol., (Wiley), pp. 87{898, 988. [7] W. Donath and A. Homan, \Lower bounds for the partitioning of graphs," IBM Journal of Research and Development, pp. 40{45, 973. [8] M. Fiedler, Special matrices and their applications in numerical mathematics. Martinus Nijho Publishers, 986.