Spectral Clustering based on the graph p-laplacian

Transcription

1 Sectral Clustering based on the grah -Lalacian Thomas Bühler Matthias Hein Saarland University Comuter Science Deartment Camus E 663 Saarbrücken Germany Abstract We resent a generalized version of sectral clustering using the grah -Lalacian a nonlinear generalization of the standard grah Lalacian. We show that the second eigenvector of the grah -Lalacian interolates between a relaxation of the normalized and the Cheeger cut. Moreover we rove that in the limit as the cut found by thresholding the second eigenvector of the grah -Lalacian converges to the otimal Cheeger cut. Furthermore we rovide an efficient numerical scheme to comute the second eigenvector of the grah - Lalacian. The exeriments show that the clustering found by -sectral clustering is at least as good as normal sectral clustering but often leads to significantly better results.. Introduction In recent years sectral clustering has become one of the major clustering methods. The reasons are its generality efficiency and its rich theoretical foundation. Sectral clustering can be alied to any kind of data with a suitable similarity measure and the clustering can be comuted for millions of oints. The theoretical background includes motivations based on balanced grah cuts random walks and erturbation theory. We refer to (von Luxburg 7) and references therein for a detailed introduction to various asects of sectral clustering. In this aer our focus lies on the motivation of sectral clustering as a relaxation of balanced grah cut criteria. It is well known that the second eigenvectors of the unnormalized and normalized grah Lalacians corresond to relaxations of the ratio cut (Hagen & Aearing in Proceedings of the 6 th International Conference on Machine Learning Montreal Canada 9. Coyright 9 by the author(s)/owner(s). Kahng 99) and normalized cut (Shi & Malik ). There are also relaxations of balanced grah cut criteria based on semi-definite rogramming (De Bie & Cristianini 6) which turn out to be better than the standard sectral ones but are comutationally more exensive. In this aer we establish a connection between the Cheeger cut and the second eigenvector of the grah -Lalacian a nonlinear generalization of the grah Lalacian. A -Lalacian which differs slightly from the one used in this aer has been used for semisuervised learning by Zhou and Schölkof (5). Our main motivation for the use of eigenvectors of the grah -Lalacian was the generalized isoerimetric inequality of Amghibech (3) which relates the second eigenvalue of the grah -Lalacian to the otimal Cheeger cut. The isoerimetric inequality becomes tight as so that the second eigenvalue converges to the otimal Cheeger cut value. In this article we extend the isoerimetric inequality of Amghibech to the unnormalized grah -Lalacian. However our key result is to show that the cut obtained by thresholding the second eigenvector of the -Lalacian converges to the otimal Cheeger cut as which rovides theoretical evidence that -sectral clustering is suerior to the standard case. Moreover we rovide an efficient algorithmic scheme for the (aroximate) comutation of the second eigenvector of the -Lalacian and the resulting clustering. This allows us to do -sectral clustering also for large scale roblems. Our exerimental results show that as one varies from (standard sectral clustering) to the value of the Cheeger cut obtained by thresholding the second eigenvector of the grah -Lalacian is always decreasing. In Section we review balanced grah cut criteria. In Section 3 we introduce the grah -Lalacian followed by the definition of eigenvectors of nonlinear oerators. In Section 4 we rovide the theoretical key result relating the cut found by thresholding the second eigenvector of the grah -Lalacian to the otimal Cheeger cut. The algorithmic scheme is resented in Section 5

2 Sectral Clustering based on the grah -Lalacian and extensive exeriments on various datasets including large scale ones are given in Section 6.. Balanced grah cut criteria Given a set of oints in a feature sace and a similarity measure the data can be transformed into a weighted undirected grah G where the vertices V reresent the oints in the feature sace and the ositive edge weights W encode the similarity of airs of oints. A clustering of the oints is then equivalent to a artition of V into subsets C... C k (which will be called clusters in the following). The usual objective for such a artitioning is to have high within-cluster similarity and low inter-cluster similarity. Additionally the clusters should be balanced in the sense that the size of the clusters should not differ too much. All the grah cut criteria resented in this section imlement these objectives with slightly different emhasis on the individual roerties. Before the definition of the balanced grah cut criteria we have to introduce some notation. The number of oints is denoted by n = V and the comlement of a set A V is written as A = V \A. The degree function d : V R of the grah is given as d i = n j= w ij and the cut of A V and A is defined as cut(a A) = w ij. i A j A Moreover we denote by A the cardinality of the set A and by vol(a) = i A d i the volume of A. In the balanced grah cut criteria one either tries to balance the cardinality or the volume of the clusters. The ratio cut RCut(C C) (Hagen & Kahng 99) and the normalized cut NCut(C C) (Shi & Malik ) for a artition of V into C C are defined as RCut(C C) = NCut(C C) = cut(c C) cut(c C) vol(c) + + cut(c C) cut(c C) vol(c). A slightly different balancing behavior is induced by the corresonding ratio Cheeger cut RCC(C C) and normalized Cheeger cut NCC(C C) defined as RCC(C C) = NCC(C C) = cut(c C) min{ C } cut(c C) min{vol(c) vol(c)}. One has the following simle relation between the normalized cut NCut(C C) and the normalized Cheeger cut NCC(C C): NCC(C C) NCut(C C) NCC(C C). The analogous result holds for the ratio cut RCut(C C) and the ratio Cheeger cut RCC(C C). It is known that finding the global otimum of all these balanced grah cut criteria is NP-hard see (von Luxburg 7). In Section 4 we will show how sectral relaxations of these criteria are related to the eigenroblem of the grah -Lalacian. U to now the cuts are just defined for a artition of V into two sets. For a artition of V into k sets C... C k the ratio and normalized cut can be generalized (von Luxburg 7) as RCut(C... C k ) = NCut(C... C k ) = k i= k i= cut(c i C i ) () C i cut(c i C i ). () vol(c i ) There seems to exist no generally acceted multiartition version of the Cheeger cuts. We come back to this issue in Section 5 when we discuss how to get multile clusters using the second eigenvector of the grah -Lalacian. 3. The grah -Lalacian It is well known see e.g. (Hein et al. 7) that the standard grah Lalacian can be defined as the oerator which induces the following quadratic form for a function f : V R: f f = n w ij (f i f j ). ij= For the standard inner roduct one gets the unnormalized grah Lalacian (u) which in matrix notation is given as (u) = D W and for the weighted inner roduct f g = n i= d i f i g i one obtains the normalized grah Lalacian (n) given as (n) = I D W. One can ask now if there exists an oerator which induces the general form (for > ) f f = n w ij f i f j. ij= It turns out that this question can be answered ositive see (Amghibech 3). The resulting oerator Note that our notation differs from the one in (Hein et al. 7) where they denote our normalized grah Lalacian as random walk grah Lalacian.

3 Sectral Clustering based on the grah -Lalacian is the grah -Lalacian (which we abbreviate as -Lalacian if no confusion is ossible). Similar to the grah Lalacian we obtain deendent on the choice of the inner roduct the unnormalized and normalized -Lalacian (u) and (n). Let i V then ( (u) f) i = j V w ij φ (f i f j ) ( (n) f) i = w ij φ (f i f j ). d i j V where φ : R R is defined for x R as φ (x) = x sign(x). Note that φ (x) = x so that we recover the standard grah Lalacians for =. In general the -Lalacian is a nonlinear oerator: (αf) α f for α R. 3.. Eigenvalues and eigenvectors of the grah -Lalacian Since our goal is to use the -Lalacian for sectral clustering the natural question arises how one can define eigenvectors and eigenvalues for such a nonlinear oerator. For notational simlicity we restrict us in this section to the case of the unnormalized - Lalacian (u) but all definitions and results carry over to the normalized version (n). Definition 3. The real number λ is called an eigenvalue for the -Lalacian (u) if there exists a function v : V R such that ( (u) v) i = λ φ (v i ) i =... n. cor- The function v is called a -eigenfunction of (u) resonding to the eigenvalue λ. The origin of this definition of an eigenvector for nonlinear oerators lies in the Rayleigh-Ritz rincile a variational characterization of eigenvalues and eigenvectors for linear oerators. For a symmetric matrix A R n n it is well-known that one can obtain the smallest eigenvalue λ () and the corresonding eigenvector v () satisfying Av () = λ () v () via the variational characterization v () = arg min f R n f A f Rn f where the -norm is defined as f := n i= f i. Note that this characterization imlies that (u to rescaling) v () is the global minimizer of f Af subject to f =. This variational characterization can now be carried over to nonlinear oerators. We define for the unnormalized -Lalacian (u) Q (f) := f (u) f = n w ij f i f j ij= and define similarly the functional F : R V R F (f) := Q (f) f. Theorem 3. The functional F has a critical oint at v R V if and only if v is a -eigenfunction of (u). The corresonding eigenvalue λ is given as λ = F (v). Moreover we have F (αf) = F (f) for all f R V and α R. Proof: One can check that the condition for a critical oint of F at v can be rewritten as v Q (v) v φ (v) =. Thus with Definition 3. v is an eigenvector of. Moreover the equation imlies that a given eigenvector v to the eigenvalue λ is a critical oint of F if λ = F (v). Summing u the eigenvector equation of Definition 3. shows this equality. The last statement follows directly from the definition. This theorem shows that in order to get all eigenvectors and eigenvalues of (u) we have to find all critical oints of the functional F. Moreover with F (αf) = F (f) we observe that the usual roerty for linear oerators that eigenvectors are invariant under scaling carries over to the nonlinear case. The following roosition is a generalization of a result by Fiedler (973) to the grah -Lalacian. It relates the connectivity of the grah to roerties of the first eigenvalue λ () of the -Lalacian. We denote by A R V the function which is one on A and zero else. Proosition 3. The multilicity of the first eigenvalue λ () = of the -Lalacian (u) is equal to the number K of connected comonents C... C K of the grah. The corresonding eigensace for λ () = is given as { K i= α i j Ci α i R i =... K}. Proof: We have Q (f) so that all eigenvalues λ of (u) are non-negative. Similar to the case = one can check that n ij= w ij f i f j = if and only if f is constant on each connected comonent. In sectral clustering the grah is usually assumed to be connected so that v () = c for c R otherwise

4 Sectral Clustering based on the grah -Lalacian sectral clustering is trivial. For the following we assume that the grah is connected. The revious roosition suggests that similar to the standard case = we need at least the second eigenvector to construct a artitioning of the grah. For = we get the second eigenvector again by the variational Rayleigh-Ritz rincile v () = arg min f R n { f (u) f f } f =.. This form is not suited for the -Lalacian since its eigenvectors are not necessarily orthogonal. However for a function with f = one has f f = n f = min c R f c. Thus we can write equivalently v () = arg min f R n f (u) f min c R f c This motivates the definition of F () : R V R F () (f) = Q (f) min c R f c. Theorem 3. The second eigenvalue λ () of the grah -Lalacian (u) is equal to the global minimum of the functional F (). The corresonding eigenvector v () of (u) is then given as v () = u c for any global minimizer u of F () where c = n arg min c R i= u i c. Furthermore the functional F () satisfies F () (tu + c) = F () (u) for all t c R. Proof: Can be found in (Bühler & Hein 9). Thus instead of solving the comlicated nonlinear equation of Definition 3. to obtain the second eigenvector of the grah -Lalacian we just have to find the global minimum of the functional F (). In the next section we discuss the relation between the second eigenvalue λ () of the grah -Lalacian and the balanced grah cuts of Section. In Section 5 we rovide an algorithmic framework to comute the second eigenvector of the -Lalacian efficiently. 4. Sectral roerties of the grah -Lalacian and the Cheeger cut Now that we have discussed the variational characterization of the second eigenvector of the -Lalacian we will rovide the relation to the relaxation of balanced grah cut criteria as it can be done for the standard grah Lalacian Sectral relaxation of balanced grah cuts It is well known that the second eigenvector of the unnormalized and normalized standard grah Lalacians ( = ) is the solution of a relaxation of the ratio cut RCut(C C) and normalized cut NCut(C C) see e.g. (von Luxburg 7). We will show now that the second eigenvector v () of the -Lalacian can also be seen as a relaxation of balanced grah cuts. Theorem 4. For > and every artition of V into C C there exists a function f C R V such that the functional F () associated to the unnormalized - Lalacian satisfies (f C ) = cut(c C) F () with the secial cases + C F () (f C ) = RCut(C C) lim F () (f C ) = RCC(C C). Moreover one has F () (f C ) RCC(C C). Equivalent statements hold for a function g C for the normalized cut and the normalized -Lalacian (n). Proof: Let > then we define for a artition C C of V the function f C : V R as (f C ) i = { / i C / C i C. One has Q (f C ) = i C j C +. Moreover one has min c R f C c = f C = +. With F () (f C ) = Q (f C )/min c R f c we get F () (f C ) = w ij + i Cy C C w ij min{ C } = RCC(C C). i Cy C The first equality shows the general result and simlifies to the ratio cut for =. The limit follows with lim α (a α + b α ) /α = max{a b}. Thus since one minimizes over all functions in the eigenroblem for the second eigenvector of the - Lalacian (u) and (n) it is a relaxation of the

5 Sectral Clustering based on the grah -Lalacian ratio/normalized cut for = and for the ratio/normalized Cheeger cut in the limit of. In the interval < < the eigenroblem can be seen as as a relaxation of the interolation between ratio/normalized cut and the ratio/normalized Cheeger cut for the functional F () F () of (u) (f C ) = cut(c C) we get + C which can be understood using the inequalities between l -norms for α β one has x β x α + ( α + ) { } α max α with α = /( ) and thus for < < one has > α >. The sectral relaxation of ratio (Hagen & Kahng 99) and normalized cut (Shi & Malik ) was one of the main motivations for standard sectral clustering. There exist other ossibilities to relax the ratio and normalized cut roblem see (De Bie & Cristianini 6) which lead to a semi-definite rogram. These relaxations give better bounds on the true cut than the standard sectral relaxation ( = ) though they are comutationally exensive. However u to our knowledge the bounds which can be achieved by semidefinite rogramming are not as tight as the ones which we rovide in the next section for the -Lalacian as. 4.. Isoerimetric Inequality - the second eigenvalue λ () and the Cheeger cut The isoerimetric inequality (Chung 997) for the grah Lalacian ( = ) rovides additional theoretical backu for the sectral relaxation. It rovides uer and lower bounds on the ratio/normalized Cheeger cut in terms of the second eigenvalue of the grah - Lalacian. We define the otimal ratio and normalized Cheeger cut values h RCC and h NCC as h RCC = inf RCC(C C) and h NCC = inf NCC(C C). C C The standard isoerimetric inequality for = (see Chung 997) is given as h NCC λ () h NCC where λ () is the second eigenvalue of the standard normalized grah Lalacian ( = ). The isoerimetric inequality for the normalized -Lalacian has been roven by Amghibech (3). Theorem 4. (Amghibech 3) Denote by λ () the second eigenvalue of the normalized -Lalacian (n). Then for any > ( hncc ) λ () h NCC. We extend the result of Amghibech to the unnormalized -Lalacian. Theorem 4.3 Denote by λ () the second eigenvalue of the unnormalized -Lalacian (u). For > ( max i d i ) ( hrcc ) λ () h RCC. Proof: Can be found in (Bühler & Hein 9). Note that h NCC < and hrcc max id i < so that in both cases the left hand side of the bound is smaller than h NCC res. h RCC. When considering the limit one observes that the bounds on λ become tight as. Thus in the limit of the second eigenvalue of the unnormalized/normalized -Lalacian aroximates the otimal ratio/normalized Cheeger cut arbitrarily well. Still the roblem remains how to transform the realvalued second eigenvector of the -Lalacian into a artitioning of the grah. We use the standard rocedure and threshold the second eigenvector v () to obtain the artitioning. The otimal threshold is determined by minimizing the corresonding Cheeger cut. For the second eigenvector v () of the unnormalized grah -Lalacian (u) we determine arg min C t={i V v () (i)>t} RCC(C t C t ) (3) and similarly for the second eigenvector v () of the normalized grah -Lalacian (n) we comute arg min C t={i V v () (i)>t} NCC(C t C t ). (4) The obvious question is how good the cut values obtained by thresholding the second eigenvector of the -Lalacian are comared to otimal Cheeger cut values. The following Theorem answers this question and rovides the key motivation for -sectral clustering. Theorem 4.4 Denote by h RCC and h NCC the ratio/normalized Cheeger cut values obtained by tresholding the second eigenvector v () of the unnormalized/normalized -Lalacian via (3) for (u) res. (4)

6 Sectral Clustering based on the grah -Lalacian Algorithm -Lalacian based Sectral Clustering : Inut: weight matrix W number of desired clusters k choice of -Lalacian. : Initialization: cluster C = V number of clusters s = 3: reeat 4: Minimize F () : R Ci R for the chosen - Lalacian for each cluster C i i =... s. 5: Comute otimal threshold for dividing each cluster C i via (3) for (u) or (4) for (n). 6: Choose to slit the cluster C i so that the total multi-artition cut criterion is minimized (ratio cut () for (u) 7: s s + 8: until number of clusters s = k for (n). Then for > and normalized cut () for (n) ). h RCC h RCC ( max i V d i ) ( h RCC ) h NCC h NCC ( h NCC ). Proof: Can be found in (Bühler & Hein 9). One observes that in the limit of both inequalities become tight which imlies that for the cut found by thresholding the second eigenvector of the -Lalacian converges to the otimal Cheeger cut. 5. -Sectral Clustering The algorithmic scheme for -Sectral Clustering is shown in Algorithm. More than two clusters are obtained by consecutive slitting of clusters until the desired number of clusters is reached. As multi-artition criterion we use the established generalized versions of ratio cut () and normalized cut (). However one could also think about multi-artition versions of the Cheeger cut. The sequential slitting of clusters is the more traditional way to do sectral clustering. Alternatively one uses for the standard grah Lalacian the first k eigenvectors to define a new reresentation of the data. In this new k-dimensional reresentation one then alies a standard clustering algorithm like k-means. This alternative is not ossible in our case since at the moment we are not able to comute higher-order eigenvectors of the -Lalacian. However as Theorem 4.4 shows there is also need for going this way since thresholding will yield the otimal Cheeger cut in the limit. The functional F () : R V R is non-convex and thus we cannot guarantee to reach the global minimum. Indeed a direct minimization for small values of leads often very fast to convergence to a non-otimal local minimum. Thus we use a different rocedure using the fact that for = we can easily comute the global minimizer of F (). It is just the second eigenvector of the standard grah Lalacian which can be efficiently comuted for sarse matrices e.g. using ARPACK. Since the functional F (f) is continuous in we can hoe for close values and that the global minimizer of F () and F () are also close (at least the local minimizer should be close). Moreover it is well known that Newton-like methods have suerlinear convergence close to the local otima (Bertsekas 999). These two facts suggest to solve the roblem F () (u) by minimizing a sequence of functionals F i F () F ()... F () with = > >... > where each ste is initialized with the solution of the revious ste and initialization is done with =. In the exeriments we found that the udate rule t+ =.9 t yields a good trade-off between decreasing too fast with the danger that the otimum for F () t is far away from the otimum of F () t+ and decreasing too slow which yields fast convergence of the Newton method but needs a lot of iterations. The minimimization of the functionals F t is done using a mixture of gradient and Newton stes. However the Hessian of F i is not sarse which causes roblems for large scale roblems but it can be decomosed into H = A + ( ab T + ba T ) + bb T where a b R n and the matrix A is sarse. Thus H is a sum of a sarse matrix lus low-rank udates. Thus we just discard the low-rank udates and use A as a surrogate for the true Hessian. We use the Minimal Residual method (Paige & Saunders 975) for solving the linear system of the Newton ste as the matrix A is symmetric but not necessarily ositive definite. In order to avoid roblems with an illconditioned matrix A we add a small ridge. Note that the term min c R f c () in the functional F (f) is itself a (convex) otimization roblem which can be solved very fast using bisection. 6. Exerimental evaluation In all exeriments we used a symmetric K-NN grah with K = and weights w ij defined as w ij = max{s i (j) s j (i)} where s i (j) = e 4 σ i x i x j with σ i being the Euclidean distance of x i to its K-nearest neighbor. We evaluate the clustering on

7 Sectral Clustering based on the grah -Lalacian datasets with known number of classes k. We then clustered the data into k clusters and checked the agreement of the found clusters C... C k with the class structure using the error measure error(c.. C k ) = V k i= j C i I Yj Y i (5) where Y j is the true label of j and Y i is the dominant label in cluster C i. 6.. High-dimensional noisy two moons The two moons dataset is generated as two half-circles in R which are embedded into a d-dimensional sace where Gaussian noise N ( σ I d ) is added. When varying d n and σ we always made the same observation: unnormalized and normalized -sectral clustering leads for decreasing values of to cuts with decreasing values of the Cheeger cuts RCC and NCC. In Fig. we illustrate this for the case d = n = and σ =.. Note that this dataset is far from being trivial since the high-dimensional noise has corruted the grah (see the edge structure in Fig. ). The histogram of the values of the second eigenvectors for equal to.7.4 and. show strong differences. For = the values are scattered over the interval whereas for =. they are almost concentrated on two eaks. This suggests that for =. the -eigenvector is quite close to the function f C as defined in Theorem 4.. The third row in Fig. shows the resulting clusters found by -sectral clustering with (n) normalized cut and error as. One observes that desite there is some variance the results of -sectral clustering are significantly better than standard sectral clustering.. For the clustering is almost erfect desite the difficulty of this dataset. In order to illustrate that this result is reresentative we have reeated the exeriment times. The lot in the bottom left of Fig. shows the mean of the normalized Cheeger cut the second eigenvalue λ () 6.. UCI-Datasets In Table we show results for -sectral clustering on several UCI datsets both for the unnormalized (right column) and the normalized -Lalacian (left column). The corresonding Cheeger-cuts (second row) are consistently decreasing as. For most of the datasets this also imlies that the ratio/normalized cut decreases. Note that the error is often constant desite the fact that the cut is still decreasing. Oosite to the other examles minimizing the cut does not necessarily lead to a smaller error. Table. To: Results of unnormalized -sectral clustering with k = for USPS and MNIST using the ratiomulti-artition criterion (). In both cases the RCut and the error significantly decrease as decreases. Bottom: confusion matrix for MNIST of the clusters found by - sectral clustering for =.. Class has been slit into two clusters and class 4 and 9 have been merged. Thus there exists no class 9 in the table. Aart from the merged classes the clustering reflects the class structure quite well. USPS MNIST RCut Error RCut Error True/Cluster USPS and MNIST We erform unnormalized -sectral clustering on the full USPS and MNIST-datasets (n = 998 and n = 7). In Table one observes that for the ratio cut as well as the error decreases for both datasets. The error is even misleading since the class searation is quite good but one class has been slit which imlies that two classes have been merged. This haens for both datasets and in Table we rovide the confusion matrix for MNIST for -sectral clustering with =.. For larger values of number of clusters k we thus exect better results. In the following table we resent the runtime behavior (in seconds) for USPS: t As the roblem becomes more difficult which is clear since one aroximates asymtotically the otimal Cheeger cut. However there is still room for imrovement to seed u our current imlementation. Acknowledgments This work has been suorted by the Excellence Cluster on Multimodal Comuting and Interaction at Saarland University.

8 Sectral Clustering based on the grah -Lalacian.6 NCut NCC nd eigenvalue Error NCut NCC nd eigenvalue Error NCut.46 NCC NCut.48 NCC NCut.55 NCC NCut.35 NCC Figure. Results for the two moons data set oints in dimensions noise variance.. First row from left to right: Second eigenvector of the -Lalacian for = Second row: Histogram of the values of the second eigenvector. Last row: Resulting clustering after finding otimal threshold according to the NCC criterion. First column () to: The values of NCC the eigenvalue λ NCut and the error for the examle shown on the right. Middle: Plot of () the edge structure. Bottom: Average values lus standard deviation of NCC NCut λ and the error for varying. Table. Results of unnormalized/normalized -sectral clustering on UCI-datasets. For each dataset the rows corresond to NCut NCC res. RCut RCC and error. Breast Heart Ring norm Two norm Wave form Normalized Unnormalized References Amghibech S. (3). Eigenvalues of the discrete Lalacian for grahs. Ars Combin Bertsekas D. (999). Nonlinear rogramming. Athena Scientific. Bu hler T. & Hein M. (9). Sulementary material. htt:// Publications/BueHei9tech.df. Chung F. (997). Sectral grah theory. AMS. De Bie T. & Cristianini N. (6). Fast SDP relaxations of grah cut clustering transduction and other combinatorial roblems. J. Mach. Learn. Res Fiedler M. (973). Algebraic connectivity of grahs. Czechoslovak Math. J Hagen L. & Kahng A. B. (99). Fast sectral methods for ratio cut artitioning and clustering. Proc. IEEE Intl. Conf. on Comuter-Aided Design 3. Hein M. Audibert J.-Y. & von Luxburg U. (7). Grah Lalacians and their convergence on random neighborhood grahs. J. Mach. Learn. Res Paige C. & Saunders M. (975). Solution of sarse indefinite systems of linear equations. SIAM J. Numer. Anal Shi J. & Malik J. (). Normalized cuts and image segmentation. IEEE Trans. Patt. Anal. Mach. Intell von Luxburg U. (7). A tutorial on sectral clustering. Statistics and Comuting Zhou D. & Scho lkof B. (5). Regularization on discrete saces. Deutsche Arbeitsgemeinschaft fu r Mustererkennung-Symosium ( ).