636 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 5, MAY 2014

Transcription

1 636 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 5, MAY 2014 Constraint Neighborhood Projetions for Sei-Supervised Custering Hongjun Wang, Tao Li, Tianrui Li, and Yan Yang Abstrat Sei-supervised ustering ais to inorporate the known prior knowedge into the ustering agorith. Pairwise onstraints and onstraint projetions are two popuar tehniques in sei-supervised ustering. However, both of the ony onsider the given onstraints and do not onsider the neighbors around the data points onstrained by the onstraints. This paper presents a new tehnique by utiizing the onstrained pairwise data points and their neighbors, denoted as onstraint neighborhood projetions that requires fewer abeed data points (onstraints) and an naturay dea with onstraint onfits. It inudes two steps: 1) the onstraint neighbors are hosen aording to the pairwise onstraints and a given radius so that the pairwise onstraint reationships an be extended to their neighbors, and 2) the origina data points are projeted into a new ow-diensiona spae earned fro the pairwise onstraints and their neighbors. A CNP-Keans agorith is deveoped based on the onstraint neighborhood projetions. Extensive experients on University of Caifornia Irvine (UCI) datasets deonstrate the effetiveness of the proposed ethod. Our study aso shows that onstraint neighborhood projetions (CNP) has soe favorabe features opared with the previous tehniques. Index Ters Constraint neighborhood projetions (CNP), pairwise onstraints, sei-supervised ustering. I. Introdution IN MANY situations when we disover new patterns using ustering, there exists known prior knowedge about the probe. We wish to inorporate the knowedge into the ustering agorith. Reenty, sei-supervised ustering (i.e., ustering with knowedge-based onstraints) has eerged as an iportant variant of the traditiona ustering paradig [1], [2] Current sei-supervised ustering ethods fous on the use of bakground inforation in the for of instane eve Manusript reeived Juy 17, 2012; revised January 19, 2013; aepted May 3, Date of pubiation January 9, 2014; date of urrent version Apri 11, This work was supported in part by the NSFC under Grants , , , and , the State Key Laboratory of Hydrauis and Mountain River Engineering, Sihuan University under Grant 1010, the Fundaenta Researh Funds for the Centra Universities under Grant SWJTU12CX092, and the Constrution Pan for Sientifi Researh Innovation Groups of Leshan Nora University. This paper was reoended by Assoiate Editor P. P. Angeov. H. Wang, T. Li, and Y. Yang are with the Shoo of Inforation Siene and Tehnoogy, Southwest Jiaotong University, Chengdu, Ihuan , China (e-ai: wanghongjun@swjtu.edu.n; tri@swjtu.edu.n; yyang@swjtu.edu.n). T. Li is with the Shoo of Coputer Siene, Forida Internationa University, Miai, FL USA (e-ai: taoi@s.fiu.edu). Coor versions of one or ore of the figures in this paper are avaiabe onine at Digita Objet Identifier /TCYB Fig. 1. Constraint projetion. (a) An exape dataset with onstraints. (b) Resuts of onstraint projetion. ust-ink (ML) and annot-ink (CL) onstraints. An ML onstraint enfores that two instanes ust be paed in the sae uster whie a CL onstraint enfores that two instanes ust not be paed in the sae uster [3]. Typia sei-supervised ustering ethods based on onstraints inude onstrained opete-ink [4], Constrained EM [5], HMRFKeans [6], MPCKeans [7], Kerne ethods [8], [9], [10], [11], [12], [13], [14], and onstraint projetion [15], [16]. Agoriths by axiizing onstraint argin are iustrated in detai by Wang [17] and Zeng et a. [18], [19], respetivey. Spetra agoriths-based parewise onstraints to earn distane etris are stated by the authors in [20], [21]. Pairwise onstraint is aso used for feature seetion [22]. Most existing sei-supervised ustering ethods ony ensure that the given onstraints are onsidered, i.e., pairs of data points with ML (or CL) are not aoated to different usters (or sae usters) or pairs of data points with ML (or CL) are ose to (or distant fro) eah other in the transfored spae or with the earned siiarity easure. They do not onsider the neighbors around the data points onstrained by the given ust/annot-inks. As a resut, in order to ensure the perforane of sei-supervised ustering, generay, a arge nuber of onstraints are needed. There are soe researh efforts reported on weighting propagated onstraints with a Gaussian funtion [23], [24]. However, these ethods are opex and annot work we if the data set is disrete. In addition, they use ots of onstraints (e.g., onstraints were used in their experients whie the nuber of instanes in Iris dataset is ony 150). In this paper, the key otivation is ess training abes and better resuts that are aso the goa of sei-supervised earning. We propose a sei-supervised ustering ethod based on onstraint neighborhood projetions (CNP), where the onstrained pairwise data points and their neighbors are IEEE. Persona use is peritted, but repubiation/redistribution requires IEEE perission. See standards/pubiations/rights/index.ht for ore inforation.

2 WANG et a.: CONSTRAINT NEIGHBORHOOD PROJECTIONS FOR SEMI-SUPERVISED CLUSTERING 637 Fig. 2. Exape iustrates onstraint neighborhood projetions that has four steps. (a) Exape dataset with onstraints. (b) Seeted neighbors. () Neighbor onstraints. (d) Resuts of projetion. used to transfor the input data into a ow-diensiona spae. The proposed ethod requires fewer abeed data points for sei-supervised earning and an naturay dea with the onstraint onfits. Consequenty, our proposed ethod has better generaization apabiity and ore fexibiity than soe state-of-the-art ethods. The rest of the paper is organized as foows. In Setion II, we introdue CNP in detai. A sei-supervised ustering ethod based on CNP is aso proposed. Experienta resuts are presented in Setion III. The paper ends with onusion in Setion IV. II. Constraint Neighborhood Projetions In onstraint projetions (CP), ony the given onstraints are used to deterine the ow-diensiona spae [15], [16]. In our proposed CNP, both the given onstraints and the neighbors of the onstraint points are used to find the transforation. An exape (Figs. 1 and 2) is epoyed to iustrate the differene between CNP and CP. Fig. 1 shows a sape dataset with two ML and one CL onstraints. The dataset is generated fro 2-D Gaussian distributions and is divided into two different usters: the seven points on the eft side of the figure beong to one uster and the eight points on the right side beong to another uster. Fig. 1 presents the resuts of CP. It an be observed fro Fig. 1 that the given onstraints do not propagate to the neighbors in the projetions. Fig. 2 presents the proess of CNP. The origina data points and onstraints are shown in Fig. 2. Aording to the neighborhood size, the seeted onstraint neighbors are shown in Fig. 2. Various derived onstraints of the seeted neighbor points are shown in Fig. 2. Finay, the projetion using a the onstraints (inuding both the input onstraints and the derived onstraints) is shown in Fig. 2. Coparing the two ethods, CNP needs fewer onstraints to ahieve siiar earning outoes as CP. Fig. 3 presents the resuts of CNP on the Wine dataset fro UCI data repository. Fro the figure, it is ear that the uster strutures of data points beoe ore evident after CNP as there are fewer overapping points in the projeted spae. A. Probe Foruation ML onstraints speify that two data points have to be in the sae uster. ML onstraints are transitive [25]. Let ki and Fig. 3. Resuts on the Wine dataset. Different oors are used to show different usters and an ova is used to approxiatey represent the range of a uster. (a) Origina datasets. (b) CNP Resuts. kj be two onneted oponents, and et xi and xj be the instanes in ki and kj, respetivey. Let M be the set of ML onstraints. Then, (xi, xj ) M, xi ki, xj kj (a, b) M, a, b : a ki, b kj. CL onstraints speify that two instanes ust be paed in two different usters and CL onstraints an aso be entaied. Let ki and kj be onneted oponents (opetey onneted subgraphs by ML onstraints), and et xi and xj be the instanes in ki and kj, respetivey. Let C be the set of CL onstraints. Then (xi, xj ) C, xi ki, xj kj (a, b) C, a, b : a ki, b kj. Given a set of p diensiona data X = {x1, x2,..., xn }, the orresponding pairwise ML onstraint set is M = {(xi, xj ) xi ki ; xj ki }, the pairwise CL onstraint set is C = {(xi, xj ) xi ki ; xj kj ; ki = kj }, and the neighbors set of xi is μ = {x ρ xi x 2 ; x X; = (1,..., n)}. CP seeks a set of projetive vetors W = [w1, w2,..., wd ], suh that the pairwise onstraints in C and M are ost faithfuy

3 638 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 5, MAY 2014 preserved in the transfored ower-diensiona representations z i = W T x i. That is, data points invoved by M shoud be ose whie data points invoved by C shoud be far away fro eah other in the ower-diensiona spae. We define the objetive funtion as axiizing J(W) w.r.t. W T W = I, where ( 1 J(W) = W T (x i x j ) 2 + 2n (x i,x j ) C 1 W T (x i x j ) 2) 2n μ (x i,x j ) μ ( 1 W T (x i x j ) 2 + 2n (x i,x j ) M 1 W T (x i x j ) 2). (1) 2n μ (x i,x j ) μ The objetive funtion in (1) an be reforuated in a ore onvenient way as J(W) = 1 2n μ W T (x i x j ) 2 (x i,x j ) C μ 1 2n μ W T (x i x j ) 2, (2) where C μ and M μ are the sets of C and M with their neighbors, respetivey. B. Inferene Ceary, the probe in (2) an be foruated as a typia Eigen-probe. It an be effiienty soved by oputing the eigenvetors and their orresponding eigenvaues. For onveniene, we define S C and S M as foows: S C = 1 2n μ W T (x i x j ) 2, (3) and S M = 1 2n μ W T (x i x j ) 2. (4) Fro (2), to axiize J(W), S C shoud be axiized whie S M shoud be iniized. There exists an anaytia soution to the optiization probe of finding the optia projetion atrix W in (3). We have S C = 1 2n μ W T (x i x j ) 2 = 1 2n μ = 1 2n μ = 1 2n μ W T W T (x 1 x 2 )(x 1 x 2 ) T W (x 1 x 2 )(x 1 x 2 ) T W W T (CC T )W = 1 2n μ W T W,. (5) Aording to (5), S M is S M = 1 2n μ W T W. (6) Fro (5) and (6), the objetion funtion now beoes J(W) = 1 2n μ W T W 1 2n μ W T W = ϑ W T ( ) W W T ( )W. (7) Using the Lagrange utipier optiization tehnique, the Lagrangian an be represented as k L W1,...,W k = J(W 1,..., W k ) δ (W T W 1). (8) By taking the partia derivative of L W1,...,W k with respet to eah W and setting it to zero, we obtain L =2 W 2δ W =0, =1,..., k W =1 W = δ W, =1,..., k. (9) It is ear fro (9) that the soution W is an eigenvetor of and δ is the orresponding eigenvaue of. To axiize J, W ust be the first k eigenvetors of that akes J the su of the k argest eigenvaues of. Suppose W =[W 1,W 2,..., W d ] is the soution to (9), and the orresponding eigenvaues are γ 1 γ 2,..., γ d. Denote = diag(γ 1,γ 2,..., γ d ). Then, the optiization probe beoes hoosing i to Maxiize( γ i ). (10) i Fro (10), if the non-negative eigenvaues of γ i are hosen for the su, it is axiized. After obtaining the onstraint-guided feature projetion as desribed above, we an represent the origina instanes in a ow-diensiona spae that onfors to the uster inforation given in the for of pairwise onstraints. If we add a saing paraeter γ to adjust ratio of onstraints, (2) an be represented as foows: J(W) = 1 2n μ W T x i W T x j 2 γ 2n μ W T x i W T x j 2. (11) Sine, the distane between data points in the sae uster is typiay saer than those in different usters, a saing paraeter is used to baane the ontributions of the two ters and its vaue an be estiated by γ = nμ x i x j 2 x i x j. (12) 2 n μ The objet funtion ontains two ters. One is to ake the average distane in the ower-diensiona spae between instanes invoved by the annot-ink set C as arge as possibe, whie the other is to et the average distanes between

4 WANG et a.: CONSTRAINT NEIGHBORHOOD PROJECTIONS FOR SEMI-SUPERVISED CLUSTERING 639 instanes invoved by the ust-ink set M as sa as possibe. Fro (12), γ > 1 shows that the ust-ink onstraints ontribute the objetion funtion ore and vise versa. At ast γ an be set aording to requireent and an aso be estiated by the two ters of the objet funtion. C. Constraint Confits If a pair (x a,x b ) beongs to C μ and M μ, we a it as a onstraint onfit. CNP an naturay hande this probe. If (x a,x b ) is a onstraint onfit (i.e., it beongs to both C μ and M μ ), we have the foowing objetive funtion J(W) = 1 2n μ W T (x i x j )(x a x b ) }{{} 2 1 2n μ W T (x i x j )(x a x b ) }{{} 2. (13) The ter of (x a x b ) is in both sets of M and C and it is a }{{} onstraint onfit. In order to sove the probe, we suppose (x a x b ) ω, (13) an then be written as J(W) = 1 2n μ W T (x i x j )ω 2 1 2n μ W T (x i x j )ω 2 = ω2 2n μ ω2 2n μ W T (x i x j ) 2 W T (x i x j ) 2. (14) Fro (2) and (14), we find that J(W) J(W) and J(W) = J(W)/ω 2,so J(W) is the sae inferene as J(W). And there are no onstraint onfits in J(W), the onstraint onfits in J(W) are soved by the natura inferene. In another way, ω an be set as a paraeter (e.g., γ in (12)) to baane the ontributions of annot-ink and ust-ink. In this way, the onstraint onfit probe has been addressed. D. Agorith Desription Aording to the above inferene, we design an agorith based on CNP for sei-supervised ustering. The agorith proedure is desried step-by-step as Agorith 1. III. Epiria Study A. Experient Setup In this setion, we perfor experients on 16 rea-word datasets fro UCI ahine earning repository. The nuber of objets, features and asses in eah data set are isted in Tabe II. For evauation, we use iro-preision [26] to Agorith 1: (Constraint Neighborhood Projetions for Sei-supervised K-eans [CNP-Keans]) Input: Training data, {x i,y i } n i=1, where x i is the data points; The set of ust-inks, M; The set of annot-inks, C; The radius of their neighborhood, L. 1) Copute J(w) aording to (5) (7). 2) Copute a of eigenvaues and eigenvetors aording to (8) and (9). 3) Choose the non-negative eigenvaues and orresponding eigenvetors for axiizing J(w). 4) Use eigenvetors orresponding to non-negative eigenvaues to get W. 5) Copute Z = W T X. 6) Run the standard K-eans agorith with input Z. Output: Custer ebership of every point. easure the auray of the uster with respet to the true abes. The iro-preision is defined as MP = k h=1 a h/n, where k is the nuber of usters and n is the nuber of objets, a h denotes the nuber of objets in the uster h that are orrety assigned to the orresponding ass. We identify the orresponding ass for a uster h as the true ass with the argest overap with the uster, and assign a objets in uster h to that ass. Note that 0 MP 1 with 1 indiating the best possibe onsensus ustering, whih has to be in fu agreeent with the ass abes. In our experients, the onstraints are generated as foows: for eah onstraint, one pair of data points are piked out randoy fro exepars of the input data sets (the abes of whih are avaiabe for evauation purpose but unavaiabe for ustering). If the abes of this pair of points are the sae, then a ML onstraint is generated. If the abes are different, a CL onstraint is generated. The aounts of onstraints are deterined by the size of input data. On a the datasets, the experients are perfored 20 ties to eiinate the differene aused by onstraints. B. Perforane Coparison To deonstrate how our ethod works for sei-supervised ustering probe and iproves the ustering perforane, we opare the foowing ethods. 1) MPC-Keans [6]: perforing MPC-Keans on the origina datasets with the nuber of onstraints in Tabe I. 2) CP-Keans [15]: perforing K-eans after onstraints projetions with the nuber of onstraints in Tabe I. 3) CNP-Keans: perforing CNP-Keans after onstraint neighbor projetions with the nuber of onstraints in Tabe I. 4) PCA-Keans: Prinipa oponent anaysis (PCA) is first appied to redue data diensionaity foowed by perforing K-eans ustering. 5) LDA-Keans [27]: adaptive data ustering by integrating K-eans ustering and inear disriinant anaysis

5 640 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 5, MAY 2014 TABLE I Average Auraies Resuts of the Experients Dataset MPC-Keans CP-Keans CNP-Keans PCA-Keans GP-Keans LDA-Keans pia ± ± ± ± ± ± iris ± ± ± ± ± ± wdb ± ± ± ± ± ± wine ± ± ± ± ± ± ionosphere ± ± ± ± ± ± gass ± ± ± ± ± ± bupa ± ± ± ± ± ± baane ± ± ± ± ± ± Kdd99sub ± ± ± ± ± ± spetheart ± ± ± ± ± ± Hepatitis ± ± ± ± ± ± donationsub ± ± ± ± ± ± MiniBooNE ± ± ± ± ± ± bank ± ± ± ± ± ± shutte ± ± ± ± ± ± agi ± ± ± ± ± ± Average ± ± ± ± ± ± TABLE II Nuber of the Instanes, Features, Casses and Constraints Used in Eah Dataset Dataset Charateristi Instanes Features Categories Constraints pia rea iris rea wdb rea wine rea ionosphere rea gass rea bupa disrete baane disrete kdd99sub disrete spetheart rea hepatitis rea donationsub1 ixed iniboone rea bank ixed shutte rea agi rea (LDA) where K-eans ustering is used to generate uster abes and LDA is used to perfor subspae seetion. 6) GP-Keans [23]: perforing GP-Keans on the origina datasets with the nuber of onstraints in Tabe I. The perforane oparison is shown in Tabe I. We an see that CNP-Keans ahieves the best average MP with the vaue of , and CNP-Keans outperfors other ustering agoriths for ost of the ties, whih is obviousy shown on the arge datasets. Aong a the agoriths on the 16 datasets, it is 11 ties that CNP-Keans ahieves the best resuts, whie the other five agoriths ony ahieve six best resuts. To ake a arefu oparison of these agoriths, we perfor a 1 n oparison by eans of the aigned Friedan test [28]. The proposed ethod CNP-Keans is the ontro ethod. Tabe III dispays the aigned observations and the aigned ranks in the parentheses onsidering the known 6 agoriths and 16 data sets. On average, CNP-Keans ranks the first with a rank of ; CP-Keans ranks the seond with a vaue of ; GP-Keans ranks the third with ; PCA-Keans ranks the fourth with ; LDA- Keans ranks the fifth; and MPC-Keans ranks the ast. The Friedan aigned rank test heks whether the easured su of aigned ranks is signifianty different fro the tota aigned ranks R j = 776 expeted under the nu hypothesis: k R 2.,j = j= = , k R 2 i,. = j=1 = , {(6 1)( ( /4)(6 16+1) 2 )} T = ((6 16( )( )) )/6 = With six agoriths and 16 data sets, T is distributed aording to the hi-square distribution with 6 1 = 5 degrees of freedo. The p-vaue oputed by using the χ 2 (5) distribution are (for one taied test) and (for two taied test). Then, the nu hypothesis is rejeted at a high eve of signifiane. We an see the vaues are far ess than 0.05 whih shows that the resuts of agoriths are signifianty different. C. Paraeter Tuning We aso report the experient resuts on the radius of onstraint neighborhood. In this experient, the sae set of onstraints is used but the radius of their neighborhoods is inreasing graduay. In both Figs. 4 and 5, the x oordinate is defined as radius =, where L is the distane MAX(L) aong any two data points. Aong the 16 datasets, ost

6 WANG et a.: CONSTRAINT NEIGHBORHOOD PROJECTIONS FOR SEMI-SUPERVISED CLUSTERING 641 TABLE III Aigned Observations of Six Agoriths Seeted in the Experienta Study Dataset pia iris wdb wine ionosphere gass bupa baane Kdd99sub spetheart Hepatitis donationsub1 iniboone bank shutte agi Tota Average rank MPC-Keans (87) (66.5) (2) (73.5) (52) (56) (85) (90.5) (15) (70.5) (59) (63) (48.5) (90.5) (54) CP-Keans (62) (50.5) (96) (3) (29) (19) (8) (94) (18) (42) (43) (58) (39) (57) (45) CNP-Keans (33) (37) (93) (73.5) (20.5) (39) (7) (1) (20.5) (30) (16) (10) (23) (6) (11) PCA-Keans (33) (50.5) (95) (82) (64) (39) (86) (12.5) (81) (70.5) (36) (44) (48.5) (9) (54) GP-Keans (33) (66.5) (4.5) (77.5) (46) (41) (83) (88) (22) (31) (61) (65) (47) (75) (54) LDA-Keans (14) (17) (92) (4.5) (77.5) (79) (84) (69) (12.5) (89) (35) (72) (60) (80) (68) (76) Tota The ranks in the parentheses are used in the oputation of the friedan aigned ranks test. The saest one is the best. Fig. 4. Experienta resuts with the inreasing radius of onstraint neighborhood. In this experient the sae onstraints are used but the radius of their neighborhood are inreasing graduay.

7 642 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 5, MAY 2014 of a the pairwise onstraints. It is ear that using ony about 1% of a the pairwise onstraints the axiu MPs are obtained in the CNP, whie CP requires about 10% of a the onstraints. In other words, CP requires as any as 10 ties the nuber of onstraints needed in CNP. Therefore, one iportant feature of our proposed CNP is that it requires ess abeed inforation. IV. Conusion Fig. 5. Average MP of 16 datasets with the inreasing radius of onstraint neighborhood. In this experient, the sae onstraints are used but the radius of their neighborhood are inreasing graduay. In this paper, a nove approah, CNP, was proposed for sei-supervised ustering. To the best of our knowedge, this is the first work in sei-supervised ustering on using pairwise onstraints and their neighbors for projetion to a ow-diensiona spae. CNP uses onstraint neighborhoods to guide the projetion. The proposed ethod opares favoraby over severa state-of-the-art ethods. In other words, it requires fewer onstraints and an aso dea with onstraint onfits. Based on CNP, we design a CNP-Keans agorith and the experienta resuts deonstrate its effetiveness. In our future work, we wi fous on the investigation of better onstraints seetion and radius seetion for sei-supervised ustering. Aknowedgent The authors woud ike to thank the anonyous reviewers for their hepfu oents and suggestions. Referenes Fig. 6. Max auray vs. nuber of onstraints. (a the datasets exept two of Bupa and Ionosphere) of their ustering auray resuts are graduay inreasing with the inreasing size of the radius. In genera, the auray wi reah axiu when the radius reahes 0.2. However, when the radius ontinues to inrease, the auray resuts wi draatiay deine. This is beause when the radius beoes arge to soe degree, the ust-ink onstraint neighborhood ay overap with the annot-ink onstraint neighborhood eah other, and the orresponding neighborhood ay ontain data points fro different usters and thus generate soe onfiting onstraints. As a resut, the ustering perforane ay be degraded. In genera, the auray of ustering inreases with the radius beoing arge between 0 and 0.2, whie the auray dereases with the radius being ore than o.2. D. Nuber of Constraints Under Maxiu MP In this subsetion, we report the resuts on the nuber of onstraints that are needed in the CP and CNP to ahieve the axiu MPs. The resuts suarizing over the 16 datasets are shown in Fig. 6 where the Y -axis is ranging fro the saest MP to the argest and the X-axis shows the perentage [1] S. Basu, I. Davidson, and K. L. Wagstaff, Constrained Custering. Boa Raton, FL, USA: CRC Press, [2] O. Chapee, A. Zien, and B. Shokopf, Sei-Supervised Learning. Cabridge, MA, USA: MIT Press, [3] K. Wagstaff, C. Cardie, S. Rogers, and S. Shroed, Constrained Keans ustering with bakground knowedge, in Pro. ICML, 2001, pp [4] D. Kein, S. D. Kavar, and C. D. Manning, Fro instane-eve onstraints to spae-eve onstraints: Making the ost of prior knowedge in data ustering, in Pro. ICML, 2002, pp [5] N. Shenta, A. Bar-Hie, T. Hertz, and D. Weinsha, Coputing gaussian ixture odes with e using equivaene onstraints, in Pro. NIPS, [6] S. Basu, M. Bienko, and R. J. Mooney, A probabiisti fraework for sei-supervised ustering, in Pro. KDD, 2004, pp [7] M. Bienko, S. Basu, and R. J. Mooney, Integrating onstraints and etri earning in sei-supervised ustering, in Pro. ICML, 2004, pp [8] B. Kuis, S. Basu, I. Dhion, and R. J. Mooney, Seisupervisedgraphustering: A kerne approah, in Pro. ICML, 2005, pp [9] B. Yan and C. Doenioni, An adaptive kerne ethod for seisupervised ustering, in Pro. ECML, 2006, pp [10] D. Y. Yeung and H. Chang, A kerne approah for sei-supervised etri earning, IEEE Trans. Neura Netw., vo. 18, no. 1, pp , Ju [11] S. C. H. Hoi, R. Jin, M. R. Lyu, and J. Wu, Learning nonparaetri kerne atries fro pairwise onstraints, in Pro. ICML, 2007, pp [12] O. Masayuki and Y. Seiji, Learning siiarity atrix fro onstraints of reationa neighbors, J. Adv. Coput. Inte. Inte. Infor., vo. 14, no. 4, pp , [13] X. Yin, S. Chen, and E. H. D. Zhang, Sei-supervised ustering with etri earning: An adaptive kerne ethod, Pattern Reognit., vo. 43, no. 4, pp , 2010.

8 WANG et a.: CONSTRAINT NEIGHBORHOOD PROJECTIONS FOR SEMI-SUPERVISED CLUSTERING 643 [14] C. Doenioni, J. Peng, and B. Yan, Coposite kernes for seisupervised ustering, Know. Infor. Syst., vo. 28, pp , Aug [15] W. Tang, H. Xiong, S. Zhong, and J. Wu, Enhaning sei-supervised ustering: A feature projetion perspetive, in Pro. KDD, 2007, pp [16] D. Zhang, S. Chen, Z. Zhou, and Q. Yang, Constraint projetions for ensebe earning, in Pro. AAAI, 2008, pp [17] F. Wang, Seisupervised etri earning by axiizing onstraint argin, IEEE Trans. Neura Netw., vo. 41, no. 4, pp , Aug [18] H. Zeng and Y.-M. Cheung, Sei-supervised axiu argin ustering with pairwise onstraints, IEEE Trans. Know. Data Eng., vo. 24, no. 5, pp , May [19] A. Mignon and F. Jurie, Pa: A new approah for distane earning fro sparse pairwise onstraints, in Pro. CVPR, 2012, pp [20] W. Chen and G. Feng, Spetra ustering: A sei-supervised approah, Neurooputing, vo. 77, no. 1, pp , Jan [21] F. Shang, Y. Liu, and F. Wang, Learning spetra ebedding for seisupervised ustering, in Pro. ICDM, 2011, pp [22] D. Zhang, S. Chen, and Z. Zhou, Constraint sore: A new fiter ethod for feature seetion with pairwise onstraints, Pattern Reognit., vo. 41, no. 5, pp , May [23] E. R. Eaton, Custering with Propagated Constraints, Thesis, Univ. Maryand, Coege Park, MD, USA, [24] J. Huang and H. Sun, Lighty-supervised ustering using pairwise onstraint propagation, in Pro. 3rd Int. Conf. Inte. Syst. Know. Eng., 2008, pp [25] Z. Li, J. Liu, and X. Tang, Pairwise onstraint propagation by seidefinite prograing for sei-supervised assifiation, in Pro. ICML, 2008, pp [26] Z. H. Zhou and W. Tang, Custerer ensebe, Knowedge-Based Syst., vo. 19, no. 1, pp , [27] C. Ding and T. Li, Adaptive diension redution using disriinant anaysis and K-eans ustering, in Pro. ICML, 2007, pp [28] S. Gara, A. Fernndez, J. Luengo, and F. Herrera, Advaned nonparaetri tests for utipe oparisons in the design of experients in oputationa inteigene and data ining: Experienta anaysis of power, Infor. Si., vo. 180, no. 10, pp , May Tao Li reeived the Ph.D. degree in oputer siene fro the Departent of Coputer Siene, University of Rohester, Rohester, NY, USA, in He is urrenty an Assoiate Professor at the Shoo of Coputing and Inforation Sienes, Forida Internationa University, Miai, FL, USA. His urrent researh interests inude data ining, oputing syste anageent, inforation retrieva, and ahine earning. Dr. Li was a reipient of NSF CAREER Award and utipe IBM Fauty Researh Awards. Tianrui Li reeived the B.S., M.S., and Ph.D. degrees fro Southwest Jiaotong University, Chengdu, China, in 1992, 1995, and 2002, respetivey. Fro 2005 to 2006, he was a Post-Dotora Researher at SCK*CEN, Begiu, and a Visiting Professor at Hasset University, Begiu, in 2008, and the University of Tehnoogy, Sydney, Austraia, in He is urrenty a Professor and the Diretor at the Key Laboratory of Coud Coputing and Inteigent Tehniques, Southwest Jiaotong University. Sine 2000, he has o-edited three textbooks, ten proeedings, five speia issues of internationa journas and pubished over 100 researh papers in refereed journas and onferenes. His urrent researh interests inude data ining and knowedge disovery, granuar oputing and rough sets, oud oputing, traffi inforation engineering and ontro. Dr. Li has been the Vie Chair of the IEEE CIS Chengdu Chapter sine 2011 and the Area Editor of the Internationa Journa of Coputationa Inteigene Systes (IJCIS, SCI indexed). He has served as ISKE2007, ISKE2008, ISKE2009, ISKE2010, ISKE2011, JRS2012, ISKE2012 progra hairs, IEEE GrC 2009 Progra Vie Chair and RSKT2008, FLINS2010 Organizing Chair. He has been a reviewer for severa eading aadei journas. aadei journas. Hongjun Wang reeived the Ph.D. degree in oputer siene fro Sihuan University, Chengdu, China, in He is urrenty an Assoiate Professor of the Key Laboratory of Coud Coputing and Inteigent Tehniques, Southwest Jiaotong University, Chengdu, China. His urrent researh interests inude ahine earning, data ining, and ensebe earning. He has pubished over 30 researh papers in journas and onferenes and he is a eber of ACM and CCF. He has been a reviewer for severa Yan Yang reeived the Ph.D. degree in traffi inforation engineering and ontro fro Southwest Jiaotong University of Chengdu, China, in She is urrenty a Professor and Assoiate Dean of the Shoo of Inforation Siene and Tehnoogy, Southwest Jiaotong University. Her urrent researh interests inude data ining, oputationa inteigene, and ensebe earning. She has been the Vie Chair of the ACM Chengdu Chapter sine 2012, eber of IEEE-CS sine 2012, and senior eber of CCF sine 2010.