Correlation Clustering with Noisy Input

Transcription

1 Correlaton Clsterng wth Nosy Inpt Clare Mathe Warren Schdy Brown Unversty SODA 2010

2 Nosy Correlaton Clsterng Model Unknown base clsterng B of n obects Nose: each edge s controlled by an adversary wth probablty p and tells the trth otherwse Problem: reconstrct B from the edge labels I n p t B = = = =

3 One of or reslts Theorem: assme p 1/3. If all clsters have sze at least 1 then the natral sem-defnte program (SDP) recovers B exactly wth hgh probablty. Prevos best: 2 n log n [Bansal, Blm, Chawla 04, Shamr and Tsr 07], combnatoral. See paper for other reslts (ncldng approxmaton algorthms) n

4 Plan The sem-defnte program Its dal Usng the dal

5 Clsterngs Clsterngs are represented by 0/1 matrces: =1: and n same clster In general a clsterng satsfes: E.g. v k k v T k for some 0/1 orthogonal vectors v, v2,..., v one per clster v cat v dog : : 1 m ,

6 Relaxaton Clsterng Relaxaton of clsterngs =1 for all 0 for all, The followng are eqvalent ( symmetrc): T vkvk for some 0/1 vectors v1, v2,..., v k T vkvk for some vectors v1, v2,..., v k s postve sem-defnte (p.s.d.) m m

7 Obectve Maxmze nmber of agreements: max f 1 f Drop the constant = I.e. max E where E 1 f = -1 f

8 Smmary of SDP max p.s.d. E s.t. Ths SDP was prevosly sed by: [Charkar, Grswam, Wrth 05] [Swamy 04] 1 0 E = = = =

9 Dscsson Algorthm: Solve SDP If ntegral, otpt t. Otherwse fal. Thm: assme p 1/3. If all clsters have sze at least 1 n then the SDP recovers B exactly wth hgh probablty. An example matrx from solver n [Elsner and Schdy 09]. That solver scales to a few thosand obects.

10 Plan The sem-defnte program Its dal Usng the dal

11 Translate SDP nto LP The followng are eqvalent ( symmetrc): postve sem-defnte SDP agan: max T 1 0 p.s.d. 0 for all E vectors s.t. max LP form: T Lnear n for fxed E s.t for all vectors

12 SDP Dal Prmal: Dal: 0, a for all ) 1( s.t. mn h E h d a d E T 0 for all, 0 for all 1for all s.t. max a h d

13 Translate dal LP nto SDP The followng are eqvalent ( symmetrc): a wth a 0 T k k k postve sem-defnte Dal agan: mn a d s.t. d 1( ) h a, h E 0 Matrx form: mn Trace ( D) E D H a D H s.t. dagonal 0 0 Arbtrary postve semdefnte matrx a T

14 The Dal SDP mn Trace( D) s.t. E D H postve sem- defnte D dagonal H 0

15 Plan The sem-defnte program Its dal Usng the dal Ths proof s nspred by a smlar reslt for the planted clqe problem [Fege and Krathgamer 00].

16 Usng the dal - overvew Prove optmalty of the base clsterng by presentng dal solton (D,H) whose vale matches vale of base clsterng B (see paper) Dffclt part: provng that E D H s p.s.d. The followng are eqvalent (Y symmetrc): Y postve sem-defnte All egenvales of Y are 0 We present b egenvectors wth egenvale 0 (see paper), where b s the nmber of clsters n B We prove that the b+1 th smallest egenvale, denoted, s postve (sketched next) b1 E D H Hence all egenvales of E D H are 0

17 Egenvale analyss E D H M M M M b 1 (mn clster sze) (see paper) 1 ( n) (next) We apply the followng: Theorem [Weyl]: If M and N are symmetrc matrces then M N) ( M) ( ) b1( 1 b1 N Hence for sffcently large mn clster sze b E D H. 1 0

18 Random matrces Theorem [Füred and Komlós 81]: Let M be a random symmetrc matrx wth ndependent entres of mean zero. Then wth hgh probablty Applcaton: 1 M E Expectaton E n 2 M O n for all. 1 To analyze M 3 we developed a generalzaton of ths theorem that handles lmted dependence between the entres.

19 Recap Theorem: assme p 1/3. If all clsters have sze at least then the SDP recovers B exactly wth hgh probablty. Proof: We wrote a dal solton matrx as a sm of 4 random matrces, sed Füred-Komlós varants to bond ther egenvales, sed Weyl to nfer bond on egenvales of the matrx, hence p.s.d., hence solton s feasble. That solton has vale eqal to the vale of B, hence by dalty B s prmal optmal B s the nqe prmal optmm (see paper), hence SDP wll exactly retrn B Hence algorthm reconstrcts B exactly when all clsters have sze at least n. 1 1 n

20 Sppose some clsters are sze c 3 n and others are sze 1. Can the SDP be sed to reconstrct the large clsters? Open Qeston 1 Software: [Elsner and Schdy 09].

21 Open Qeston 2 Planted clqe problem = correlaton clsterng wth only one non-sngleton and no corrpton of wthn-clster edges Exst polynomal-tme algorthm when clqe sze = c 1 n O(log n) Exsts n -tme algorthm when clqe sze = c1 log n Can polynomal-tme algorthms beat the n barrer? c 1

22 Clsterng References Nr Alon, Moses Charkar, and Alantha Newman. Aggregatng nconsstent nformaton: rankng and clsterng. In STOC 05, pages , Nkhl Bansal, Avrm Blm, and Shch Chawla. Correlaton clsterng. Mach. Learn., 56(1-3):89 113, Moses Charkar, Venkatesan Grswam, and Anthony Wrth. Clsterng wth qaltatve nformaton. J. Compt. Syst. Sc., 71(3): , M. Elsner and W. Schdy. Bondng and Comparng Methods for Correlaton Clsterng Beyond ILP. In ILP-NLP 09: Proc. NAACL/HLT 2009 Workshop on Integer Lnear Programmng for Natral Langage Processng, pages 19 27, Ron Shamr and Dekel Tsr. Improved algorthms for the random clster graph model. Random Strctres and Algorthms, 31(4): , 2007.

23 Other References F. Alzadeh. Interor pont methods n semdefnte programmng wth applcatons to combnatoral optmzaton. SIAM Jornal on Optmzaton, 5(1):13 51, 1995 Urel Fege and Robert Krathgamer. Fndng and certfyng a large hdden clqe n a semrandom graph. Random Strct. Algorthms, 16(2): , 2000 Zoltán Füred and János Komlós. The egenvales of random symmetrc matrces. Combnatorca, 1(3): , 1981