Density Evolution in the Degree-correlated Stochastic Block Model

Transcription

1 JMLR: Workshop and Conference Proceedings vol 9:1 3, 016 Densiy Evoluion in he Degree-correlaed Sochasic Block Model Elchanan Mossel Deparmen of Saisics, The Wharon School, Universiy of Pennsylvania, Philadelphia, PA Jiaming Xu Simons Insiue for he Theory of Compuing, Universiy of California, Berkeley, Berkeley, CA Absrac There is a recen surge of ineres in idenifying he sharp recovery hresholds for cluser recovery under he sochasic block model. In his paper, we address he more refined quesion of how many verices ha will be misclassified on average. We consider he binary form of he sochasic block model, where n verices are pariioned ino wo clusers wih edge probabiliy a/n wihin he firs cluser, c/n wihin he second cluser, and b/n across clusers. Suppose ha as n, a = b + µ b, c = b + ν b for wo fixed consans µ, ν, and b wih b = n o(1). When he cluser sizes are balanced and µ ν, we show ha he minimum fracion of misclassified verices on average is given by Q( v ), where Q(x) is he Q-funcion for sandard normal, v is he unique fixed poin of v = (µ ν) 16 + (µ+ν) 16 E[anh(v + vz)], and Z is sandard normal. Moreover, he minimum misclassified fracion on average is aained by a local algorihm, namely belief propagaion, in ime linear in he number of edges. Our proof echniques are based on connecing he cluser recovery problem o ree reconsrucion problems, and analyzing he densiy evoluion of belief propagaion on rees wih Gaussian approximaions. Keywords: Belief propagaion, Densiy evoluion, Communiy deecion 1. Inroducion The problem of cluser recovery under he sochasic block model has been inensely sudied in saisics Holland e al. (193); Snijders and Nowicki (1997); Bickel and Chen (009); Cai and Li (01); Zhang and Zhou (015); Gao e al. (015), compuer science (where i is known as he planed pariion problem) Dyer and Frieze (199); Jerrum and Sorkin (199); Condon and Karp (001); McSherry (001); Coja-Oghlan (005, 010); Chen e al. (01); Anandkumar e al. (01); Chen and Xu (01), and heoreical saisical physics Decelle e al. (011a); Zhang e al. (01); Decelle e al. (011b). In he simples binary form, he sochasic block model assumes ha n verices are pariioned ino wo clusers wih edge probabiliy a/n wihin he firs cluser, c/n wihin he second cluser, and b/n across he wo clusers. The goal is o reconsruc he underlying clusers from he observaion of he graph. Differen reconsrucion goals can be considered depending on how he model parameers a, b, c scale wih n (See Abbe and Sandon (015) for more discussions): Exac recovery (srong consisency). If he average degree is Ω(log n), i is possible o exacly recover he clusers (up o a permuaion of cluser indices) wih high probabiliy. In he case wih wo equal-sized clusers, and a = c = α log n/n and b = β log n/n for wo fixed α, β > 0, a sharp exac recovery hreshold ( α β) has been found in Mossel e al. (015b); Abbe e al. (01) and i is furher shown ha semi-definie programming can achieve he sharp hreshold in Hajek e al. (01); Bandeira (015). The hreshold for c 016 E. Mossel & J. Xu.

2 MOSSEL XU wo unequal-sized clusers is proved in Hajek e al. (015a). Exac recovery hreshold wih a fixed number of clusers has been idenified in Hajek e al. (015a); Yun and Prouiere (01a); Agarwal e al. (015), and more generally in Abbe and Sandon (015); Perry and Wein (015) wih heerogeneous cluser sizes and edge probabiliies. Weak recovery (weak consisency). If he average degree is Ω(1), one can hope for misclassifying only o(n) verices wih high probabiliy, which is known as weak recovery or weak consisency. In he seing wih wo approximaely equal-sized clusers and a = c, i is shown in Yun and Prouiere (01b); Mossel e al. (015b) ha weak recovery is possible if and only (a b) /(a + b). Correlaed recovery (non-rivial deecion). If he average degree is Θ(1), exac recovery or weak recovery becomes hopeless as he resuling graph under he sochasic block model will have a leas a consan fracion of isolaed verices. Moreover, i is easy o see ha even verices wih consan degree canno be labeled accuraely given all he oher verices labels are revealed. Thus one goal in he sparse graph regime is o find a pariion posiively correlaed wih he rue one (up o a permuaion of cluser indices), which is also called nonrivial deecion. In he seing wih wo approximaely equal-sized clusers and a = c, i was firs conjecured in Decelle e al. (011a) and laer proven in Mossel e al. (015a, 013b); Massoulié (01) ha correlaed recovery is feasible if and only if (a b) > (a + b). A specral mehod based on he non-backracking marix is recenly shown o achieve he sharp hreshold in Bordenave e al. (015). In pracice, one may be ineresed in he finer quesion of how many verices ha will be misclassified on expecaion or wih high probabiliy. In he wo equal-sized clusers seing, previous resuls on exac recovery, weak recovery, and correlaed recovery provide condiions under which he minimum fracion of misclassified verices on average is o(1/n), o(1), and sricly smaller han 1/, respecively. By assuming (a b) /(a + b), recen work Zhang and Zhou (015); Gao e al. (015) showd ha he expeced misclassified fracion decays o zero exponenially fas and gives a sharp characerizaion of he decay exponen under a minimax framework. However, all hese previous resuls do no shed ligh on he imporan quesion of when i is possible o misclassify only ɛ fracion of verices on expecaion, for any finie ɛ (0, 1/). To he bes of our knowledge, i is an open problem o find a closed-form expression of he expeced misclassified fracion in erms of he model parameers. In his paper, we give such a simple formula in he special case of wo approximaely equal-sized clusers. Specifically, suppose ha a = b + bµ, c = b + bν, b, b = n o(1), (1) for wo fixed consans µ, ν. We furher assume ha µ ν so ha he verex degrees are saisically correlaed wih he cluser srucure, and hence he name of he degree-correlaed sochasic block model. We show ha he minimum fracion of misclassified verices on average is given by Q( v ), where Q(x) is he Q-funcion for sandard normal, v is he unique fixed poin of v = (µ ν) 16 + (µ+ν) 16 E [anh(v + vz)], and Z is sandard normal. Moreover, he minimum expeced misclassified fracion can be aained by a local algorihm, namely belief propagaion (BP) algorihm (See Algorihm 1), in ime O(nb ). The local belief propagaion algorihm can be viewed as an ieraive algorihm which improves on he misclassified fracion on average sep by sep; running belief propagaion for one ieraion reduces o he simple hresholding algorihm

3 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL based on verex degrees. I is crucial o assume µ ν for he above resuls o hold, oherwise i is well-known (see e.g. Kanade e al. (01)) ha no local algorihm can even achieve he non-rivial deecion. Neverheless, under a slighly sronger assumpion ha b and b = o(log n), we show ha if µ = ν wih µ >, local belief propagaion combined wih a global algorihm capable of non-rivial deecion when µ >, aains he minimum expeced misclassified fracion Q( v) in polynomial-ime, where v is he larges fixed poin of v = µ E [anh(v + vz)]. When he clusers sizes are unbalanced, i.e., one cluser is of size approximaely ρn for ρ (0, 1/), we give a lower bound on he minimum expeced misclassified fracion, and an upper bound aained by he local belief propagaion algorihm. However, we are unable o prove ha he upper bound maches he lower bound. In fac, numerical experimens sugges ha here exiss a gap beween he upper and lower bound when he cluser sizes are very unbalanced, i.e., ρ is close o 0. Our proofs are mainly based on wo useful echniques inroduced in previous work. Firs, in he regime (1), he observed graph is locally ree-like, so we connec he cluser recovery problem o reconsrucion problems on rees, and for he ree problems, he opimal esimaor can be compued by belief propagaion algorihm. Such connecion has been invesigaed before in Mossel e al. (015a, 013a); Mossel and Xu (015). Second, we characerize he densiy evoluion of belief propagaion on rees wih Gaussian approximaions, and as a resul, we ge a recursion wih he larges fixed poin corresponding o a lower bound on he minimum expeced misclassified fracion, and he smalles fixed poin correspond o he expeced misclassified fracion aained by he local belief propagaion algorihm. Densiy evoluion has been widely used for he analysis of muliuser deecion Monanari and Tse (006) and sparse graph codes Richardson and Urbanke (00); Mezard and Monanari (009), and more recenly has been inroduced for he analysis of finding a single communiy in a sparse graph Monanari (015). As a final piece, we prove ha in he balanced cluser case, he recursion has a unique fixed poin using he ideas of symmeric random variables Richardson and Urbanke (00); Monanari (005) and he firs-order sochasic dominance, hus esablishing he ighness of he lower bound and he opimaliy of he local BP simulaneously. We poin ou ha local algorihm by iself is a hriving research area (see Lyons and Nazarov (011); Haami e al. (01); Gamarnik and Sudan (01) and he references herein). Inuiively, local algorihms are one ype of algorihms ha make decision for each verex jus based on he neighborhood of small radius around he verex; hese algorihms are by design easy o run in a disribued fashion. Under he conex of communiy deecion, local algorihms deermine which communiy each verex lies in jus based on he local neighborhood around each verex (see Monanari (015) for a formal definiion). Recen work Mossel and Xu (015) shows ha wih he aid of exra noisy label informaion on cluser srucure, he local algorihms can be opimal in minimizing he expeced misclassified fracion in he sochasic block model. In comparison, we show ha when he verex degrees are correlaed wih he cluser srucure, he local algorihms can be opimal even wihou he exra noisy label informaion. In closing, we compare our resuls wih he recen resuls in Monanari (015); Hajek e al. (015b), which sudied he problem of finding a single communiy of size ρn in a sparse graph. When ν = 0, i.e. b = c, he sochasic block model considered in his paper, specializes o he single communiy model sudied in Monanari (015), and he recursion of densiy evoluion derived in his paper reduces o he recursion derived in (Monanari, 015, Eq. (36)). I is shown in Monanari (015); Hajek e al. (015b) ha he local algorihm is sricly subopimal comparing o he global exhausive search when ρ 0. In conras, we show ha if ρ = 1/, he local algorihm is opimal 3

4 MOSSEL XU in minimizing he expeced fracion of misclassified verices as long as µ ν, and give a sharp characerizaion of he minimum expeced misclassified fracion. Parallel Independen Work Zhang e al. (015) independenly sudied he problem of cluser recovery under he degree-correlaed sochasic block model wih muliple clusers. Based on he caviy mehod and numerical simulaions, i is shown ha wih a mos four clusers of unequal sizes bu same in and ou degrees, he non-rivial deecion hreshold phenomenon disappears, making he minimum fracion of misclassified verices on average a coninuous funcion of model parameers. In comparison, in he regime (1) wih wo equal-sized clusers and µ ν, we give a more precise answer, showing ha he fracion of misclassified verices on average is Q( v ), where v is he unique fixed poin of v = (µ ν) 16 + (µ+ν) 16 E [anh(v + vz)]. Moreover, i is shown in Zhang e al. (015) ha wih more han four clusers of unequal sizes, here exiss a regime where wo sable fixed poins coexis, wih he smaller one corresponding o he performance of local belief propagaion, and he larger one corresponding o he performance of belief propagaion iniialized based on he rue cluser srucure. We find ha he same phenomenon also happens in he case of wo clusers wih very unbalanced sizes and differen in and ou degrees (See Secion. for deails). We recenly became aware of he work Deshpande e al. (015), who sudied he problem of cluser recovery under he sochasic block model in he symmeric seing wih wo equal-sized (a b) clusers and a = c. By assuming ha (a+b)(1 (a+b)/n) µ for a fixed consan µ and (a + b)(1 (a + b)/n), a sharp characerizaion of he per-verex muual informaion beween he verex labels and he graph is given in erms of µ and v, where v is he larges fixed poin of v = µ E [anh(v + vz)]. In comparison, we show ha he minimum fracion of verices misclassified on expecaion is given by Q( v) and i is aainable in polynomial ime wih an addiional echnical assumpion ha b = o(log n). Ineresingly, he poin (a) of Lemma 6.1 in Deshpande e al. (015) is a special case of Lemma 1 wih ρ = 1/ in our paper. The proof of Lemma 6.1 given in Deshpande e al. (015) and he proof of Lemma 1 given in his paper are similar: boh used he ideas of symmeric random variables Richardson and Urbanke (00); Monanari (005). One sligh difference is ha o prove he concaviy of he mapping in he recursion when ρ = 1/, we used he firs-order sochasic dominance, while Deshpande e al. (015) compues he second-order derivaive.. Model and Main Resuls We consider he binary sochasic block model wih n verices pariioned ino wo clusers, where each verex is independenly assigned ino he firs cluser wih probabiliy ρ (0, 1) and he second cluser wih probabiliy ρ 1 ρ. 1 Each pair of verices is conneced independenly wih probabiliy a/n if wo verices are in he firs cluser, wih probabiliy c/n if hey are in he second cluser, and wih probabiliy b/n if hey are in wo differen clusers. Le G = (V, E) denoe he observed graph and A denoe he adjacency marix of he graph G. Le σ denoe he underlying verex labeling such ha σ i = + if verex i is in he firs cluser and σ i = oherwise. The model parameers {ρ, a, b, c 1. Noice ha he cluser sizes are random and concenrae on ρn and (1 ρ)n. A slighly differen model assumes ha he verices are pariioned ino wo clusers of deerminisic sizes, exacly given by ρn and (1 ρ)n. The wo models behave similarly, bu for ease of analysis, we focus on he random cluser size model in his paper.

5 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL are assumed o be known, and he goal is o esimae he verex labeling σ from he observaion of G. More precisely, we have he following definiion. Definiion 1 The reconsrucion problem on he graph is he problem of inferring σ from he observaion of G. The expeced fracion of verices misclassified by an esimaor σ is given by p G ( σ) = 1 n n P {σ i σ i. () i=1 Le p G denoe he minimum expeced misclassified fracion among all possible esimaors based on G. The opimal esimaor in minimizing he error probabiliy P {σ i σ i is he maximum a poserior (MAP) esimaor, which is given by 1 {P{σi =+ G P{σ i = G 1, and he minimum error probabiliy is given by 1 1 E [ P {σ i = + G P {σ i = G ]. Hence, he minimum expeced misclassified fracion p G is given by p G = 1 1 n n E [ P {σ i = + G P {σ i = G ] i=1 = 1 1 E [ P {σ i = + G P {σ i = G ], (3) where he second equaliy holds due o he symmery among verices. In he special case wih ρ = 1/ and a = c, he wo clusers are symmeric; hus p G = 1/ and one can only hope o esimae σ up o a global flip of sign. In general, compuing he MAP esimaor is compuaionally inracable and i is unclear wheher he minimum expeced misclassified fracion p G can be achieved by some esimaor compuable in polynomial-ime. Throughou his paper, we assume ha ρ is fixed and focus on he regime (1). As he average degree is n o(1), i is well-known ha a local neighborhood of a verex is a ree wih high probabiliy. Thus, i is naural o sudy he local algorihms. More precisely, we consider a local belief propagaion algorihm o approximae he MAP esimaor in he nex subsecion..1. Local Belief Propagaion Algorihm Our local belief propagaion algorihm ( is given) in Algorihm 1. Specifically, le i denoe he se of neighbors of i, and F (x) = 1 log e x ρa+ ρb. Le d e x ρb+ ρc + = ρa + ρb and d = ρb + ρc denoe he expeced verex degree in he firs and second cluser, respecively. Define he message ransmied from verex i o verex j a -h ieraion as R i j = d + + d + l i\{j F (R 1 l i ), () wih iniial condiions Ri j 0 = 0 for all i [n] and j i. Then we define he belief of verex u a -h ieraion Ru o be R u = d + + d + l u F (R 1 l u ). (5) 5

6 MOSSEL XU Algorihm 1 Belief propagaion for cluser recovery 1: Inpu: n N, ρ (0, 1), a/b, c/b, adjacency marix A {0, 1 n n, and N. : Iniialize: Se Ri j 0 = 0 for all i [n] and j i. 3: Run 1 ieraions of message passing as in () o compue Ri j 1 for all i [n] and j i. : Compue Ri for all i [n] as per (5). 5: Reurn σ BP wih σ BP (i) = 1 {Ri ϕ 1, where ϕ = 1 log ρ 1 ρ. As we will show in Secion 3.1, he message passing as in () and (5) exacly compues he log likelihood raio for a problem of inferring σ u on a suiably defined ree model wih roo u. Moreover, in he regime (1), here exiss a coupling such ha he local neighborhood of a fixed verex u is he same as he ree model rooed a u wih high probabiliy. These wo observaions ogeher sugges ha Ru is a good approximaion of 1 P{G σu=+ log P{G σ, and hus we can esimae σ u= u by runcaing Ru a he opimal hreshold ϕ = 1 1 ρ log ρ, according o he MAP rule. We can see from () ha in each BP ieraions, each verex i needs o compue i ougoing messages. To his end, i can firs compue Ri according o (5), and hen subrac F (R 1 j i ) from R i o ge Ri j for every neighbor j of i. In his way, each BP ieraion runs in ime O(m), where m is he oal number of edges. Hence σ BP can be compued in ime O(m). Finally, noice ha Algorihm 1 needs o know he parameers {ρ, a/b, c/b. For he main resuls of his paper coninue o hold, he values of he parameers are only needed o know up o o(1) addiive errors. In fac, here exiss a fully daa-driven procedure o consisenly esimae hose parameers, see e.g., Hajek e al. (015a)[Appendix B]... Main Resuls The following heorem characerizes he expeced fracion of verices misclassified by σ BP as n ; i also gives a lower bound on he minimum expeced misclassified fracion as n. Furhermore, in he case ρ = 1/ and µ ν, σ BP achieves he lower bound as afer n. Theorem Assume ρ (0, 1) is fixed and consider he regime (1). Le h(v) = E [ anh(v + vz + ϕ) ], ρ where Z N (0, 1) and ϕ = 1 log ρ(µ+ν) 1 ρ. Le λ = and θ = ρ(µ ν) and v o be he smalles and larges fixed poin of v = θ + λh(v), + (1 ρ)ν. Define v respecively. Define (v : 0) recursively by v 0 = 0 and v +1 = θ + λh(v ). Le σ BP denoe he esimaor given by Belief Propagaion applied for ieraions, as defined in Algorihm 1. Then. The exisence of fixed poins of v θ + λh(v) follows from Brouwer s fixed-poin heorem and he fac ha h(v) 1. 6

7 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL lim v = v, (ρµ ρν) / v v (ρµ + ρν )/, and ( ) ( lim p G( σ v + ϕ v ϕ n BP) = ρq + (1 ρ)q v ( ) ( ) v + ϕ v ϕ + (1 ρ)q, v v where Q(x) = + x lim inf n p G ρq v ), 1 π e y / dy. Moreover, if ρ = 1/ and µ ν, hen v = v = v, and hus lim lim p G( σ n BP) = lim n p G = Q( v ), where v is he unique fixed poin of v = (µ ν) 16 + (µ+ν) 16 E [anh(v + vz)]. If ρµ ρν so ha he verex degrees are saisically correlaed wih he cluser srucure, we have v > 0 and hus lim lim n p G ( σ BP ) min{ρ, 1 ρ. Hence, he local applicaion of BP sricly ouperforms he rivial esimaor, which always guesses he label of all verices o be +1 if ρ 1/ and 1 if ρ < 1/. In he balanced case ρ = 1/, he local BP achieves he minimum expeced misclassified fracion. Numerical experimens furher indicae ha he local BP is sill opimal in he unbalanced case provided ha ρ is no close o 0 or 1; however, we do no have a proof (See Secion. for more discussions). If ρµ = ρν, hen v = 0 and hus ( ) ( ) v + ϕ v ϕ ρq + (1 ρ)q = min{ρ, 1 ρ. v v In his case, our local applicaion of BP canno do beer han he rivial esimaor. In fac, he local neighborhoods are saisically uncorrelaed wih he cluser srucure, and one can furher argue ha no local algorihm can achieve non-rivial deecion (see e.g. Kanade e al. (01)). Alhough local algorihms are bound o fail, here migh sill exis some efficien global algorihms which achieve he minimum expeced misclassified fracion. The following heorem shows ha his is indeed he case when ρ = 1/, µ = ν and b = o(log n). Theorem 3 Assume ρ = 1/, a = c = b + bµ for some fixed consan µ, and b such ha b = o(log n). For an esimaor σ based on graph G, define he fracion of verices misclassified by σ as { n O( σ, σ) = 1 n n min 1 {σi σ i,. (6) i=1 i=1 1 {σi σ i If µ >, hen ( ) lim inf E [O( σ, σ)] = Q v, (7) n σ where he infimum ranges over all possible esimaors σ based on graph G; v > 0 is he larges fixed poin of v = µ E [anh(v + vz)]. Moreover, here is a polynomial-ime esimaor such ha for every ɛ > 0, i misclassifies a mos Q ( v ) ɛ fracion of verices on expecaion. 7

8 MOSSEL XU In conras o (), he fracion of verices misclassified by σ is defined up o a global flip of signs of σ in (6). This is because in he case ρ = 1/ and a = c, due o he symmery beween + and, σ and σ have he same disribuion condiional on graph G. Thus, i is impossible o reliably esimae he sign of σ based on graph G. Noe ha µ = corresponds o he Kesen-Sigum bound Kesen and Sigum (1966). I is shown in Mossel e al. (015a) ha if µ <, correlaed recovery is impossible and hus he minimum expeced misclassified fracion is 0; Remarkably, Massoulié (01); Mossel e al. (013b); Bordenave e al. (015) prove ha correlaed recovery is efficienly achievable if µ >. Our resuls furher show ha in his case wih b and b = o(log n), he minimum expeced misclassified fracion is Q ( v ) and i can also be aained in polynomial-ime. The proof of Theorem 3 is mainly based on wo observaions. Firs, i is shown in Mossel e al. (013a) ha he local BP is able o improve a clusering ha is slighly beer han a random guess o achieve he minimum expeced misclassified fracion if µ > C for a universal consan C > 0. Second, we find ha if µ >, he recursion v = µ E [anh(v + vz)] derived in he densiy evoluion analysis of local BP has only wo fixed poins: 0 and v > 0, where 0 is unsable and v is sable. These wo resuls ogeher esablish ha if µ >, hen running he local BP for ieraions wih a correlaed iniializaion provided by a non-rivial deecion algorihm is able o aain he minimum expeced misclassified fracion Q ( v ) as..3. Proof Ideas The proof of Theorem is based on wo useful ools. Firs, we connec he cluser recovery problem o he reconsrucion problem on rees. Second, we use he densiy evoluion wih Gaussian approximaions o give a sharp characerizaion of error probabiliies of ree reconsrucion problems, in erms of fixed poins of a recursion. To bound from below he minimum expeced misclassified fracion, we bound he error probabiliy of inferring σ u for a specific verex u. Following Mossel e al. (015a), we consider an oracle esimaor, which in addiion o he graph srucure, he exac labels of all verices a disance exacly from u are also revealed. As in Mossel e al. (015a), i is possible o show ha he bes esimaor in his case is given by BP for levels iniialized using he exac labels a disance. In conras, he expeced fracion of verices misclassified by he local BP algorihm approximaely equals o he error probabiliy of inferring σ u solely based on he neighborhood of verex u of radius, wihou having access o he exac labels of verices a disance. We characerize he densiy evoluion of he local BP and he BP lower bound using Gaussian approximaions, and ge a recursion wih he larges fixed poin corresponding o he BP lower bound, and he smalles fixed poin corresponding o he expeced fracion of verices misclassified by he local BP as. In he balanced cluser cases, we furher show ha here is a unique fixed poin for he recursion, and hus he BP lower bound maches he expeced fracion of verices misclassified by he local BP... Numerical Experimens and Open Problems In he case wih ρ = 1/ and µ ν, we show ha v = θ + λh(v) has a unique fixed poin and hus he local BP is opimal; he key idea is o prove ha h(v) is concave in his case. Numerical

9 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL ρ=0. ρ=0. ρ=0.1 ρ= Figure 1: Numerical calculaions of h (v) (y axis) versus v [0, 6] (x axis) wih differen ρ. I shows ha h(v) is concave when ρ 0. and h(v) becomes convex for v small when ρ Figure : The plo of θ + λh(v) (y axis) versus v (x axis) in he case ρ = Lef frame: µ = 50 and ν = 0; righ frame: µ = 0 and ν = 1.5. I shows ha v = θ+λh(v) has hree fixed poins: The smalles one is v corresponding o he performance of local BP; he larges one is v corresponding he lower bound on expeced misclassified fracion; he inermediae one is unsable. calculaions, depiced in Fig. 1, show ha h(v) is sill concave if ρ 0., suggesing ha he local BP is sill opimal in roughly balanced cluser size cases. However, h(v) becomes convex for v small when ρ 0.1. I is inriguing o invesigae when v = θ + λh(v) has a unique fixed poin. If ρ = 0.01, numerical experimens, depiced in Fig., shows ha v = θ + λh(v) may have muliple fixed poins, suggesing ha he local BP may be subopimal. However, in he case wih µ = ν and ρ 1/, numerical experimens indicae ha here is always a unique fixed poin. Conjecure If µ = ν, hen v = θ + λh(v) has a unique fixed poin for all ρ (0, 1/) (1/, 1). 9

10 MOSSEL XU Noice ha in he case wih µ = ν and ρ = 1/, θ = 0, λ = µ /, and h(v) = E [anh(v + vz)]. We have shown in Lemma 1 ha h is non-decreasing, concave, and lim v 0 h (v) = 1. Thus if µ <, here is a unique fixed poin a zero, which is sable; if µ >, here are wo fixed poins: one is zero which is unsable and he oher is v > 0 which is sable..5. Noaion and Organizaion of he Paper For any posiive ineger n, le [n] = {1,..., n. For any se T [n], le T denoe is cardinaliy and T c denoe is complemen. We use sandard big O noaions, e.g., for any sequences {a n and {b n, a n = Θ(b n ) if here is an absolue consan c > 0 such ha 1/c a n /b n c. Le Bern(p) denoe he Bernoulli disribuion wih mean p and Binom(n, p) denoe he binomial disribuion wih n rials and success probabiliy p. All logarihms are naural and we use he convenion 0 log 0 = 0. We say a sequence of evens E n holds wih high probabiliy if P {E n 1. The res of his paper is organized as follows. Secion 3 focuses on he inference problems on he ree model. The analysis of he belief propagaion algorihm on rees and he proofs of our main heorems are given in Secion. The proofs of auxiliary lemmas can be found in Appendix A. 3. Inference Problems on Galon-Wason Tree Model In his secion, we firs inroduce he inference problems on Galon-Wason rees, and hen relae i o he cluser recovery problem under he sochasic block model. Definiion 5 For a verex u, we denoe by (T u, τ) he following Poisson wo-ype branching process ree rooed a u, where τ is a {± labeling of he verices of T. Le τ u = + wih probabiliy ρ and τ u = 1 wih probabiliy ρ, where ρ = 1 ρ. Now recursively for each verex i in T u, given is label τ i = +, i will have L i Pois(ρa) children j wih τ j = + and M i Pois( ρb) children j wih τ j = ; given is label τ i = 1, i will have L i Pois(ρb) children j wih τ j = + and M i Pois( ρc) children j wih τ j =. For any verex i in T u, le Ti denoe he subree of T u of deph rooed a verex i, and Ti denoe he se of verices a he boundary of Ti. Wih a bi abuse of noaion, le τ A denoe he vecor consising of labels of verices in A, where A could be eiher a se of verices or a subgraph in T u. We firs consider he problem of esimaing he label of roo u given he observaion of Tu and τ T u. Noice ha he labels of verices in Tu 1 are no observed. Definiion 6 The deecion problem on he ree wih exac informaion a he boundary is he problem of inferring τ u from he observaion of T u and τ T u. The error probabiliy for an esimaor τ u (T u, τ T u ) is defined by p T ( τ u ) = ρp { τ u = τ u = + + ρp { τ u = + τ u =. Le p T denoe he minimum error probabiliy among all esimaors based on T u and τ T u. The opimal esimaor in minimizing p T, is he maximum a poserior (MAP) esimaor, which can be expressed in erms of log likelihood raio: τ ML = 1 {Λ u ϕ 1, 10

11 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL where for all i in T u, and ϕ = 1 log { Λ i 1 P T log i, τ Ti τ i = + {, P Ti, τ Ti τ i = ρ 1 ρ. Thus, he minimum error probabiliy p T is given by p T = 1 1 E [ P { τ u = + T u, τ T u P { τu = T u, τ T u ]. () We hen consider he problem of esimaing τ u given observaion of T u. Noice ha in his case he rue labels of verices in T u are no observed. Definiion 7 The deecion problem on he ree is he problem of inferring τ u from he observaion of T u. The error probabiliy for an esimaor τ u (T u) is defined by q T ( τ u ) = ρp { τ u = τ u = + + ρp { τ u = + τ u =. Le q T denoe he minimum error probabiliy among all esimaors based on T u. In passing, we remark ha he only difference beween Definiion 6 and Definiion 7 is ha he exac labels a he boundary of he ree is revealed o esimaors in he former and hidden in he laer. The opimal esimaor in minimizing qt, is he maximum a poserior (MAP) esimaor, which can be expressed in erms of he log likelihood raio: where τ MAP = 1 {Γ u ϕ 1, Γ i 1 log P { Ti τ u = + P {Ti τ u =. for all i in T u, and ϕ = 1 log ρ 1 ρ. The minimum error probabiliy q T is given by q T = 1 1 E [ P { τu = + T u P { τu = T u ], (9) If d + = d, hen he disribuion of Tu condiional on τ u = + is he same as ha condiional on τ u =. Thus, Γ u = 0 and he MAP esimaor reduces o he rivial esimaor, which always guesses he label o be + if ρ 1/ and if ρ < 1/, and qt = min{ρ, ρ. If d + d, hen T u becomes saisically correlaed wih τ u, and i is possible o do beer han he rivial esimaor based on Tu. For he ree model, he likelihoods can be compued exacly via a belief propagaion algorihm. The following lemma gives a recursive formula o compue Λ i and Γ i ; no approximaions are needed. Le i denoe he se of children of verex i. ( ) Lemma Recall F (x) = 1 log e x ρa+ ρb. For 0, e x ρb+ ρc Λ +1 i Γ +1 i = d + + d = d + + d wih Λ 0 i = if τ i = + and Λ 0 i = if τ i = ; Γ 0 i + j i F (Λ j), (10) + j i F (Γ j), (11) = 0 for all i. 11

12 MOSSEL XU 3.1. Connecion beween he Graph Problem and Tree Problems For he reconsrucion problem on graph, recall ha p G ( σ BP ) denoe he expeced fracion of verices misclassified by σ BP as per (); p G is he minimum expeced misclassified fracion. For he reconsrucion problems on ree, recall ha p T is he minimum error probabiliy of esimaing τ u based on Tu and τ T u as per (); qt is minimum error probabiliy of esimaing τ u based on Tu as per (9). In his secion, we show ha in he limi n, p G ( σ BP ) equals o q T, and p G is bounded by p T from he below for any 1. Noice ha q T and p T depend on n only hrough he parameers a, b, and c. A key ingredien is o show ha G is locally ree-like wih high probabiliy in he regime b = n o(1). Le G u denoe he subgraph of G induced by verices whose disance o u is a mos and le G u denoe he se of verices whose disance from u is precisely. In he following, for ease of noaion, we wrie Tu as T and G u as G when here is no ambiguiy. Wih a bi abuse of noaion, le σ A denoe he vecor consising of labels of verices in A, where A could be eiher a se of verices or a subgraph in G. The following lemma proved in Mossel e al. (015a) shows ha we can consruc a coupling such ha (G, σ G ) = (T, τ T ) wih probabiliy converging o 1 when b = n o(1). Lemma 9 For = (n) such ha b = n o(1), here exiss a coupling beween (G, σ) and (T, τ) such ha (G, σ G ) = (T, τ T ) wih probabiliy converging o 1. Suppose ha (G, σ G ) = (T, τ T ) holds, hen by comparing BP ieraions () and (5) wih he recursions of log likelihood raio Γ given in (11), we find ha R u exacly equals o Γ u, i.e., he BP algorihm defined in Algorihm 1 exacly compues he log likelihood raio Γ u for he ree model. Building upon his inuiion, he following lemma shows ha p G ( σ BP ) equals o q T as n. Lemma 10 For = (n) such ha b = n o(1), lim p G( σ n BP) qt = 0. Proof In view of Lemma 9, we can consruc a coupling such ha (G, σ G ) = (T, τ T ) wih probabiliy converging o 1. On he even (G, σ G ) = (T, τ T ), we have ha R u = Γ u. Hence, where o(1) erm comes from he coupling error. p G ( σ BP) = qt + o(1), (1) The following lemma shows ha p G is lower bounded by p T as n. Lemma 11 For = (n) such ha b = n o(1), lim sup (p G p T ) 0. n We pause a while o give some inuiion behind he lemma. To lower bound p G, i suffices o lower bound he error probabiliy of esimaing σ u for a given verex u based on graph G. To his end, we consider an oracle esimaor, which in addiion o he graph srucure, he exac labels of all verices a disance exacly from u are also revealed. We furher show ha once he exac 1

13 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL labels a disance are condiioned, σ u becomes asympoically independen of he labels of all verices a disance larger han from u. Hence, effecively he oracle esimaor is equivalen o he MAP esimaor solely based on he graph srucure in G u and he exac labels a disance. By he coupling lemma, G u is a ree wih high probabiliy, and hus he error probabiliy of he oracle esimaor asympoically equals o p T.. Gaussian Densiy Evoluion In he previous subsecion, we have argued ha in he limi n, p G ( σ BP ) equals o q T, and p G is bounded by p T from he below. In his secion, we analyze recursions (10) and (11) using densiy evoluion analysis wih Gaussian approximaions, and derive simple formulas for p T and qt in he limi n. Aferwards, we give he proof of Theorem. Noice ha Γ i is a funcion of T i alone. Since he subrees {T i i u condiional on τ u are independen and idenically disribued, {Γ i i u condiional on τ u are also independen and idenically disribued. Thus, in view of he recursion (11), Γ u can be viewed as a sum of i.i.d. random variables. When he expeced degree of u ends o infiniy, due o he cenral limi heorem, we expec ha he disribuion of Γ u condiional on τ u is approximaely Gaussian. Moreover, he consrucion of he subree Ti condiional on τ i is he same as he consrucion of Tu condiional on τ u. Therefore, for any i u, he disribuion of Γ i condiional on τ i is he same as he disribuion of Γ u condiional on τ u. Similar conclusions hold for Λ i as well. Le Z± (W±) denoe a random variable ha has he same disribuion as Γ u (Λ u) condiional on τ u = ±. The following lemma provides expressions of he mean and variance of Z+ and Z. Recall ha λ = ρ(µ+ν) and θ = ρ(µ ν) + (1 ρ)ν. Lemma 1 For all 0, E [ Z± +1 ] [ = ±θ ± λe anh(z + + ϕ) ] + O(b 1/ ), (13) var ( Z± +1 ) [ = θ + λe anh(z + + ϕ) ] + O(b 1/ ). (1) Recall ha (v : 0) saisfies v 0 = 0 and v +1 = θ + λh(v ) = θ + λe [anh(v + v Z + ϕ], where Z N (0, 1). Similarly, define (w : 1) by w 1 = θ + λ = ρµ + ρν and w +1 = θ + λh(w ) = θ + λe [anh(w + w Z + ϕ]. The following lemma shows ha for any fixed 0, Z ± and W ± are approximaely Gaussian. Lemma 13 For any 0, as n, { sup Z P ± v x v Similarly, for any 1, as n, { sup W P ± w x w x x P {Z x = O(b 1/ ). (15) P {Z x = O(b 1/ ). (16) 13

14 MOSSEL XU Before proving Theorem, we also need a key lemma, which shows ha h is coninuous and non-decreasing, and h is concave if ϕ = 0. Lemma 1 h(v) is coninuous on [0, ) and for v (0, + ), h (v) = E [( 1 anh(v + vz + ϕ) ) ( 1 anh (v + vz + ϕ) )] 0. (17) Furhermore, if ϕ = 0, hen h (v) h (w) for 0 < v < w <. Finally, we are ready o prove Theorem based on Lemma 13 and Lemma 1. Proof [Proof of Theorem ] In view of Lemma 13, lim P { Γ n u ϕ τ u = ( ) v ϕ = Q, v lim P { Γ n u ϕ τ u = + ( ) v + ϕ = Q. v Hence, i follows from Lemma 10 ha [ ( )] lim p G( σ n BP) = lim n q T = E v + U Q, v where U = ϕ wih probabiliy 1 ρ and U = ϕ wih probabiliy ρ. We prove ha v +1 v for 0 by inducion. Recall ha v 0 = 0 (ρµ ρν) / = θ + λh(v 0 ) = v 1. Suppose v +1 v holds; we shall show he claim also holds for + 1. In paricular, since h is coninuous on [0, ) and differenial on (0, ), i follows from he mean value heorem ha v + v +1 = λ (h(v +1 ) h(v )) = λh (x)(v +1 v ), for some x (v, v +1 ). Lemma 1 implies ha h (x) 0 for x (0, ), i follows ha v + v +1. Hence, v is non-decreasing in. Nex we argue ha v v for all 0 by inducion, where v is he smalles fixed poin of v = θ + λh(v). For he base case, v 0 = 0 v. If v v, hen by he monooniciy of h, v +1 = θ + λh(v ) θ + λh(v) = v. Hence, v v and hus lim v v. By he coninuiy of h, lim v is also a fixed poin of v = θ + λh(v), and consequenly lim v = v. Therefore, [ ( )] v + U lim lim p G( σ n BP) = lim lim n q T = E Q. v Nex, we prove he claim for p G. In view of Lemma 13, lim P { Λ n u ϕ τ u = ( ) w ϕ = Q, w lim P { Λ n u ϕ τ u = + ( ) w + ϕ = Q. w 1

15 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL Hence, i follows from Lemma 11 ha [ ( )] lim inf n p G lim n p T = E w + U Q. w Recall ha w 1 = θ + λ w. By he same argumen of proving v is non-decreasing, one can show ha w is non-increasing in. Also, by he same argumen of proving v is upper bounded by v, one can show ha w is lower bounded by v, where v is he larges fixed poin of v = θ + λh(v). Thus, lim w = v and v w 1 = θ + λ = (ρµ + ρν )/. Therefore, lim inf n p G = lim lim inf n p G lim lim n p T = E [ Q ( )] v + U. v If ϕ = 0 and µ ν, hen v 1 > 0 and Lemma 1 shows ha h (v) h (w) for all 0 < v < w <. Since v 1 = θ + λh(0) > 0 and v = θ + λh(v), i mus hold ha λh (v) < 1. Thus λh (v) < 1 for all v v and consequenly θ + λh(v) < v for all v > v. Hence, v = v = v, where v is he unique fixed poin of v = (µ ν) 16 + (µ+ν) 16 E [anh(v + vz)]. Therefore, lim inf n p G lim lim n p T = lim lim n q T = lim lim n p G( σ BP) = Q( v ). Since p G is he minimum expeced misclassified fracion, i also holds ha lim sup n p G lim n p G ( σ BP ) for all 1 and consequenly lim sup p G lim lim p n G( σ n BP). Combing he las wo displayed equaions gives ha lim n p G = lim lim p G( σ n BP) = Q( v )..1. Degree-Uncorrelaed Case As remarked in Secion., in he case ρµ = ρν, he verex degrees are saisically uncorrelaed wih he cluser srucure, and no local algorihms is capable of non-rivial deecion. However, i is sill possible ha local algorihms combined wih some efficien global algorihms achieve he minimum expeced misclassified fracion. In his subsecion, we show ha i is indeed he case, if ρ = 1/, µ = ν wih µ >, and b = o(log n). The algorihm as described in Algorihm is inroduced in Mossel e al. (013a) and we give he full descripion for compleeness. Noice ha Algorihm runs in ime polynomial in n. The algorihm consiss of wo main seps. Firs, we apply some global algorihm o ge a correlaed clusering when µ >, for example, he algorihm sudied in Mossel e al. (013c). Then, we apply he local BP algorihm o boos he correlaed clusering in he firs sep o achieve he minimum expeced misclassified fracion. To ensure he firs and second sep are independen of each oher, for each verex u, we firs wihhold he ( 1)-local neighborhood of u, and hen apply he global deecion algorihms on he reduced se of verices. The clusering on he reduced se of verices is used as he iniializaion o he local 15

16 MOSSEL XU Algorihm Local Belief propagaion Plus Correlaed Recovery 1: Inpu: n N, a = c, b > 0, ρ = 1/, adjacency marix A {0, 1 n n, N. : Take U V o be a random subse of size n. Le u U be a random verex in U wih a log n leas log(log n/b) neighbors in V \U. 3: For u V \U do 1. Run a polynomial-ime esimaor capable of correlaed recovery on he subgraph induced by verices no in G 1 u and U, and le W + u and W u denoe he oupu of he pariion.. Relabel W u + and Wu such ha if a > b, hen u has more neighbors in W u + han Wu ; oherwise, u has more neighbors in Wu han W u +. Le α u denoe he fracion of verices misclassified by he pariion (W u +, Wu ). 3. For all i G u and j G 1 u, define Ri j 0 = αu log α u if i Wu. log 1 αu α u if i W + u, and R 0 i j =. Run 1 ieraions of message passing as in () o compue Ri u 1 for all u s neighbors i. 5. Compue Ru as per (5), and le σ BP (u) = + if R u ϕ; oherwise le σ BP (u) =. : For u U, le σ BP (u) equal o + or uniformly a random. Oupu σ BP. belief propagaion algorihm running on he wihheld ( 1)-local neighborhood of u. In his way, he oucome of he global deecion algorihm based on he reduced se of verices is independen of he edges beween he wihheld -local neighborhood of u and he reduced se of verices, as well as he edges wihin he wihheld se. There is also a suble issue o overcome. We run he global deecion algorihm once for each verex, and he global deecion algorihm canno reliably esimae he sign of he rue σ due o he symmery beween + and. Therefore, differen runs of he global deecion algorihm may have differen esimaes of he sign of σ. We need a way o coordinae differen runs of he global deecion algorihms o have he same esimae of he sign of σ. To his end, a small random subse U is reserved and a verex of high degree u in U is served as an anchor. In every runs of he global deecion algorihms, we relabel he pariion if necessary, o ensure ha u will always have more neighbors wih esimaed + labels han neighbors wih esimaed labels if a > b, and he oher way around if a < b. Finally, we cauion he reader ha in addiion o he model parameers a, b, afer each run of he global deecion algorihm, he algorihm requires knowing α u, which is he fracion of verices misclassified by he pariion (W u +, Wu ). In he main analysis, we assume he exac value of α u is known for simpliciy. One can check ha only an esimaor α u = α u + o(1) wih high probabiliy is needed for Theorem 3 o hold. In Appendix B, we give an efficien and daa-driven procedure o consruc such a consisen esimae of α u. Nex, in he limi n, we give a lower bound on he minimum expeced misclassified fracion, and an upper bound aainable by σ BP. Then we show ha he lower and upper bound mach wih each oher in he double limi, where firs n and hen. 16

17 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL Recall ha he fracion of verices misclassified by σ is defined up o a global flip of signs of σ as in (6). The following lemma shows ha he minimum expeced misclassified fracion is sill lower bounded by p T. Is proof is very similar o he proof of Lemma 11. The key new challenge is ha E [O(σ, σ)] does no reduce o he error probabiliy of esimaing σ u for a given verex u direcly. Lemma 15 For = (n) such ha b = n o(1), lim sup n ( inf E [O(σ, σ)] σ p T ) 0, where p T is defined in () under he ree model wih ρ = 1/ and a = c defined in Definiion 5. In he following, we relae he expeced fracion of verices misclassified by σ BP as defined in Algorihm o an esimaion problem on he ree model. In paricular, consider he ree model wih ρ = 1/ and a = c as defined in Definiion 5. Fix an α [0, 1/]. Le τ i = τ i wih probabiliy 1 α and τ i = τ i wih probabiliy for α, independenly for all i T u. Then τ T u is a α-noisy version of τ T u. Le q T,α denoe he minimum error probabiliy of inferring τ u based on Tu and τ T u. The opimal esimaor achieving q T,α is he MAP esimaor given by where τ MAP = 1 { Γ u ϕ 1, { Γ i 1 P T log i, τ Ti τ u = +1 { P Ti, τ Ti τ u = 1 for all i in T u. The minimum error probabiliy q T,α is given by q T,α = 1 { Γ P u < ϕ τ u = { Γ P u ϕ τ u = = 1 1 E [ P { τu = + T u, τ T u P { τ u = T u, τ T u ], I follows from he definiion ha q T,α is non-decreasing in α. Also, q T,α = p T if α = 0 and q T,α = q T if α = 1/. The following lemma shows ha he fracion of verices misclassified by σ BP as defined in Algorihm is asympoically no larger han q T,α for some α [0, 1/). Lemma 16 There exiss an α [0, 1/) such ha for = (n) wih b = n o(1), ( [ lim sup E O(σ, σ BP ) ] ) q T,α 0. n The following lemma gives a characerizaion of he disribuion of Γ u based on he densiy evoluion wih Gaussian approximaions. Lemma 17 Le Z + and Z 1 denoe a random variable ha has he same disribuion as Γ u condiioning on τ u = + and τ u =, respecively. For any 1, as n, { sup Z P ± u x P {Z x x u = O(b 1/ ), (1) where u 1 = (1 α) µ and u +1 = µ E [ anh(u + u Z) ]. 17

18 MOSSEL XU We are ready o prove Theorem 3 by combing Lemma 15, Lemma 16, and Lemma 17. Proof [Proof of Theorem 3] In view of Lemma 17, for 0, lim { Γ P n u 0 τ u = = lim { Γ P n u 0 τ u = + = Q( u ), I follows from Lemma 16 ha here exiss an α [0, 1/) such ha lim sup E [ O(σ, σ BP) ] lim q T n n,α = Q( u ). Le h(v) = E [anh(v + vz)]. In view of Lemma 1, h is non-decreasing and concave in [0, ), and lim v 0 h (v) = 1. Noice ha h(0) = 0, and hus 0 is a fixed poin of v = µ h(v). Moreover, by he mean value heorem, for v > 0, h(v) = h(0) + h (ξ)v for some ξ (0, v). Thus µ h(v) = µ h (ξ)v. By he assumpion ha µ >, and lim h v 0 (v) = 1, i follows ha here exiss a v > 0 such ha µ h(v) > v for all v (0, v ). Furhermore, h(v) 1 and hence µ h(v) < v for all v sufficienly large. Since h is coninuous, v = µ h(v) mus have nonzero fixed poins. Le v denoe he smalles nonzero fixed poin. Then v > 0, µ h(v) > v for all v (0, v), and µ h (v) < 1. Because h is concave, h (v) h (v) for all v v. Thus µ h(v) > v for all v > v. Therefore, v is he unique nonzero fixed poin and also he larges fixed poin. I follows ha if u 1 < v, hen {u is a non-decreasing sequence upper bounded by v. If u 1 > v, hen {u is a non-increasing sequence lower bounded by v. Since u 1 > 0, i follows ha lim u = v. Hence, lim sup inf n σ E [O(σ, σ)] lim I follows from Theorem and Lemma 15 ha lim sup E [ O(σ, σ BP) ] lim lim q T n n,α = Q( v). (19) lim inf inf E [O(σ, σ)] lim lim n σ n p T = Q( v). (0) The heorem follows by combining he las wo displayed equaions. 5. Acknowledgemen Research suppored by NSF gran CCF , DOD ONR gran N , and gran 305 from he Simons Foundaion. J. Xu would like o hank Bruce Hajek and Yihong Wu for numerous discussions on belief propagaion algorihms and densiy evoluion analysis. References E. Abbe and C. Sandon. Communiy deecion in general sochasic block models: fundamenal limis and efficien recovery algorihms. arxiv , March 015. E. Abbe, A. S. Bandeira, and G. Hall. Exac recovery in he sochasic block model. arxiv , Ocober 01. N. Agarwal, A. S. Bandeira, K. Koiliaris, and A. Kolla. Mulisecion in he sochasic block model using semidefinie programming. arxiv , July

19 DENSITY EVOLUTION IN STOCHASTIC BLOCK MODEL A. Anandkumar, R. Ge, D. Hsu, and S. M. Kakade. A ensor specral approach o learning mixed membership communiy models. Journal of Machine Learning Research, 15:39 31, June 01. A.S. Bandeira. Random Laplacian marices and convex relaxaions. arxiv , April 015. P. J. Bickel and A. Chen. A nonparameric view of nework models and Newman-Girvan and oher modulariies. Proceedings of he Naional Academy of Science, 106(50): , 009. C. Bordenave, M. Lelarge, and L. Massoulié. Non-backracking specrum of random graphs: communiy deecion and non-regular Ramanujan graphs. ArXiv , January 015. URL hp://arxiv.org/abs/ T. Cai and X. Li. Robus and compuaionally feasible communiy deecion in he presence of arbirary oulier nodes. arxiv preprin arxiv: , 01. Y. Chen and J. Xu. Saisical-compuaional radeoffs in planed problems and submarix localizaion wih a growing number of clusers and submarices. In Proceedings of ICML 01 (Also arxiv:10.167), Feb 01. Y. Chen, S. Sanghavi, and H. Xu. Improved graph clusering. IEEE Transacions on Informaion Theory. An earlier version of his work appeared under he ile Clusering Sparse Graphs a NIPS 01, 60(10):60 655, Oc 01. ISSN A. Coja-Oghlan. A specral heurisic for bisecing random graphs. In Proceedings of he sixeenh annual ACM-SIAM symposium on Discree algorihms, SODA 05, pages 50 59, Philadelphia, PA, USA, 005. Sociey for Indusrial and Applied Mahemaics. A. Coja-Oghlan. Graph pariioning via adapive specral echniques. Comb. Probab. Compu., 19 ():7, March 010. A. Condon and Richard M. Karp. Algorihms for graph pariioning on he planed pariion model. Random Srucures and Algorihms, 1():116 10, 001. A. Decelle, F. Krzakala, C. Moore, and L. Zdeborová. Asympoic analysis of he sochasic block model for modular neworks and is algorihmic applicaions. Phys. Rev. E, :066106, Dec 011a. A. Decelle, F. Krzakala, C. Moore, and L. Zdeborová. Inference and phase ransiions in he deecion of modules in sparse neworks. Phys. Rev. Le., 107:065701, 011b. Y. Deshpande, E. Abbe, and A. Monanari. Asympoic muual informaion for he wo-groups sochasic block model. arxiv: , July 015. M. E. Dyer and A. M. Frieze. The soluion of some random NP-hard problems in polynomial expeced ime. Journal of Algorihms, 10():51 9, 199. D. Gamarnik and M. Sudan. Limis of local algorihms over sparse random graphs. In Proceedings of he 5h conference on Innovaions in heoreical compuer science, pages ACM,

20 MOSSEL XU C. Gao, Z. Ma, A. Y. Zhang, and H. H. Zhou. Achieving opimal misclassificaion proporion in sochasic block model. arxiv: , 015. B. Hajek, Y. Wu, and J. Xu. Achieving exac cluser recovery hreshold via semidefinie programming. arxiv , Nov. 01. B. Hajek, Y. Wu, and J. Xu. Achieving exac cluser recovery hreshold via semidefinie programming: Exensions. arxiv: , 015a. B. Hajek, Y. Wu, and J. Xu. Recovering a hidden communiy beyond he specral limi in O( E log V ) ime. arxiv , Ocober 015b. H. Haami, L. Lovász, and B. Szegedy. Limis of local-global convergen graph sequences. arxiv preprin arxiv: , 01. P. W. Holland, K. B. Laskey, and S. Leinhard. Sochasic blockmodels: Firs seps. Social Neworks, 5(): , 193. M. Jerrum and G. B. Sorkin. The Meropolis algorihm for graph bisecion. Discree Applied Mahemaics, (1 3): , 199. V. Kanade, E. Mossel, and T. Schramm. Global and local informaion in clusering labeled block models. In Proceedings of Approximaion, Randomizaion, and Combinaorial Opimizaion. Algorihms and Techniques, pages , 01. H. Kesen and B. P. Sigum. Addiional limi heorems for indecomposable mulidimensional Galon-Wason processes. Ann. Mah. Sais., 37:163 11, V. Korolev and I. Shevsova. An improvemen of he Berry Esseen inequaliy wih applicaions o Poisson and mixed Poisson random sums. Scandinavian Acuarial Journal, 01():1 105, 01. R. Lyons and F. Nazarov. Perfec machings as iid facors on non-amenable groups. European Journal of Combinaorics, 3(7): , 011. L. Massoulié. Communiy deecion hresholds and he weak Ramanujan propery. In Proceedings of he Symposium on he Theory of Compuaion (STOC), 01. F. McSherry. Specral pariioning of random graphs. In Proceedings of IEEE Conference on he Foundaions of Compuer Science (FOCS), pages , 001. M. Mezard and A. Monanari. Informaion, Physics, and Compuaion. Oxford Universiy Press, Inc., New York, NY, USA, 009. ISBN X, A. Monanari. Tigh bounds for LDPC and LDGM codes under MAP decoding. IEEE Transacions on Informaion Theory, 51(9):31 36, Sep 005. A. Monanari. Finding one communiy in a sparse random graph. arxiv , Feb 015. A. Monanari and D. Tse. Analysis of belief propagaion for non-linear problems: The example of cdma (or: How o prove anaka s formula). In Informaion Theory Workshop, 006. ITW 06 Puna del Ese. IEEE, pages , March