Improved Hidden Clique Detection by Optimal Linear Fusion of Multiple Adjacency Matrices

Transcription

1 Iproed Hidden Clique Detection by Optial Linear Fusion of Multiple Adjacency Matrices Hianshu Nayar, Benjain A. Miller, Kelly Geyer, Rajonda S. Caceres, Steen T. Sith, and Raj Rao Nadakuditi Departent of Electrical Engineering and Coputer Science, Uniersity of Michigan Ann Arbor, Michigan 4809 Lincoln Laboratory, Massachusetts Institute of Technology Lexington, MA Abstract Graph fusion has eerged as a proising research area for addressing challenges associated with noisy, uncertain, ulti-source data. While any ad-hoc graph fusion techniques exist in the current literature, an analytical approach for analyzing the fundaentals of the graph fusion proble is lacking. We consider the setting where we are gien ultiple Erdős-Rényi odeled adjacency atrices containing a coon hidden or planted clique. The objectie is to cobine the linearly so that the principal eigenectors of the resulting atrix best reeal the ertices associated with the clique. We utilize recent results fro rando atrix theory to derie the optial weighting coefficients and use these insights to deelop a data-drien fusion algorith. We deonstrate the iproed perforance of the algorith relatie to other siple heuristics. I. INTRODUCTION In a wide ariety of applications, iportant data take the for of connections, relationships, or interactions between discrete entities. This relational structure proides additional context, iproing situational awareness and enhancing the inforation that can be inferred fro the data. A dataset rich in relational inforation is represented atheatically as a graph. One particular proble of interest when working with graph-based data is subgraph detection []. Gien a graph, the objectie is to deterine whether the relationships obsered are consistent with noral behaior in the network, or if there is a subset of ertices coprising a subgraph within the oerall graph that exhibits a topology that is contrary to the expectation. Seeral applications are focused on finding denser-than-usual subgraphs within larger graphs, such as detecting counities in social networks [2] and finding highly interactie subsets aong proteins [3]. A siplified for of this proble is planted clique detection, where the objectie is to detect a fully connected subgraph placed within a larger graph with edges that occur with equal probability. This subset of the ore general subgraph detection proble The Lincoln Laboratory portion of this work is sponsored by the Assistant Secretary of Defense for Research & Engineering under Air Force Contract FA C Opinions, interpretations, conclusions and recoendations are those of the authors and are not necessarily endorsed by the United States Goernent. The Uniersity of Michigan portion of this work was sponsored by the Office of the Assistant Secretary of Defense for Research and Engineering. proides atheatical tractability that enables analysis of the fundaental liits of subgraph detectability. Planted clique detection has a long history within theoretical coputer science. Algoriths hae been deeloped using a ariety of techniques, including cobinatorial search ethods [4], spectral ethods [5], and statistical query ethods [6]. Recent work has focused on deriing analytical bounds for subgraph detectability, including technique-independent bounds [7], and bounds on detection using spectral techniques [8]. Spectral ethods are particularly interesting, since they are coputationally efficient and yield principled analytical bounds using rando atrix theory. Siilar bounds for the planted partition odel hae also been shown to predict a detectability phase transition in other efficient detection ethods [9]. While the planted clique proble has traditionally been studied in the context of detection within a single graph, in practice, a network is often the result of fused inforation fro ultiple disparate sources. Understanding the iplications that the algoriths for a single obseration hae on ulti-source data is extreely iportant as data analysts are increasingly required to ake decisions in this setting. The objectie of this paper is to extend the recent spectral detection bounds to cases where ultiple graphs are obsered. In this setting, the clique exists in each graph, each of which has a different edge probability. We adopt a linear fusion odel in which we analyze a conex cobination of the adjacency atrices of the graphs. Within this context, we deonstrate that the optial fusion ethod is highly intuitie: weighting the graphs in inerse proportion to their expected background degrees. In a set of experients on rando graphs, we test subgraph detectability at seeral leels of difficulty, arying the relatie density of the background graphs, the clique size, and the weights. In all cases, the optiality of the solution we derie is erified. The reainder of this paper is organized as follows. In Section II, we define the proble odel and foralize the atheatical context in which the graphs are fused. Section III proides a deriation of the forula for optial fusion within this context. In Section IV, we outline seeral experients that epirically alidate the deried optial fusion ethod /5/$ IEEE 520 Asiloar 205

2 Section V concludes the paper with a brief suary and directions for future work. A. Planted Clique Detection II. PROBLEM MODEL In the planted clique proble, we are gien a graph G = (V,E), where V is the set of ertices (representing the entities) and E is the set of edges (representing connections). We will denote N = V and M = E. The degree of a ertex is the nuber of edges connected to it. We will denote the aerage degree of the graph by c. A subset of ertices, V S V, coprise the clique, eaning that for all, u 2 V S, there is an edge between and u in E. We will denote the clique size by k = V S. In this paper, we will only consider graphs that are undirected (so edges are unordered pairs of ertices) and unweighted (so edges either exist or not, with no notion of connection strength). If either /2 V S or u/2 V S, then an edge exists between and u with probability p, which is constant across all pairs of ertices and independent of the existence of other edges. Spectral ethods ake use of atrix representations of a graph. The ost basic atrix representation of a graph is the adjacency atrix. For an unweighted, undirected graph, the adjacency atrix A = {a ij } is an N N binary atrix where a ij = if there is an edge between ertices i and j, and a ij =0otherwise. (This requires an arbitrary labeling of ertices with integers fro to N.) Since we consider undirected graphs, A will be syetric. A odification of the adjacency atrix used for counity detection is the odularity atrix [0]. The odularity atrix is a residuals atrix: the obsered adjacency atrix inus its expected alue. Since the planted clique proble assues a background with equal probability, we use the odularity atrix with respect to the Erdős-Rényi odel: B := A p N N. () This technique cancels out the effects of typical background behaior and allows the detection of deiations fro the expectation; in this case the planted clique. The for of () is a rank- perturbation of a Wigner atrix, and recent work has defined a sharp threshold for detectability of such a perturbation []. This analysis was applied to planted clique detection in [8], where a siple algorith was used to detect the clique: copute the (unit-noralized) principal eigenector of B, denoted by u. Since the entries in the principal eigenector of a Wigner atrix appear norally distributed, u is thresholded, with a false alar rate based on this distribution, and the estiate of the clique ertices is gien by: ˆV S = n i : p Nu i >F N (0,) o, (2) 2 where F N (0,) is the inerse cuulatie density function of a standard noral distribution and is the desired false alar probability. Using this algorith, the following bound was deried. Clai 2. (Nadakuditi [8]): Consider a k-ertex clique planted in an N-ertex graph with edge probability p, where the clique ertices are identified using (2) for a significance leel. Then, for fixed p, as k, N such that k/ p N 2 (0, ) we hae P(clique discoered) a.s. B. Multi-Source Graph Fusion ( if > crit. := otherwise. q p p Our objectie is to derie a bound analogous to Clai 2. for fusion of ultiple graphs. For the ulti-source setting, we assue we hae graphs G i =(V,E i ) for apple i apple. Note that the ertex set is the sae for all graphs (and, in atrix for, the indices are consistent across obserations). The nonclique edges are generated independently in each graph. To analyze the ultigraph in the sae context as the clai, we cobine the adjacency atrices of the graphs into a single atrix, and copute the principal eigenector of its residuals. We take a linear cobination of the adjacency atrices A i with weights w i to create a fused adjacency atrix: à = (3) w i A i. (4) We consider only positie weights that su to, eaning that à is a conex cobination of the adjacency atrices. Using this conention, the alue on the edges between clique ertices reains, since these edges exist in each obseration. Applying the sae weighting to the expected alues (p i being the background edge probability for G i ), we aintain a residuals atrix where the ajority of the entries (all of those not part of the clique) hae the sae zero-ean distribution. The fused residuals atrix is gien by: B := (A i p i N N )=à w i p i N N. (5) By applying the algorith defined in Section II-A to the principal eigenector of B, we can iproe detection perforance oer what would be possible with a single obseration. As deonstrated in the next section, we can derie analytically optial weights in this setting by iniizing the ariance of the entries. III. OPTIMAL LINEAR FUSION We begin by odeling the fused adjacency atrix à in a way that enables rando atrix theoretic analysis. Without 52

3 loss of generality, we can perute the ertex indices so that the clique ertices hae indices to k. We hae: apple à = w k k B k N 0(p i ) i B N 0 k(p i ) B N 0 N 0(p (6) apple P = w P P i k k w ib k N 0(p i ) w P ib N 0 k(p i ) w ib N 0 N 0(p (7) apple P = w P i k k w ip i k N 0 P w P ip i N 0 k w (8) ip i N 0 N apple P P k k w ib c k N 0(p w ib c N 0 k (p P w ib c N 0 N 0(p (9) i =E hã + X, (0) where B N N 2 (p) is an N N 2 atrix of Bernoulli rando ariables, each drawn independently with probability p, B c N N 2 (p) is the centered ersion of this atrix, where the rando ariables hae had their expected alue subtracted. For conenience, we define X to be the rando deiations fro the ean fro the atrix on line (9). Thus, E[Ã] is a rank-2 atrix, and à is a rando deiation fro this lowrank structure. The rando entries in X (i.e., those outside of the clique) hae ariance P w2 i p i( p i ). If k grows ore slowly than N, then X will tend toward a Wigner atrix, where the support of the eigenalue distribution (which deterines the noise power, and, thus, the detectability of the clique) scales with the standard deiation of these entries. Subtracting the expected alue of the fused atrix, we are left with a rank- perturbation of the P rando atrix. This rank- perturbation has alues of w ip i in the entries where both ertices are part of the clique, and 0 elsewhere. The nonzero eigenalue of this atrix is: = k w i p i. At this point, we hae a siilar setting as in [8]. In this case, the Wigner atrix has an eigenalue distribution that tends to a seicircle with radius: u X R = t 4N wi 2p u X i ( p i )= t 4 wi 2c i ( p i ), () whereas in the single-source case the radius is p 4Np( p). The rank- perturbation in the new setting is rather than siply k( p). We can apply siilar reasoning to deelop a new bound for planted clique detection using the fused odularity atrix with gien weights. First, we introduce an approxiation that will enhance the interpretability of the result. As graphs grow large, their density tends to decrease. That is, the aerage degree of the ertices grows slowly, not proportionally to N as it would if p reained constant. Thus, we will focus on a case where p 0 as N. Specifically, we consider the case where the aerage degree c reains constant. This allows us to express the detectability bound in ters of the aerage degrees of the graphs, c i, for large alues of N. The relationship between the axiu eigenalue of the rank- perturbation and the radius of the seicircle in Clai 2. is: k( p) > p Np( p). Thus, in the siilar setting for fused graphs, we want > R/2. Since we are assuing c i reains constant, we can use ( p i ) to approxiate the quantities of interest as k and R/2 p P w2 i c i. Since the approxiate eigenalue of the rank- perturbation is independent of the weights, the objectie to axiize detectability is equialent to that of iniizing R. To incorporate the constraint that the weights su to, we optiize the Lagrange function: ux L(w, )= t wi 2c i + w i. (2) j to zero yields: ux w j c j = t wi 2c i. (3) Since the right hand side of (3) is constant across weights, this critical point exists where each weight is inersely proportional to the aerage degree of its associated graph. Thus, the optial weighting schee for iniizing the support of the eigenalues is gien by: w j = /c P j /c = i + P i6=j c. (4) j/c i This result proides a atheatical justification for what would be an intuitie heuristic: giing the obserations that are noisier a proportionally lower weight. For exaple, if all graphs hae equal aerage degree, then they will be weighted equally. Conersely, if one graph has a uch saller aerage degree than the others, its weight will be close to while the others will be close to 0. In conjunction with the alue of R, this also deonstrates how additional inforation iproes detectability. Indeed, if G i has substantially lower degree than the other graphs, the detection threshold will reain approxiately p c i, since the other inforation is uch noisier and ost of the ephasis will be placed on the graph with the sparsest background. On the other hand, if all of the graphs hae equal aerage degree, the detection threshold will be reduced by p. As we show in the next section, the optiality of this weighting schee is erified by epirical detection perforance. IV. SIMULATION RESULTS In each of the following experients, we fuse two Erdős- Rényi graphs, each with a planted clique. The graphs each hae 0,000 ertices, and we ary the clique size, the aerage degrees of the backgrounds, and the weights to deonstrate the optiality of the solution deried in Section III. In each case, we set the false alars rate fro (2) to 0.05, and 522

4 Fig.. Probability of detection for a clique of arying size, ebedded into one graph with aerage degree 70 and one with aerage degree that aries. The optiized weights are used to fuse the graphs. The detection probability is shown on a base-0 logarithic scale, and the detection threshold is drawn in white. copute the epirical probability of detection of the clique ertices when thresholding the principal eigenector of B at the corresponding leel. We begin by deonstrating that the iproed bound correctly predicts where the clique becoes detectable. In this experient, we set the aerage degree of one graph to 70, and ary the aerage degree of the other graph fro 0 to 20. We independently ary the size of the clique fro 2 to 2. For each trial, the optiized weights fro (4) are used. The epirical probability of detection is shown in Fig.. Probability of detection is copared to the detectability threshold: q k = w 2c + w2 2c 2. Once k becoes larger than the threshold, its detectability begins to increase until the probability of detection eentually reaches. Using only the graph with arying degree, the detection threshold would be the square root of the aerage degree on the horizontal axis (e.g., a clique of size 0 at an aerage degree of 00). Using the additional inforation proided by the other graph, the detection threshold is substantially lowered on the right side of the plot, where the graph with ariable aerage degree is the densest. Our second experient deonstrates the optiality of the deried solution. We again fix the aerage degree of one of the graphs (in this case c 2 = 80), and now fix the clique size to k = 7. We then ary the aerage degree of G and its corresponding weight w = w 2. We consider c 2 {20, 40, 60, 80}. Probability of detection is shown in Fig. 2, where the epirical results are copared to the deried optial weight w =/(+(c /70)). For all alues of c, the epirical detection rate is axiized at the deried weight. This is true when c is sall, and ost of the weight is placed on w, and when c is large and the weights are approxiately equal. This behaior holds for arious clique sizes. Fig. 3 illustrates detection probability for three cases. In each case, two graphs Fig. 2. Probability of detection for a 7-ertex clique, ebedded into one graph with aerage degree 80 and one with aerage degree that aries. The weights used for fusion are aried to deonstrate optiality of the deried solution, which is drawn in white. are fused, with a planted clique whose size is at the detection threshold for the sparser graph (i.e., k = p c, where c apple c 2 ). We choose a case where the optial weighting has w =0.5, one with w =0.7, and one with w =0.9. By siply aeraging the graphs, we would achiee the detection probability at w =0.5, and the only case where this is optial is the case where the aerage degrees are equal. In the other cases, while the iproeent oer only considering the sparser graph (the detection probability achieed at the extree right of the plot), is ore subtle, there is a substantial iproeent that is axiized at the theoretically deterined weighting. Finally, we consider a case where we ary the aerage degrees of both graphs independently, and use the optial weighting. In this experient, the size of the planted clique is k =9. Detection probabilities are shown in Fig. 4. First, note that the probability of detection increases as either graph gets sparser. Cures indicating the detection thresholds for cliques of size 5 to 8 are oerlayed in the plot. These cures follow the forula: c 2 = /k 2. /c As we expect, at the thresholds for saller cliques, the 9- ertex clique is ore likely to be detected. It is also noteworthy that, along the cure where the threshold is the sae, the epirical detection probability reains consistent. This proides additional alidation that the optial weighting is correct: Using the optial weighting at a gien threshold gies a consistent detection probability, regardless of the densities of the indiidual graphs. V. CONCLUSION This paper extends recent spectral bounds for planted clique detection to cases where ultiple graphs are obsered. Operating in a ulti-source setting has becoe extreely iportant in recent years, as correlating obserations fro ultiple datasets has becoe ore coon. We deonstrate that the intuitie approach of weighting each graph in inerse proportion to its aerage degree is, in fact, the optial technique 523

5 Fig. 3. Probability of detection with respect to the weight of G in three scenarios. In each case, the theoretically optial weighting is indicated by a ertical dashed line of the sae color as the cure. beneficial. Fig. 4. Probability of detection for a 9-ertex clique, ebedded into two graphs of arying degree. Optiized weights are used to fuse the graphs. Detection thresholds for saller cliques (5 to 8 ertices) are drawn in white. under a linear weighting schee. Epirical results confir the theory in a wide range of settings. There are any possible future directions for this work. A siple extension of the result to dense subgraphs, rather than cliques, follows rather directly. Extending results to ore coplicated background odels such as Chung-Lu odels, with arbitrary aerage degree, or stochastic blockodels, with inherent counity structure would be another natural progression. This could also apply to cases where the subgraph changes oer the obserations, as in [2]. As data are often correlated across sources, extending to a setting where there are dependencies across the obserations would also be REFERENCES [] B. A. Miller, M. S. Beard, P. J. Wolfe, and N. T. Bliss, A spectral fraework for anoalous subgraph detection, IEEE Trans. Signal Process., ol. 63, no. 6, pp , Aug 205. [2] M. E. J. Newan and M. Giran, Finding and ealuating counity structure in networks, Phys. Re. E, ol. 69, no. 2, [3] D. Bu, Y. Zhao, L. Cai, H. Xue, X. Zhu, H. Lu, J. Zhang, S. Sun, L. Ling, N. Zhang, G. Li, and R. Chen, Topological structure analysis of the protein protein interaction network in budding yeast, Nucleic Acids Research, ol. 3, no. 9, pp , [4] M. Jerru, Large cliques elude the Metropolis process, Rando Structures & Algoriths, ol. 3, pp , 992. [5] N. Alon, M. Krieleich, and B. Sudako, Finding a large hidden clique in a rando graph, in Proc. ACM-SIAM Syp. Discrete Algoriths, 998, pp [6] V. Feldan, E. Grigorescu, L. Reyzin, S. Vepala, and Y. Xiao, Statistical algoriths and a lower bound for detecting planted clique, 205, preprint: arxi.org: [7] E. Arias-Castro and N. Verzelen, Counity detection in rando networks, 203, preprint: arxi.org: [8] R. R. Nadakuditi, On hard liits of eigen-analysis based planted clique detection, in Proc. IEEE Statistical Signal Process. Workshop, 202, pp [9] R. R. Nadakuditi and M. E. J. Newan, Graph spectra and the detectability of counity structure in networks, Phys. Re. Lett., ol. 08, p. 8870, May 202. [Online]. Aailable: [0] M. E. J. Newan, Finding counity structure in networks using the eigenectors of atrices, Phys. Re. E, ol. 74, no. 3, [] F. Benaych-Georges and R. R. Nadakuditi, The eigenalues and eigenectors of finite, low rank perturbations of large rando atrices, Adances in Math., ol. 227, no., pp , May 20. [2] B. A. Miller and N. T. Bliss, Toward atched filter optiization for subgraph detection in dynaic networks, in Proc. IEEE Statistical Signal Process. Workshop, 202, pp