Percolation Thresholds of Updated Posteriors for Tracking Causal Markov Processes in Complex Networks

Transcription

1 Percolation Thresholds of Udated Posteriors for Tracing Causal Marov Processes in Comlex Networs We resent the adative measurement selection robarxiv: v1 stat.ml 14 May 29 Patric L. Harrington Jr. Bioinformatics Graduate Program Deartment of Statistics University of Michigan Ann Arbor, MI 4819 Abstract Percolation on comlex networs has been used to study comuter viruses, eidemics, and other casual rocesses. Here, we resent conditions for the existence of a networ secific, observation deendent, hase transition in the udated osterior of node states resulting from actively monitoring the networ. Since traditional ercolation thresholds are derived using observation indeendent Marov chains, the threshold of the osterior should more accurately model the true hase transition of a networ, as the udated osterior more accurately tracs the rocess. These conditions should rovide insight into modeling the dynamic resonse of the udated osterior to active intervention and control olicies while monitoring large comlex networs. 1 INTRODUCTION Increasingly often, researchers are confronted with monitoring the states of nodes in large comuter, social, or ower networs where these states dynamically change due to viruses, rumors, or failures that roagate according to the grah toology 2, 5, 8. This class of networ dynamics has been extensively modeled as a ercolation henomenon, where nodes on a grah can randomly infect their neighbors. Percolation across networs has a rich history in the field of statistical hysics, comuter science, and mathematical eidemiology. Here, researchers are tyically confronted with a networ, or a distribution over the networ toology, and extract fixed oint attractors of node configurations, thresholds for hase transitions in node states, or distributions of node state lhjr@umich.edu hero@umich.edu Alfred O. Hero III. Deartment of EECS Deartment of Statistics University of Michigan Ann Arbor, MI, 4819 configurations 4, 1, 9. In the field of fault detection, the nodes or edges can fail, and the goal is to activate a subset of sensors in the networ which yield high quality measurements that identify these failures 12. While the former field of research concerns itself with extracting offline statistics about roerties of the ercolation henomenon on networs, devoid of any measurements, the latter field addresses online measurement selection tass. Here, we roose a methodology that actively tracs a causal Marov rocess across a comlex networ such as the one in Figure 3, where measurements are adatively selected. We extract conditions such that the udated osterior robability of all nodes infected is driven to one in the limit of large observation time. In other words, we derive conditions for the existence of an eidemic threshold on the udated osterior distribution over the states. The roosed ercolation threshold should more accurately reflect the true conditions that cause a hase transition in a networ, e.g., node status changing from healthy/normal to infected/failed, than traditional thresholds derived from conditions on redictive distributions which are devoid of observations or controls. Since most ractical networs of interest are large, such as electrical grids, it is usually infeasible to samle all nodes continuously, as such measurements are either exensive or bandwidth is limited. Given these, or other resource constraints, we resent an information theoretic samling strategy that selectively targets secific nodes that will yield the largest information gain, and thus, better detection erformance. The roosed samling strategy balances the tradeoff between trusting the redictions from the nown model dynamics from ercolation theory and exending recious resources to select a set of nodes for measurement.

2 lem and give two tractable aroximations to this subset selection roblem based uon the joint and marginal osterior distribution, resectively. A set of decomosable Bayesian filtering equations are resented for this adative samling framewor and the necessary tractable inference algorithms for comlex networs are discussed. We resent analytical worst case erformance bounds for our adative samling erformance, which can serve as samling heuristics for the activation of sensors or trusting redictions generated from revious measurements. To the author s nowledge, this is the first attemt to extract a ercolation threshold of an actively monitored networ using the udated osterior distribution instead of the observation indeendent redictive distributions. may be modeled as a decomosable artially observed Marov rocess: Y i = fz i + w i 4 Z i = h Z 1, i {Z j 1 } j ηi. 5 Here, ηi = {j : E V i, V j / } is the neighborhood of i, fz i is a non-random vector-valued function, wi is measurement noise, and h Z 1 i, {Zj 1 } j ηi is a stochastic equation encoding the transition dynamics of the Marov rocess see Figure 1 for a two node grahical model reresentation. Z i 1 Z j 1 2 PROBLEM FORMULATION Y i 1 Y j 1 Z i Z j The objective of actively monitoring the n node networ is to recursively udate the osterior distribution of each hidden node state given various measurements. Secifically, the next set of m measurement actions nodes to samle, m n, at next discrete time are chosen such that they yield the highest quality of information about the n hidden states. The condition on m n simulates the reality of fixed resource constraints, where tyically only a small subset of nodes in a large networ can be observed at any one time. Here, the hidden states are discrete random variables that corresond to the states encoded by the ercolation rocess on the grah. Here, the grah G = V, E, with V reresenting the set of nodes and E corresonding to the set of edges. Formally, we will assume a state-sace reresentation of a discrete time, finite state, artially observed Marov decision rocess POMDP. Here, Z = {Z 1,..., Z n } 1 reresents the joint hidden states, e.g., healthy or infected Y = {Y 1,..., Ym } 2 reresents the m observed measurements obtained at time, e.g., biological assays or PINGing an IP address, and a = {a 1,..., a m } 3 reresents the m actions taen at time, i.e., which nodes to samle. Here, Y j, continuous/categorical valued vector of measurements, which is induced by action a j, aj A, with A = {1,..., n} confined to be the set of all n individuals in the grah, and Z i {, 1,..., r}. Since the toology of G encodes the direction of flow for the rocess, the state equations Figure 1: Partially Observed Marov Structure for i and j for E V i, V j / Y i 2.1 BAYESIAN FILTERING In our roosed framewor for actively monitoring the hidden node states in the networ, the osterior distribution is the sufficient statistic for inferring these states. The general recursion for udating the joint osterior robability given all ast and resent observations is given by the standard Bayes udate formula: with Z Y = fy Z gy Y 1 Z Y 1 6 Z Y 1 = z {,1,...,r} n Y j Z Z 1 = zz 1 = z Y 1. 7 The Chaman-Kolmogorov equations rovide the connection between the osterior udate 7 and the distribution resulting from the standard ercolation equations. In the former, the udates are conditional robabilities that are conditional on ast observations, while in the latter, the udates are not deendent on observations. The local interactions in the grah G imly the following conditional indeendence assumtions: fy Z = n fy i Z. i 8 i=1

3 Z Z 1 = n i=1 Z Z i 1, i {Z j 1 } j ηi 9 where the lielihood term is defined in 4 and the transition dynamics are defined in 5. This decomosable structure allows the belief state osterior excluding time observations udate, for the i th node in G, to be written as: Z Y 1 = Z Z a 1 = zza 1 = z Y 1 z {,1,...,r} a 1 with the arent set, a = {ηi, i}. Unfortunately, for highly connected nodes in G, this marginal udate becomes intractable. It thus must be aroximated 3, 1, INFORMATION THEORETIC ADAPTIVE SAMPLING In most real world situations, acquiring measurements from all n nodes at any time is unrealistic, and thus, a samling olicy must be exloited for measuring a subset of nodes 6, 12. Since we are concerned with monitoring the states of the nodes in the networ, an aroriate reward is the exected information gain between the udated osterior, = Z {Y i } i a, Y 1, and the belief state, 1 = Z Y 1 : a = argmax a A E D α {Y i } i a Y 1 11 D α {Y i } i a = Dα 1, < α < 1 12 with α-divergence D α q = 1 α 1 log E q /q α 13 for distributions and q with identical suort. The reward in 11 has been widely alied to multi-target, multi-sensor tracing for many roblems including, sensor management and surveillance 6, 11. Note that lim α 1 D α q D KL q, where D KL q is the Kullbac-Leibler divergence between and q. The exectation in 11 is taen with resect to the conditional distribution gy Y 1 given the revious measurements Y 1 and actions a. In ractice, the exected information divergence in 11 must be evaluated via Monte-Carlo methods. Also, the maximization in 11 requires enumeration over all n m actions for subsets of size m, and therefore, we must resort to Greedy aroximations. We roose incrementally constructing the set of actions at time, a, for j = 1,..., m, according to: a j = argmax i A\a E D α Y i, {Y j } j a Y Both 11 and 14 are selecting the nodes to samle which yield maximal divergence between the ercolation rediction distribution belief state and the udated osterior distribution, averaged over all ossible observations. Thus 11 rovides a metric to assess whether to trust the redictor and defer actions until a future time or choose to tae action, samle a node, and udate the osterior Lower Bound on Exected α-divergence Since the exected α-divergence in 11 is not closed form, we could resort to numerical methods for estimating this quantity. Alternatively, one could secify an analytical lower-bound that could be used in-lieu of numerically comuting the exected information gain in 11 or 14. We begin by noting that the exected divergence between the udated osterior and the redictive distribution conditioned on revious observations differ only through the measurement udate factor, f /g 1 11 re-written: E g 1 Dα 1 = E g 1 1 α 1 log E 1 f g 1 α. 15 So, if there is significant overla between the lielihood distributions of the observations, the exected divergence will tend to zero, imlying that there is not much value-added in taing measurements, and thus, it is sufficient to use the ercolation redictions for inferring the states. It would be convenient to interchange the order of the conditional exectations in 15. It is easily seen that Jensen s inequality yields the following lower bound for the exected information gain E g 1 Dα 1 1 α 1 log E 1 E g 1 f g 1 α. 16 Here, the inner conditional exectation can be obtained from D α f g 1, which has a closed form for common distributions e.g., multivariate Gaussians 6. 3 ASYMPTOTIC ANALYSIS OF MARGINAL POSTERIOR For tracing the ercolation rocess across G, we have discussed recursive udating of the belief state. However, comuting these udates exactly is in general intractable. For the remainder of the aer, we will use

4 4 and 5 to directly udate the marginal osterior distribution using the following matrix reresentation: z = D z 1 z 17 with udated marginal osterior z = 1, z,..., n, z T with i, z = Z i = z Y i, Y 1, D z = diag f z i, /g i, 1, and marginal belief state 1 z = 1, 1 z,..., n, 1 z T with i, 1 z = Z i = z Y 1. Note that for i / a, D z i,i = 1, and i, z = i, 1 z. Given that we can find an efficient way of udating 1 z, according to the transition dynamics 5, we can solve a modified version of 14, for j = 1,..., m: a j = argmax i A\a E D α Y i Y 1 18 D α Y i = Dα i, z i, 1 z, < α < TOTAL DIVERGENCE OF UPDATED POSTERIOR One interesting roerty of the Bayesian filtering equations is that the udated osterior can be written as a erturbation of the redictive ercolation distribution through the following relationshi z omitted for clarity: = D 1 = 1 + D I 1. 2 Hence, when the sensors do a oor job in discriminating the observations, D I, we have 1. It is of interest to determine when there is significant difference between the osterior udate and the rior udate secified by the standard ercolation equations. Recall that the udated osterior is, in the mean, equal to the redictive distribution, E Y 1 1. The total deviation of the udated osterior from the ercolation distribution can be summarized by comuting the trace of the following conditional covariance:» h tr E E h tr R Y 1 Y 1 i = i = 21 h i «T E Y 1 Y 1. Using 2 and roerties of the trace oerator, we obtain the following measure of total deviation of the udated osterior from the redictive distribution in terms of f and g 1 : tr R Y 1 = tr E D I 2 Y 1 P 1 22 with P 1 = 1 T 1. The conditional exectation in 22 is the Pearson χ 2 divergence between distributions f i, and g i, 1, for all i. This joint measure of deviation is analytical for articular families of distributions and thus can be used as an alternative measure of divergence for activation of sensors PERCOLATION THRESHOLD OF UPDATED POSTERIOR There has recently been significant interest in deriving the conditions of a ercolation/eidemic threshold in terms of transition arameters and the grah adjacency matrix sectra for two state causal Marov rocesses 1, 4, 9. Such thresholds yield conditions necessary for an eidemic to arise from a small number of infections. Knowledge of these conditions are articularly useful for designing robust networs, where the robability of eidemics is minimized. Percolation thresholds are tyically obtained by extracting the sufficient conditions of the networ and model arameters for the node states to be driven to their stationary oint, with high robability. The robability of these events are usually comuted using the observation indeendent ercolation distribution 1, 4, 9. We use the results in 1, 4 to derive a ercolation threshold based uon the udated osterior distribution assuming a restricted class of two-state Marov rocesses. These conditions should more accurately model the current networ threshold since the osterior distribution tracs a articular disease trajectory better than the observation indeendent ercolation distribution. Formally, Z i z {, 1}, f i, = fy i Z i = z is the conditional lielihood for node i, i, = Z i = 1 Y i, Y 1, and i, = Z i = 1 Y 1. Here, we will assume that Z = is the unique absorbing state of the system. The Bayes udate for i, can be written as i subscrit omitted for clarity: = = = f 1 f f f f 1 /f f f 1 1 f 1 /f 1 + f f There are three different samling/observation deendent ossibilities for each individual at time : case

5 1, i is not samled and therefore, = 1, case 2, f >, and case 3, f <. We first derive a tight-uer bound for cases 2 and 3 of the form c 1. For the remainder of the analysis we will assume that f 1 < 1 for cases 2 and 3 see Aendix. f Using the uer-bounds derived in the Aendix, and after gathering all n nodes, we have the following element-wise uer-bound on the udated belief state: C 1 = B + O with B = diag b i, and O = f diag I i, { fi, <}O f i, 1 where I { fi, <} i, is the indicator function for the event f i, <. Thus far, we have established, under the assumtions of f 1 < 1, an uer-bound for the udated f osterior in terms of observation lielihoods and the belief state 24. Next, consider the restricted class of two-state Marov rocesses on G, for which we can roduce a bound of the form 1 S 1 25 where S contains information about the transition arameters and the toology of the networ. It turns out that the SIS model of mathematical eidemiology falls within this restricted class of ercolation roblems 1. The SIS model on a grah G, assumes that each of the n individuals are in states or 1, where corresonds to suscetible and 1 corresonds to infected. At any time, an individual can receive the infection from their neighbors, ηi, based uon their states at 1. Under this SIS model in 1 S = 1 γi + βa 26 where the Marov transition arameters γ is the robability of i transitioning from 1 to, β is the robability of transmission between neighbors i and j, and A is the grah adjacency matrix see Figure 2. Returning to the derivation, using the bound 25, we have, by induction, the following recursion: C 1 C S 1 C S C 1 S = B S B 1 S + O C S 27 where we have lumed the higher order modes and higher order cross-terms into O C S. The dominant mode of decay of the udated osterior may be found by investigating the following eigendecomosition: n n B S = b j, e j e T j λ j u j u T j 28 j=1 j=1 with e j =,...,, 1,,..., T 1 at j th element. Without loss of generality, we can assume the eigenvalues of S are listed in decreasing order, λ 1 λ n. Now rewriting 28, we have B S = b j e j e T j + O B λ1 u 1 u T 1 + O S = λ 1 b j e j e T j u 1 u T 1 + O BS 29 where b j = max j {1,...,n} b j, and the O B, O S, O BS variables corresonds to the higher order terms. Inserting 29 into 27, and matching the largest eigenvalues of B with λ 1 we obtain B S... B 1 S + O C S = λ 1 b jl ejl e T j l u 1 u T 1 +Oϕ. 3 l=1 l=1 Thus, at large, the dominant mode of the osterior goes as λ 1 l=1 b j l the modes in Oϕ decay faster than the dominant mode resented above. We can see that if the sectral radius of S is less than one, λ 1 < 1, then for large,, which is the unique absorbing state of the system. This eidemic threshold condition on λ 1 has been reviously established for unforced SIS-ercolation rocesses 1. However, in the tracing framewor, the rate at which the osterior decays to the suscetible state is erturbed by an additional measurement deendent factor, l=1 b j l. This measurement-deendent dominant mode of the osterior should more accurately model the true dynamic resonse of the node states better than that in 1 since the osterior better tracs the truth than the unforced redictive distribution. Additionally, this dominant mode of the udated osterior distribution allows one to simulate the resonse of the ercolation threshold to intervention and control actions which are designed to increase the threshold, such that the robability of eidemics is minimized. 4 NUMERICAL EXAMPLE Here, we resent results of simulations of our adative samling for the active tracing of a causal Marov ground truth rocess across a random 2 node, scalefree networ Figure 3. Since the goal in tracing

6 1 q I 1 S I η q γ I 1 γ Figure 2: SIS Marov Chain for Node i Interacting with the Infected States of its Neighbors is to accurately classify the states of each node, we are interested in exloring the detection erformance as the lielihood of an eidemic increases through the ercolation threshold for this grah. One would exect different hase transitions thresholds in detection erformance for various samling strategies, ranging from the lowest threshold for unforced ercolation distributions to highest for a continuous monitoring of all n nodes. We will resent a few of these detection surfaces that deict these hase transitions for the unforced ercolation distribution, random m = 4 node samling, and our roosed information theoretic adative samling of m = 4. Here, we will restrict our simulations to the two-state SIS model of mathematical eidemiology described above. Figure 3: 2 Node Scale-Free Grah G = V, E The sensor models 4, are of the form of twodimensional multivariate Guassians with common covariance and shifted mean vector. The transition dynamics of the i th individual 5, for the SIS model is given by: 2 3 Z Z i {i,ηi} 1 1 γz 1+1 Z i 1 i 41 Y 1 βz j 1 5. j ηi 31 where Z 1 i {, 1} is the indicator function of i being infected at time 1. The transmission term between i and ηi is nown the Reed-Frost model 1, 4, 8. Since the tail of the degree distribution of our synthetic scale-free grah contains nodes with degree greater than 1, udating 1 exactly is unrealistic and we must resort to aroximate algorithms. Here, we will assume the mean field aroximation used by 1 for this SIS model, resulting in the following marginal belief state udate for the i th node of infected Z i = 1: " # i, 1 = 1 γ i, 1 +1 i, 1 1 Y j η1 β j, 1 32 Equation 32 allows us to efficiently udate the marginal belief state directly for all n nodes which are then used for estimating the best m measurements using 18. As we are interested in detection erformance as a function of time and eidemic intensity, the Area Under the ROC Curve AUR is a natural statistic to quantify the detection ower detection of the infected state. The AUR is evaluated at each time, each SIS ercolation intensity arameter τ = β/γ 33 and over 5 random initial states of the networ. For the SIS model, τ is the single arameter aside from the toology of the grah that characterizes the intensity of the ercolation/eidemic. It is useful to understand how the detection erformance varies as a function of eidemic intensity, as it indicates how well the udated osteriors are laying catch-u in tracing the true dynamics on the networ. For this SIS model, the ercolation threshold is defined as τ c = 1/λ 1 A where λ 1 A = max i {1,...,n} λ i is the sectral radius of the grah adjacency matrix, A 1. Values of τ greater than τ c imly that any infection tend to become an eidemic, whereas those values less than τ c imly that small eidemics tend to die out. For the networ under investigation Figure 3, τ c = We see from Figure 4a that a hase transition in detection ower AUR for the unforced ercolation distribution does indeed coincide with the eidemic threshold τ c. While the eidemic threshold for the random and adative samling olicies is still τ c =.1819, the measurements acquired allow the osterior to better trac the truth, but only u to their resective hase transitions in detection ower see Figures 4b and 4c. Figure 4c confirms that the adative samling better tracs the truth than randomly samling nodes, while ushing the hase transition in detection erformance to higher ercolation intensities, τ. We see that the major benefit of the adative samling is aarent when conditions of the networ are changing.

7 Relative Frequency ling method. AUR a Tau.3333 Relative Frequnecy.6.9 a AUR Surface for Unforced Prediction Distribution no evidence acquired throughout the monitoring Relative Frequency Degree b c AUR Degree Figure 5: Relative Frequency of Nodes Samled zaxis of a Given Degree x-axis Over y-axis for m = 4 Adative Samling Strategy: a. τ =.125, b. τ =.2143, and c. τ = Tau.3333 It is often useful for develoing samling heuristics and offline control/intervention olicies to insect what b AUR Surface for Udated Posterior Distribution with m = tye of nodes, toologically seaing, is the adative 4 Random Measurements at Each samling strategy targeting, under various networ conditions different values of τ. In Figure 5, the relative frequency of nodes samled with a articular degree is lotted against time under the m = 4 adative samling strategy for three different values of τ over 5 random initial conditions of the networ AUR.9.8 For the larger of the three values exlored τ =.5 > τc we see that the samling is aroximately uniform across the nodes of each degree on the grah Figure 5c. Therefore, under extremely intense eidemic conditions, the adative samling strategy is targeting all nodes of each degree equally, and therefore, it c AUR Surface for Udated Posterior Distribution with is sufficient to erform random samling. For the two m = 4 Information Theoretic Adative Measurements at lower values of τ, Figure 5a and Figure 5b near Each τc, we see that adative olicy targets highly connected nodes more frequently than those of lesser defigure 4: Area under the ROC curve surface as a funcgree and thus, it is more advantageous to exloit such tion of ercolation arameter τ = β/γ and time a strategy, as comared to random samling see AUR surface in Figure 4c Tau moderately, at medium eidemic conditions. Beyond a certain level of ercolation intensity, more resources will need to be allocated to samling to maintain a high level of detection erformance. A heuristic samling strategy based on the toology of G was also exlored results not shown by samling the hubs highly-connected nodes. However, detection erformance was only slightly better than random samling and oorer than our adative sam- 5 DISCUSSION In this aer, we have derived the conditions for a networ secific ercolation threshold using exressions for the udated osterior distribution resulting from actively tracing the rocess. These conditions recover the unforced ercolation threshold derived in 1 but with an additional factor involving sensor lielihood terms due to measurements obtained throughout the Degree

8 monitoring. A term of the form λ 1 l=1 b j l derived in 3 was shown to be the dominant mode of the udated osterior dynamic resonse to active intervention of immunizing the nodes holding node states constant. The conditions of the ercolation using the udated osterior should more accurately model the hase transition corresonding to a articular disease trajectory and therefore, enable a better assessment of immunization strategies and any subsequent observations resulting from such actions. The framewor resented above, along with the new osterior ercolation threshold, should rovide additional insight into active monitoring of large comlex networs under resource constraints. 6 APPENDIX In case 2, when f >, we can re-write 23 in terms of an alternating geometric series: = f 1 l 1 l f f l= f 1 1 f 1 f 1 + f f where we have used the fact that 1/1 + a 1 + a. Recalling that 2 for 1, we have f f f f In case 3, when f <, and 23 can be reresented as a geometric series: 2! 3 = f 1 l X 4 f f l= f = f 1 f 41 + f f 1 + X l=2 f f! 3 l Once again, using the 2 bound, we obtain: f f f f f 1 + O f A general inequality, that is equality for case 1, is of the form c 1 with 1, i / a b = f f, f f f > or f < with c = b for cases 1 and 2 and c = b + f O f 1 for case 3. References 1 D. Charabarti, Y. Wang, C. Wang, J. Lesovec, and C. Faloutsos, Eidemic thresholds in real networs, ACM Trans. Inf. Syst. Secur., 1 28, R. Cohen, S. Havlin, and D. ben Avraham, Efficient immunization strategies for comuter networs and oulations, Physical Review Letters, 91 23, A. Doucet, S. Godsill, and C. Andrieu, On sequential monte carlo samling methods for bayesian filtering, Statistics and Comuting, 1 2, M. Draief, A. J. Ganesh, and L. Massoulié, Thresholds for virus sread on networs, Ann. Al. Probab., 18 28, S. Euban, H. Guclu, Anil, M. V. Marathe, A. Srinivasan, Z. Toroczai, and N. Wang, Modelling disease outbreas in realistic urban social networs, Nature, , A. O. Hero, D. Castenon, D. Cochran, and K. D. Kastella, Foundations and alications of sensor management, Sringer, NY, D. J. C. Macay, Information Theory, Inference & Learning Algorithms, Cambridge University Press, M. E. J. Newman, The sread of eidemic disease on networs, Physical Review E, 66 22, M. E. J. Newman, Threshold effects for two athogens sreading on a networ, Physical Review Letters, 95 25, B. Ng, L. Peshin, and A. Pfeffer, Factored articles for scalable monitoring, in In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, 22, Z. Rabinovich and J. S. Rosenschein, Extended marov tracing with an alication to control, in The Third International Joint Conference on Autonomous Agents and Multiagent Systems, 24, A. X. Zheng, I. Rish, and A. Beygelzimer, Efficient test selection in active diagnosis via entroy aroximation, in Proceedings of the 21th Annual Conference on Uncertainty in Artificial Intelligence, 25,. 675.