Goodness of fit of logistic models for random graphs: a variational Bayes approach

Size: px

Start display at page:

Download "Goodness of fit of logistic models for random graphs: a variational Bayes approach"

Thomas Palmer
5 years ago
Views:

1 Goodness of fit of logistic models for random graphs: a variational Bayes approach S. Robin Joint work with P. Latouche and S. Ouadah INRA / AgroParisTech MIAT Seminar, Toulouse, November 2016 S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

2 Problem Motivating example Data: n = 51 tree species, Y ij = number of common parasites [23]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

3 Problem Motivating example λ kl T1 T2 T3 T4 T5 T6 T7 Data: n = 51 tree species, Y ij = number of common parasites [23]. T T T T T T T π k Valued stochastic block model. If i C k, j C l : Y ij P(λ kl ), λ kl = mean number of shared parasites. Pseudo-BIC: K = 7 groups S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

Problem Motivating example λ kl T1 T2 T3 T4 T5 T6 T7 Data: n = 51 tree species, Y ij = number of common parasites [23]. T1 14.46 4.19 5.99 7.67 2.44 0.13 1.43 T2 14.13 0.68 2.79 4.84 0.53 1.54 T3 3.

4 Problem Motivating example λ kl T1 T2 T3 T4 T5 T6 T7 Data: n = 51 tree species, Y ij = number of common parasites [23]. T T T T T T T π k Valued stochastic block model. If i C k, j C l : Y ij P(λ kl ), λ kl = mean number of shared parasites. Pseudo-BIC: K = 7 groups S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

5 Problem Accounting for the taxonomic distance d ij Model: [19] Y ij P[λ kl e βd ij ]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

6 Problem Accounting for the taxonomic distance d ij λ kl T 1 T 2 T 3 T 4 Model: [19] T T T T π k β Y ij P[λ kl e βd ij ]. Results: β = Pseudo-BIC: K = 4 groups. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

7 Problem Accounting for the taxonomic distance d ij λ kl T 1 T 2 T 3 T 4 Model: [19] T T T T π k β Y ij P[λ kl e βd ij ]. Results: β = Pseudo-BIC: K = 4 groups. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

8 Problem Problem Ecological network Y = (Y ij ) 1 i,j n : interactions between n tree species: Y ij = I{i j} = I{i and j share some fungal parasite} in this talk, binary networks only. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

9 Problem Problem Ecological network Y = (Y ij ) 1 i,j n : interactions between n tree species: Y ij = I{i j} = I{i and j share some fungal parasite} in this talk, binary networks only. + Edge covariates x = (x ij ) 1 i,j n : x 1 ij = phylogenetic distance between i and j, x 2 ij = geographic distance between the typical habitat of i and j,... S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

10 Problem Problem Ecological network Y = (Y ij ) 1 i,j n : interactions between n tree species: Y ij = I{i j} = I{i and j share some fungal parasite} in this talk, binary networks only. + Edge covariates x = (x ij ) 1 i,j n : x 1 ij = phylogenetic distance between i and j, x 2 ij = geographic distance between the typical habitat of i and j,... Questions: Can we explain the topology of Y based on x? Is there a residual heterogeneity in the network Y? S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

11 Problem Problem Ecological network Y = (Y ij ) 1 i,j n : interactions between n tree species: Y ij = I{i j} = I{i and j share some fungal parasite} in this talk, binary networks only. + Edge covariates x = (x ij ) 1 i,j n : x 1 ij = phylogenetic distance between i and j, x 2 ij = geographic distance between the typical habitat of i and j,... Questions: Can we explain the topology of Y based on x? Is there a residual heterogeneity in the network Y? logistic regression W -graph S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

12 Problem Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

13 Reminder of variational Bayes inference Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

14 Reminder of variational Bayes inference Variational Bayes inference: General principle Y = observed data, Z = latent variable, θ = parameter. Frequentist or Bayesian inference requires p(z Y ), p(θ Y ), p(θ, Z Y ). S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

15 Reminder of variational Bayes inference Variational Bayes inference: General principle Y = observed data, Z = latent variable, θ = parameter. Frequentist or Bayesian inference requires p(z Y ), p(θ Y ), p(θ, Z Y ). Variational inference: find p( ) p( Y ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

16 Reminder of variational Bayes inference Variational Bayes inference: General principle Y = observed data, Z = latent variable, θ = parameter. Frequentist or Bayesian inference requires p(z Y ), p(θ Y ), p(θ, Z Y ). Variational inference: find p( ) p( Y ) Typically [11,21,25] D[q p] = KL[q p]; p(θ) = N ; p = arg min D[q( ) p( Y )] q Q p(z) = i q i(z i ); p(θ, Z) = p(θ) p(z). S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

17 Reminder of variational Bayes inference Variational Bayes inference: Some tools Latent variable models: mean field approximation The VBEM algorithm [1] achieves p(θ, Z) := p(θ) p(z), min KL[ p(θ, Z) p(θ, Z Y )] max log p(y ) KL[ p(θ, Z) p(θ, Z Y )] p p S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

18 Reminder of variational Bayes inference Variational Bayes inference: Some tools Latent variable models: mean field approximation The VBEM algorithm [1] achieves p(θ, Z) := p(θ) p(z), min KL[ p(θ, Z) p(θ, Z Y )] max log p(y ) KL[ p(θ, Z) p(θ, Z Y )] p p Bayesian model averaging (BMA) [8]: Consider models M 1,..., M K,... E[h(θ) Y ] = K p(k Y ) E[h(θ) Y, K] S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

19 Reminder of variational Bayes inference Variational Bayes inference: Some tools Latent variable models: mean field approximation The VBEM algorithm [1] achieves p(θ, Z) := p(θ) p(z), min KL[ p(θ, Z) p(θ, Z Y )] max log p(y ) KL[ p(θ, Z) p(θ, Z Y )] p p Bayesian model averaging (BMA) [8]: Consider models M 1,..., M K,... E[h(θ) Y ] = K p(k Y ) E[h(θ) Y, K] Variational approximation of p(k Y ) [24]: p(k) p(k Y ) e KL K p(k) e log p(y K) KL K where KL K = minimized KL divergence for model M K. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

20 Logistic regression Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

21 Logistic regression Logistic regression Forget about the network structure: (Y ij ) are independent Y ij B[g(x ij β)], g(u) = 1/(1 + e u ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

22 Logistic regression Logistic regression Forget about the network structure: (Y ij ) are independent Y ij B[g(x ij β)], g(u) = 1/(1 + e u ) Bayesian inference. Prior on β: β N (m, G 1 ) Posterior distribution: p(β Y ) p(β)p(y β) No nice conjugacy holds for the normal prior / logistic regression model. No close form for the posterior p(β Y ). S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

23 Logistic regression Variational Bayes inference Nasty term in log p(y β): log(1 + e x ij β ). S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

24 Logistic regression Variational Bayes inference Nasty term in log p(y β): log(1 + e x ij β ). [12]: quadratic lower bound of log p(β, Y ) to get p(β Y ) p(β) = N (β; m, G 1 ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

25 Logistic regression Variational Bayes inference Nasty term in log p(y β): log(1 + e x ij β ). [12]: quadratic lower bound of log p(β, Y ) to get p(β Y ) p(β) = N (β; m, G 1 ) Example: Tree network. n = 51 tree species, (N = 1275 pairs), 3 distances: intercept genetic geographic taxonomic µ β σ β S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

26 W -graphs and graphon Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

27 W -graphs and graphon Latent space models Generic framework [2]: Latent variable for node i: Z i π. Edges Y ij conditionally independent: (Z i ) iid π, Y ij (Z i ) : Y ij B[γ(Z i, Z j )] S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

28 W -graphs and graphon Latent space models Generic framework [2]: Latent variable for node i: Z i π. Edges Y ij conditionally independent: (Z i ) iid π, Y ij (Z i ) : Y ij B[γ(Z i, Z j )] Includes [20]: stochastic block-model [5], latent position models [9,7,3], W -graph [18,4] limit model of most of these models. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

29 W -graphs and graphon Latent space models Generic framework [2]: Latent variable for node i: Z i π. Edges Y ij conditionally independent: (Z i ) iid π, Y ij (Z i ) : Y ij B[γ(Z i, Z j )] Includes [20]: stochastic block-model [5], latent position models [9,7,3], W -graph [18,4] limit model of most of these models. Intricate dependency structure: sampling and/or approximation is required for the inference of p(z, θ Y ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

30 W -graphs and graphon W -graph (graphon) model Latent variables: Graphon function γ(u, v) (U i ) iid U [0,1], Graphon function γ: γ(u, v) : [0, 1] 2 [0, 1] Edges: P(Y ij = 1 U i, U j ) = γ(u i, U j ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

31 W -graphs and graphon Some typical graphon functions Graphon function γ: a global picture of the network s topology. Scale free Community Small world S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

32 W -graphs and graphon Some typical graphon functions Graphon function γ: a global picture of the network s topology. Scale free Community Small world Remark: γ(, ) =cst ER model S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

33 W -graphs and graphon Variational Bayes inference of the graphon (1/3) Stochastic Block-model (SBM) is a W -graph model. Latent variables: (Z i ) iid M(1, π) Blockwise constant graphon: γ(z, z ) = γ kl Edges: P(Y ij = 1 Z i, Z j ) = γ(z i, Z j ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

34 W -graphs and graphon Variational Bayes inference of the graphon (1/3) Stochastic Block-model (SBM) is a W -graph model. Latent variables: (Z i ) iid M(1, π) Blockwise constant graphon: γ(z, z ) = γ kl Edges: P(Y ij = 1 Z i, Z j ) = γ(z i, Z j ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

35 W -graphs and graphon Variational Bayes inference of the graphon (1/3) Stochastic Block-model (SBM) is a W -graph model. Latent variables: Graphon function of SBM K (Z i ) iid M(1, π) Blockwise constant graphon: γ(z, z ) = γ kl Edges: P(Y ij = 1 Z i, Z j ) = γ(z i, Z j ) block widths = π k, block heights γ kl S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

36 W -graphs and graphon Variational Bayes inference of the graphon (2/3) VBEM for SBM: conjugate priorexponential family setting [13]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

37 W -graphs and graphon Variational Bayes inference of the graphon (2/3) VBEM for SBM: conjugate priorexponential family setting [13]. Approximate posteriors p(π) = D( π) p(γ kl ) = B( γ kl, 0 γ kl) 1 p(z i ) = M(1; τ i ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

38 W -graphs and graphon Variational Bayes inference of the graphon (2/3) VBEM for SBM: conjugate priorexponential family setting [13]. Posterior mean ẼSBM K [γ(u, v)] Approximate posteriors p(π) = D( π) p(γ kl ) = B( γ kl, 0 γ kl) 1 p(z i ) = M(1; τ i ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

39 W -graphs and graphon Variational Bayes inference of the graphon (3/3) Variational Bayes model averaging. Each auxiliary model SBM K provides an estimates of γ as: γ K (u, v) = ẼSBM K [γ(u, v)]. VB inference also provides an approximation of the posterior probability of each auxiliary model SBM K : p(k) p(k Y ) A averaged estimate of γ is given by [15]: Ẽ[γ(u, v)] = K p(k) ẼSBM K [γ(u, v)]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

40 W -graphs and graphon Example: Tree network n = 51 tree species. Inferred graphon function: S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

41 W -graphs and graphon Example: Political blog network n = 196 blogs. Inferred graphon function: S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

42 W -graphs and graphon Example: Political blog network n = 196 blogs. Inferred graphon function: Graphical representation of the network. Interpretability... S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

43 Goodness-of-fit Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

44 Goodness-of-fit Back to the original problem Data: Y = observed (binary) network x = covariates Questions: Does x explain the topology of Y? Residual (wrt x) heterogeneity in Y? S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

45 Goodness-of-fit Combined model Logistic regression: logit P(Y ij = 1) = β 0 + k β k x k ij S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

46 Goodness-of-fit Combined model Logistic regression: logit P(Y ij = 1) = β 0 + k β k x k ij Logistic regression + graphon residual term: (U i ) iid U[0, 1], logit P(Y ij = 1 U i, U j ) = φ(u i, U j ) + k β k x k ij S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

47 Goodness-of-fit Combined model Logistic regression: logit P(Y ij = 1) = β 0 + k β k x k ij Logistic regression + graphon residual term: (U i ) iid U[0, 1], logit P(Y ij = 1 U i, U j ) = φ(u i, U j ) + k β k x k ij Goodness of fit [16]: Check if φ(u, v) = cst (= β 0 ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

48 Goodness-of-fit Goodness-of-fit as model comparison Auxiliary model M K : logit P(Y ij = 1 U i, U j, K) = φ SBM K (U i, U j ) + k β k x k ij. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

49 Goodness-of-fit Goodness-of-fit as model comparison Auxiliary model M K : logit P(Y ij = 1 U i, U j, K) = φ SBM K (U i, U j ) + k β k x k ij. Goodness of fit: H 0 = {logistic regression is sufficient} = M 1 H 1 = {logistic regression is not sufficient} = K>1 M K GOF is a assessed if p(h 0 Y ) = p(m 1 Y ) is large. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

50 Goodness-of-fit Variational Bayes inference VBEM: A global variational Bayes EM algorithm can be designed to obtain all needed (approximate) posteriors: p(θ, Z, K Y ) p(θ, Z, K) = p(k) p(θ K) p(z K) ( ) = p(k) ( p(α K) p(β K) p(π K)) p(z i K) i S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

51 Goodness-of-fit Variational Bayes inference VBEM: A global variational Bayes EM algorithm can be designed to obtain all needed (approximate) posteriors: p(θ, Z, K Y ) p(θ, Z, K) = p(k) p(θ K) p(z K) ( ) = p(k) ( p(α K) p(β K) p(π K)) p(z i K) i Posterior quantities of interest: Goodness of fit criterion: p(h 0 Y ) P(K = 1) Residual graphon: φ(u, v) = K p(k) Ẽ[φSBM K (u, v)] S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

52 Illustrations Outline Reminder of variational Bayes inference Logistic regression W -graphs and graphon Goodness-of-fit Illustrations S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

53 Illustrations Some examples network size (n) nb. covariates (d) density ˆp(H 0 Y ) Blog e-172 Tree e-115 Karate e-2 Florentine (marriage) Florentine (business) Faux Dixon High CKM AddHealth e-25 S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

54 Illustrations Political blog network n = 196 blogs (N = pairs), 3 covariates, density =.075 Inferred graphon (no covariate) Residual graphon (3 covariates) P(H 0 ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

55 Illustrations Tree network n = 51 species (N = 1275 pairs), 3 covariates, density =.54 Inferred graphon (no covariate) Residual graphon (3 covariates) P(H 0 ) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

56 Illustrations Florentine business n = 16 families (N = 120 pairs), 3 covariates, density =.12 Inferred graphon (no covariate) Residual graphon (3 covariates) P(H 0 ) =.991 S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

57 Illustrations Physicians friendship network n = 219 individuals, 24 covariates (39 df), density =.015 Inferred graphon (no covariate) Residual graphon (3 covariates) P(H 0 ) 1 S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

58 Discussion Discussion Contribution: Generic logistic regression model with a network-oriented residual term. Detour through SBM provides a natural goodness-of-fit criterion. R package on github.com/platouche/gofnetwork (soon on CRAN) S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

59 Discussion Discussion Contribution: Generic logistic regression model with a network-oriented residual term. Detour through SBM provides a natural goodness-of-fit criterion. R package on github.com/platouche/gofnetwork (soon on CRAN) Comments: Strongly relies on variational Bayes approximation of the posteriors. VBEM asymptotically accurate for logistic regression and SBM separately. No clue about accuracy in the combined model. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

60 Discussion Discussion Contribution: Generic logistic regression model with a network-oriented residual term. Detour through SBM provides a natural goodness-of-fit criterion. R package on github.com/platouche/gofnetwork (soon on CRAN) Comments: Strongly relies on variational Bayes approximation of the posteriors. VBEM asymptotically accurate for logistic regression and SBM separately. No clue about accuracy in the combined model. On-going work: Relevant edge covariates made of node covariates [10]. Efficient sampling in the true posterior using sequential importance sampling: Move from p( ) to p( Y ) using a tempering scheme. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

61 Discussion References I J. Beal, M. and Z. Ghahramani. The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures. Bayes. Statist., 7:543 52, B. Bollobás, S. Janson, and O. Riordan. The phase transition in inhomogeneous random graphs. Rand. Struct. Algo., 31(1):3 122, J.-J. Daudin, L. Pierre, and C. Vacher. Model for heterogeneous random networks using continuous latent variables and an application to a tree fungus network. Biometrics, 66(4): , P. Diaconis and S. Janson. Graph limits and exchangeable random graphs. Rend. Mat. Appl., 7(28):33 61, O. Frank and F. Harary. Cluster inference by using transitivity indices in empirical graphs. J. Amer. Statist. Assoc., 77(380): , Steven Gazal, Jean-Jacques Daudin, and Stéphane Robin. Accuracy of variational estimates for random graph mixture models. Journal of Statistical Computation and Simulation, 82(6): , M. S. Handcock, A. E. Raftery, and J. M. Tantrum. Model-based clustering for social networks. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(2): , doi: /j X x. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

62 Discussion References II J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky. Bayesian model averaging: A tutorial. Statistical Science, 14(4): , P. D. Hoff, A. E. Raftery, and M. S. Handcock. Latent space approaches to social network analysis. J. Amer. Statist. Assoc., 97(460): , D. R. Hunter, S. M. Goodreau, and M. S. Handcock. Goodness of fit of social network models. Journal of the American Statistical Association, 103(481): , T. Jaakkola. Advanced mean field methods: theory and practice, chapter Tutorial on variational approximation methods. MIT Press, T. S. Jaakkola and M. I. Jordan. Bayesian parameter estimation via variational methods. Statistics and Computing, 10(1):25 37, P. Latouche, E. Birmelé, and C. Ambroise. Variational bayesian inference and complexity control for stochastic block models. Statis. Model., 12(1):93 115, P. Latouche and S Robin. Bayesian model averaging of stochastic block models to estimate the graphon function and motif frequencies in a W-graph model. Technical report, arxiv: , to appear in Stat. Comput. P. Latouche and S. Robin. Variational bayes model averaging for graphon functions and motif frequencies inference in W -graph models. Statistics and Computing, pages 1 13, S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

63 Discussion References III P. Latouche, S. Robin, and S. Ouadah. Goodness of fit of logistic models for random graphs. Technical report, arxiv: , S.L. Lauritzen. Graphical Models. Oxford Statistical Science Series. Clarendon Press, L. Lovász and B. Szegedy. Limits of dense graph sequences. Journal of Combinatorial Theory, Series B, 96(6): , M. Mariadassou, S. Robin, and C. Vacher. Uncovering structure in valued graphs: a variational approach. Ann. Appl. Statist., 4(2):715 42, C. Matias and S. Robin. Modeling heterogeneity in random graphs through latent space models: a selective review. ESAIM: Proc., 47:55 74, Tom Minka. Divergence measures and message passing. Technical Report MSR-TR , Microsoft Research Ltd, F. Picard, J.-J. Daudin, M. Koskas, S. Schbath, and S. Robin. Assessing the exceptionality of network motifs,. J. Comp. Biol., 15(1):1 20, C. Vacher, D. Piou, and M.-L. Desprez-Loustau. Architecture of an antagonistic tree/fungus network: The asymmetric influence of past evolutionary history. PLoS ONE, 3(3):1740, e1740. doi: /journal.pone S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

64 Discussion References IV S. Volant, M.-L. Martin Magniette, and S. Robin. Variational bayes approach for model aggregation in unsupervised classification with markovian dependency. Comput. Statis. & Data Analysis, 56(8): , M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1 2):1 305, S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

65 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

66 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

67 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, p(y ij Z i, Z j ), S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

68 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, p(y ij Z i, Z j ), p(z i, Z j Y ): graph moralization, S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

69 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, p(y ij Z i, Z j ), p(z i, Z j Y ): graph moralization, this holds for each pair (i, j), S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

70 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, p(y ij Z i, Z j ), p(z i, Z j Y ): graph moralization, this holds for each pair (i, j), Dependency graph of p(z Y, θ) = clique. No factorization can be hoped (unlike for HMM). p(z Y, θ) can not be computed (efficiently). S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

71 Appendix Variational Bayes for heterogeneous network models Graphical model formalism [17]. Frequentist setting (p( θ)): iid Z i s, p(y ij Z i, Z j ), p(z i, Z j Y ): graph moralization, this holds for each pair (i, j), Dependency graph of p(z Y, θ) = clique. No factorization can be hoped (unlike for HMM). p(z Y, θ) can not be computed (efficiently). Bayesian setting. Things get worst. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

72 Appendix Accuracy of VBEM estimates for SBM: Simulation study Credibility intervals: π 1 : +, γ 11 :, γ 12 :, γ 22 : S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

73 Appendix Accuracy of VBEM estimates for SBM: Simulation study Credibility intervals: π 1 : +, γ 11 :, γ 12 :, γ 22 : Width of the posterior credibility intervals. π 1, γ 11, γ 12, γ 22 [6] S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

74 Appendix A first criterion for goodness-of-fit: Motifs frequency Network motifs have a biological or sociological interpretation in terms of building blocks of the global network Triangles = friends of my friends are my friends. Latent space graph models only describe binary interactions, conditional on the latent positions Goodness of fit criterion based on motif frequencies? [15]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

75 Appendix Moments of motif counts First moments EN(m), VN(m) known for exchangeable graphs (incl. SBM) [22]: E SBM N(m) µ SBM (m), µ SBM (m) = motif occurrence probability. VN(m) depends on the frequency of super-motifs: Motif 1-node overlap 2-node overlap Moments under W -graph: Motif probability under the W -graph can be estimated as µ(m) = P(K)Ẽ(µK SBM(m)) k Estimates of E W N(m) and V W N(m): see [14]. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

76 Appendix Network frequencies in the blog network Motif Count Mean Std. dev. Rel. diff. ( 10 3 ) ( 10 3 ) ( 10 3 ) (N E)/ V No specific structure seems to be exceptional wrt the model s expectations. S. Robin (INRA / AgroParisTech) GOF for graph models MIAT, Nov / 32

Deciphering and modeling heterogeneity in interaction networks

Deciphering and modeling heterogeneity in interaction networks (using variational approximations) S. Robin INRA / AgroParisTech Mathematical Modeling of Complex Systems December 2013, Ecole Centrale de