Curved exponential family models for networks

Transcription

1 Curved exponential family models for networks David R. Hunter, Penn State University Mark S. Handcock, University of Washington February 18, 2005 Available online as Penn State Dept. of Statistics Technical report from Statistical Models, Sunbelt 2005 p. 1

2 Overview This talk will focus on the alternating k-star and alternating k-triangle statistics of Snijders et al. General goals: 1. Present equivalent (and preferable) formulations of these statistics alternating k-stars d G (y, θ) alternating k-triangles p G (y, θ) 2. Introduce the mathematical issues that make model-fitting particularly challenging when using these statistics Snijders et al. is W.P. # 42 at Statistical Models, Sunbelt 2005 p. 2

3 Alternating k-star statistic The alternating k-star statistic is defined as s 2 (y) s 3(y) γ + + ( 1) n s n 1(y) γ n 2, where s k (y) denotes the number of k-stars in the network y. Statistical Models, Sunbelt 2005 p. 3

4 Alternating k-star statistic The alternating k-star statistic is defined as s 2 (y) s 3(y) γ + + ( 1) n s n 1(y) γ n 2, where s k (y) denotes the number of k-stars in the network y. Consider the γ parameter. What does it do? How do we choose it? Note in particular that if the alternating k-star statistic is used in a model, γ enters in a nonlinear way. Statistical Models, Sunbelt 2005 p. 3

5 A small undirected network Statistical Models, Sunbelt 2005 p. 4

6 A small undirected network Degree distribution: (d 0,...,d n 1 ) = (0, 1, 1, 3, 0) Statistical Models, Sunbelt 2005 p. 4

7 A small undirected network Degree distribution: (d 0,...,d n 1 ) = (0, 1, 1, 3, 0) k-star distribution: (s 1,...,s n 1 ) = (6, 10, 3, 0) Statistical Models, Sunbelt 2005 p. 4

8 A small undirected network Degree distribution: (d 0,...,d n 1 ) = (0, 1, 1, 3, 0) k-star distribution: (s 1,...,s n 1 ) = (6, 10, 3, 0) Edgewise shared partner distribution: (p 0,...,p n 2 ) = (1, 4, 1, 0) Statistical Models, Sunbelt 2005 p. 4

9 A small undirected network Degree distribution: (d 0,...,d n 1 ) = (0, 1, 1, 3, 0) k-star distribution: (s 1,...,s n 1 ) = (6, 10, 3, 0) Edgewise shared partner distribution: (p 0,...,p n 2 ) = (1, 4, 1, 0) k-triangle distribution: (t 1,...,t n 2 ) = (2, 1, 0) Statistical Models, Sunbelt 2005 p. 4

10 A small undirected network Degree distribution: (d 0,...,d n 1 ) = (0, 1, 1, 3, 0) k-star distribution: (s 1,...,s n 1 ) = (6, 10, 3, 0) Edgewise shared partner distribution: (p 0,...,p n 2 ) = (1, 4, 1, 0) k-triangle distribution: (t 1,...,t n 2 ) = (2, 1, 0) Relationship between edgewise shared partners and k-triangles: analagous to the relationship between degrees and k-stars (and also the relationship between dyadic shared partners and alternating independent 2-paths.) Statistical Models, Sunbelt 2005 p. 4

11 Rewriting alternating k-stars The alternating k-star statistic s 2 (y) s 3(y) γ + + ( 1) n 1 s n 1(y) γ n 3 may be rewritten (brace yourself): Statistical Models, Sunbelt 2005 p. 5

12 Rewriting alternating k-stars The alternating k-star statistic s 2 (y) s 3(y) γ + + ( 1) n 1 s n 1(y) γ n 3 may be rewritten (brace yourself): where: d G (y; θ) = 2e θ s 1 (y) n 1 i=1 γ is replaced by e θ (to ensure γ > 0) e 2θ [ 1 ( 1 e θ) i ] d i (y), s k (y) = # of k-stars in the graph y. (In particular, s 1 = # of edges.) d k (y) = # of nodes of degree k in y. Statistical Models, Sunbelt 2005 p. 5

13 Alternating k-triangle statistic, rewritten The alternating k-triangle statistic of Snijders et al. (2004) is 3t 1 (y) t 2(y) γ + + ( 1) n 1 t n 2(y) γ n 3. In analogy with the alternating k-star case, we rewrite: Statistical Models, Sunbelt 2005 p. 6

14 Alternating k-triangle statistic, rewritten The alternating k-triangle statistic of Snijders et al. (2004) is 3t 1 (y) t 2(y) γ + + ( 1) n 1 t n 2(y) γ n 3. In analogy with the alternating k-star case, we rewrite: p G (y; θ) = n 2 i=1 e θ { 1 ( 1 e θ) i } p i (y), where γ is replaced by e θ (to ensure γ > 0) t k (y) = # of k-triangles in the graph y. p k (y) = # of nodes with k edgewise shared partners in y. Statistical Models, Sunbelt 2005 p. 6

15 An important question We have shown that the alternating k-star statistic is the same as d G (y, θ) the alternating k-triangle statistic is the same as p G (y, θ) where θ = log γ. Suppose we wish to include d G (y, θ 1 ) and/or p G (y, θ 2 ) in an ERGM, but we wish to estimate θ 1 and θ 2. How do we do it? Statistical Models, Sunbelt 2005 p. 7

16 ERGM specification For a (random, as-yet-unobserved) graph Y, we assume P(Y = y) = exp{ηt g(y)} c(η) = exp{η 1g 1 (y) + + η p g p (y)} c(η) for all possible realizations y. The name ERGM (exponential random graph model) arises because this model is based on a statistical exponential family. Statistical Models, Sunbelt 2005 p. 8

17 ERGM specification For a (random, as-yet-unobserved) graph Y, we assume P(Y = y) = exp{ηt g(y)} c(η) = exp{η 1g 1 (y) + + η p g p (y)} c(η) for all possible realizations y. The name ERGM (exponential random graph model) arises because this model is based on a statistical exponential family. As usual, g(y) is a vector of statistics to be specified by the modeler (and p is the number of statistics). The vector η is sometimes called the canonical parameter. Statistical Models, Sunbelt 2005 p. 8

18 Not quite an ERGM? The ERGM says P(Y = y) = exp{η 1g 1 (y) + + η p g p (y)}. c(η) But suppose g(y) consists of only the statistic p G (y, θ). Thus, we wish to estimate the parameters θ 1 and θ 2 in the model P(Y = y) = exp{θ 1p G (y, θ 2 )}. c(θ) Statistical Models, Sunbelt 2005 p. 9

19 Not quite an ERGM? The ERGM says P(Y = y) = exp{η 1g 1 (y) + + η p g p (y)}. c(η) But suppose g(y) consists of only the statistic p G (y, θ). Thus, we wish to estimate the parameters θ 1 and θ 2 in the model P(Y = y) = exp{θ 1p G (y, θ 2 )}. c(θ) The second equation is not in ERGM form because the parameters are mixed up with the statistics! Statistical Models, Sunbelt 2005 p. 9

20 η vs. θ Earlier, we showed p G (y; θ) = n 2 i=1 e θ { 1 ( 1 e θ) i } p i (y). Thus, the model we wish to fit turns into P(Y = y) = exp{θ 1p G (y, θ 2 )} c [ n 2 { exp i=1 θ 1e θ 2 1 ( ) } ] 1 e θ 2 i p i (y) =. c Thus, we can write η i (the coefficient of the ith statistic) as a function of θ (the parameter vector we want to estimate). Statistical Models, Sunbelt 2005 p. 10

21 Curved exponential family models The original model has turned into P(Y = y) = exp{η 1(θ)p 1 (y) + + η n 2 (θ)p n 2 (y)}. c[η(θ)] Statistical Models, Sunbelt 2005 p. 11

22 Curved exponential family models The original model has turned into P(Y = y) = exp{η 1(θ)p 1 (y) + + η n 2 (θ)p n 2 (y)}. c[η(θ)] Thus, η is the vector of coefficients, whereas θ is the parameter vector to be estimated. Often, the two vectors are the same so this distinction is ignored. Statistical Models, Sunbelt 2005 p. 11

23 Curved exponential family models The original model has turned into P(Y = y) = exp{η 1(θ)p 1 (y) + + η n 2 (θ)p n 2 (y)}. c[η(θ)] Thus, η is the vector of coefficients, whereas θ is the parameter vector to be estimated. Often, the two vectors are the same so this distinction is ignored. But sometimes, η(θ) is a nonlinear function; the equation above imposes nonlinear constraints on η. In that case, statisticians call the model a curved exponential family. Statistical Models, Sunbelt 2005 p. 11

24 Staving off death The complication of a nonlinear constraint on η can actually be a good thing: What happens if we try to estimate the unconstrained η vector in P(Y = y) = exp{η 1p 1 (y) + + η n 2 p n 2 (y)}? c(η) Statistical Models, Sunbelt 2005 p. 12

25 Staving off death The complication of a nonlinear constraint on η can actually be a good thing: What happens if we try to estimate the unconstrained η vector in P(Y = y) = exp{η 1p 1 (y) + + η n 2 p n 2 (y)}? c(η) Answer: In the words of Pip Pattison, Death By Parameter! Statistical Models, Sunbelt 2005 p. 12

26 Staving off death The complication of a nonlinear constraint on η can actually be a good thing: What happens if we try to estimate the unconstrained η vector in P(Y = y) = exp{η 1p 1 (y) + + η n 2 p n 2 (y)}? c(η) Answer: In the words of Pip Pattison, Death By Parameter! Boiling the entire η vector down into a function of just the (θ 1, θ 2 ) is actually healthy. But nothing that s good for you is ever fun, so there s more work to be done at the estimation step. See paper for details. Statistical Models, Sunbelt 2005 p. 12

27 Lazega s lawyer collaboration data F F F Sizes indicate seniority (larger=more recent); colors indicate office location; F indicates female; shapes indicate practice (circle=litigation, square=corporate) Statistical Models, Sunbelt 2005 p. 13

28 Coefficient estimates (generated by statnet) F F F Model 1 Model 2 Parameter est. s.e. est. s.e. Alternating k-triangles Rate of transitivity Seniority main effect Practice main effect Same practice Same gender Same office Statistical Models, Sunbelt 2005 p. 14

29 Deviance analysis F F F Model Residual Deviance Deviance Residual d.f. p-value NULL Covariates Full model Covariates : The model with only the covariate terms Full model : The model with covariate terms plus p G (y, θ) Statistical Models, Sunbelt 2005 p. 15

30 Conclusion Other things covered in our paper: Statistical Models, Sunbelt 2005 p. 16

31 Conclusion Other things covered in our paper: A general formulation of the problem of fitting curved exponential family models Statistical Models, Sunbelt 2005 p. 16

32 Conclusion Other things covered in our paper: A general formulation of the problem of fitting curved exponential family models Numerical algorithms for estimating curved EF parameters and their standard errors Statistical Models, Sunbelt 2005 p. 16

33 Conclusion Other things covered in our paper: A general formulation of the problem of fitting curved exponential family models Numerical algorithms for estimating curved EF parameters and their standard errors How to estimate likelihood ratio statistics and loglikelihoods using MCMC Statistical Models, Sunbelt 2005 p. 16

34 Conclusion Other things covered in our paper: A general formulation of the problem of fitting curved exponential family models Numerical algorithms for estimating curved EF parameters and their standard errors How to estimate likelihood ratio statistics and loglikelihoods using MCMC Huge thanks to: Tom Snijders for extremely helpful suggestions about the manuscript; Steve Goodreau for blackboard brainstorming sessions; Martina Morris, Garry Robins, and Pip Pattison for insightful comments. Statistical Models, Sunbelt 2005 p. 16