Quantitative Metrics: II

Size: px

Start display at page:

Download "Quantitative Metrics: II"

Marjory Jacobs
5 years ago
Views:

1 Network Observational Methods and Topics Quantitative Metrics: II Degree correlation Exploring whether the sign of degree correlation can be predicted from network type or similarity to regular structures, or details about the network itself, or maybe nothing Calculating degree correlation for simple regular structures like trees and grids 2/16/2011 Degree Correlation Daniel E Whitney /24

2 Summary Properties of Several Big Networks (Newman) Network Type n m z l α C (1) C (2) r Ref(s). Social Film actors Company directors Math coauthorship Physics coauthorship Biology coauthorship Telephone call graph messages address books Student relationships Sexual contacts Undirected 449, 913 Undirected Undirected Undirected Undirected Undirected Directed Directed Undirected Undirected 7, , , 909 1, 520, , 000, , , , , 516, , , , , 803, , 000, , , _ _ _ _ / , , , , , 313 8, , 266 Information Citation network Roget's thesaurus Word co-occurrence Directed Directed Directed Directed Undirected 269, , 549, , 339 1, , 902 1, 497, 135 2,130, 000, 000 6, 716, 198 5, , 000, / / / , , 157 Technological Internet Power grid Train routes Software packages Software classes Electronic circuits Peer-to-peer network Undirected Undirected Undirected Directed Directed Undirected Undirected 10, 697 4, , 439 1, , , 992 6, , 603 1, 723 2, , 248 1, /1.4 _ , , 354 Biological Metabolic network Protein interactions Marine food web Freshwater food web neural network Undirected Undirected Directed Directed Directed 765 3, 686 2, 115 2, , _ , 421 Basic statistics for a number of published networks. The properties measured are: type of graph, directed or undirected; total number of vertices n; total number of edges m; mean degree z; mean vertex-vertex distance l; exponent α of degree distribution if the distribution follows a power law (or " _ " if not; in/out-degree exponents are given for directed graphs); clustering coefficient C (1) from Eq. (3); clustering coefficient C (2) from Eq. (6); and degree correlation coefficient r, Sec. III.F. The last column gives the citation(s) for the network in the bibliography. Blank entries indicate unavailable data. Degree Correlation 2/16/2011 Daniel E Whitney /24 Courtesy of Society for Industrial and Applied Mathematics. Used with permission. Image by MIT OpenCourseWare. Table 2 in Mark E. J. Newman s. The Structure and Function of Complex Networks. SIAM Review 45, no. 2 (2003):

3 Community-finding and Pearson Coefficient r Newman says technological networks seem to have r < 0 while social networks seem to have r > 0 Newman and Park sought an explanation in community structure and clustering: Why Social Networks are Different From Other Kinds of Networks Phys Rev E, 68, (2003) Social networks can arise by people joining multiple groups and generating multiple connections Networks derived from these multiple connections have positive r Networks coauthconn and coauthwhole are from this paper coauthconn is the connected portion with 147 nodes coauthwhole has 42 clusters, smallest has 2 nodes, biggest has 5 2/16/2011 Degree Correlation Daniel E Whitney /24

4 Why Social Networks are Different Left to their own devices, we conjecture, networks normally have negative values of r. In order to show a positive value of r, a network must have some specific additional structure that favors assortative mixing. Special structure that explains networks with r > 0: Large clustering coeff compared to random network with same degree sequence Community structure (No special structure needed to explain r < 0) 2/16/2011 Degree Correlation Daniel E Whitney /24

Physics Coauthors Network 41 separate clusters coauthconn: r = 0.

5 Physics Coauthors Network 41 separate clusters coauthconn: r = coauthwhole: r = /16/2011 Degree Correlation Daniel E Whitney /24

6 Work with David Alderson Are Social Networks Really Different? ICCS 2006 paper, published in NECSI journal 2/16/2011 Degree Correlation Daniel E Whitney /24

7 Our Observations Data show a mixture of r > 0 and r < 0 for all kinds of networks (38 simple connected networks) Many with r < 0 have community structure There is a structural explanation for this, based on a structural property that all networks have: the variability of the degree sequence, but not related to category: social technological Can use it to show that certain networks cannot possibly have r > 0 Also, some canonical structures have r < 0 or r > 0 and real networks share properties with these canonical structures: trees have r < 0 and grids have r > 0 2/16/2011 Degree Correlation Daniel E Whitney /24

8 Full Range of r 2/16/2011 Degree Correlation Daniel E Whitney /24

9 Negative r 2/16/2011 Degree Correlation Daniel E Whitney /24

10 Positive r 2/16/2011 Degree Correlation Daniel E Whitney /24

11 More Data on r vs Network Type Network n m < k > fraction of nodes : r C Random clustering Generalized random clustering coeff k > x coeff < k > < k 2 2 > <k > < k > /n n < k > 2 Karate Club J. Tirole ŅErdos NetworkÓ2 J. Stiglitz ŅErdos NetworkÓ2 Scheduled Air Routes, US Littlerock Lake food web* Grand Piano Action 1 key Santa Fe coauthors V8 engine Grand Piano Action 3 keys Exercise walker Abeline Bike Six speed transmission ŅHOTÓ Car Door DSM* Jet Engine DSM* TV Circuit* Tokyo Regional Rail FAA Nav Aids, Unscheduled Canton food web* Color code: Ņsocial,Óassemblies, rail lines, trophic food webs, software call graphs, power grids, internet/phone, Design Structure Matrix, air routes, electric circuits 2/16/2011 Degree Correlation Daniel E Whitney /24

12 Network n m < k > fraction of TV Circuit* nodes : k > x r C Random clustering coeff < k > /n Tokyo Regional Rail FAA Nav Aids, Unscheduled Canton food web* Mozilla * Mozilla all comp* Munich Schnellbahn FAA Nav Aids, Scheduled St. Marks food web * Western Power Grid Unscheduled Air Routes, US Apache call list* Physics coauthors Tokyo Regional Rail plus Subways Traffic Light controller* Berlin U- & S Bahn London Underground Regional Power Grid Generalized random clustering coeff < k > < k 2 > <k > n < k > 2 2/16/2011 Degree Correlation Daniel E Whitney /24 2 Data Continued

13 Data Continued Network n m < k > fraction of nodes : r C Random clustering Generalized random clustering coeff k > x coeff < k > < k 2 > <k > < k > /n n < k > 2 Moscow Subways Nano-bell Broom food web* Company directors Moscow Subways and Regional Rail 2 2/16/2011 Degree Correlation Daniel E Whitney /24

14 How to Calculate r in Closed Form for Canonical Structures 2/16/2011 Degree Correlation Daniel E Whitney /24

3 Review: Calculating r from the Edge 1 List 2 4 5 (x x)(y y) r = (x x) 2 (y y) 2 x = 2.2 y = 2.2 r = -0.

15 3 Review: Calculating r from the Edge 1 List (x x)(y y) r = (x x) 2 (y y) 2 x = 2.2 y = 2.2 r = using Pearson function in Excel Note: if all nodes have the same k then r = 0/0 2/16/2011 Degree Correlation Daniel E Whitney /24

16 n=1 n=2 n=3 n=4 n=5 binary tree with n= rows like this - ignore 1 6 rows like this - ignore 2 3 All other rows like this: 3-3 (except last two sets) 2 n-2 rows of 3--3 means (3 x) 2 2*2 n-2 rows of = 1-3 and means (3 x)(1 x) 2 n-1 rows of 3-1 Ksum total rows 2 n total rows of 3-1 ~Ksum - 2 n rows of Pure Binary Tree 2/16/2011 Degree Correlation Daniel E Whitney /24 Census of Pairs for

17 Result of Census Sum of row entries = k i 2 = 10*2 n 1 14 = ksqsum Total number of rows = k i = 2 n = ksum < k 2 > k 2 x = = = 2.5 in the limit of large n < k > k Also <k>=2 Total 2 n rows of 3-1 Approx (ksum - 2 n ) rows of 3-3 Denominator = (x x) 2 (y y) 2 = (x x) 4 = (x x) 2 This is just one column s entries squared r = for this tree with 5 layers 2/16/2011 Degree Correlation Daniel E Whitney /24

18 Closed Form Results Property Pure Binary Tree Binary Tree with Cross-linking ksum 2 n * 2 n 10 ksqsum 10 * 2 n * 2 n 64 x 2.5 as n becomes large (>~ 6) 13 as n becomes large (>~6) 3 Pearson numerator ~2 n (3 x)(1 x) + (ksum 2 n )(3 x) 2 ~2 n (5 x)(1 x) + (ksum 2 n )(5 x) 2 Pearson denominator ~2 n 1 (1 x) 2 + (ksum 2 n 1 )(3 x) 2 ~2 n 1 (1 x) 2 + (ksum 2 n 1 )(5 x) 2 r 1 3 as n becomes large 1 5 as n becomes large l Note: Western Power Grid r = (2 x)(3 x) + 8( 3)(3 x) 2 2 Bounded grid r = 2(2 x) 2 +12( 2)(3 x) 2 3 x = 4 so all terms in (4 x) disappear 2/16/2011 Degree Correlation Daniel E Whitney /24

19 HOT Network: A WAN Num_rows=n*(n-1+m*I/n)+m*(I+k+2)+m*k Row_sum=n*(n-1+m*I/n)^2+m*(I+k+2)^2+m*k numerator=(n-1+m*i/n-xbar)^2*n*(n-1)+2*(n-1+m*i/n-xbar)*(i+k+2-xbar)*m*i +(I+k+2-xbar)^2*2*m+(I+k+2-xbar)*(1-xbar)*2*k*m denom=(n-1+m*i/n-xbar)^2*n*(n-1+m*i/n) +(I+k+2-xbar)^2*m*(I+k+2)+(1-xbar)^2*m*k HOTnet r for HOT Plus Circle n = core nodes m = intermediate nodes I = 2 core nodes/intermediate node k = 2 stub nodes/intermediate node r > r < n = 4 m = 8 k = 3 i = 2 m= n=4 Image by MIT OpenCourseWare. r =

20 HOT is a Tree with the Core at the Top HOTnet n = 4 m = 8 k = 3 i = 2 Image by MIT OpenCourseWare. 2/16/2011 Degree Correlation Daniel E Whitney /24

21 Actual Metro System Histories Berlin Metro History Moscow Metro History ClustCoeff(5) Pearson r ClustCoeff(5) Pearson r Year 2/16/2011 Degree Correlation Daniel E Whitney /24

22 Networks Observed r Against Background Found by Rewiring While Preserving the Degree Sequence Observed r rmax rmin Minibell Karate Tirole Stiglitz Santa Fe Coauthors V8 engine Walker Hyperengine Abeline Bike Six Speed Transmission Winchester Tokyo Regional Rail Trafficlight Controller Munich S- and U-bahn Western Power Grid Physics Coauthors Tokyo Rail & Subway Berlin S & U- Bahn London Underground Regional Power Grid Moscow Subway Nanobell + leased 2005 Directors Moscow Rail & Subway Nanobell + leased For r < -0.1, the background range is restricted to mostly negative values For r > -0.1, the background covers most of [-1, 1] 2/16/2011 Degree Correlation Daniel E Whitney /24

23 Node Degree How Degree Sequence Constrains r for x i > x One Network But Not for Another Almost no x i > x Many x i > x V8 Engine x i > x x i > x x i < x Node Degree x i > x Physics Coauthors x i > x x i > x x i < x Fraction of Nodes Fraction of Nodes r = r = Range = [ , ] Range = [ , 0.553] 2/16/2011 Degree Correlation Daniel E Whitney /24

24 Conclusions A network s domain ( social, technical, etc.) is not a reliable predictor of the sign of r The degree sequence imposes considerable structural constraint on networks whose observed r < 0 But it does not impose much constraint on networks whose observed r > 0 Each network s actual circumstances impose constraint, but circumstances are stronger than the degree sequence when r >0 and vice-versa when r < 0 Example: cost of connection may be high for technological systems but not for social systems like coauthor or movie actor networks Similarly, the exact connections matter for the bike but not for the coauthors, who could in principle collaborate with anyone 2/16/2011 Degree Correlation Daniel E Whitney /24

25 Backups 2/16/2011 Degree Correlation Daniel E Whitney /24

26 Zachary s Karate Club: A Social Network with r < 0 (from UCINET) r = C = There is no link between 23 and 34. Every later scholar has this error. 2/16/2011 Degree Correlation Daniel E Whitney /24

27 Moscow Metro Image of Moscow Metro map removed due to copyright restrictions. See Moscow Metro. 2/16/2011 Degree Correlation Daniel E Whitney /24

28 Moscow Regional Rail Map of Moscow Regional Rail removed due to copyright restrictions. Please refer to: The Mappery 2/16/2011 Degree Correlation Daniel E Whitney /24

29 r subway + rail r subway Regional Rail 2/16/2011 Degree Correlation Daniel E Whitney /24 Moscow Metro and = =

30 Bike High-k nodes do not link to each other! r = -0.2 bike newman girvan 2.jpg 2/16/2011 Degree Correlation Daniel E Whitney /24

V8 Engine High-k nodes do not link to each other! r = -0.

31 V8 Engine High-k nodes do not link to each other! r = /16/2011 Degree Correlation Daniel E Whitney /24

32 Does a Network Have to Have a Particular Value of r Given its D? Technological networks have to perform a function, use scarce resources efficiently, or satisfy some other structural or functional constraint, so their observed wiring and r are probably necessary Social networks do not have to do any of these things so their structure is more subject to circumstances, such as communication or collaboration habits; thus their observed wiring and r are probably circumstantial We can test by seeing if rewired versions are plausible 2/16/2011 Degree Correlation Daniel E Whitney /24

33 Example Domain Sources for Constraint in D Leading to r < 0 High x with respect to x in mechanical assemblies comes from need to provide a foundation part to absorb loads and locate other parts to each other Engine block Bike and walker frame High x with respect to x in some social networks reflects hierarchy or dominance Karate instructor and club president in Zachary s club Tree-like structure of wireline phone networks causes them to have r < 0 because trees have r < 0 These networks can t be rewired plausibly 2/16/2011 Degree Correlation Daniel E Whitney /24

34 Example Domain Sources for Constraint in D Leading to r > 0 Planar transport networks are grid-like, and grids have r > 0 Modern fiber-optic phone networks are built on loops or chains (trunks) that link clusters (central offices) leading to r > 0 2/16/2011 Degree Correlation Daniel E Whitney /24

35 MIT OpenCourseWare ESD.342 Network Representations of Complex Engineering Systems Spring 2010 For information about citing these materials or our Terms of Use, visit:

Network Observational Methods and. Quantitative Metrics: II

Network Observational Methods and Whitney topics Quantitative Metrics: II Community structure (some done already in Constraints - I) The Zachary Karate club story Degree correlation Calculating degree