CS 124/LINGUIST 180 From Languages to Information

Size: px

Start display at page:

Download "CS 124/LINGUIST 180 From Languages to Information"

Ashley Garrison
6 years ago
Views:

1 CS 124/LINGUIST 180 From Languages to Information Christopher Manning Lecture 19: (Social) Networks: Fat Tails, Small Worlds, and Weak Ties Slides from Dan Jurafsky who mainly got them from Lada Adanic; some also from James Moody, Bing Liu, James Moody

2 Networks We ve looked a lot at informaeon in language We ve seen that you can also get informaeon from the pagerns of pages across the Web PageRank But there are also pagerns of people who interact Maybe it s also useful to study and understand that? Many goals: Sociology BeGer ad targeeng 2 Slide from Chris Manning

3 Social network analysis Social network analysis is the study of eneees (people in an organizaeon), and their interaceons and relaeonships. The interaceons and relaeonships can be represented with a network or graph, each vertex (or node) represents an actor and each link represents a relaeonship. May be directed or not. CS583, Bing Liu, UIC Nov 5, 2009

4 Centrality Important or prominent actors are those that are involved with other actors extensively. A person with extensive contacts or communicaeons with many other people in the organizaeon is considered more important The contacts can also be called links or &es. A central actor is one involved in many Ees. Degree centrality: The number of direct conneceons a node has What really magers is where those conneceons lead to and how they connect the otherwise unconnected. CS583, Bing Liu, UIC

5 Prestige PresEge is a more refined measure of prominence of an actor than centrality. DisEnguish: Ees sent (out- links) and Ees received (in- links). A presegious actor is one who is the object of extensive Ees as a recipient. To compute the presege: we use only in- links. PageRank is based on presege CS583, Bing Liu, UIC

6 Social Network Analysis 2. Betweenness Centrality: A node with high betweenness has great influence over what flows in the network indicaeng important links and single point of failure. 3. Closeness Centrality: The measure of nodes which are close to everyone else. The pagern of direct and indirect Ees allows the nodes to get to any other node in the network more quickly than anyone else. They have the shortest paths to all others. from Amit Sharma UT Austin

7 Outline Sketch of some real networks Fat Tails Small Worlds Weak Ties

8 High school dating Slide from Drago Radev Peter S. Bearman, James Moody and Katherine Stovel Chains of affection: The structure of adolescent romantic and sexual networks American Journal of Sociology (2004) Image drawn by Mark Newman

10 More networks hgp://oracleo\acon.org/

11 5 million edges Graph Colors: Asia Pacific Red Europe/Middle East/ Central Asia/Africa Green North America Blue Latin American and Caribbean Yellow RFC1918 IP Addresses Cyan Unknown - White Slide from Drago Radev

12 The Harlem Shake: Anatomy of a Viral Meme by Gilad Lotan (SocialFlow) 12

13 13 Slide from Chris Manning

14 Degree of nodes Many nodes on the internet have low degree One or two conneceons A few (hubs) have very high degree The number P(k) of nodes with degree k follows a power law: P(k) k α Where alpha for the internet is about 2.1

15 What is a heavy tailed-distribution? Unlike a normal distribueon, power- law distribueons have no scale [I ll explain that later ] Right skew normal distribueon (not heavy tailed) Zipf s or power- law distribueon (heavy tailed) High raeo of max to min human heights vs. city sizes

16 Normal (Gaussian) distribution of human heights average value close to most typical distribution close to symmetric around average value

17 Power-law distribution linear scale n log-log scale n high skew (asymmetry) n straight line on a log-log plot

18 Logarithmic axes powers of a number will be uniformly spaced n 2 0 =1, 2 1 =2, 2 2 =4, 2 3 =8, 2 4 =16, 2 5 =32, 2 6 =64,.

19 Power laws are seemingly everywhere note: these are cumulative distributions Moby Dick scientific papers AOL users visiting sites 97 bestsellers AT&T customers on 1 day California

20 Yet more power laws Moon Solar flares wars ( ) richest individuals 2003 US family names 1990 US cities 2003

21 Power law distribution Straight line on a log- log plot ln( p( x)) = ExponenEate both sides to get that p(x), the probability of observing an item of size x is given by α p ( x) c α ln( x) = Cx normalization constant (probabilities over all x must sum to 1) power law exponent α

22 What does it mean to be scale free? A power law looks the same no mater what scale we look at it on (2 to 50 or 200 to 5000) Only true of a power- law distribueon! p(bx) = g(b) p(x) shape of the distribueon is unchanged except for a muleplicaeve constant p(bx) = (bx) -α = b -α x -α log(p(x)) x b*x log(x)

23 Many real world networks are power law film actors co-appearance 2.3 telephone call graph 2.1 networks 1.5/2.0 sexual contacts 3.2 WWW 2.3/2.7 internet 2.5 peer-to-peer 2.1 metabolic network 2.2 protein interactions 2.4 exponent α (in/out degree)

24 Power laws and fat tails

25 Fat tails in the news

26 Hey, not everything is a power law number of sighengs of 591 bird species in the North American Bird survey in cumulaeve distribueon n another examples: n size of wildfires (in acres)

27 Not every network is power law distributed address books power grid Roget s thesaurus company directors

28 Zipf s law is a power-law Zipf George Kingsley Zipf how frequent is the 3rd or 8th or 100th most common word? IntuiEon: small number of very frequent words ( the, of ) lots and lots of rare words ( expressive, Jurafsky ) Zipf's law: the frequency of the r'th most frequent word is inversely proporeonal to its rank: y ~ r - β, with β close to unity.

29 Pareto s law and power-laws Pareto The Italian economist Vilfredo Pareto was interested in the distribueon of income. Pareto s law is expressed in terms of the cumulaeve distribueon (the probability that a person earns X or more). P[X > x] ~ x - k Here we recognize k as just α - 1, where α is the power- law exponent

30 80/20 rule The fraceon W of the wealth in the hands of the richest P of the populaeon is given by W = P (α-2)/(α-1) Example: US wealth: α = 2.1 richest 20% of the populaeon holds 86% of the wealth

31 Generative processes for power-laws Many different processes can lead to power laws There is no one unique mechanism that explains it all

32 One mechanism for generating a power-law distribution: Preferential attachment First considered by [Price 65] as a model for citation networks each new paper is generated with m citations (mean) new papers cite previous papers with probability proportional to their in-degree (citations) what about papers without any citations? each paper is considered to have a default citation probability of citing a paper with degree k, proportional to k+1 Power law with exponent α = 2+1/m

33 Small worlds

34 Small world experiments MA NE Stanley Milgram s experiment (1967): Given a target individual and a particular property, pass the message to a person you correspond with who is closest to the target.

35 Milgram s small world experiment Target person worked in Boston as a stockbroker. 296 senders from Boston and Omaha. 20% of senders reached target. average chain length = 6.5. Six degrees of separaeon

36 Milgram s Findings: Distance to target person, by sending group. Slide from James Moody

37 Duncan Watts: Networks, Dynamics and the Small-World Phenomenon Asks why we see the small world pagern and what implicaeons it has for the dynamical properees of social systems. His contribueon is to show that globally significant changes can result from locally insignificant network change. Slide from James Moody

38 Duncan Watts: Networks, Dynamics and the Small-World Phenomenon WaGs says there are 4 condieons that make the small world phenomenon intereseng: 1) The network is large - O(Billions) 2) The network is sparse - people are connected to a small fraceon of the total network 3) The network is decentralized - - no single (or small #) of stars 4) The network is highly clustered - - most friendship circles are overlapping Slide from James Moody

39 Duncan Watts: Networks, Dynamics and the Small-World Phenomenon Formally, we can characterize a graph through 2 staesecs. 1) The characteristic path length, L (the diameter) The average length of the shortest paths connecting any two actors. 2) The clustering coefficient, C Is the average local density. That is, C v = ego-network density, and C = C v /n A small world graph is any graph with a relatively small L and a relatively large C. Slide from James Moody

40 Local clustering coefficient (Watts&Strogatz 1998) For a vertex i The fraceon pairs of neighbors of the node that are themselves connected Let n i be the number of neighbors of vertex i C i = number of conneceons between i s neighbors maximum number of possible conneceons between i s neighbors Ci directed = # directed conneceons between i s neighbors n i * (n i - 1) Ci undirected = # undirected conneceons between i s neighbors n i * (n i - 1)/2

41 Local clustering coefficient (Watts&Strogatz 1998) Average over all n vereces C = 1 n i C i i n i = 4 max number of connections: 4*3/2 = 6 3 connections present C i = 3/6 = 0.5 link present link absent

42 The most clustered graph is Watt s Caveman graph: C caveman 6 1 k 2 1 L caveman n 2 ( k + 1)

43 Why does this work? Key is fraction of shortcuts in the network In a highly clustered, ordered network, a single random conneceon will create a shortcut that lowers L dramaecally WaGs demonstrates that Small world graphs occur in graphs with a small number of shortcuts

probability p, rewire every edge (or, add a shortcut) to a

44 Watts and Strogatz model [WS98] Start with a ring, where every node is connected to the next z nodes ( a regular layce) With probability p, rewire every edge (or, add a shortcut) to a uniformly chosen desenaeon. order p = 0 0 < p < 1 p = 1 Small world randomness

45 Clustering and Path Length Regular Slide Graphs from have Lada Adamic a high clustering coefficient but also a high diameter Random Graphs have a low clustering coefficient but a low diameter

46 Weak links Mark GranoveGer (1960s) studied how people find jobs. He found out that most job referrals were through personal contacts But more by acquaintances and not close friends. Accepted by the American Journal of Sociology azer 4 years of unsuccessful agempts elsewhere. One of the most cited papers in sociology. Slide from Drago Radev

47 Mark Granovetter: The strength of weak ties Strength of ties amount of time spent together emotional intensity intimacy (mutual confiding) reciprocal services Many strong ties are transitive we meet our friends through other friends if we spend a lot of time with our strong ties, they will tend to overlap balance theory if two of my friends do not like each other we will all be unhappy. Triad closure is the happy solution = Homophily Slide from James Moody

48 Strength of weak ties Weak ties can occur between cohesive groups old college friend former colleague from work weak ties will tend to have low transitivity Slide from James Moody

49 Strength of weak ties Evidence from small world experiments Small world experiment at Columbia: acquaintanceship ties more effective than family, close friends Slide from James Moody

50 Strength of weak ties how to get a job Granovetter: How often did you see the contact that helped you find the job prior to the job search 16.7% often (at least once a week) 55.6% occasionally (more than once a year but less than twice a week) 27.8% rarely once a year or less Weak ties will tend to have different information than we and our close contacts do Long paths rare 39.1 % info came directly from employer 45.3 % one intermediary 3.1 % > 2 (more frequent with younger, inexperienced job seekers) Compatible with Watts/Strogatz small world model: short average shortest paths thanks to shortcuts that are nontransitive Slide from James Moody

51 Finding paths WaGs model shows how these short paths can exist small world networks But how do people find the paths? People seem to be successful by making greedy local decisions The existence of findable short paths depends on further elucidaeng the structure of the network

52 Spatial search Kleinberg, The Small World Phenomenon, An Algorithmic Perspective Proc. 32nd ACM Symposium on Theory of Computing, (Nature 2000) The geographic movement of the [message] from Nebraska to Massachusetts is striking. There is a progressive closing in on the target area as each new person is added to the chain S.Milgram The small world problem, Psychology Today 1,61,1967 nodes are placed on a lattice and connect to nearest neighbors additional links placed with p uv ~ r d uv

53 Increasing r favors near nodes d -r (u,v) =1, Uniform Distribution r=0, Link to each other node equally likely r=1, inverse of distance If a node is twice as far away, 1/2 as likely r=2, inverse squared If a node is twice as far away, 1/4 as likely

54 Kleinberg s SW network is Greedy Routable iff r=2 t Greedy roueng algorithm using local informaeon only, find a short path from s to t s u v w When u is the current node, choose next v: the closest to t (use lattice distance) with (u,v) a local or random edge.

55 Kleinberg s SW network is Greedy Routable iff r=2 t A greedy roueng algorithm using local informaeon only, find a short path from s to t s u v " The number of hops is the the delivery time " This greedy routing achieves n expected delivery time of O(log 2 n), i.e. the s-t paths have expected length O(log 2 n).

56 Kleinberg s SW network is Greedy Routable iff r=2 t A greedy roueng algorithm using local informaeon only, find a short path from s to t s u v " This greedy routing achieves n n expected `delivery time of O(log 2 n), i.e. the s-t paths have expected length O(log 2 n). This does not work unless r=2 : for r 2, α>0 such that the expected delivery time of any decentralized algorithm is Ω(n α ).

57 Overly localized links on a lattice When r>2 expected search time ~ N (r-2)/(r-1) p ~ 1 d 4

58 no locality When r=0, links are randomly distributed, ASP ~ log(n), n size of grid When r=0, any decentralized algorithm is at least a 0 n 2/3 When r<2, expected time at least α r n (2-r)/3 p ~ p 0 Good paths exist, but are not greedily findable

59 Links balanced between long and short range When r=2, expected time of a DA is at most C (log N) 2 p ~ 1 d 2

60 Outline Quick sketch of some networks Fat Tails Small Worlds Weak Ties

6.207/14.15: Networks Lecture 12: Generalized Random Graphs

6.207/14.15: Networks Lecture 12: Generalized Random Graphs 1 Outline Small-world model Growing random networks Power-law degree distributions: Rich-Get-Richer effects Models: Uniform attachment model