Handout: Power Laws and Preferential Attachment

Transcription

1 CS224W: Analysis of Neworks Fall 2017 Handou: Power Laws and Preferenial Aachmen 1 Preferenial Aachmen Empirical sudies of real world neworks revealed ha degree disribuion ofen follows a heavyailed disribuion, a power law. A ha ime, here were wo kinds of nework models: he Erdos-Renyi random graph G n,p and he Small World graphs of Was and Srogaz. In boh models he degrees were very close o he mean degree and here was lile variaion. Thus, here was he quesion of finding naural processes ha could generae graphs wih power law degree disribuions. In 1999 Barabasi and Alber reinvened he model of Preferenial Aachmen de Solla Price had inroduced a similar model called Price s model in 1976, which in fac generalized he Simon s model inroduced by Herber Simon in 1955) ha exhibied power-law degree disribuion and renewed ineres in he sudy of neworks. Definiion 1 Preferenial Aachmen PA)) Consider he sequence of direced graphs {G } 0 where G V, E ), V is he verex se and E n he edge se. Given G 0, he graph G +1 is consruced from G according o he following rule: 1. A new verex v +1 is inroduced: V +1 V {v +1 } 2. We add a single direced edge from v +1 o a verex u in V, E +1 E {v +1, u)}, according o he following scheme. Uniform Aachmen) Wih probabiliy p < 1 we pick a verex u uniformly a random. Preferenial Aachmen) Wih probabiliy q 1 p we pick a verex u V wih probabiliy proporional o is in-degree qu) d u, The inuiion behind he model semmed from he general belief ha power-laws correspond o underlying organizaion principles and feedback mechanisms. In paricular, in he PA model he rich ge richer effec is explicily incorporaed hrough he growh process. Since, hen preferenial aachmen models and heir varians have been exensively sudied and heir various descendans are employed o generae realisic nework models. The challenge wih he PA model and similar growh models is o make precise quaniaive predicions abou hem despie heir inricae incremenal consrucion. The basic ool ha enables he analysis of such models is he Differenial Equaion Mehod. 1.1 Differenial Equaion Mehod In order o analyze any complex process one needs o disinguish beween he few variables ha really maer and omi minor deails or inricacies. In our case, incremenal growh processes can be seen as mappings from one graph o a new graph. The hope is ha each sep will no change he properies of he graph oo much and hus we hope o be able o make predicions. The abiliy

2 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen 2 of analysis depends on wheher we can rack he evoluion of he process using only a handful of manageable quaniies. To his end, given he probabilisic naure of he PA model, he following concep is indispensable: Definiion 2 A sequence of random variables or vecors) {X)} 0 is called Markov iff PX + 1) X1),..., X)) P X + 1) X)) 1) Inuiively, our aim is o find a vecor) X such ha we can summarize he sae of our process, in he sense ha in order o rack how X +1 changes we only need he curren values X and nohing else. This is essenially he usefulness of he Markov propery. Nex, we show how o uilize his concep o analyze he Preferenial Aachmen model hrough he Differenial Equaion Mehod. The basic seps of he mehod are: 1. Markovian Dynamics: Find a vecor) funcion Z fg) of a graph G such ha he sequence {Z } 0 {fg )} 0 is appromaely) Markov. 2. Condiional Change: assuming ha he sae Z is known, compue he condiional expecaion a ime + 1: E[Z +1 Z Z ] fz ) 2) 3. Rae Equaion: If he funcion f is appromaely) linear fz ) A Z +b, se z) E[Z ] and ake expecaion wih respec o Z : z + 1) z) A z) + b 3) 4. Fluid limi: consider he ordinary differenial equaion appromaion for large 1: ż) A z) + b) 4) 5. Soluion of Ordinary Differenial Equaion ODE): use he boundary condiions and solve he ODE for z). 6. Concenraion: argue ha he probabiliy ha he random vecor Z deviaes significanly from is expecaion z) is very small. The las sep is beyond he scope of he class a is very echnically involved. Wha we gain by using he differenial equaion mehod is ha we are able o work effecively wih expeced deerminisic quaniies and rack heir changes insead of working wih random variables. Tha is he power of he mehod. Wha allows us o carry ou Sep 2 is he Markov propery and Sep 3 is possible because of lineariy of expecaion and he following propery. Lemma 1 Tower Propery) Given random variables X, Y, i holds ha E[X] E[E[X Y ]]

3 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen 3 Proof: Here, we prove he lemma only for discree random variables bu he general case follows easily. E[X] x xp X x) 5) x x y P X x, Y y) 6) x x y P X x Y y)p Y y) 7) y P Y y) x xp X x y y) 8) y P Y y)e[x Y y] 9) E[E[X Y ]] 10) where in Equaion 6) we used he law of oal probabiliy and Bayes rule in Eq. 7). Nex, we will concreely insaniae he above framework in our analysis of Preferenial Aachmen. 1.2 Power Law degree disribuion of PA graphs Before saring he analysis we firs compue he normalizing consan involved in he Preferenial Aachmen sep of he growh process. In our model, a ime 1 here are exacly verices and 1 direced edges. The exac probabiliy of connecing o a node of degree k is: We sar now wih he firs sep of he mehod: 1 d u 1 E 1 Z E 1) Z Z u Markovian Dynamics: Le D k ) be he number of verices of in-degree k of he graph G and consider he random vecor D D 0 ),..., D 1 )). A momen s hough reveals ha: i) D is he in-degree disribuion of he graph G, ii) he sequence {D } 0 is Markov wih respec o he preferenial aachmen process. Condiional Change: To calculae he expeced change of D +1 given D we focus on D k + 1) he number of nodes wih a paricular degree k: Increase: D k + 1) can only increase by one if he new verex insered a ime connecs o one verex wih degree k 1 a ime. This happens wih probabiliy: p D k 1) + qk 1) D k 1) as wih probabiliy p we would have o selec one of D k 1 ) ou of verices in he graph, and wih probabiliy q he oal weigh of he verices in he preferenial aachmen scheme is given by k 1)D k 1 ). 11)

4 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen 4 Decrease: respecively D k + 1) can only decrease by one if he new edge insered lands on one of he verices of degree k. This happens wih probabiliy: p D k) + qk D k) 12) Rae Equaion: Define d k ) E[D k )] o be he expeced number of verices having in-degree k a ime. Adding Eq. 11), subracing Eq. 12) and aking expecaions, we obain: ) ) dk 1 ) d k ) k 1)dk 1 ) kd k ) d k + 1) d k ) p + q 13) Fluid Limi: For 1, he lef hand side appromaes he derivaive: ) ) d d d dk 1 ) d k ) k 1)dk 1 ) kd k ) k) p + q 14) The previous equaion holds for all k 1, for k 0 we have: d d d 0) p 1 d ) 0) + q 1 d ) 0) 15) as he only way o increase he number of verices of degree 1 is for he new verex o connec o a verex wih degree greaer han one. Soluion of ODE: To solve he differenial equaion we assume he following form for he soluion d k ) p k. Subsiuing in equaions Eq. 14) and Eq. 15) we ge: 1 + p + kq) p k p + k 1)q) p k 1 16) 1 + p + q) p ) 1 + p + 1 p) p ) The above equaions allow us o form a recurrence for p k : ) p + k 1)q p k p k 1 19) 1 + p + kq q ) p k 1 20) 1 + p + kq ) 1 + q)/q 1 p k 1 21) k ) 1+q k 1 q p k 1 22) k From 18) we see ha p 0 1 2, so ieraing 22) we ge ha: p k k 1+q q k 2 p 1 p k p 23) Tha is we have shown heurisically ha he degree disribuion of he preferenial aachmen model follows a power law wih exponen α 2 p 1 p p.

5 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen 5 2 Power Laws Saisical modeling of daa involves gahering enough daa such ha reasonable models can be buil and from here informed inferences can be carried ou. One can make a very broad disincion beween differen random variables as o he exen ha hey are close heir expeced behavior value). People speak broadly of heavy-ailed phenomena where exremes are possible even frequen) and ligh ailed phenomena where hings are well behaved wihin limis. The prooypical example of heavy-ailed behavior are power-law disribued random variables. Definiion 3 A random variable X is said o follow a power law wih exponen α if for large x 1, PX x) x α. The pas decades numerous empirical sudies have idenified power laws in almos every imaginable aspec of he world. In our course, power-laws will show up when looking a various saisics of neworks, mos predominanly he degree-disribuion. There are wo flavors of power-laws one for coninuous and one for discree daa. For he purposes of his noe, we will focus almos exclusively on coninuous random variables and simply allude o he fac ha discree daa can be very accuraely simulaed by appropriaely binning rounding o he closer ineger) he coninuous daa. We sar wih some calculaions. 2.1 Normalizing Consan A coninuous random variable would have a densiy px)dx P x X x + dx) Cx α. Since he densiy diverges for x 0 we need o impose a lower bound on he values of X. For all α > 1 and given > 0 he normalizaion consan Cα, ) is given by: px)dx C x α dx C 1 α 1 The final form of a coninuous pure) power law disribuion is: px) α 1 [ x a+1 ] 1 Cα, ) α 1 ) α 1 x ) α 24) For a discree random variable following a power law pk) Dk α we ge he following expression for he normalizing consan Dα, ): pk) D + n) α 1 Dα, ) n0 1 ζα, ) The funcion ζα, ) n0 n + ) α is called he Hurwiz zea funcion. The final form for a discree pure) power law disribuion is: px) x α ζα, ) 25)

6 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen Tail Disribuion Power laws and heavy-ailed disribuions in general are ypically conrased o he Normal disribuion. The pronounced difference is in heir ail disribuion. Definiion 4 The ail disribuion T x) of a random variable x is defined as T x) PX x). Definiion 5 A random variable is heavy-ailed if λ > 0, lim x [ T x) e λx ]. From he definiion i is easy o see ha Normal disribuion does no have heavy-ails whereas power laws do. The ail disribuion for coninuous and discree power-law disribuions are: T c x) PX x) T d x) PX x) x α 1 zx z ) α x dz z α ζα, x) ζα, ) ζα, ) ) α Momens One of he consequences of power-law behaviour is ha depending on he exponen α cerain momens diverge. E[X k ] px)x k dx 26) x min α 1 x 1 α x k α dx 27) min { α 1 α k 1 ) k k < α 1 28) k α 1 For insance if α 3 he variance diverges, whereas for α 2 he mean diverges. 2.4 Generaion For a coninuous CDF, he funcion F x) is one-o-one. Using he ransformaion mehod we can generae power-law disribued daa using only a uniform random variable: ) X α+1 U 1 ln1 U) 1 α)[lnx) ln)] 29) 1 α 1 ln1 U) + ln) lnx) 30) X 1 U) 1 α 1 31)

7 CS224W: Analysis of Neworks: Power Laws and Preferenial Aachmen MLE for he Scaling Parameer Due o he peculiar saisical properies of power-law random variables, cauion mus be praciced when esimaing he parameers. The bes way o esimae he scaling exponen is o use Mamum Likelihood Esimaion MLE). We firs derive an expression for he log-likelihood for n independen samples: Lα) ln px 1,..., x n ) 32) [ n ] ln px i ) 33) n ln[px i )] 34) n [ lnα 1) ln ) α ln n[lnα 1) ln )] α Nex we are going o find he value â ha mamizes he likelihood: Lα) α 0 n n αˆα ˆα 1 ˆα 1 + n n ln [ n )] ) ln 35) 36) ) 0 37) ) ] 1 ln 38)