Power laws. Leonid E. Zhukov

Size: px

Start display at page:

Download "Power laws. Leonid E. Zhukov"

Rose Anderson
5 years ago
Views:

Department of Computer Science National Research University Higher

1 Power laws Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural Analysis and Visualization of Networks Leonid E. Zhukov (HSE) Lecture / 31

2 Table of contents 1 Probability basics 2 Power law distribution 3 Scale-free networks 4 Parameter estimation 5 Zipf s law Leonid E. Zhukov (HSE) Lecture / 31

3 Continuous distribution Continuous random variable X Probability density function p(x) (PDF): Pr(a X b) = p(x) 0 p(x)dx = 1 Cumulative distribution function (CDF) F (x) = Pr(X x) = x b a p(x)dx; p(x)dx d F (x) = p(x) dx Complementary cumulative distribution function (ccdf) F (x) = Pr(X > x) = 1 F (x) = x p(x)dx Leonid E. Zhukov (HSE) Lecture / 31

4 Continuous distribution Gaussian: p(x) = 1 (x µ) 2 σ e 2σ 2, F (x) = 1 x µ 2π 2 [1 + erf ( σ )] 2 Exponential (x 0): p(x) = λe λx, F (x) = 1 e λx, F (x) = e λx Leonid E. Zhukov (HSE) Lecture / 31

5 Discrete distribution Discrete random variable X i Probability mass function (PMF) p(x): p(x) = Pr(X i = x) p(x) 0 p(x) = 1 Cumulative distribution function (CDF) x F (x) = Pr(X i x) = x x p(x ) Leonid E. Zhukov (HSE) Lecture / 31

6 Empirical distributions Newman et.al, 2005 Leonid E. Zhukov (HSE) Lecture / 31

7 Power Laws Continuous approximation Power law p(x) = Cx α = C x α, for x Normalization (α > 1) 1 = p(x)dx = C dx x α = C α 1 x α+1 min Power law PDF C = (α 1)x α 1 min p(x) = α 1 ( x ) α Leonid E. Zhukov (HSE) Lecture / 31

8 Power Laws poisson: p(k) = λk k! e λ, exponent: p(x) = Ce λx, power law: p(x) = Cx α Leonid E. Zhukov (HSE) Lecture / 31

9 Power Laws Power law PDF p(x) = Cx α = α 1 ( x ) α Complimentary cumulative distribution function ccdf F (x) = Pr(X > x) = F (x) = Cx (α 1) = x p(x)dx C ( ) x (α 1) α 1 x (α 1) = Leonid E. Zhukov (HSE) Lecture / 31

10 Power Laws Power law: p(x) = Cx α, log p(x) = log C α log x, F (x) = Cx (α 1) log F (x) = log C (α 1) log x Leonid E. Zhukov (HSE) Lecture / 31

11 Empirical distributions log-log scale Leonid E. Zhukov (HSE) Lecture / 31

12 Moments PDF p(x) = C x α, x First moment (mean value), α > 2: x = Second moment, α > 3: x 2 = xp(x)dx = C k-th moment, α > k + 1: x 2 p(x)dx = C x k = α 1 α 1 k x k min dx x α 1 = α 1 α 2 dx x α 2 = α 1 α 3 2 Leonid E. Zhukov (HSE) Lecture / 31

13 Moments Fisrt moment (mean): x = C xmax ( ) dx x α 1 = α 1 x α 1 min α 2 xmax α 2 Clauset et.al, 2009 Leonid E. Zhukov (HSE) Lecture / 31

14 Scale invariance Scaling of the density x bx, Scale invariance p(bx) = C(bx) α = b α Cx α p(x) p(100x) p(10x) = p(10x) p(x) Leonid E. Zhukov (HSE) Lecture / 31

15 Power law histograms Newman et.al, 2005 Leonid E. Zhukov (HSE) Lecture / 31

16 Scale-free networks Leonid E. Zhukov (HSE) Lecture / 31

17 Node degree distribution k i - node degree, i.e. number of nearest neighbors, k i = 1, 2,...k max n k - number of nodes with degree k, n k = i I(k i == k) total number of nodes n = k n k Degree distribution P(k i = k) P(k) P(k) = n k k n k = n k n CDF ccdf F (k) = P(k ) = 1 n k k F (k) = 1 P(k ) = 1 n k k k k n k k >k n k Leonid E. Zhukov (HSE) Lecture / 31

18 Discrete power law distribution Power law distribution Normalization P(k) = C k=1 P(k) = Ck γ = C k γ k=1 Riemann zeta function, γ > 1 Log-log coordinates k γ = Cζ(γ) = 1; C = 1 ζ(γ) P(k) = k γ ζ(γ) log(p(k)) = γ log k + log C Leonid E. Zhukov (HSE) Lecture / 31

19 Power law networks Probability mass function PMF/mPDF Leonid E. Zhukov (HSE) Lecture / 31

20 Power law networks Complementary cumulative distribution function ccdf Leonid E. Zhukov (HSE) Lecture / 31

21 Power law networks Actor collaboration graph, N=212,250 nodes, k = 28.8, γ = 2.3 WWW, N = 325,729 nodes, k = 5.6, γ = 2.1 Power grid data, N = 4941 nodes, k = 5.5, γ = 4 Barabasi et.al, 1999 Leonid E. Zhukov (HSE) Lecture / 31

22 Power law networks In- and out- degrees of WWW crawl 1999 Broder et.al, 1999 Leonid E. Zhukov (HSE) Lecture / 31

23 Parameter estimation: α Maximum likelihood estimation of parameter α Let {x i } be a set of n observations (points) independently sampled from the distribution Probability of the sample P(x i ) = α 1 ( xi ) α P({x i } α) = n i α 1 ( xi ) α Bayes theorem P(α {x i }) = P({x i } α) P(α) P({x i }) Leonid E. Zhukov (HSE) Lecture / 31

24 Maximum likelihood log-likelihood L = ln P(α {x i }) = n ln(α 1) n ln α maximization L α = 0 α = 1 + n [ n i=1 ln x i ] 1 n i=1 ln x i error estimate σ = n [ n i=1 ] 1 ln x i = α 1 n Leonid E. Zhukov (HSE) Lecture / 31

25 Parameter estimation: Kolmogorov-Smirnov test (compare model and experimental CDF) D = max F (x α, ) F exp (x) x Clauset et.al, 2009 Leonid E. Zhukov (HSE) Lecture / 31

26 Empirical models Clauset et.al, 2009 Leonid E. Zhukov (HSE) Lecture / 31

27 Word counting Word frequency table (6318 unique words, min freq 800, corpus size > 85mln): the be of and a in to incredibly 801 historically 801 decision-making 800 wildly 800 reformer 800 quantum Leonid E. Zhukov (HSE) Lecture / 31

28 Zips f law Zipf s law - the frequency of a word in an natural language corpus is inversely proportional to its rank in the frequency table f (k) 1/k. f (k) = 1/k s N k=1 (1/ks ) Leonid E. Zhukov (HSE) Lecture / 31

29 Rank-frequency plot Sort items by their frequency in decreasing order (frequency table) Fraction of the words with frequencies higher or equal to the k-th word is ccdf F (k) = Pr(X k). The number of the words with frequency above k-th word is its rank k! Plot word rank as a function of the word frequency: rank k - y axis, frequency - x axis. Use rank-frequency plot instead of computing and plotting cumulative distribution of a quantity the be of and a in Leonid E. Zhukov (HSE) Lecture / 31

30 Rank-frequency plot Leonid E. Zhukov (HSE) Lecture / 31

31 References Power laws, Pareto distributions and Zipf s law, M. E. J. Newman, Contemporary Physics, pages , Power-Law Distribution in Empirical Data, A. Clauset, C.R. Shalizi, M.E.J. Newman, SIAM Review, Vol 51, No 4, pp , A Brief History of Generative Models for Power Law and Lognormal Distributions, M. Mitzenmacher, Internet Mathematics Vol 1, No 2, pp Leonid E. Zhukov (HSE) Lecture / 31

Network models: random graphs

Network models: random graphs Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural Analysis