Principles of Complex Systems CSYS/MATH 300, Fall, Prof. Peter Dodds

Transcription

1 Principles of Complex Systems CSYS/MATH, Fall, Prof. Peter Dodds Department of Mathematics & Statistics Center for Complex Systems Vermont Advanced Computing Center University of Vermont Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike. License. of 6

2 Outline of 6

3 Size distributions The sizes of many systems elements appear to obey an inverse power-law size distribution: P(size = x) c x γ where < x min < x < x max and γ > Exciting class exercise: sketch this function. x min = lower cutoff x max = upper cutoff Negative linear relationship in log-log space: log P(x) = log c γ log x We use base because we are good people. of 6

4 Size distributions Usually, only the tail of the distribution obeys a power law: P(x) c x γ for x large. Still use term power law distribution. Other terms: Fat-tailed distributions. Heavy-tailed distributions. Beware: Inverse power laws aren t the only ones: lognormals ( ), Weibull distributions ( ),... of 6

5 Size distributions Many systems have discrete sizes k: Word frequency Node degree in networks: # friends, # hyperlinks, etc. # citations for articles, court decisions, etc. P(k) c k γ where k min k k max Obvious fail for k =. Again, typically a description of distribution s tail. 5 of 6

6 The statistics of surprise words: Brown Corpus ( ) ( 6 words): rank word % q. the of.589. and.8. to a in. 7. that.8 8. is was.966. he.99. for.9. it.86. with.776. as his.6886 rank word % q 95. apply vital September review wage motor fifteen regarded draw wheel organized vision wild Palmer intensity.55 6 of 6

7 The statistics of surprise words: First a Gaussian example: P(x)dx = πσ e (x µ) /σ dx linear: P(x).... log-log log P(x) x mean µ =, variance σ =. 5 log x 7 of 6

8 The statistics of surprise words: Raw probability (binned): linear: log-log N q 8 6 log N q 6 8 q log q 8 of 6

9 The statistics of surprise words: Exceedance probability : linear: N > q q log-log log N > q log q 9 of 6

10 My, what big words you have... Test capitalizes on word frequency following a heavily skewed frequency distribution with a decaying power law tail. Let s do it collectively... ( ) of 6

11 The statistics of surprise: Gutenberg-Richter law ( ) Log-log plot Base Slope = - N(M > m) m From both the very awkwardly similar Christensen et al. and Bak et al.: Unified scaling law for earthquakes [, ] of 6

12 The statistics of surprise: From: Quake Moves Japan Closer to U.S. and Alters Earth s Spin ( ) by Kenneth Chang, March,, NYT: What is perhaps most surprising about the Japan earthquake is how misleading history can be. In the past years, no earthquake nearly that large nothing larger than magnitude eight had struck in the Japan subduction zone. That, in turn, led to assumptions about how large a tsunami might strike the coast. It did them a giant disservice, said Dr. Stein of the geological survey. That is not the first time that the earthquake potential of a fault has been underestimated. Most geophysicists did not think the Sumatra fault could generate a magnitude 9. earthquake,... of 6

13 Great: Two things we have poor cognitive understanding of:. Probability Ex. The Monty Hall Problem ( ) Ex. Son born on Tuesday ( ).. Logarithmic scales. On counting and logarithms: Listen to Radiolab s Numbers. ( ). Later: Benford s Law ( ). of 6

14 9 net worth in US dollars (j) 5 peak intensity name frequency telephone calls received (h) (k) web hits 5 (f) 5 6 intensity population of city 7 (l) (c) population of city 5 7 earthquake magnitude (i) (l) 7 FIG. Cumulative distributions or rank/frequency plots of twelve quantities reputed to follow power laws. The distributions were computed as described in Appendix A. Data in the shaded regions were excluded from the calculations of the exponents in Table I. Source references for the data are given in the text. (a) Numbers of occurrences of words in the novel Moby Dick by Hermann Melville. (b) Numbers of citations to scientific papers published in 98, from time of publication until June 997. (c) Numbers of hits on web sites by 6 users of the America Online Internet service for the day of December 997. (d) Numbers of copies of bestselling books sold in the US between 895 and 965. (e) Number of calls received by AT&T telephone customers in the US for a single day. (f) Magnitude of earthquakes in California between January 9 and May 99. Magnitude is proportional to the logarithm of the maximum amplitude of the earthquake, and hence the distribution obeys a power law even though the horizontal axis is linear. (g) Diameter of craters on the moon. Vertical axis is measured per square kilometre. (h) Peak gamma-ray intensity of solar flares in counts per second, measured from Earth orbit between February 98 and November 989. (i) Intensity of wars from 86 to 98, measured as battle deaths per of the population of the participating countries. (j) Aggregate net worth in dollars of the richest individuals in the US in October. (k) Frequency of occurrence of family names in the US in the year 99. (l) Populations of US cities in the year. 6 name frequency 5 (e) 6 (b) net worth in US dollars intensity (k) 5 (d) 9 (i) 7 peak intensity (j) (h) earthquake magnitude word frequency citations crater diameter in km. crater diameter in km. (g) books sold 6 (a). 6 (g) telephone calls received 7 books sold (f). (e) 6 6 (d) of 6

15 Size distributions : Earthquake magnitude (Gutenberg-Richter law ( )): [] P(M) M Number of war deaths: [9] P(d) d.8 Sizes of forest fires [] Sizes of cities: [] P(n) n. Number of links to and from websites [] See in part Simon [] and M.E.J. Newman [6] Power laws, Pareto distributions and for more. Note: Exponents range in error 5 of 6

16 Size distributions : Number of citations to papers: [7, 8] P(k) k. Individual wealth (maybe): P(W ) W. of tree trunk diameters: P(d) d. The gravitational force at a random point in the universe: [5] P(F) F 5/. Diameter of moon craters: [6] P(d) d. Word frequency: [] e.g., P(k) k. (variable) 6 of 6

17 Power law distributions Gaussians versus power-law distributions: Mediocristan versus Extremistan Mild versus Wild (Mandelbrot) Example: Height versus wealth. See The Black Swan by Nassim Taleb. [] 7 of 6

18 Turkeys... From The Black Swan [] 8 of 6

19 Taleb s table [] Mediocristan/Extremistan Most typical member is mediocre/most typical is either giant or tiny Winners get a small segment/winner take almost all effects When you observe for a while, you know what s going on/it takes a very long time to figure out what s going on Prediction is easy/prediction is hard History crawls/history makes jumps Tyranny of the collective/tyranny of the rare and accidental 9 of 6

20 Size distributions Power law size distributions are sometimes called Pareto distributions ( ) after Italian scholar Vilfredo Pareto. ( ) Pareto noted wealth in Italy was distributed unevenly (8 rule; misleading). Term used especially by practitioners of the Dismal Science ( ). of 6

21 Devilish power law distribution details: Exhibit A: Given P(x) = cx γ with < x min < x < x max, the mean is (γ ): x = c ( ) xmax γ x γ γ min. Mean blows up with upper cutoff if γ <. Mean depends on lower cutoff if γ >. γ < : Typical sample is large. γ > : Typical sample is small. Insert question from assignment ( ) of 6

22 And in general... Moments: All moments depend only on cutoffs. No internal scale that dominates/matters. Compare to a Gaussian, exponential, etc. For many real size distributions: < γ < mean is finite (depends on lower cutoff) σ = variance is infinite (depends on upper cutoff) Width of distribution is infinite If γ >, distribution is less terrifying and may be easily confused with other kinds of distributions. Insert question from assignment ( ) of 6

23 Moments Standard deviation is a mathematical convenience: Variance is nice analytically... Another measure of distribution width: Mean average deviation (MAD) = x x For a pure power law with < γ < : x x is finite. But MAD is mildly unpleasant analytically... We still speak of infinite width if γ <. Insert question from assignment ( ) of 6

24 How sample sizes grow... Given P(x) cx γ : We can show that after n samples, we expect the largest sample to be x c n /(γ ) Sampling from a finite-variance distribution gives a much slower growth with n. e.g., for P(x) = λe λx, we find x ln n. λ Insert question from assignment ( ) of 6

25 Complementary Cumulative Distribution Function: CCDF: P (x) = P(x x) = P(x < x) = = x =x x =x P(x )dx (x ) γ dx γ + (x ) γ+ x γ+ x =x 5 of 6

26 Complementary Cumulative Distribution Function: CCDF: P (x) x γ+ Use when tail of P follows a power law. Increases exponent by one. Useful in cleaning up data. PDF: CCDF: log N q log N > q log q log q 6 of 6

27 Complementary Cumulative Distribution Function: Discrete variables: P (k) = P(k k) = P(k) k =k k γ+ Use integrals to approximate sums. 7 of 6

28 Zipfian rank-frequency plots George Kingsley Zipf: Noted various rank distributions followed power laws, often with exponent - (word frequency, city sizes...) Zipf s 99 Magnum Opus ( ): Human Behaviour and the Principle of Least-Effort [] We ll study in depth... 8 of 6

29 Zipfian rank-frequency plots Zipf s way: Given a collection of entities, rank them by size, largest to smallest. x r = the size of the rth ranked entity. r = corresponds to the largest size. Example: x could be the frequency of occurrence of the most common word in a text. Zipf s observation: x r r α 9 of 6

30 Size distributions Brown Corpus (,5,95 words): CCDF: log N > q Zipf: log q i log q log rank i The, of, and, to, a,... = objects Size = word frequency Beep: CCDF and Zipf plots are related... of 6

31 Size distributions Brown Corpus (,5,95 words): CCDF: log N > q Zipf (axes flipped): log rank i log q log q i The, of, and, to, a,... = objects Size = word frequency Beep: CCDF and Zipf plots are related... of 6

32 Observe: NP (x) = the number of objects with size at least x where N = total number of objects. If an object has size x r, then NP (x r ) is its rank r. So x r r α = (NP (x r )) α x ( γ+)( α) r since P (x) x γ+. We therefore have = ( γ + )( α) or: α = γ A rank distribution exponent of α = corresponds to a size distribution exponent γ =. of 6

33 The Don. ( ) Extreme deviations in test cricket: Don Bradman s batting average ( ) = 66% next best. of 6

34 I [] P. Bak, K. Christensen, L. Danon, and T. Scanlon. Unified scaling law for earthquakes. Phys. Rev. Lett., 88:785,. pdf ( ) [] A.-L. Barabási and R. Albert. Emergence of scaling in random networks. Science, 86:59 5, 999. pdf ( ) [] K. Christensen, L. Danon, T. Scanlon, and P. Bak. Unified scaling law for earthquakes. Proc. Natl. Acad. Sci., 99:59 5,. pdf ( ) [] P. Grassberger. Critical behaviour of the Drossel-Schwabl forest fire model. New Journal of Physics, :7. 7.5,. pdf ( ) of 6

35 II [5] J. Holtsmark.? Ann. Phys., 58:577, 97. [6] M. E. J. Newman. Power laws, pareto distributions and zipf s law. Contemporary Physics, pages 5, 5. pdf ( ) [7] D. J. d. S. Price. Networks of scientific papers. Science, 9:5 55, 965. pdf ( ) [8] D. J. d. S. Price. A general theory of bibliometric and other cumulative advantage processes. J. Amer. Soc. Inform. Sci., 7:9 6, of 6

36 III [9] L. F. Richardson. Variation of the frequency of fatal quarrels with magnitude. J. Amer. Stat. Assoc., :5 56, 99. pdf ( ) [] H. A. Simon. On a class of skew distribution functions. Biometrika, :5, 955. pdf ( ) [] N. N. Taleb. The Black Swan. Random House, New York, 7. [] G. K. Zipf. Human Behaviour and the Principle of Least-Effort. Addison-Wesley, Cambridge, MA, of 6