The Best, The Hottest, & The Luckiest: A Statistical Tale of Extremes

Transcription

1 The Best, The Hottest, & The Luckiest: A Statistical Tale of Extremes Sid Redner, Boston University, physics.bu.edu/~redner Extreme Events: Theory, Observations, Modeling and Simulations Palma de Mallorca, November 10-14, 2008

2 The Best with S. Maslov, P. Chen, H. Xie J. Informetrics 1, 8-15 (2007) Observations about scientific citations Google page rank analysis for hidden gems

3 Phys. Rev. Citation Data (as of July 03) 353,268 papers, 3,110,839 cites # cites =8.81, cite age =6.20 N.B.: Internal citations only; undercount by factor of 3-5. (for highly cited HEP papers; SPIRES)

4 Phys. Rev. Citation Data (as of July 03) 353,268 papers, 3,110,839 cites # cites =8.81, cite age =6.20 N.B.: Internal citations only; undercount by factor of 3-5. (for highly cited HEP papers; SPIRES) 11 papers with > 1000 citations 79 papers with > 500 citations 237 papers with > 300 citations 2340 papers with > 100 citations 8073 papers with > 50 citations

5 Phys. Rev. Citation Data (as of July 03) 353,268 papers, 3,110,839 cites # cites =8.81, cite age =6.20 N.B.: Internal citations only; undercount by factor of 3-5. (for highly cited HEP papers; SPIRES) 11 papers with > 1000 citations 79 papers with > 500 citations 237 papers with > 300 citations 2340 papers with > 100 citations 8073 papers with > 50 citations papers with < 10 citations papers with 1 citation papers with 0 citations

6 PR papers with >1000 cites cites age article title author(s) 1 PR 140 A1133 (1965) Self Consistent Equations.. W. Kohn & L. J. Sham

7 PR papers with >1000 cites cites age article title author(s) 1 PR 140 A1133 (1965) Self Consistent Equations.. W. Kohn & L. J. Sham

8 PR papers with >1000 cites article cites age title author(s) 1 PR 140 A1133 (1965) Self Consistent Equations.. W. Kohn & L. J. Sham 2 PR 136 B864 (1965) Inhomogeneous Electron Gas.. P. Hohenberg & W. Kohn 3 PRB (1981) Self-Interaction Correction to.. J. P. Perdew & A. Zunger 4 PRL (1980) Ground State of the Electron.. D. M. Ceperley & B. J. Alder 5 PR (1957) Theory of Superconductivity Bardeen, Cooper, Schrieffer 6 PRL (1967) A Model of Leptons S. Weinberg 7 PRB (1975) Linear Methods in Band Theory O. K. Andersen 8 PR (1961) Effects of Configuration.. U. Fano 9 RMP (1985) Disordered Electronic Systems P. A. Lee & T. V. Ramakrishnan 10 RMP (1982) Electronic Properties of.. T. Ando, A. B. Fowler, & F. Stern 11 PRB (1976) Special Points for Brillouin.. H. J. Monkhorst & J. D. Pack

9 Google Page Rank for Citations Brin & Page (1999)

10 Google Page Rank for Citations Brin & Page (1999) Idea: launch many surfers that surf forever until a steady-state surfer number Gᵢ is reached at each site i of the network.

11 Google Page Rank for Citations Brin & Page (1999) Idea: launch many surfers that surf forever until a steady-state surfer number Gᵢ is reached at each site i of the network. Equation for number of surfers: G i =(1 d) j nn i G j k out j + d N

12 Google Page Rank for Citations Brin & Page (1999) Idea: launch many surfers that surf forever until a steady-state surfer number Gᵢ is reached at each site i of the network. Equation for number of surfers: random walk propagation G i =(1 d) j nn i G j k out j + d N manna from heaven

13 Google Page Rank for Citations Brin & Page (1999) Idea: launch many surfers that surf forever until a steady-state surfer number Gᵢ is reached at each site i of the network. Equation for number of surfers: random walk propagation G i =(1 d) j nn i G j k out j + d N manna from heaven Brin/Page: d=0.15 (bored after 6 clicks) We use: d=0.50 (don t cite beyond 2 generations)

14 Correlation between Google & citation counts 10 3 average google number slope number of citations

15 google number number of citations

16 500 Cabibbo 400 google number number of citations

17 Cabibbo BCS google number number of citations

18 Cabibbo BCS K/S google number H/K number of citations

19 500 Cabibbo 400 BCS K/S google number Onsager Weinberg H/K number of citations

20 500 Cabibbo BCS 400 K/S google number Onsager Slater W/S F/GM Anderson G4 Weinberg H/K number of citations

21 500 Cabibbo 400 BCS K/S google number Onsager Slater W/S F/GM Anderson G4 GM/B W/S Hi Tc DLA Glauber S Weinberg Fano H/K number of citations

22 The Hottest with M. Petersen Phys. Rev. E 74, (2006) Some facts about urban temperatures Evolution of record temperature events Role of global warming on records

23 Philadelphia Temperatures annual temperature pattern long-term temperature trends Aug 7, F record high annual high 80 daily temperature av. high av. low record low average temperature annual middle 0 20 Feb 9, F day of the year 40 annual low year

24 Philadelphia Temperatures annual temperature pattern Aug 7, F long-term temperature trends IPCC 2001: global warming 1.1 over past century record high 3.8 annual high daily temperature av. high av. low record low average temperature annual middle 0 20 Feb 9, F day of the year 40 annual low year

25 Philadelphia Temperatures annual temperature pattern Aug 7, F long-term temperature trends IPCC 2001: global warming 1.1 over past century record high 3.8 annual high daily temperature av. high av. low record low average temperature annual middle 0 20 Feb 9, F day of the year annual low year

26 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables

27 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day year

28 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 0 t 0 year

29 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 0 t 0 year

30 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 0 t 0 year

31 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

32 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

33 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

34 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

35 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

36 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

37 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

38 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 1 T 0 t 0 t 1 year

39 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 2 T 1 T 0 t 0 t 1 t 2 year

40 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 2 T 1 T 0 t 0 t 1 t 2 year

41 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 3 T 2 T 1 T 0 t 0 t 1 t 2 t 3 year

42 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 3 T 2 T 1 T 0 t 0 t 1 t 2 t 3 year

43 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 3 T 2 T 1 T 0 t 0 t 1 t 2 t 3 year Goal: compute T k, P k (T ) t k,p k (t)

44 Evolution of Temperature Records (Arnold et al., Records, 1998) assumption: daily temperatures are iid continuous variables temperature on a fixed day T 3 T 2 T 1 T 0 t 0 t 1 t 2 t 3 year Goal: compute T k, P k (T ) t k,p k (t) } distribution independent!

45 Record Time Statistics (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar)

46 Record Time Statistics (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) define σ i = { 1 if record in i th year 0 otherwise

47 Record Time Statistics (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) define σ i = { 1 if record in i th year 0 otherwise largest then σ i = 1 i i

48 define σ i = Record Time Statistics { 1 if record in i th year 0 otherwise (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) 2nd largest new largest then σ i = 1 i σ i σ j = σ i σ j i j

49 define σ i = Record Time Statistics { 1 if record in i th year 0 otherwise (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) 2nd largest new largest then σ i = 1 i +1 therefore n(t) = σ i σ j = σ i σ j t i=1 σ i ln t i P (n) = (ln t)n n! e ln t j

50 define σ i = Record Time Statistics { 1 if record in i th year 0 otherwise (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) 2nd largest new largest then σ i = 1 i +1 therefore n(t) = σ i σ j = σ i σ j t i=1 σ i ln t i P (n) = (ln t)n n! e ln t probability that current record is broken in ith year:... 2nd j 1st i

51 define σ i = Record Time Statistics { 1 if record in i th year 0 otherwise (Glick 1978, Sibani et al 1997, Krug & Jain 05, Majumdar) 2nd largest new largest then σ i = 1 i +1 therefore n(t) = σ i σ j = σ i σ j t i=1 σ i ln t i P (n) = (ln t)n n! e ln t probability that current record is broken in ith year:... 2nd j 1st = 1 i(i +1) time until next record = i

52 Records If Global Warming Is Occurring assume temperature distribution as a soluble example only p(t ; t) = { e (T vt) T>vt 0 T<vt exceedance probability prob of record at year n (over)estimate of record time p > (T k ; t k + j) = T k e [T v(t k+j)] dt = e (T k vt k ) e jv X e jv q n (T k ) p > (T k ) p < (T k ) n 1 e nv X n 1 j=1 (1 e jv X) }{{} when 0 record must occur e jv X =1 (t k+1 t k )v = T k vt k t k k v records ultimately occur at constant rate (Ballerini & Resnick, 85; Borokov, 99)

53 Frequency of Record High Temperatures 10 1 probability v=0.012 v= v=0.003 v= time

54 Frequency of Record Low Temperatures 10 1 probability 10 2 v= v=0.003 v=0.006 v= time probability v=0.012 v= v=0.003 v= time

55 Record Frequencies: Multiple Stations Potsdam CET Brussels years after 1893 years after 1878 years after Milan 100 Prague highs lows 1 years after years after

56 The Luckiest with C. Sire Eur. Phys. J B to appear very soon Statistics of team win/loss records Consecutive-game winning/losing streaks

57 Fun Facts About Team Win/Loss Records Best all-time record: 1906 Chicago Cubs: 116/ Worst all-time record: 1916 Philadelphia Athletics: 36/ Historically most successful : NY Highlanders/Yankees : 5105/ Historically least successful : Philadelphia Phillies : 3828/ Boston Braves : 2627/

58 Bradley-Terry Competition Model Zermelo (1929) Bradley & Terry (1952)

59 Bradley-Terry Competition Model Zermelo (1929) Bradley & Terry (1952) each team i has strength xᵢ, i=1,2,...,n (fixed in each season, can change between seasons)

60 Bradley-Terry Competition Model Zermelo (1929) Bradley & Terry (1952) each team i has strength xᵢ, i=1,2,...,n (fixed in each season, can change between seasons) probability that team i beats j: actually, assume only: p ij = x i p ij = f(x i ) x i + x j f(x i )+f(x j ) numerical studies: Anthology of Statistics in Sports, Albert et al. (2005)

61 Bradley-Terry Competition Model Zermelo (1929) Bradley & Terry (1952) each team i has strength xᵢ, i=1,2,...,n (fixed in each season, can change between seasons) probability that team i beats j: actually, assume only: p ij = x i p ij = f(x i ) x i + x j f(x i )+f(x j ) numerical studies: Anthology of Statistics in Sports, Albert et al. (2005) uniform strength distribution: x [ɛ, 1] with 0 ɛ 1 fundamental parameter: ε common assumption: log-normal distribution

62 Win Fraction Versus Rank MLB data for 1905, 10, 15,..., W(r) r data from

63 Average Win Fraction Versus Rank 0.7 MLB W(r) r

64 Average Win Fraction Versus Rank MLB W(r) r

65 Average Win Fraction Versus Rank MLB , W(r) r

66 Average Win Fraction Versus Rank Simulations of BT model for 10⁶ seasons ε=0.278 balanced schedule W(r) ε= r

67 Theory for the Average Win Fraction W

68 Theory for the Average Win Fraction W for a team of strength x: N x W (x) = 1 N j=1 x + x j

69 Theory for the Average Win Fraction W for a team of strength x: N x W (x) = 1 N 1 1 ɛ j=1 1 x + x j ɛ x x + y dy infinite season length infinite number of teams balanced schedule uniform strengths on [ε,1]

70 Theory for the Average Win Fraction W for a team of strength x: N x W (x) = 1 N 1 1 ɛ = j=1 1 ɛ x 1 ɛ ln x + x j x x + y dy [ x +1 x + ɛ ] infinite season length infinite number of teams balanced schedule uniform strengths on [ε,1]

71 Theory for the Average Win Fraction W for a team of strength x: N x W (x) = 1 N 1 1 ɛ = j=1 1 ɛ x 1 ɛ ln x + x j x x + y dy [ x +1 x + ɛ ] infinite season length infinite number of teams balanced schedule uniform strengths on [ε,1] strength x rank r: x=ε+(1-ε)r [ 1+ɛ +(1 ɛ)r ] W (r) =[ɛ +(1 ɛ)r] ln 2ɛ +(1 ɛ)r

72 Average Win Fraction Versus Rank ε= W(r) ε= r

73 Average Win Fraction Versus Rank BT model analytics ε= W(r) ε= r

74 Role of Season Length on Win Fraction 0.6 game data BT model W(r) r

75 Role of Season Length on Win Fraction 0.6 game data BT model 300 game season W(r) r

76 Role of Season Length on Win Fraction 0.6 game data BT model 300 game season 500 game season W(r) r

77 Role of Season Length on Win Fraction W(r) game data BT model 300 game season 500 game season 1000 game season r

78 Role of Season Length on Win Fraction W(r) game data BT model 300 game season 500 game season 1000 game season game season + theory r

79 Fun Facts About Winning and Losing Streaks

80 Fun Facts About Winning and Losing Streaks longest win streaks New York Giants (1 tie) Chicago Cubs Oakland Athletics Chicago White Sox (1 tie) New York Yankees New York Giants New York Yankees New York Giants Washington Senators New York Giants Philadelphia Athletics Pittsburgh Pirates New York Giants New York Yankees New York Giants Kansas City Royals Pittsburgh Pirates New York Highlanders Philadelphia Athletics Brooklyn Dodgers Chicago Cubs New York Giants Boston Red Sox New York Yankees Minnesota Twins Atlanta Braves Seattle Mariners

81 Fun Facts About Winning and Losing Streaks longest win streaks New York Giants (1 tie) Chicago Cubs Oakland Athletics Chicago White Sox (1 tie) New York Yankees New York Giants New York Yankees New York Giants Washington Senators New York Giants Philadelphia Athletics Pittsburgh Pirates New York Giants New York Yankees New York Giants Kansas City Royals Pittsburgh Pirates New York Highlanders Philadelphia Athletics Brooklyn Dodgers Chicago Cubs New York Giants Boston Red Sox New York Yankees Minnesota Twins Atlanta Braves Seattle Mariners longest losing streaks Philadelphia Phillies Baltimore Orioles Boston Americans Philadelphia As Philadelphia As Montreal Expos (first year) Boston Beaneaters Cincinnati Reds Detroit Tigers Kansas City Royals Philadelphia As Washington Senators Washington Senators Boston Red Sox NY Mets (first year) Atlanta Braves Boston Braves Boston Doves Boston Americans (2 ties) Brooklyn Dodgers (1 made-up game) St. Louis Browns Boston Rustlers Boston Braves Boston Red Sox Boston Braves Philadelphia As Tampa Bay Texas Rangers (first year)

82 Fun Facts About Winning and Losing Streaks longest win streaks New York Giants (1 tie) Chicago Cubs Oakland Athletics Chicago White Sox (1 tie) New York Yankees New York Giants New York Yankees New York Giants Washington Senators New York Giants Philadelphia Athletics Pittsburgh Pirates New York Giants New York Yankees New York Giants Kansas City Royals Pittsburgh Pirates New York Highlanders Philadelphia Athletics Brooklyn Dodgers Chicago Cubs New York Giants Boston Red Sox New York Yankees Minnesota Twins Atlanta Braves Seattle Mariners longest losing streaks Philadelphia Phillies Baltimore Orioles Boston Americans Philadelphia As Philadelphia As Montreal Expos (first year) Boston Beaneaters Cincinnati Reds Detroit Tigers Kansas City Royals Philadelphia As Washington Senators Washington Senators Boston Red Sox NY Mets (first year) Atlanta Braves Boston Braves Boston Doves Boston Americans (2 ties) Brooklyn Dodgers (1 made-up game) St. Louis Browns Boston Rustlers Boston Braves Boston Red Sox Boston Braves Philadelphia As Tampa Bay Texas Rangers (first year) game win/loss streaks between between between 1961-present

83 Winning and Losing Streak Distributions 10 0 MLB data 10-1 P n n

84 Winning and Losing Streak Distributions 10 0 MLB data 10-1 P n n n

85 Winning and Losing Streak Distributions 10 0 BT model simulations 10-1 P n n n ε=0.435 ε=0.278

86 Streak Length Distribution in BT Model P n (x) = n j=1 n wins in a row initial loss final loss x x + x j x 0 x + x 0 x n+1 x + x n+1

87 P n (x) = Streak Length Distribution in BT Model n j=1 n wins in a row initial loss P n (x) {xj } = x n 1 x + y final loss x x + x j x 0 x + x 0 x n+1 + x n+1 n y x + y 2 1 x + y y x + y = 1 1 ɛ 1 ɛ =1 x 1 ɛ ln dy x + y = 1 1 ɛ ln ( ) x +1 x + ɛ ( ) x +1 x + ɛ

88 P n (x) = Streak Length Distribution in BT Model n j=1 n wins in a row initial loss P n (x) {xj } = x n 1 x + y final loss x x + x j x 0 x + x 0 x n+1 + x n+1 n y x + y 2 1 x + y y x + y = 1 1 ɛ 1 ɛ =1 x 1 ɛ ln dy x + y = 1 1 ɛ ln ( ) x +1 x + ɛ ( ) x +1 x + ɛ P n = 1 1 ɛ 1 ɛ [ 1 x 1 ɛ ln ( )] 2 x +1 e ng(x) dx x + ɛ g(x) = ln x + ln [ 1 1 ɛ ln ( x +1 x + ɛ )] monotonic

89 P n (x) = Streak Length Distribution in BT Model n j=1 n wins in a row initial loss P n (x) {xj } = x n 1 x + y final loss x x + x j x 0 x + x 0 x n+1 + x n+1 n y x + y 2 1 x + y y x + y = 1 1 ɛ 1 ɛ =1 x 1 ɛ ln dy x + y = 1 1 ɛ ln ( ) x +1 x + ɛ ( ) x +1 x + ɛ P n = 1 1 ɛ 1 ɛ [ 1 x 1 ɛ ln 1 ng (1) eng(1) ( )] 2 x +1 e ng(x) dx x + ɛ g(x) = ln x + ln [ 1 1 ɛ ln monotonic ( x +1 x + ɛ )]

90 Asymptotic Behavior P n 1 ng (1) eng(1) g(x) = ln x + ln [ 1 1 ɛ ln g(1) = ln(1 ɛ) + ln ln ( )] x +1 x + ɛ ( 2 1+ɛ )

91 Asymptotic Behavior P n 1 ng (1) eng(1) g(x) = ln x + ln [ 1 1 ɛ ln g(1) = ln(1 ɛ) + ln ln ( )] x +1 x + ɛ ( 2 1+ɛ ) fastest decay when ε=1: P n 2 n slowest decay when ε=0: P n (ln 2) n (0.693) n

92 Winning and Losing Streak Distributions P n n n

93 Winning and Losing Streak Distributions numerically integrate P n = 1 1 ɛ 1 ɛ [ 1 x 1 ɛ ln ( x +1 x + ɛ )] 2 e ng(x) dx P n n n

94 Winning and Losing Streak Distributions numerically integrate P n = 1 1 ɛ 1 same as: 10⁵ randomizations of all 2166 win/loss histories ɛ [ 1 x 1 ɛ ln ( x +1 x + ɛ )] 2 e ng(x) dx P n n n

95 Summary/Outlook

96 Summary/Outlook Citations: Page rank analysis: helps uncover hidden gems Future: contextual analysis, specialization

97 Summary/Outlook Citations: Page rank analysis: helps uncover hidden gems Future: contextual analysis, specialization Climate: Global warming seems to affect temperature records Open: seasonality, high/low asymmetry, changing variability

98 Inter-day Temperature Correlations 10 0 C α (t) = i=1 T i T i+t T i T i+t T 2 i T i 2 α = low, middle, high slope -4/3 (other work: slope 0.7 e.g., Bunde et al.) 10 1 C α (t) middle low 10 2 high t 100 t

99 Seasonal Variance 12 daily variances (10-day averages) 10 8 low high average day of the year

100 Seasonal Variance & Record Numbers daily variances (10-day averages) low high average number of record highs (20-day average) day of the year

101 Summary/Outlook Citations: Page rank analysis: helps uncover hidden gems Future: contextual analysis, specialization Climate: Global warming seems to affect temperature records Open: seasonality, high/low asymmetry, changing variability

102 Summary/Outlook Citations: Page rank analysis: helps uncover hidden gems Future: contextual analysis, specialization Climate: Global warming seems to affect temperature records Open: seasonality, high/low asymmetry, changing variability Competition: Statistical model win/loss records, streak distributions keener competition; harder to maintain advantage