The Fundamentals of Heavy Tails Properties, Emergence, & Identification. Jayakrishnan Nair, Adam Wierman, Bert Zwart
|
|
|
- Helen Daniel
- 6 years ago
- Views:
Transcription
1 The Fundamentals of Heavy Tails Properties, Emergence, & Identification Jayakrishnan Nair, Adam Wierman, Bert Zwart
2 Why am I doing a tutorial on heavy tails? Because we re writing a book on the topic Why are we writing a book on the topic? Because heavy-tailed phenomena are everywhere!
3 Why am I doing a tutorial on heavy tails? Because we re writing a book on the topic Why are we writing a book on the topic? Because heavy-tailed phenomena are everywhere! BUT, they are extremely misunderstood.
4 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial
5 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial Our intuition is flawed because intro probability classes focus on light-tailed distributions Simple, appealing statistical approaches have BIG problems
6 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial BUT Similar stories in electricity nets, citation nets, 2005, ToN
7 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
8 What is a heavy-tailed distribution? A distribution with a tail that is heavier than an Exponential
9 What is a heavy-tailed distribution? A distribution with a tail that is heavier than an Exponential Pr ( > ) Exponential:
10 What is a heavy-tailed distribution? A distribution with a tail that is heavier than an Exponential Pr ( > ) Exponential: Canonical Example: The Pareto Distribution a.k.a. the power-law distribution Pr > =() = density: () = for
11 What is a heavy-tailed distribution? A distribution with a tail that is heavier than an Exponential Pr ( > ) Exponential: Canonical Example: The Pareto Distribution a.k.a. the power-law distribution Many other examples: LogNormal, Weibull, Zipf, Cauchy, Student s t, Frechet, : log / =
12 What is a heavy-tailed distribution? A distribution with a tail that is heavier than an Exponential Pr ( > ) Exponential: Canonical Example: The Pareto Distribution a.k.a. the power-law distribution Many other examples: LogNormal, Weibull, Zipf, Cauchy, Student s t, Frechet, Many subclasses: Regularly varying, Subexponential, Long-tailed, Fat-tailed,
13 Heavy-tailed distributions have many strange & beautiful properties The Pareto principle : 80% of the wealth owned by 20% of the population, etc. Infinite variance or even infinite mean Events that are much larger than the mean happen frequently. These are driven by 3 defining properties 1) Scale invariance 2) The catastrophe principle 3) The residual life blows up
14 Scale invariance
15 Scale invariance is scale invariant if there exists an and a such that = () for all, such that. change of scale
16 Scale invariance is scale invariant if there exists an and a such that = () for all, such that. Theorem: A distribution is scale invariant if and only if it is Pareto. Example: Pareto distributions = = 1
17 Scale invariance is scale invariant if there exists an and a such that = () for all, such that. Asymptotic scale invariance is asymptotically scale invariant if there exists a continuous, finite such that lim () = for all.
18 Example: Regularly varying distributions is regularly varying if = (), where () is slowly varying, () i.e., lim () =1for all >0. Theorem: A distribution is asymptotically scale invariant iff it is regularly varying. Asymptotic scale invariance is asymptotically scale invariant if there exists a continuous, finite such that lim () = for all.
19 Example: Regularly varying distributions is regularly varying if = (), where () is slowly varying, () i.e., lim () =1for all >0. Regularly varying distributions are extremely useful. They basically behave like Pareto distributions with respect to the tail: Karamata theorems Tauberian theorems
20 Heavy-tailed distributions have many strange & beautiful properties The Pareto principle : 80% of the wealth owned by 20% of the population, etc. Infinite variance or even infinite mean Events that are much larger than the mean happen frequently. These are driven by 3 defining properties 1) Scale invariance 2) The catastrophe principle 3) The residual life blows up
21 A thought experiment During lecture I polled my 50 students about their heights and the number of twitter followers they have The sum of the heights was ~300 feet. The sum of the number of twitter followers was 1,025,000 What led to these large values?
22 A thought experiment During lecture I polled my 50 students about their heights and the number of twitter followers they have The sum of the heights was ~300 feet. The sum of the number of twitter followers was 1,025,000 A bunch of people were probably just over 6 tall (Maybe the basketball teams were in the class.) Conspiracy principle One person was probably a twitter celebrity and had ~1 million followers. Catastrophe principle
23 Example Consider + i.i.d Weibull. Given + =, what is the marginal density of? () Light-tailed Weibull Conspiracy principle Exponential 0 Heavy-tailed Weibull Catastrophe principle
24
25 Extremely useful for random walks, queues, etc. Principle of a single big jump
26 Subexponential distributions is subexponential if for i.i.d., Pr + + > ( >)
27 Weibull Heavy-tailed Pareto Regularly Varying Subexponential LogNormal Subexponential distributions is subexponential if for i.i.d., Pr + + > ( >)
28 Heavy-tailed distributions have many strange & beautiful properties The Pareto principle : 80% of the wealth owned by 20% of the population, etc. Infinite variance or even infinite mean Events that are much larger than the mean happen frequently. These are driven by 3 defining properties 1) Scale invariance 2) The catastrophe principle 3) The residual life blows up
29 A thought experiment residual life What happens to the expected remaining waiting time as we wait for a table at a restaurant? for a bus? for the response to an ? The remaining wait drops as you wait If you don t get it quickly, you never will
30 The distribution of residual life The distribution of remaining waiting time given you have already waited time is = () (). Examples: Exponential : = () = memoryless Pareto : = = 1+ Increasing in x
31 The distribution of residual life The distribution of remaining waiting time given you have already waited time is = () (). Mean residual life = > = Hazard rate = = (0)
32 What happens to the expected remaining waiting time as we wait for a table at a restaurant? for a bus? for the response to an ? BUT: not all heavy-tailed distributions have DHR / IMRL some light-tailed distributions are DHR / IMRL
33 Long-tailed distributions () is long-tailed if lim = lim () =1for all BUT: not all heavy-tailed distributions have DHR / IMRL some light-tailed distributions are DHR / IMRL
34 Long-tailed distributions () is long-tailed if lim = lim () =1for all Weibull Heavy-tailed Pareto Regularly Varying Subexponential LogNormal Long-tailed distributions
35 Weibull Heavy-tailed Difficult to work with in general Pareto Regularly Varying Subexponential LogNormal Long-tailed distributions
36 Asymptotically scale invariant Useful analytic properties Weibull Heavy-tailed Difficult to work with in general Pareto Regularly Varying Subexponential LogNormal Long-tailed distributions
37 Catastrophe principle Useful for studying random walks Weibull Heavy-tailed Difficult to work with in general Pareto Regularly Varying Subexponential LogNormal Long-tailed distributions
38 Residual life blows up Useful for studying extremes Weibull Heavy-tailed Difficult to work with in general Pareto Regularly Varying Subexponential LogNormal Long-tailed distributions
39 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
40 We ve all been taught that the Normal is normal because of the Central Limit Theorem But the Central Limit Theorem we re taught is not complete!
41 A quick review Consider i.i.d.. How does grow? Law of Large Numbers (LLN):.. when < = +() =15 = 300
42 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 grow? ~(0, ) when Var = <. = + + ( ) [ ]
43 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 grow? Two key assumptions ~(0, ) when Var = <. = + + ( ) [ ]
44 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 What if grow? ~(0, ) when Var = <. = + + ( ) [ ]
45 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 What if grow? ~(0, ) when Var = <. = + + ( ) [ ] = 300
46 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 What if grow? ~(0, ) when Var = <. The Generalized Central Limit Theorem (GCLT): 1 (0, ) (0,2) = + / +( / )
47 A quick review Consider i.i.d.. How does Central Limit Theorem (CLT): 1 What if grow? ~(0, ) when Var = <. The Generalized Central Limit Theorem (GCLT): 1 (0, ) (0,2)
48 Returning to our original question Consider i.i.d.. How does grow? Either the Normal distribution OR a power-law distribution can emerge!
49 Returning to our original question Consider i.i.d.. How does grow? but this isn t the only question one can ask about Either the Normal distribution OR a power-law distribution can emerge!. What is the distribution of the ruin time? The ruin time is always heavy-tailed!
50 Consider a symmetric 1-D random walk 1/2 1/2 The distribution of ruin time satisfies Pr > / What is the distribution of the ruin time? The ruin time is always heavy-tailed!
51 We ve all been taught that the Normal is normal because of the Central Limit Theorem Heavy-tails are more normal than the Normal! 1. Additive Processes 2. Multiplicative Processes 3. Extremal Processes
52 A simple multiplicative process =, where are i.i.d. and positive Ex: incomes, populations, fragmentation, twitter popularity Rich get richer
53 Multiplicative processes almost always lead to heavy tails An example:, () Pr > Pr > = is heavy-tailed!
54 Multiplicative processes almost always lead to heavy tails = log =log +log + +log Central Limit Theorem log = + +,where (0, ) when Var = <. / (0, ) where = [ ] and Var log = <.
55 / Multiplicative central limit theorem / (0, ) where = [ ] and Var log = <.
56 Satisfied by all distributions with finite mean and many with infinite mean. Multiplicative central limit theorem / (0, ) where = [ ] and Var log = <.
57 A simple multiplicative process =, where are i.i.d. and positive Ex: incomes, populations, fragmentation, twitter popularity Rich get richer LogNormals emerge Heavy-tails
58 A simple multiplicative process =, where are i.i.d. and positive Ex: incomes, populations, fragmentation, twitter popularity Multiplicative process with a lower barrier =min(,) Multiplicative process with noise = + Distributions that are approximately power-law emerge
59 A simple multiplicative process =, where are i.i.d. and positive Ex: incomes, populations, fragmentation, twitter popularity lim Multiplicative process with a lower barrier =min(,) Under minor technical conditions, such that () = where = sup ( 0 1) Nearly regularly varying
60 We ve all been taught that the Normal is normal because of the Central Limit Theorem Heavy-tails are more normal than the Normal! 1. Additive Processes 2. Multiplicative Processes 3. Extremal Processes
61 A simple extremal process =max(,,, ) Ex: engineering for floods, earthquakes, etc. Progression of world records Extreme value theory
62 =max(,,, ) How does scale? A simple example () Pr max,, > + = + =1, =log = 1 e = 1 e : Gumbel distribution
63 =max(,,, ) How does scale? Extremal Central Limit Theorem Heavy-tailed Heavy or light-tailed Light-tailed
64 =max(,,, ) How does scale? Extremal Central Limit Theorem iff are regularly varying e.g. when are Uniform e.g. when are LogNormal
65 A simple extremal process =max(,,, ) Ex: engineering for floods, earthquakes, etc. Progression of world records Either heavy-tailed or light-tailed distributions can emerge as but this isn t the only question one can ask about. What is the distribution of the time until a new record is set? The time until a record is always heavy-tailed!
66 : Time between & +1 record Pr > 2 What is the distribution of the time until a new record is set? The time until a record is always heavy-tailed!
67 We ve all been taught that the Normal is normal because of the Central Limit Theorem Heavy-tails are more normal than the Normal! 1. Additive Processes 2. Multiplicative Processes 3. Extremal Processes
68 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
69 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1999 Sigcomm paper citations! BUT Similar stories in electricity nets, citation nets, 2005, ToN
70 A typical approach for identifying of heavy tails: Linear Regression frequency Heavy-tailed or light-tailed? value frequency plot
71 A typical approach for identifying of heavy tails: Linear Regression Heavy-tailed or light-tailed? log-linear scale Log(frequency) value
72 A typical approach for identifying of heavy tails: Linear Regression Heavy-tailed or light-tailed? log-linear scale Linear Exponential tail = log = log
73 A typical approach for identifying of heavy tails: Linear Regression Heavy-tailed or light-tailed? = log = log ( + 1) log log-log scale Linear Power-law tail log-linear scale Linear Exponential tail = log = log
74 = log = log ( + 1) log log-log scale Linear Power-law tail Regression Estimate of tail index ()
75 = log = log ( + 1) log log-log scale Is it really linear? Is the estimate of accurate? Linear Power-law tail Regression Estimate of tail index ()
76 Pr > = = log-log scale = log = log ( + 1) log log-log scale Pr > =1.9 =1.4 rank plot True =2.0 frequency plot
77 This simple change is extremely important log-log scale log-log scale Pr > rank plot frequency plot
78 This simple change is extremely important Does this look like a power law? log-log scale log-log scale Pr > rank plot frequency plot The data is from an Exponential!
79 This mistake has happened A LOT! Electricity grid degree distribution log-linear scale log-log scale Pr > rank plot =3 (from Science) frequency plot
80 Pr > This mistake has happened A LOT! =1.7 rank plot log-log scale WWW degree distribution log-log scale =1.1 (from Science) frequency plot
81 This simple change is extremely important But, this is still an error-prone approach log-log scale Pr > rank plot Regression Estimate of tail index () Linear Power-law tail other distributions can be nearly linear too log-log scale log-log scale Lognormal Weibull
82 This simple change is extremely important But, this is still an error-prone approach log-log scale Pr > rank plot Regression Estimate of tail index () assumptions of regression are not met tail is much noisier than the body Linear Power-law tail other distributions can be nearly linear too
83 A completely different approach: Maximum Likelihood Estimation (MLE) What is the for which the data is most likely? ; = log (; ) = log log Maximizing gives = log( / ) This has many nice properties: is the minimal variance, unbiased estimator. is asymptotically efficient.
84 not so A completely different approach: Maximum Likelihood Estimation (MLE) Weighted Least Squares Regression (WLS) asymptotically for large data sets, when weights are chosen as =1/(log log ). ) = log( / log( / ) log( / ) =
85 not so A completely different approach: Maximum Likelihood Estimation (MLE) Weighted Least Squares Regression (WLS) asymptotically for large data sets, when weights are chosen as =1/(log log ). WLS, =2.2 log-log scale LS, =1.5 rank true plot =2.0 Listen to your body
86 A quick summary of where we are: Suppose data comes from a power-law (Pareto) distribution = Then, we can identify this visually with a log-log plot, and we can estimate using either MLE or WLS..
87 What if the data is not exactly a power-law? What if only the tail is power-law? Suppose data comes from a power-law (Pareto) distribution = Then, we can identify this visually with a log-log plot, and we can estimate using either MLE or WLS.. log-log scale Can we just use MLE/WLS on the tail? But, where does the tail start? rank plot Impossible to answer
88 An example Suppose we have a mixture of power laws: = + 1 < We want as. but, suppose we use as our cutoff: 1 ( ) + (1 ) ( )
89 Identifying power-law distributions Listen to your body MLE/WLS v.s. Identifying power-law tails Let the tail do the talking Extreme value theory
90 Returning to our example Suppose we have a mixture of power laws: = + 1 < We want as. but, suppose we use as our cutoff: 1 ( ) + (1 ) ( ) The bias disappears as!
91 The idea: Improve robustness by throwing away nearly all the data!, where as. + Larger () Small bias - Larger () Larger variance
92 The idea: Improve robustness by throwing away nearly all the data!, where as. The Hill Estimator, = log where () is the th largest data point Looks almost like the MLE, but uses order th order statistic
93 The idea: Improve robustness by throwing away nearly all the data!, where as. The Hill Estimator, = where () is the th largest data point log Looks almost like the MLE, but uses order th order statistic how do we choose?, as if / 0 & throw away nearly all the data, but keep enough data for consistency
94 The idea: Improve robustness by throwing away nearly all the data!, where as. The Hill Estimator, = where () is the th largest data point log Looks almost like the MLE, but uses order th order statistic how do we choose?, as if / 0 & Throw away everything except the outliers!
95 Choosing in practice: The Hill plot
96 Choosing in practice: The Hill plot Pareto, =2 Exponential
97 Choosing in practice: The Hill plot log-log scale Pareto, =2 2.5 Mixture, with Pareto-tail, =
98 but the hill estimator has problems too =1.4 = 300 Hill horror plot =0.92 = This data is from TCP flow sizes!
99 Identifying power-law distributions Listen to your body MLE/WLS Identifying power-law tails Let the tail do the talking Hill estimator It s dangerous to rely on any one technique! (see our forthcoming book for other approaches)
100 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
101 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial Heavy-tailed 1. Properties Weibull Pareto Regularly Varying 2. Emergence 3. Identification Subexponential LogNormal Long-tailed distributions
102 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
103 Heavy-tailed phenomena are treated as something Mysterious, Surprising, & Controversial 1. Properties 2. Emergence 3. Identification
104 The Fundamentals of Heavy Tails Properties, Emergence, & Identification Jayakrishnan Nair, Adam Wierman, Bert Zwart
Model Fitting. Jean Yves Le Boudec
Model Fitting Jean Yves Le Boudec 0 Contents 1. What is model fitting? 2. Linear Regression 3. Linear regression with norm minimization 4. Choosing a distribution 5. Heavy Tail 1 Virus Infection Data We
Heavy Tails: The Origins and Implications for Large Scale Biological & Information Systems
Heavy Tails: The Origins and Implications for Large Scale Biological & Information Systems Predrag R. Jelenković Dept. of Electrical Engineering Columbia University, NY 10027, USA {predrag}@ee.columbia.edu
Approximating the Integrated Tail Distribution
Approximating the Integrated Tail Distribution Ants Kaasik & Kalev Pärna University of Tartu VIII Tartu Conference on Multivariate Statistics 26th of June, 2007 Motivating example Let W be a steady-state
MFM Practitioner Module: Quantitiative Risk Management. John Dodson. October 14, 2015
MFM Practitioner Module: Quantitiative Risk Management October 14, 2015 The n-block maxima 1 is a random variable defined as M n max (X 1,..., X n ) for i.i.d. random variables X i with distribution function
Financial Econometrics and Volatility Models Extreme Value Theory
Financial Econometrics and Volatility Models Extreme Value Theory Eric Zivot May 3, 2010 1 Lecture Outline Modeling Maxima and Worst Cases The Generalized Extreme Value Distribution Modeling Extremes Over
Large deviations for random walks under subexponentiality: the big-jump domain
Large deviations under subexponentiality p. Large deviations for random walks under subexponentiality: the big-jump domain Ton Dieker, IBM Watson Research Center joint work with D. Denisov (Heriot-Watt,
Introduction to Algorithmic Trading Strategies Lecture 10
Introduction to Algorithmic Trading Strategies Lecture 10 Risk Management Haksun Li haksun.li@numericalmethod.com www.numericalmethod.com Outline Value at Risk (VaR) Extreme Value Theory (EVT) References
Approximation of Heavy-tailed distributions via infinite dimensional phase type distributions
1 / 36 Approximation of Heavy-tailed distributions via infinite dimensional phase type distributions Leonardo Rojas-Nandayapa The University of Queensland ANZAPW March, 2015. Barossa Valley, SA, Australia.
We're in interested in Pr{three sixes when throwing a single dice 8 times}. => Y has a binomial distribution, or in official notation, Y ~ BIN(n,p).
Sampling distributions and estimation. 1) A brief review of distributions: We're in interested in Pr{three sixes when throwing a single dice 8 times}. => Y has a binomial distribution, or in official notation,
Power Laws & Rich Get Richer
Power Laws & Rich Get Richer CMSC 498J: Social Media Computing Department of Computer Science University of Maryland Spring 2015 Hadi Amiri hadi@umd.edu Lecture Topics Popularity as a Network Phenomenon
15-388/688 - Practical Data Science: Basic probability. J. Zico Kolter Carnegie Mellon University Spring 2018
15-388/688 - Practical Data Science: Basic probability J. Zico Kolter Carnegie Mellon University Spring 2018 1 Announcements Logistics of next few lectures Final project released, proposals/groups due
CPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining MLE and MAP Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due tonight. Assignment 5: Will be released
Introduction to Scientific Modeling CS 365, Fall 2012 Power Laws and Scaling
Introduction to Scientific Modeling CS 365, Fall 2012 Power Laws and Scaling Stephanie Forrest Dept. of Computer Science Univ. of New Mexico Albuquerque, NM http://cs.unm.edu/~forrest forrest@cs.unm.edu
CPSC 340: Machine Learning and Data Mining. MLE and MAP Fall 2017
CPSC 340: Machine Learning and Data Mining MLE and MAP Fall 2017 Assignment 3: Admin 1 late day to hand in tonight, 2 late days for Wednesday. Assignment 4: Due Friday of next week. Last Time: Multi-Class
CS 124 Math Review Section January 29, 2018
CS 124 Math Review Section CS 124 is more math intensive than most of the introductory courses in the department. You re going to need to be able to do two things: 1. Perform some clever calculations to
11. Further Issues in Using OLS with TS Data
11. Further Issues in Using OLS with TS Data With TS, including lags of the dependent variable often allow us to fit much better the variation in y Exact distribution theory is rarely available in TS applications,
Overview of Extreme Value Theory. Dr. Sawsan Hilal space
Overview of Extreme Value Theory Dr. Sawsan Hilal space Maths Department - University of Bahrain space November 2010 Outline Part-1: Univariate Extremes Motivation Threshold Exceedances Part-2: Bivariate
Sampling. Where we re heading: Last time. What is the sample? Next week: Lecture Monday. **Lab Tuesday leaving at 11:00 instead of 1:00** Tomorrow:
Sampling Questions Define: Sampling, statistical inference, statistical vs. biological population, accuracy, precision, bias, random sampling Why do people use sampling techniques in monitoring? How do
Operational risk modeled analytically II: the consequences of classification invariance. Vivien BRUNEL
Operational risk modeled analytically II: the consequences of classification invariance Vivien BRUNEL Head of Risk and Capital Modeling, Société Générale and Professor of Finance, Léonard de Vinci Pôle
Theory and Applications of Complex Networks
Theory and Applications of Complex Networks 1 Theory and Applications of Complex Networks Classes Six Eight College of the Atlantic David P. Feldman 30 September 2008 http://hornacek.coa.edu/dave/ A brief
Lecture 14 October 13
STAT 383C: Statistical Modeling I Fall 2015 Lecture 14 October 13 Lecturer: Purnamrita Sarkar Scribe: Some one Disclaimer: These scribe notes have been slightly proofread and may have typos etc. Note:
review session gov 2000 gov 2000 () review session 1 / 38
review session gov 2000 gov 2000 () review session 1 / 38 Overview Random Variables and Probability Univariate Statistics Bivariate Statistics Multivariate Statistics Causal Inference gov 2000 () review
Using R in Undergraduate Probability and Mathematical Statistics Courses. Amy G. Froelich Department of Statistics Iowa State University
Using R in Undergraduate Probability and Mathematical Statistics Courses Amy G. Froelich Department of Statistics Iowa State University Undergraduate Probability and Mathematical Statistics at Iowa State
Chapter 18. Sampling Distribution Models /51
Chapter 18 Sampling Distribution Models 1 /51 Homework p432 2, 4, 6, 8, 10, 16, 17, 20, 30, 36, 41 2 /51 3 /51 Objective Students calculate values of central 4 /51 The Central Limit Theorem for Sample
We're in interested in Pr{three sixes when throwing a single dice 8 times}. => Y has a binomial distribution, or in official notation, Y ~ BIN(n,p).
Sampling distributions and estimation. 1) A brief review of distributions: We're in interested in Pr{three sixes when throwing a single dice 8 times}. => Y has a binomial distribution, or in official notation,
Introduction to Maximum Likelihood Estimation
Introduction to Maximum Likelihood Estimation Eric Zivot July 26, 2012 The Likelihood Function Let 1 be an iid sample with pdf ( ; ) where is a ( 1) vector of parameters that characterize ( ; ) Example:
Efficient rare-event simulation for sums of dependent random varia
Efficient rare-event simulation for sums of dependent random variables Leonardo Rojas-Nandayapa joint work with José Blanchet February 13, 2012 MCQMC UNSW, Sydney, Australia Contents Introduction 1 Introduction
Introduction to Rare Event Simulation
Introduction to Rare Event Simulation Brown University: Summer School on Rare Event Simulation Jose Blanchet Columbia University. Department of Statistics, Department of IEOR. Blanchet (Columbia) 1 / 31
Fundamentals of Machine Learning. Mohammad Emtiyaz Khan EPFL Aug 25, 2015
Fundamentals of Machine Learning Mohammad Emtiyaz Khan EPFL Aug 25, 25 Mohammad Emtiyaz Khan 24 Contents List of concepts 2 Course Goals 3 2 Regression 4 3 Model: Linear Regression 7 4 Cost Function: MSE
12 Statistical Justifications; the Bias-Variance Decomposition
Statistical Justifications; the Bias-Variance Decomposition 65 12 Statistical Justifications; the Bias-Variance Decomposition STATISTICAL JUSTIFICATIONS FOR REGRESSION [So far, I ve talked about regression
MS&E 226: Small Data. Lecture 11: Maximum likelihood (v2) Ramesh Johari
MS&E 226: Small Data Lecture 11: Maximum likelihood (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 18 The likelihood function 2 / 18 Estimating the parameter This lecture develops the methodology behind
STATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
Overview of Extreme Value Analysis (EVA)
Overview of Extreme Value Analysis (EVA) Brian Reich North Carolina State University July 26, 2016 Rossbypalooza Chicago, IL Brian Reich Overview of Extreme Value Analysis (EVA) 1 / 24 Importance of extremes
On the estimation of the heavy tail exponent in time series using the max spectrum. Stilian A. Stoev
On the estimation of the heavy tail exponent in time series using the max spectrum Stilian A. Stoev (sstoev@umich.edu) University of Michigan, Ann Arbor, U.S.A. JSM, Salt Lake City, 007 joint work with:
MATH 10 INTRODUCTORY STATISTICS
MATH 10 INTRODUCTORY STATISTICS Tommy Khoo Your friendly neighbourhood graduate student. It is Time for Homework! ( ω `) First homework + data will be posted on the website, under the homework tab. And
ACTEX CAS EXAM 3 STUDY GUIDE FOR MATHEMATICAL STATISTICS
ACTEX CAS EXAM 3 STUDY GUIDE FOR MATHEMATICAL STATISTICS TABLE OF CONTENTS INTRODUCTORY NOTE NOTES AND PROBLEM SETS Section 1 - Point Estimation 1 Problem Set 1 15 Section 2 - Confidence Intervals and
robustness, efficiency, breakdown point, outliers, rank-based procedures, least absolute regression
Robust Statistics robustness, efficiency, breakdown point, outliers, rank-based procedures, least absolute regression University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html
Ordinary Least Squares Linear Regression
Ordinary Least Squares Linear Regression Ryan P. Adams COS 324 Elements of Machine Learning Princeton University Linear regression is one of the simplest and most fundamental modeling ideas in statistics
Importance Sampling Methodology for Multidimensional Heavy-tailed Random Walks
Importance Sampling Methodology for Multidimensional Heavy-tailed Random Walks Jose Blanchet (joint work with Jingchen Liu) Columbia IEOR Department RESIM Blanchet (Columbia) IS for Heavy-tailed Walks
6.207/14.15: Networks Lecture 12: Generalized Random Graphs
6.207/14.15: Networks Lecture 12: Generalized Random Graphs 1 Outline Small-world model Growing random networks Power-law degree distributions: Rich-Get-Richer effects Models: Uniform attachment model
Stat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
appstats27.notebook April 06, 2017
Chapter 27 Objective Students will conduct inference on regression and analyze data to write a conclusion. Inferences for Regression An Example: Body Fat and Waist Size pg 634 Our chapter example revolves
Sampling Distribution Models. Chapter 17
Sampling Distribution Models Chapter 17 Objectives: 1. Sampling Distribution Model 2. Sampling Variability (sampling error) 3. Sampling Distribution Model for a Proportion 4. Central Limit Theorem 5. Sampling
Chapter 27 Summary Inferences for Regression
Chapter 7 Summary Inferences for Regression What have we learned? We have now applied inference to regression models. Like in all inference situations, there are conditions that we must check. We can test
Business Statistics. Lecture 10: Course Review
Business Statistics Lecture 10: Course Review 1 Descriptive Statistics for Continuous Data Numerical Summaries Location: mean, median Spread or variability: variance, standard deviation, range, percentiles,
MATH 10 INTRODUCTORY STATISTICS
MATH 10 INTRODUCTORY STATISTICS Ramesh Yapalparvi It is Time for Homework! ( ω `) First homework + data will be posted on the website, under the homework tab. And also sent out via email. 30% weekly homework.
Complex Systems Methods 11. Power laws an indicator of complexity?
Complex Systems Methods 11. Power laws an indicator of complexity? Eckehard Olbrich e.olbrich@gmx.de http://personal-homepages.mis.mpg.de/olbrich/complex systems.html Potsdam WS 2007/08 Olbrich (Leipzig)
STA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
Statistics 135 Fall 2007 Midterm Exam
Name: Student ID Number: Statistics 135 Fall 007 Midterm Exam Ignore the finite population correction in all relevant problems. The exam is closed book, but some possibly useful facts about probability
Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, 2013
Bayesian Methods Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2013 1 What about prior n Billionaire says: Wait, I know that the thumbtack is close to 50-50. What can you
Sequence convergence, the weak T-axioms, and first countability
Sequence convergence, the weak T-axioms, and first countability 1 Motivation Up to now we have been mentioning the notion of sequence convergence without actually defining it. So in this section we will
1 Introduction to Generalized Least Squares
ECONOMICS 7344, Spring 2017 Bent E. Sørensen April 12, 2017 1 Introduction to Generalized Least Squares Consider the model Y = Xβ + ɛ, where the N K matrix of regressors X is fixed, independent of the
Discrete Dependent Variable Models
Discrete Dependent Variable Models James J. Heckman University of Chicago This draft, April 10, 2006 Here s the general approach of this lecture: Economic model Decision rule (e.g. utility maximization)
Introduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
δ -method and M-estimation
Econ 2110, fall 2016, Part IVb Asymptotic Theory: δ -method and M-estimation Maximilian Kasy Department of Economics, Harvard University 1 / 40 Example Suppose we estimate the average effect of class size
Lecture 19 Inference about tail-behavior and measures of inequality
University of Illinois Department of Economics Fall 216 Econ 536 Roger Koenker Lecture 19 Inference about tail-behavior and measures of inequality An important topic, which is only rarely addressed in
E cient Simulation and Conditional Functional Limit Theorems for Ruinous Heavy-tailed Random Walks
E cient Simulation and Conditional Functional Limit Theorems for Ruinous Heavy-tailed Random Walks Jose Blanchet and Jingchen Liu y June 14, 21 Abstract The contribution of this paper is to introduce change
An overview of applied econometrics
An overview of applied econometrics Jo Thori Lind September 4, 2011 1 Introduction This note is intended as a brief overview of what is necessary to read and understand journal articles with empirical
1 Degree distributions and data
1 Degree distributions and data A great deal of effort is often spent trying to identify what functional form best describes the degree distribution of a network, particularly the upper tail of that distribution.
Descriptive Statistics (And a little bit on rounding and significant digits)
Descriptive Statistics (And a little bit on rounding and significant digits) Now that we know what our data look like, we d like to be able to describe it numerically. In other words, how can we represent
{X i } realize. n i=1 X i. Note that again X is a random variable. If we are to
3 Convergence This topic will overview a variety of extremely powerful analysis results that span statistics, estimation theorem, and big data. It provides a framework to think about how to aggregate more
Machine Learning CSE546 Sham Kakade University of Washington. Oct 4, What about continuous variables?
Linear Regression Machine Learning CSE546 Sham Kakade University of Washington Oct 4, 2016 1 What about continuous variables? Billionaire says: If I am measuring a continuous variable, what can you do
CS224W: Analysis of Networks Jure Leskovec, Stanford University
CS224W: Analysis of Networks Jure Leskovec, Stanford University http://cs224w.stanford.edu 10/30/17 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 2
1 Random walks and data
Inference, Models and Simulation for Complex Systems CSCI 7-1 Lecture 7 15 September 11 Prof. Aaron Clauset 1 Random walks and data Supposeyou have some time-series data x 1,x,x 3,...,x T and you want
Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, What about continuous variables?
Linear Regression Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2014 1 What about continuous variables? n Billionaire says: If I am measuring a continuous variable, what
Week 9 The Central Limit Theorem and Estimation Concepts
Week 9 and Estimation Concepts Week 9 and Estimation Concepts Week 9 Objectives 1 The Law of Large Numbers and the concept of consistency of averages are introduced. The condition of existence of the population
Introduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation. EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016
Introduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016 EPSY 905: Intro to Bayesian and MCMC Today s Class An
ECE521 lecture 4: 19 January Optimization, MLE, regularization
ECE521 lecture 4: 19 January 2017 Optimization, MLE, regularization First four lectures Lectures 1 and 2: Intro to ML Probability review Types of loss functions and algorithms Lecture 3: KNN Convexity
Talk Science Professional Development
Talk Science Professional Development Transcript for Grade 4 Scientist Case: The Heavy for Size Investigations 1. The Heavy for Size Investigations, Through the Eyes of a Scientist We met Associate Professor
p y (1 p) 1 y, y = 0, 1 p Y (y p) = 0, otherwise.
1. Suppose Y 1, Y 2,..., Y n is an iid sample from a Bernoulli(p) population distribution, where 0 < p < 1 is unknown. The population pmf is p y (1 p) 1 y, y = 0, 1 p Y (y p) = (a) Prove that Y is the
Economics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
Common ontinuous random variables
Common ontinuous random variables CE 311S Earlier, we saw a number of distribution families Binomial Negative binomial Hypergeometric Poisson These were useful because they represented common situations:
Bayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
Gradualism versus Catastrophism Curriculum written by XXXXX
Gradualism versus Catastrophism Curriculum written by XXXXX A curriculum written with the goal of educating 8 th grade science students on the difference between gradual and cataclysmic geological events
Solve Wave Equation from Scratch [2013 HSSP]
![Solve Wave Equation from Scratch [2013 HSSP]](/thumbs/75/72028107.jpg "Solve Wave Equation from Scratch [2013 HSSP]")
1 Solve Wave Equation from Scratch [2013 HSSP] Yuqi Zhu MIT Department of Physics, 77 Massachusetts Ave., Cambridge, MA 02139 (Dated: August 18, 2013) I. COURSE INFO Topics Date 07/07 Comple number, Cauchy-Riemann
5.2 Fisher information and the Cramer-Rao bound
Stat 200: Introduction to Statistical Inference Autumn 208/9 Lecture 5: Maximum likelihood theory Lecturer: Art B. Owen October 9 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
CHAPTER 1: Preliminary Description of Errors Experiment Methodology and Errors To introduce the concept of error analysis, let s take a real world
CHAPTER 1: Preliminary Description of Errors Experiment Methodology and Errors To introduce the concept of error analysis, let s take a real world experiment. Suppose you wanted to forecast the results
Extreme Value Theory and Applications
Extreme Value Theory and Deauville - 04/10/2013 Extreme Value Theory and Introduction Asymptotic behavior of the Sum Extreme (from Latin exter, exterus, being on the outside) : Exceeding the ordinary,
DEVELOPING MATH INTUITION
C HAPTER 1 DEVELOPING MATH INTUITION Our initial exposure to an idea shapes our intuition. And our intuition impacts how much we enjoy a subject. What do I mean? Suppose we want to define a cat : Caveman
Exploring EDA. Exploring EDA, Clustering and Data Preprocessing Lecture 1. Exploring EDA. Vincent Croft NIKHEF - Nijmegen
Exploring EDA, Clustering and Data Preprocessing Lecture 1 Exploring EDA Vincent Croft NIKHEF - Nijmegen Inverted CERN School of Computing, 23-24 February 2015 1 A picture tells a thousand words. 2 Before
Reduction of Variance. Importance Sampling
Reduction of Variance As we discussed earlier, the statistical error goes as: error = sqrt(variance/computer time). DEFINE: Efficiency = = 1/vT v = error of mean and T = total CPU time How can you make
Computer Systems on the CERN LHC: How do we measure how well they re doing?
Computer Systems on the CERN LHC: How do we measure how well they re doing? Joshua Dawes PhD Student affiliated with CERN & Manchester My research pages: http://cern.ch/go/ggh8 Purpose of this talk Principally
Maximum-Likelihood Estimation: Basic Ideas
Sociology 740 John Fox Lecture Notes Maximum-Likelihood Estimation: Basic Ideas Copyright 2014 by John Fox Maximum-Likelihood Estimation: Basic Ideas 1 I The method of maximum likelihood provides estimators
Two hours. To be supplied by the Examinations Office: Mathematical Formula Tables THE UNIVERSITY OF MANCHESTER. 21 June :45 11:45
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS 21 June 2010 9:45 11:45 Answer any FOUR of the questions. University-approved
What is a random variable
OKAN UNIVERSITY FACULTY OF ENGINEERING AND ARCHITECTURE MATH 256 Probability and Random Processes 04 Random Variables Fall 20 Yrd. Doç. Dr. Didem Kivanc Tureli didemk@ieee.org didem.kivanc@okan.edu.tr
Biostatistics and Design of Experiments Prof. Mukesh Doble Department of Biotechnology Indian Institute of Technology, Madras
Biostatistics and Design of Experiments Prof. Mukesh Doble Department of Biotechnology Indian Institute of Technology, Madras Lecture - 39 Regression Analysis Hello and welcome to the course on Biostatistics
Lecture 11: Linear Regression
Lecture 11: Linear Regression Background Suppose we have an independent variable x (time for example). And, we have some other variable y, and we want to ask how the variable y depends on x (maybe y are
Zwiers FW and Kharin VV Changes in the extremes of the climate simulated by CCC GCM2 under CO 2 doubling. J. Climate 11:
Statistical Analysis of EXTREMES in GEOPHYSICS Zwiers FW and Kharin VV. 1998. Changes in the extremes of the climate simulated by CCC GCM2 under CO 2 doubling. J. Climate 11:2200 2222. http://www.ral.ucar.edu/staff/ericg/readinggroup.html
Correlation: Copulas and Conditioning
Correlation: Copulas and Conditioning This note reviews two methods of simulating correlated variates: copula methods and conditional distributions, and the relationships between them. Particular emphasis
Nonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
Swarthmore Honors Exam 2012: Statistics
Swarthmore Honors Exam 2012: Statistics 1 Swarthmore Honors Exam 2012: Statistics John W. Emerson, Yale University NAME: Instructions: This is a closed-book three-hour exam having six questions. You may
Gaussians Linear Regression Bias-Variance Tradeoff
Readings listed in class website Gaussians Linear Regression Bias-Variance Tradeoff Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University January 22 nd, 2007 Maximum Likelihood Estimation
MS&E 226: Small Data
MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the
Math 576: Quantitative Risk Management
Math 576: Quantitative Risk Management Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 11 Haijun Li Math 576: Quantitative Risk Management Week 11 1 / 21 Outline 1
Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata
Maura Department of Economics and Finance Università Tor Vergata Hypothesis Testing Outline It is a mistake to confound strangeness with mystery Sherlock Holmes A Study in Scarlet Outline 1 The Power Function
Statistics 253/317 Introduction to Probability Models. Winter Midterm Exam Friday, Feb 8, 2013
Statistics 253/317 Introduction to Probability Models Winter 2014 - Midterm Exam Friday, Feb 8, 2013 Student Name (print): (a) Do not sit directly next to another student. (b) This is a closed-book, closed-note
Central limit theorem. Paninski, Intro. Math. Stats., October 5, probability, Z N P Z, if
Paninski, Intro. Math. Stats., October 5, 2005 35 probability, Z P Z, if P ( Z Z > ɛ) 0 as. (The weak LL is called weak because it asserts convergence in probability, which turns out to be a somewhat weak
Lecture: Local Spectral Methods (1 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 18-03/31/2015 Lecture: Local Spectral Methods (1 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
{X i } realize. n i=1 X i. Note that again X is a random variable. If we are to
3 Convergence This topic will overview a variety of extremely powerful analysis results that span statistics, estimation theorem, and big data. It provides a framework to think about how to aggregate more
Learning Energy-Based Models of High-Dimensional Data
Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal