ECON Fundamentals of Probability

Transcription

1 ECON Fundamentals of Probability Maggie Jones 1 / 32

2 Random Variables A random variable is one that takes on numerical values, i.e. numerical summary of a random outcome e.g., prices, total GDP, number of people who show up for a flight, if it rains today or not, etc A discrete random variable is one that takes on only a finite (or countably infinite) number of values e.g., number of people who show up for a flight, if it rains today or not A continuous random variable can take on a continuum of possible values prices, total GDP 2 / 32

3 Probability Density Functions (PDF) The probability density function of X summarizes information concerning all possible realizations of X e.g., input xj and the function returns the probability of observing x j f(x j ) = p j, j = 1,..., k, 3 / 32

4 Discrete Random Variable Bernoulli random variable used to model outcome of flipping a coin { heads with probability 0.5 X = tails with probability / 32

5 PDF of Bernoulli Random Variable probability Bernoulli Random Variable 5 / 32

6 Cumulative Distribution Function The cumulative distribution function of X gives the probability that X is less than or equal to any real number x F (x) = P (X x) Note that for a discrete random variable the CDF is obtained by summing the pdf over each value of x Two important properties of CDFs: For any number c, P (X > c) = 1 F (c) For any numbers a and b, where a < b, P (a < X b) = F (b) F (a) 6 / 32

7 CDF of Bernoulli Random Variable cumulative prob Bernoulli Random Variable 7 / 32

8 Joint Distributions The probability that event A and B occur simultaneously e.g., the probability that someone earns over the median income and has a university degree If X and Y are discrete random variables then their joint probability density function is: f X,Y (x, y) = P (X = x, Y = y) If X and Y are independent then the joint pdf is: f X,Y (x, y) = f X (x) f Y (y) 8 / 32

9 Conditional Distributions The probability that event A occurs, given that B has occurred e.g., given than someone has a university degree, what is the likelihood that they earn over the median income Given random variables X and Y the conditional probability density function of Y given X is: f Y X (y x) = f X,Y (x, y) f X (x) Note that if X and Y are independent, then knowledge of X does not convey additional information about the value taken on by Y so that f Y X (y x) = f Y (y) 9 / 32

10 First Moment: Central Tendency Given a random variable X with pdf f(x), the expected value is an average of all possible values of X weighted by their probability If X is a discrete random variable then E(X) is computed as: E(X) = x 1 f(x 1 ) + x 2 f(x 2 )... x k f(x k ) k x j f(x j ) j=1 If X is a continuous random variable then E(X) is computed as: E(X) = xf(x)dx 10 / 32

11 Useful Properties of Expected Values For any constant c, E(c) = c For any constants a and b, E(aX + b) = ae(x) + b If {a 1, a 2,..., a n } are constants and {X 1, X 2,..., X n } are random variables, then E(a 1 X 1 + a 2 X a n X n ) = a 1 E(X 1 ) + a 2 E(X 2 ) + + a n E(X n ) 11 / 32

12 Second Moment: Variability The variance of a random variable tells us the expected distance the r.v. is from its mean (alternatively, you can think of the variance as a way to summarize the spread of the distribution) Var(X) E [ (X µ) 2] 12 / 32

13 Useful Properties of the Variance For any constant c, Var(c) = 0 (Note that the variance can only be zero for a constant). For any constants a and b, Var(aX + b) = a 2 Var(X) For any constants a and b and random variables X and Y then Var(aX + by ) = a 2 Var(X) + b 2 Var(Y ) + 2abCov(X, Y ) where Cov(X, Y ) is the covariance between X and Y : Cov(X, Y ) = E [(X µ x )(Y µ y )] 13 / 32

14 Covariance The covariance of two random variables measures the amount of linear dependence between the two variables Some useful properties of the covariance are: If X and Y are independent, then Cov(X, Y ) = 0 For any constants a, b, c, and d Cov(aX + b, cy + d) = accov(x, Y ) Cov(X, Y ) sd(x)sd(y ) 14 / 32

15 Correlation Coefficient The correlation coefficient measures dependence between two random variables X and Y with a useful interpretation ρ Cov(X, Y ) sd(x)sd(y ) = σ XY σ X σ Y { 1, 1} For any constants a, b, c, and d, If ac > 0, then Corr(aX + b, cy + d) = Corr(X, Y ) If ac < 0 then Corr(aX + b, cy + d) = Corr(X, Y ) 15 / 32

16 Standardizing a Random Variable Given a random variable X with mean µ and standard deviation σ, then define new variable Z = X µ σ where E(Z) = 0 and Var(Z) = 1 16 / 32

17 Conditional Expectation In economics we are generally interested in explaining one variable Y in terms of another variable X. The conditional expectation of Y given X provides a useful way to summarize the relationship between Y and X. If Y is a discrete random variable then E(Y x) = m y i f Y X (y j x) j=1 17 / 32

18 Properties of Conditional Expectations Given function c(x), E [c(x) X] = c(x) For functions a(x) and b(x), E [a(x)y + b(x) X] = a(x)e(x Y ) + b(x) If X and Y are independent, then E(Y X) = E(Y ) E [E(Y X)] = E(Y ) known as the law of iterated expectations 18 / 32

19 Common Probability Distributions 19 / 32

20 Normal Distribution In econometrics we rely on the normal distribution to conduct inference If X is a normally distributed random variable with mean µ and variance σ 2 then the pdf of X is defined over the interval x {, } by: f(x) = 1 σ 2π e (x µ) 2 2σ 2 Examples of (roughly) normally distributed variables include heights, weights, test scores, etc 20 / 32

21 PDF of Normal Variable probability Normal Random Variable 21 / 32

22 CDF of Normal Variable cumulative prob Normal Random Variable 22 / 32

23 Properties of Normal Distribution If X N(µ, σ 2 ) then Z = X µ σ N(0, 1) If X N(µ, σ 2 ) then ax + b N(aµ + b, a 2 σ 2 ) Any linear combination of independent, identically distributed normal random variables has a normal distribution 23 / 32

24 Chi-Square Distribution The chi-square distribution is obtained by summing across the square of independent standard normal random variables X = n i=1 Z 2 i χ 2 n Note that if X χ 2 n then E(X) = n 24 / 32

25 PDF of Chi-Squared Variable / 32

26 CDF of Chi-Squared Variable / 32

27 t Distribution One of the most commonly used distributions in econometrics Obtained from a standard normal random variable and a chi-square random variable Suppose Z N(0, 1) and X χ 2 n then the random variable T = Z X/n t n If n > 1 then E(T ) = 0 If n > 2 then Var(T ) = n n 2 Note as df then T t n Z N(0, 1) 27 / 32

28 PDF of t Distribution / 32

29 CDF of t Distribution / 32

30 F Distribution Another important distribution for econometrics Let X χ 2 n and Y χ 2 m then the random variable F = X/n Y/m F n,m 30 / 32

31 PDF of F Distribution / 32

32 CDF of F Distribution / 32