Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.

Transcription

1 Probability UBC Economics 326 January 23, 2018

2 1 2 3

3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections Diez, Barr, and Cetinkaya-Rundel (2012) chapter 2 Grinstead and Snell (2003) chapters 1-7

4 Probability Purpose: system for quantifying chance and making predictions about future events Interpretations: Relative frequency of an event in many repeated trials Subjective assessment of the likelihood of an event

5 Section 1

6 Basic definitions experiment: procedure that has well-defined set of outcomes and could be infinitely repeated Sample space: set of possible outcomes of an experiment, S = {a 1, a 2,..., a J } Event: any subset of sample space, A S Probability: function from all subsets of the sample space, S, to [0, 1] such that 1 P(S) = P(A) 0 for all A S 3 If A 1, A 2,... are disjoint events then P(A 1 A 2...) = P(A 1 ) + P(A 2 ) +... variable: function from S to a number

7 Section 2

8 of probability 1 P( ) = 0 2 P(A c ) = 1 P(A) 3 A B implies P(A) P(B) 4 P(A B) = P(A) + P(B) P(A B)

9 probability probability: P(A B) = P(A B)/P(B) Satisfies axioms of unconditional probability (and so also has properties on previous slide) Bayes Rule P(A B) = P(B A)P(A) P(B) A is independent of B iff P(A B) = P(A)P(B), denote as A B A B implies P(A B) = P(A) and P(B A) = P(B).

10 Section 3

11 random Recall: a random variable is a function from S to a number A random variable is discrete if it can only take countably many different values E.g. X {x 1,..., x m } Probability mass function (PMF) of X, p i = P(X = x i ) gives the probability that X equals each of its possible values Cumulative distribution function (CDF): F(x) = P(X x) = m i=1 p i1{x i x}

12 random A random variable is continuous if it takes on any single real value with zero probability Set of possible values is uncountably infinite (e.g. real line or line segment) CDF is continuous and differentiable Probability density function (PDF): the derivative of the CDF f(x) = df x (x) and F(x) = f(t)dt dx

13 : Joint PMF f X,Y (x, y) = P(X = x, Y = y) Marginal PMF f X (x) = P(X = x) = y all values of Y P(X = x, Y = y) PMF f X Y (x y) = f X,Y(x,y) f Y (y) : joint CDF F X,Y (x, y) = P(X x, Y y) = x y f X,Y(u, t)dudt joint PDF f X,Y (x, y) = 2 F X,Y x y (x, y) Marginal CDF for X, F X (x) = P(X x) = F X,Y (x, ) Marginal PDF for X, f X (x) = df X dx (x) = f X,Y(x, y)dy PDF f X Y (x y) = f X,Y(x,y) f Y (y)

14 Independence X, Y are independent if f X,Y (x, y) = f X (x)f Y (y) for all x, y X Y implies f X Y (x y) = f X (x)

15 The of X is Expectation E[X] = { m i=1 x if X (x i ) if X discrete xf X(x)dx if X continuous Is a constant a.k.a. expected value, population average, population mean, first moment Expectation is linear: E[aX + by] = ae[x] + be[y] Expectation of function g: E[g(X)] = g(x)f X (x)dx

16 Variance and other moments kth moment of X: E[X k ] kth central moment of X: E [(X E[X]) k] Variance is the 2nd central moment Var(X) =E [ (X E[X]) 2] =E[X 2 ] E[X] 2 Var(a + bx) = b 2 Var(X) Standard deviation is Var(X)

17 Covariance Covariance of X and Y: Cov(X, Y) =E [(X E[X]) (Y E[Y])] =E[XY] E[X]E[Y] =E[(X E[X]) Y] = E[X (Y E[Y])] Cov(X, X) = Var(X) Cov(a 1 + b 1 X + c 1 Y, a 2 + b 2 X + c 2 Y) = b 1 b 2 Var(X) + c 1 c 2 Var(Y) + (b 1 c 2 + c 1 b 2 )Cov(X, Y) Var(a + bx + cy) = b 2 Var(X) + c 2 Var(Y) + 2bcCov(X, Y) ( N ) Var i=1 b ix i = ( N N ) i=1 j=1 b ib j Cov(X i, X j )

18 Correlation Correlation of X and Y is Corr(X, Y) = Cauchy-Schwartz inequality: Cov(X, Y) Var(X)Var(Y) Implies 1 Corr(X, Y) 1 Corr(X, Y) = ±1 iff Y = a + bx Cov(X,Y) Var(X)Var(Y)

19 of Y given X = x: { m i=1 E[Y X = x] = y if Y X (y i x) if discrete yf Y X(y x)dy if continuous E[Y X] is a function of X Has all properties of unconditional : E[g 1 (X) + g 2 (X)Y X] = g 1 (X) + g 2 (X)E[Y X] Law of iterated s: E X [ EY X [Y X] ] = E[Y] E [X (Y E[Y X])] = 0

20 variance variance: Var(Y X) = E [ (Y E[Y X]) 2 X ] Relation to variance: Var(Y) = E X [Var(Y X)] + Var (E[Y X]) Analysis of variance (ANOVA) E X [Var(Y X)] is within-x variance Var (E[Y X]) is between-x variance

21 Abbring, Jaap An Introduction to Econometrics: Lecture notes. URL courses/b44old/lect210.pdf. Diez, David M, Christopher D Barr, and Mine Cetinkaya-Rundel OpenIntro Statistics. OpenIntro. URL Grinstead, Charles M and J. Laurie Snell Introduction to Probability. American Mathematical Society. URL books_articles/probability_book/book.html. Linton, Oliver B Probability, Statistics and Econometrics. Academic Press. URL Probability%2C%20statistics%20and%20econometrics.

22 Stock, J.H. and M.W. Watson Introduction to Econometrics, 2/E. Addison-Wesley. Wooldridge, J.M Introductory econometrics: A modern approach. South-Western.