More on Distribution Function

Transcription

1 More on Distribution Function The distribution of a random variable X can be determined directly from its cumulative distribution function F X. Theorem: Let X be any random variable, with cumulative distribution function F X. Let B be any subset of real numbers. Then P( X B) can be determined solely from the values of F X (x). Proof: STA347 - week 5 1

2 Expectation In the long run, rolling a die repeatedly what average result do you expect? In 6,000,000 rolls expect about 1,000,000 1 s, 1,000,000 s etc. Average is For a random variable X, the Expectation (or expected value or mean) of X is the expected average value of X in the long run. Symbols: μ, μ X, E(X) and EX. STA347 - week 5

3 Expectation of Discrete Random Variable For a discrete random variable X with pmf whenever the sum converge absolutely. STA347 - week 5 3

4 Examples 1) Roll a die. Let X = outcome on 1 roll. Then E(X) = 3.5. ) Bernoulli trials and. Then 3) X ~ Binomial(n, p). Then 4) X ~ Geometric(p). Then 5) X ~ Poisson(λ). Then STA347 - week 5 4

5 Expectation of Continuous Random Variable For a continuous random variable X with density whenever this integral converge absolutely. STA347 - week 5 5

6 Examples 1) X ~ Uniform(a, b). Then ) X ~ Exponential(λ). Then 3) X- is a random variable with density (i) Check if this is a valid density. (ii) Find E(X) STA347 - week 5 6

7 4) X ~ Gamma(α, λ). Then 5) X ~ Beta(α, β). Then STA347 - week 5 7

8 Theorem For g: R R If X is a discrete random variable then [ g( X )] g( x) p X ( x) = If X is a continuous random variable E x Proof: E [ g( X )] g( x) f ( x) = dx X STA347 - week 5 8

9 Examples 1. Suppose X ~ Uniform(0, 1). Let Y = X then, Y = e. Suppose X ~ Poisson(λ). Let, then X STA347 - week 5 9

10 Properties of Expectation For X, Y random variables and a, b R constants, E(aX + b) = ae(x) + b Proof: Continuous case E(aX + by) = ae(x) + be(y) Proof to come If X is a non-negative random variable, then E(X) = 0 if and only if X = 0 with probability 1. If X is a non-negative random variable, then E(X) 0 E(a) = a STA347 - week 5 10

11 Moments The k th moment of a distribution is E(X k ). We are usually interested in 1 st and nd moments (sometimes in 3 rd and 4 th ) Some second moments: 1. Suppose X ~ Uniform(0, 1), then 1 E( X ) = 3. Suppose X ~ Geometric(p), then E ( ) X = = x 1 x pq x 1 = STA347 - week 5 11

12 Variance The expected value of a random variable E(X) is a measure of the center of a distribution. The variance is a measure of how closely concentrated to center (µ) the probability is. It is also called nd central moment. Definition The variance of a random variable X is Var [ ] = E[ ( μ) ] ( X ) E ( X E( X )) = X Claim: Proof: Var ( X ) = E( X ) ( E( X )) = E( X ) μ We can use the above formula for convenience of calculation. The standard deviation of a random variable X is denoted by σ X ; it is the square root of the variance i.e.. σ X = Var ( X ) STA347 - week 5 1

13 Properties of Variance For X, Y random variables and are constants, then Var(aX + b) = a Var(X) Proof: Var(aX + by) = a Var(X) + b Var(Y) + abe[(x E(X ))(Y E(Y ))] Proof: Var(X) 0 Var(X) = 0 if and only if X = E(X) with probability 1 Var(a) = 0 STA347 - week 5 13

14 Examples Suppose X ~ Uniform(0, 1), then ( X ) = and E X = therefore Var( X ) = = 3 1 E ( ) 1 1+ q. Suppose X ~ Geometric(p), then E( X ) = and E( X ) = therefore p p 1 + q 1 q 1 p Var( X ) = = = p p p p ( ) p 3. Suppose X ~ Bernoulli(p), then X and therefore, Var ( X ) = p p = p( 1 p) E = E ( X ) = 1 p + 0 q = p STA347 - week 5 14

15 Joint Distribution of two or More Random Variables Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution. Joint behavior of random variable (continuous or discrete), X and Y is determined by their joint cumulative distribution function n dimensional case ( x, y) = P( X x, Y ). F X, Y y ( x,..., x ) = P( X x1, K, X x ). F n X1,..., X 1 n 1 n n STA347 - week 5 15

16 Properties of Joint Distribution Function For random variables X, Y, F X,Y : R [0,1] given by F X,Y (x,y) = P(X x,y x) ( x, y) 0 lim, = F X Y x y ( x, y) 1 lim, = F X Y x y F X,Y (x,y) is non-decreasing in each variable i.e. F if x 1 x and y 1 y. ( x, y ) F ( x y ) X, Y 1 1 X, Y, lim, ( ) ( ) F ( x, y) = F ( x) F x, y = F y and x X Y Y lim X, Y y X STA347 - week 5 16

17 Discrete case Suppose X, Y are discrete random variables defined on the same probability space. The joint probability mass function of discrete random variables X and Y is the function p X,Y (x,y) defined for all pairs of real numbers x and y by ( x, y) = P( X = x and Y y) p X, Y = For a joint pmf p X,Y (x,y) we must have: p X,Y (x,y) 0 for all values of x,y and x y p ( x, y) 1 X, Y = STA347 - week 5 17

18 Example for illustration Toss a coin 3 times. Define, X: number of heads on 1st toss, Y: total number of heads. The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}. We display the joint distribution of X and Y in the following table Can we recover the probability mass function for X and Y from the joint table? To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function. Similarly, to find the mass function for Y we sum the appropriate columns. STA347 - week 5 18

19 Marginal Probability Function The marginal probability mass function for X is ( x) p ( x y) p X = X, Y, y The marginal probability mass function for Y is ( y) p ( x y) p Y = X, Y, x STA347 - week 5 19

20 Case of several discrete random variables is analogous. If X 1,,X m are discrete random variables on the same sample space with joint probability function X1,... X ( x x ) = P( X = x,..., X x ) p n = 1,... m 1 1 The marginal probability function for X 1 is p X ( x ) = p ( x x ) 1 1 X1,... X n 1,... x,..., x m The -dimentional marginal probability function for X 1 and X is p X ( x, x ) = p ( x, x, x x ) 1 X 1 X1,... X n 1 3,..., x,..., x 3 m m m m m STA347 - week 5 0

21 Example Roll a die twice. Let X: number of 1 s and Y: total of the die. There is no available form of the joint mass function for X, Y. We display the joint distribution of X and Y with the following table. The marginal probability mass function of X and Y are Find P(X 1 and Y 4) STA347 - week 5 1

22 The Joint Distribution of two Continuous R.V s Definition Random variables X and Y are (jointly) continuous if there is a non-negative function f X,Y (x,y) such that (( X Y ) A) f ( x y) P,, X, Y A = for any reasonable -dimensional set A. dxdy f X,Y (x,y) is called a joint density function for (X, Y). In particular, if A = {(X, Y): X x, Y x}, the joint CDF of X,Y is X, Y ( x y) f ( u, v) F du, dv, = x y X, Y From Fundamental Theorem of Calculus we have f x y y x ( x y) = F ( x, y) = F ( x y) X. Y, X, Y X, Y, STA347 - week 5

23 Properties of joint density function ( x, y) 0 f X, Y for all x, y R It s integral over R is f X Y ( x, y), dxdy = 1 STA347 - week 5 3

24 Example Consider the following bivariate density function f X, Y ( x, y) ( x + xy) Check if it s a valid density function. = x 1, 0 otherwise y 1 Compute P(X > Y). STA347 - week 5 4

25 Marginal Densities and Distribution Functions The marginal (cumulative) distribution function of X is X x ( x) = P( X x) = f ( u y) The marginal density of X is then f F, Similarly the marginal density of Y is f ' ( x) F ( x) = f ( x y) X X X, Y, X, Y = dy ( y) f ( x y) Y X, Y, = dx dydu STA347 - week 5 5

26 Example Consider the following bivariate density function f X, Y ( x, y) 6xy = Check if it is a valid density function. 0 x 1, 0 y 1 0 otherwise Find the joint CDF of (X, Y) and compute P(X ½,Y ½). Compute P(X,Y ½). Find the marginal densities of X and Y. STA347 - week 5 6

27 Generalization to higher dimensions Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then Marginal density of X is given by: ( x) f ( x, y z) f X =, dydz Marginal density of X, Y is given by : ( x, y) f ( x, y z) f X, Y, = dz STA347 - week 5 7

28 Example Given the following joint CDF of X, Y ( x, y) = x y + y x x y 0 x 1, 0 y 1 F X, Y Find the joint density of X, Y. Find the marginal densities of X and Y and identify them. STA347 - week 5 8

29 Example Consider the joint density f X, Y ( x, y) = λ e 0 λ y 0 x y otherwise where λ is a positive parameter. Check if it is a valid density. Find the marginal densities of X and Y and identify them. STA347 - week 5 9

30 Independence of random variables Recall the definition of random variable X: mapping from Ω to R such that ω Ω : X ω = x for x R. { ( ) } F By definition of F, this implies that (X > 1.4) is an event and for the discrete case (X = ) is an event. ( ) { ( ) } In general X A = ω : X ω A is an event for any set A that is formed by taking unions / complements / intersections of intervals from R. Definition Random variables X and Y are independent if the events ( A) and are independent. X ( Y B) STA347 - week 5 30

31 Theorem Two discrete random variables X and Y with joint pmf p X,Y (x,y) and marginal mass function p X (x) and p Y (y), are independent if and only if Proof: p ( x, y) p ( x) p ( y) X, Y = X Y Question: Back to the rolling die times example, are X and Y independent? STA347 - week 5 31

32 Theorem Suppose X and Y are jointly continuous random variables. X and Y are independent if and only if given any two densities for X and Y their product is the joint density for the pair (X,Y) i.e. Proof: f ( x, y) f ( x) f ( y) X, Y = X Y If X and Y are independent random variables and Z =g(x), W = h(y) then Z, W are also independent. STA347 - week 5 3

33 Example Suppose X and Y are discrete random variables whose values are the nonnegative integers and their joint probability function is p X 1 x y ( λ+ μ ), Y ( x, y) = λ μ e x, y = x! y! 0,1,... Are X and Y independent? What are their marginal distributions? Factorization is enough for independence, but we need to be careful of constant terms for factors to be marginal probability functions. STA347 - week 5 33

34 Example and Important Comment The joint density for X, Y is given by f X, Y ( x, y) 4 = ( x + y ) 0 x, y > 0, x + otherwise y 1 Are X, Y independent? Independence requires that the set of points where the joint density is positive must be the Cartesian product of the set of points where the marginal densities are positive i.e. the set of points where f X,Y (x,y) >0 must be (possibly infinite) rectangles. STA347 - week 5 34