Summary of Random Variable Concepts March 17, 2000

Transcription

1 Summar of Random Variable Concepts March 17, 2000 This is a list of important concepts we have covered, rather than a review that devives or eplains them. Tpes of random variables discrete A random variable X is discrete if there is a discrete set A (i.e. finite our countabl infinite) such that Pr(X A) = 1. continuous A random variable X is continuous if Pr(X=) = 0 for all values. mied (other) A random variable X is continuous if it is neither discrete nor continuous. In other words there is a least one value such that Pr(X=)>0 and the sum of the probabilities of all values with positive probabilit is not one. The Probabilit Distribution of a Random Variable The probabilit distribution of a random variable X can be described b the following three tpes of functions probabilit mass function (pmf) p X (): for discrete random variables onl Defn: p X () = Pr(X=) Ke propert: Pr(X B) = p X (), B probabilit densit function (pdf) f X (): basicall for continuous random variables, but with delta functions a densit can also be used for discrete and mied random variables. f X () is defined to be a function such that Pr(X B) = B f X () d, this is also how it is principall used. cumulative distribution function (cdf) F X () Defn: F X () = Pr(X ) Pr(a<X b) = F X (b)-f X (a), Relationships between pdf and cdf: F X () = f X (') d' d f X () = d F X() Some common probabilit distributions: Discrete: binar, binomial, Poisson, geometric Continuous: uniform, Gaussian, eponential, Laplacian EECS 401 1

2 Functions of Random Variables Suppose X is a random variable and Y = g(x) Ke fact: Pr(Y B) = Pr(X { : g() B}) Common question: Given knowledge of the function g and the distribution of X (i.e. of its pdf, pmf or cdf), find the probabilit distribution of Y (i.e. its pdf, pmf or cdf) Important preliminar steps: 1. Identif the possible values of Y. 2. Determine the tpe of Y (discrete, continuous or mied). Special case, if X is continuous r.v. with densit f X () and g() is a function with no flat segments ecept those on which f X () = 0, then Y is a continuous r.v. with pdf f Y () = i where 1, 2,... are the values of such that g() =. f X ( i ) 1 g'( i ) If g() has flat segments, then it is possible for f Y () to contain delta functions. Epected Values For a random variable X with densit f X () (X could be continuous, discrete or mied), the epected value or mean value of X is E[X] = f X () d For a discrete random variable with pmf p X (), an alternative and usuall more useful formula is E[X] = p X (), where means to sum over all the possible values of Fundamental theorem of epectation: When Y = g(x), E[Y] = E[g(X)] = g() f X () d Linearit of epectation: E[aX+b] = a E[X] + b (Generall speaking, for nonlinear functions, E[g(X)] g(e[x]) ) Moments: The nth moment of X is E[X n ] Central moments: The nth central moment of X is E[(X-E[X]) n ] Variance of X: var(x) = E[(X-E[X]) 2 ] = E[X 2 ] - (E[X]) 2 Chebchev's Inequalit Pr( X-E[X] ε) σ2 X ε 2 EECS 401 2

3 Pairs of Random Variables Let X and Y be random variables. Each could be discrete, continuous or mied. The need not be of the same tpe. The Joint Distribution of a Pair of Random Variables: Can be described b the following three tpes of functions Joint pmf: applies onl when both X and Y are discrete Defn: p XY (,) = Pr(X=, Y=) Ke propert: Pr((X,Y) A) = p XY (,) (,) A Joint pdf: applies onl when both X and Y are continuous AND Pr((X,Y) A) = 0 for ever set A having zero area. Defn: the joint pdf is a function f XY (,) such that Pr((X,Y) A) = A f XY (,) d d An eample of a pair of continuous random variables that do not have joint densit: X is continuous and Y = g(x). Note: We have not introduced delta functions for use in joint densities. For pairs of random variables, the are too complicated to be of use. Joint cdf: Defn: F XY (,) = Pr(X, Y ) Propert: Pr(a X b and c Y d) = F XY (b,d) - F XY (a,d) - F XY (b,c) + F XY (a,c) Interrelationships F XY (,) = f XY (',') d' d' if X and Y have joint densit F XY (,) = p XY (,) if X and Y are discrete ',' f XY (,) = d 2 d d F XY(,) if X and Y have joint densit marginal distribution of X (similar relationships for Y) F X () = F XY (,) f X () = f XY (,) d p X () = p XY (, EECS 401 3

4 A pair of Gaussian random variables has joint densit of the form f XY (,) = 1 (- X) _ 2 σ 2-2r(-_ X)(- Y) _ X σ X σ Y + (- _ Y) 2 2σ 2 Y 2πσ X σ ep Y 1-r 2 - where -1 r 1. The marginal densit of X is f X () = 1 ep - (-_ X) 2 2πσ2 2σ 2 X X 2(1-r 2 ) Independence Defn: Random variables X and Y are independent if the events {X Α} and {Y B} are indepenendent for all A and B. In other words X and Y are independent if Pr(X A, Y B) = Pr(X A) Pr(X B) for all A and B Each of the following is an equivalent conditions for independence: p XY (,) = p X () p Y () all, (applies onl if X and Y are discrete) f XY (,) = f X () f Y () all, (applies onl if X and Y are continuous with a joint densit) F XY (,) = F X () F Y () all, (applies to an pair of random variables) Equal vs. Identical (not covered on Eam 2) Defn: Random variables X and Y are equal, i.e. X = Y, if Pr(X=Y)=1, i.e. the produce the same value with probabilit one. Defn: Random variables X and Y are identical if the have the same probabilit distribution, i.e. the have the same cdf and pdf and if discrete the have the same pmf. equal identical. identical \ equal. independent not equal; equivalentl equal not independent Conditional Probabilit Distributions tpe of functtion describing conditional probabilit distribution tpe of conditioning pmf pdf cdf X Β p X ( B) f X ( B) F X ( B) Y B p X ( Y B) f X ( Y B) F X ( Y B) Y= p X Y ( ) f X Y ( ) F X Y ( ) nonumerical event p X ( event) p X ( event) F X ( event) EECS 401 4

5 Conditional pmf's: These are defined b p X ( B) = Pr(X= X B) p X ( Y B) = Pr(X= X B) p X Y ( ) = Pr(X= Y=) p X ( event) = Pr(X= event) As functions of, the conditional pmf's have all the usual properties of pmf's. Most importantl the are summed to compute conditional probabilities, e.g. Pr(X A Y=) = p X Y ( ) A Conditional pdf's: The conditional pdf's are defined as functions that one integrates to compute a conditional probabilit. Specificall, Pr(X A X B) = A f X ( B) d Pr(X A Y B) = A f X ( Y B) d Pr(X A Y=) = A f X Y ( ) d Pr(X A event) = f X ( event) d A As functions of, the conditional pmf's have all the usual properties of pdf's. Conditional cdf's: These are defined b F X ( B) = Pr(X X B) F X ( Y B) = Pr(X X B) F X Y ( ) = Pr(X Y=) F X ( event) = Pr(X event) As functions of, the conditional pmf's have all the usual properties of cdf's. Notes: Of all the above functions, the following are generall the most useful and nicest to work with: p X (), p Y (), p XY (,), p X Y ( ), p Y X ( ), f X (), f Y (), f XY (,), f X Y ( ), f Y X ( ) Conditioning can change the tpe of a random variable X. For eample, X could be continuous, but conditioned on Y =, X could be discrete. We use a conditional pmf when X is conditionall discrete. We use a conditional pdf when X is conditionall continuous. We can also use a conditional pmf when X is conditionall discrete or mied, but it will contain delta functions. EECS 401 5

6 The Big Four Rrelationships X&Y discrete X&Y continuous with a joint densit 1. p XY (,) = p Y () p X Y ( ) f XY (,) = f Y () f X Y ( ) 2. p Y X ( ) = p XY(,) p X () 3. Baes rule p X Y ( ) = p Y X( )p X () p Y () f Y X ( ) = f XY(,) f X () f X Y ( ) = f Y X( )f X () f Y () 4. Total prob. p X () = p XY (,) = p Y () p X Y ( ) Other relationships: f X (), B f X ( B) = Pr(X B, else) F X ( B) = f X (' B) d' when X is conditionall continuous = p X (' B) when X is conditionall discrete ' f X ( B) = d d F X( B) f X () = f XY (,) d = f Y ()f X Y ( ) d Same as the above but for conditioning on Y B or Y= or nonnumerical conditioning. f X ( Y B) = f XY (,) d B Pr(Y B) Other total probabilit laws: These can be straightforwardl derived, though we haven't derived them in class. If B 1,...,B n form a partition, then n f X () = fx ( B i ) Pr(X B i ) i=1 n p X () = px ( B i ) Pr(X B i ) i=1 Pr(X A) = Pr(X A Y=) f Y () d Pr(X A) = Pr(X A Y=) p Y () EECS 401 6