RS Chapter Random Variables 9/8/017 Chapter Random Variables Random Variables A random variable is a convenient way to epress the elements of Ω as numbers rather than abstract elements of sets. Definition: Let A X ; A Y be nonempty families of subsets of X and Y, respectively. A function f: X Y is (A X ;A Y )-measurable if f -1 A A X for all A A Y. Definition: Random Variable A random variable X is a measurable function from the probability space (Ω, Σ,P) into the probability space (χ, A X, P X ), where χ in R is the range of X (which is a subset of the real line) A X is a Borel field of X, and P X is the probability measure on χ induced by X. Specifically, X: Ω χ. 1
RS Chapter Random Variables 9/8/017 Random Variables - Remarks Remarks: - A random variable X is a function. - It is a numerical quantity whose value is determined by a random eperiment. - It takes single elements in Ω and maps them to single points in R. - P is the probability measure over the sample space and P X is the probability measure over the range of the random variable. - The induced measure P X is just a way of relating measure on the real line --the range of X-- back to the original probability measure over the abstract events in the σ-algebra of the sample space. Random Variables - Interpretation Interpretation The induced measure P X allows us to relate a measure on the real line --the range of X-- back to the original probability measure over the abstract events in the σ-algebra of the sample space: P X [A] = P [X -1 (A)] =P[{ω Ω: X(ω) A}. That is, we take the probability weights associated with events and assign them to real numbers. Recall that when we deal with probabilities on some random variable X, we are really dealing with the P X measure. We measure the "size" of the set (using P as our measure) of ω's such that the random variable X returns values in A.
RS Chapter Random Variables 9/8/017 Random Variables - Interpretation Interpretation P is the probability measure over the sample space and P X is the probability measure over the range of the random variable. Thus, we write P[A] (where A is a subset of the range of X) but we mean P X [A], which is equivalent to P[{ω Ω: X(ω) A}]. Notational shortcut: We use P[A] instead of P X [A] (This notation can be misleading if there's confusion about whether A is in the sample space or in the range of X.) Random Variables Eample 1 Eample: Back to the previous eample where two coins are tossed. We defined the sample space (Ω) as all possible outcomes and the sigma algebra of all possible subsets of the sample space. A simple probability measure (P) was applied to the events in the sigma algebra. Let the random variable X be "number of heads." Recall that X takes Ω into χ and induces P X from P. In this eample, χ = {0; 1; } and A = {Φ; {0}; {1}; {}; {0;1}; {0;}; {1;}; {0;1;}}. The induced probability measure P X from the measure defined above would look like: 3
RS Chapter Random Variables 9/8/017 Random Variables Eample 1 Eample: Back to the previous eample where two coins are tossed. Prob. of 0 heads = P X [0] = P[{TT}] = 1/4 Prob. of 1 heads = P X [1] = P[{HT; TH}] = 1/ Prob. of heads = P X [] = P[{HH}] = ¼ Prob. of 0 or 1 heads = P X [{0; 1}] = P[{TT; TH; HT}] = 3/4 Prob. of 0 or heads = P X [{0; }] = P[{TT; HH}] = 1/ Prob. of 1 or heads = P X [{1; }] = P[{TH; HT; HH}] = 3/4 Prob. of 1,, or 3 heads = P X [{0; 1; }] = P[{HH; TH; HT; TT}] = 1 Prob. of "nothing" = P X [Φ] = P[Φ] = 0 The empty set is simply needed to complete the σ-algebra. Its interpretation is not important since P[Φ] = 0 for any reasonable P. Random Variables Eample Eample: Probability Space One standard probability space is the Borel field over the unit interval of the real line under the Lebesgue measure λ. That is ([0; 1]; B; λ). The Borel field over the unit interval gives us a set of all possible intervals taken from [0,1]. The Lebesgue measure measures the size of any given interval. For any interval [a; b] in [0,1] with b a, λ[[a,b]] = b - a. This probability space is well known: uniform distribution, the probability of any interval of values is the size of the interval. 4
RS Chapter Random Variables 9/8/017 Random Variables Eample 3 1. Two dice are rolled and X is the sum of the two upward faces.. A coin is tossed n = 3 times and X is the number of times that a head occurs. 3. A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower left hand corner. X point 4. Xis the number of times the price of IBM increases during a time interval, say a day. 5. Today, the DJ Inde is 9,504.17, X is the value of the inde in thirty days. Random Variables: Summary Ω is the sample space - the set of possible outcomes from an eperiment. - An event A is a set containing outcomes from the sample space. Σ is a σ -algebra of subsets of the sample space. Think of Σ as the collection of all possible events involving outcomes chosen from. P is a probability measure over Σ. P assigns a number between [0,1] to each event in Σ. We have functions (random variables) that allow us to look at real numbers instead of abstract events in Σ. 5
RS Chapter Random Variables 9/8/017 Random Variables: Summary For each random variable X, there eists a new probability measure P X : P X [A] where A R simply relates back to P[{ω Ω: X(ω) A}. We calculate P X [A], but we are really interested in the probability P[{ω Ω: X(ω) A}, where A simply represents {ω Ω: X(ω) A} through the inverse transformation X -1. Random Variables: Probability Function & CDF Definition - The probability function, p(), of a RV, X. For any random variable, X, and any real number,, we define p P X P X where {X = } = the set of all outcomes (event) with X =. Definition The cumulative distribution function (CDF), F(), of a RV, X. For any random variable, X, and any real number,, we define F P X P X where {X } = the set of all outcomes (event) with X. 6
RS Chapter Random Variables 9/8/017 Probability Function & CDF: Eample I Two dice are rolled and X is the sum of the two upward faces. Sample space S = { :(1,1), 3:(1,;,1), 4:(1,3; 3,1;,), 5:(1,4;,3; 3,; 4,1), 6, 7, 8, 9, 10, 11, 1}. Graph: Probability function: p() 0.18 0.1 0.06 0.00 3 4 5 6 7 8 9 10 11 1 Probability Function & CDF: Eample I Probability function: 1 p PX P 1,1 36 p3 PX 3 P 1,,,1 36 3 p4 PX 4 P 1,3,,, 3,1 36 4 5 6 5 4 p5, p6, p7, p8, p9 36 36 36 36 36 3 1 p10, p11, p1 36 36 36 and p 0 for all other Note: X for all other 7
RS Chapter Random Variables 9/8/017 Probability Function & CDF: Eample I Graph: CDF 0 1 36 3 3 36 3 4 6 36 4 5 10 36 5 6 15 36 6 7 1 36 7 8 6 36 8 9 30 9 10 36 33 10 11 36 35 11 1 36 1 1 F 1. 1 0.8 0.6 0.4 0. 0 0 5 10 F() is a step function Probability Function & CDF: Eample II A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower le{t hand corner. X point An event, E, is any subset of the square, S. P[E] = (area of E)/(Area of S) = area of E E S 8
RS Chapter Random Variables 9/8/017 Probability Function & CDF: Eample II The probability function is given by: set of all points a dist p PX P 0 from lower left corner S Thus p() = 0 for all values of. The probability function for this eample is not very informative. The Cumulative distribution function is given by: set of all points within a F PX P dist from lower left corner S 0 1 1 9
RS Chapter Random Variables 9/8/017 0 0 0 1 4 F PX Area A 1 1 S A 0 1 1 Computation of Area A 1 A 1 1 1 1 tan 1 1 1 A 1 1 1 tan 1 4 4 1 1 tan 1 10
RS Chapter Random Variables 9/8/017 0 0 0 1 4 F PX 1 1 tan 1 1 4 1 1 F 0-1 0 1 Random Variables: PDF for a Continuous RV Definition: Suppose that X is a random variable. Let f() denote a function defined for - < < with the following properties: 1. f() 0. f d1. 3. P a X b f d. b a Then f() is called the probability density function of X.. The random variable X is called continuous. 11
RS Chapter Random Variables 9/8/017 Random Variables: PDF for a Continuous RV f d1. b. P a X b f d a Random Variables: CDF for a Continuous RV. F P X f t dt F 1
RS Chapter Random Variables 9/8/017 CDF and PDF for a Continuous RV: Relation Thus if X is a continuous random variable with probability density function, f(), the cumulative distribution function of X is given by:. F P X f t dt Also because of the FTC (fundamental theorem of calculus): df F f d CDF and PDF for a Continuous RV: Relation Eample: Deriving a pdf from a CDF A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower left hand corner. point X 0 0 4 0 1 F PX 1 1 1 tan 1 4 1 13
RS Chapter Random Variables 9/8/017 CDF and PDF for a Continuous RV: Relation Now 0 0 or f F 0 1 d 1 1 tan 1 1 d 4 CDF and PDF for a Continuous RV: Relation Also d d 1 tan 1 4 1 3 1 1 d 1 1 tan 1 tan 1 d 1 3 tan 1 1 d 1 tan d 1 14
RS Chapter Random Variables 9/8/017 CDF and PDF for a Continuous RV: Relation Now d du 1 1 tan u 1 u d d 1 1 1 1 1 tan 1 1 3 and d d d d 1 tan 1 1 tan 1 4 3 1 1 1 tan 1 CDF and PDF for a Continuous RV: Relation Finally 0 0 or f F 01 1 tan 1 1 15
RS Chapter Random Variables 9/8/017 CDF and PDF for a Continuous RV: Relation Graph of f() 1.5 1 0.5 0-1 0 1 Discrete Random Variables A random variable X is called discrete if p p i1 i 1 All the probability is accounted for by values,, such that p() > 0. For a discrete random variable X the probability distribution is described by the probability function p(), which has the following properties: 1. 0 p 1 pi. p 1 3. i1 P a b p ab 16
RS Chapter Random Variables 9/8/017 Discrete Random Variables: Graph p() Pa b p ab a b Discrete Random Variables: Details Recall p() = P[X = ] = the probability function of X. This can be defined for any random variable X. For a continuous random variable p() = 0 for all values of X. Let S X ={ p() > 0}. This set is countable -i. e., it can be put into a 1-1 correspondence with the integers. S X ={ p() > 0}= { 1,, 3, 4, } Thus, we can write p p i1 i 17
RS Chapter Random Variables 9/8/017 Discrete Random Variables: Details Proof: (that the set S X ={ p() > 0} is countable -i. e., it can be put into a 1-1 correspondence with the integers.) S X = S 1 S S 3 S 3 where 1 1 Si p i 1 i That is, 1 S1 p 1 Note: ns1 1 1 S p Note: ns3 3 3 1 1 S3 p Note: ns3 4 4 3 Thus the number of elements of S n S i 1 (is finite) i i Discrete Random Variables: Details Thus the elements of S X = S 1 S S 3 S 3 can be arranged { 1,, 3, 4, } by choosing the first elements to be the elements of S 1, the net elements to be the elements of S, the net elements to be the elements of S 3, the net elements to be the elements of S 4, etc This allows us to write for p p i1 i 18
RS Chapter Random Variables 9/8/017 Discrete & Continuous Random Variables A Probability distribution is similar to a distribution of mass. A Discrete distribution is similar to a point distribution of mass. => Positive amounts of mass are put at discrete points. p( 1 ) p( ) p( 3 ) p( 4 ) 1 3 4 Discrete & Continuous Random Variables A Continuous distribution is similar to a continuous distribution of mass. The total mass of 1 is spread over a continuum. The mass assigned to any point is zero but has a non-zero density f() 19
RS Chapter Random Variables 9/8/017 Distribution function F(): Properties This is defined for any random variable, X: F() = P[X ] Properties 1. F(- ) = 0 and F( ) = 1. Since {X - }= and {X } =S => F(- ) = 0 and F( ) = 1.. F() is non-decreasing (i. e., if 1 < then F( 1 ) F( ) ) If 1 < then {X } = {X 1 } { 1 < X } Thus P[X ] = P[X 1 ] + P[ 1 < X ] or F( ) = F( 1 ) + P[ 1 < X ] Since P[ 1 < X ] 0 then F( ) F( 1 ). Distribution function F(): Properties 3. F(b) F(a) = P[a < X b]. If a < b then using the argument above F(b) = F(a) + P[a < X b] => F(b) F(a) = P[a < X b]. 4. p() = P[X = ] =F() F(-) F lim F u Here u 5. If p() = 0 for all (i.e., X is continuous) then F() is continuous. A function F is continuous if lim lim F F u F F u u u One can show that p() = 0 implies F F F 0
RS Chapter Random Variables 9/8/017 Distribution function F(): Discrete RV F P X p u u F() is a non-decreasing step function with F F 0 and 1 p F F jump in F at. F() 1. 1 0.8 0.6 0.4 0. p() 0-1 0 1 3 4 Distribution function F(): Continuous RV F P X f u du F() is a non-decreasing continuous function with F F 0 and 1. f F F() f() slope 1 0-1 0 1 1
RS Chapter Random Variables 9/8/017 Some Important Discrete Distributions The Binomial distribution Jacob Bernoulli (1654 1705)
RS Chapter Random Variables 9/8/017 Bernouille Distribution Suppose that we have a Bernoulli trial (an eperiment) that has results: 1. Success (S). Failure (F) Suppose that p is the probability of success (S) and q = 1 p is the probability of failure (F). Then, the probability distribution with probability function q p PX p is called the Bernoulli distribution. 0 1 Now assume that the Bernoulli trial is repeated independently n times. Let X be the number of successes ocurring in the n trials. (The possible values of X are {0, 1,,, n}) The Binomial Distribution Suppose we have n = 5 the outcomes together with the values of X and the probabilities of each outcome are given in the table below: FFFFF 0 q 5 SFFSF p q 3 SSSFF 3 p 3 q FSFSS 3 p 3 q SFFFF 1 pq 4 SFFFS p q 3 SSFSF 3 p 3 q FFSSS 3 p 3 q FSFFF 1 pq 4 FSSFF p q 3 SSFFS 3 p 3 q SSSSF 4 p 4 q FFSFF 1 pq 4 FSFSF p q 3 SFSSF 3 p 3 q SSSFS 4 p 4 q FFFSF 1 pq 4 FSFFS p q 3 SFSFS 3 p 3 q SSFSS 4 p 4 q FFFFS 1 pq 4 FFSSF p q 3 SFFSS 3 p 3 q SFSSS 4 p 4 q SSFFF p q 3 FFSFS p q 3 FSSSF 3 p 3 q FSSSS 4 p 4 q SFSFF p q 3 FFFSS p q 3 FSSFS 3 p 3 q SSSSS 5 p 5 3
RS Chapter Random Variables 9/8/017 The Binomial Distribution For n = 5 the following table gives the different possible values of X,, and p() = P[X = ] 0 1 3 4 5 p() = P[X = ] q 5 5pq 4 10p 3 q 10p q 3 5p 4 q p 5 For general n, the outcome of the sequence of n Bernoulli trials is a sequence of S s and F s of length n: SSFSFFSFFF FSSSFFSFSFFS The value of X for such a sequence is k = the number of S s in the sequence. The probability of such a sequence is p k q n k ( a p for each S and a q for each F) There are n k such sequences containing eactly k S s The Binomial Distribution n k is the number of ways of selecting the k positions for the S s (the remaining n k positions are for the F s). Thus, n k nk pk PX k p q k 0,1,,3,, n1, n k These are the terms in the epansion of (p + q) n using the Binomial Theorem n n n n 0 1 n For this reason the probability function n 0 n 1 n1 n n 0 pq p q p q p q p q n n p PX p q 0,1,,, n is called the probability function for the Binomial distribution 4
RS Chapter Random Variables 9/8/017 The Binomial Distribution Summary We observe a Bernoulli trial (S,F) n times. Let X denote the number of successes in the n trials. Then, X has a binomial distribution: n n p PX p q 0,1,,, n where 1. p = the probability of success (S), and. q = 1 p = the probability of failure (F) The Binomial Distribution Eample If a firm announces profits and they are surprising, the chance of a stock price increase is 85%. Assume there are n=0 (independent) announcements. Let X denote the number of increases in the stock price following surprising announcements in the n = 0 trials. Then, X has a binomial distribution, with p = 0.85 and n = 0. Thus n n p PX p q 0,1,,, n 0.85.15 0 0,1,,, 0 5
RS Chapter Random Variables 9/8/017 0 1 3 4 5 p ( ) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 6 7 8 9 10 11 p ( ) 0.0000 0.0000 0.0000 0.0000 0.000 0.0011 1 13 14 15 16 17 p ( ) 0.0046 0.0160 0.0454 0.108 0.181 0.48 18 19 0 p ( ) 0.93 0.1368 0.0388 0.3000 p() 0.500 0.000 0.1500 0.1000 0.0500-0 1 3 4 5 6 7 8 9 10 11 1 13 14 15 16 17 18 19 0 The Poisson distribution Siméon Denis Poisson (1781-1840) 6
RS Chapter Random Variables 9/8/017 The Poisson distribution Suppose events are occurring randomly and uniformly in time. The events occur with a known average. Let X be the number of events occurring (arrivals) in a fied period of time (time-interval of given length). Typical eample: X = number of crime cases coming before a criminal court per year (original Poisson s application in 1838.) Then, X will have a Poisson distribution with parameter. 0,1,,3, 4, p e! The parameter λ represents the epected number of occurrences in a fied period of time. The parameter λ is a positive real number. The Poisson distribution Eample: On average, a trade occurs every 15 seconds. Suppose trades are independent. We are interested in the probability of observing 10 trades in a minute (X=10). A Poisson distribution can be used with λ=4 (4 trades per minute). Poisson probability function 7
RS Chapter Random Variables 9/8/017 Properties: 1. p e 1! Thus 0 0 0 0 3 4 e e 1!! 3! 4! 3 4 e e 1 e!! 3! 4! e 1 3 4 using e u 1 u u u u! 3! 4! n n. If pbin p, n p 1 p is the probability function for the Binomial distribution with parameters n and p. Let n and p 0 such that np = a constant (=λ, say) then lim pbin p, n ppoisson e n, p0! Proof: p p, n p 1 p Bin Suppose np or p n n n n n! pbin p, n pbin, n 1! n! n n n n! 1 1! n n! n n nn n 1 1 1 1! n nn n n 1 1 11 1 1 1! n n n n n n 8
RS Chapter Random Variables 9/8/017 Now lim p Bin, n n 1 1 lim 1 1 1 1! n n n n n n lim 1! n n n u u Using the classic limit lim 1 e n n lim pbin, n lim 1 e p n! n n! n Poisson Note: In many applications, when n is large and p is very small --and the epectation np is not big. Then, the binomial distribution may be approimated by the easier Poisson distribution. This is called the law of rare events, since each of the n individual Bernoulli events rarely occurs. n The Poisson distribution: Graphical Illustration Suppose a time interval is divided into n equal parts and that one event may or may not occur in each subinterval. n subintervals - Event occurs - Event does not occur time interval X = # of events is Bin(n,p) As n, events can occur over the continuous time interval. X = # of events is Poisson() 9
RS Chapter Random Variables 9/8/017 The Poisson distribution: Comments The Poisson distribution arises in connection with Poisson processes - a stochastic process in which events occur continuously and independently of one another. It occurs most easily for time-events; such as the number of calls passing through a call center per minute, or the number of visitors passing through a turnstile per hour. However, it can apply to any process in which the mean can be shown to be constant. It is used in finance (number of jumps in an asset price in a given interval); market microstructure (number of trades per unit of time in a stock market); sports economics (number of goals in sports involving two competing teams); insurance (number of a given disaster -volcano eruptions/hurricanes/floods- per year); etc. Poisson Distribution - Eample: Hurricanes The number of Hurricanes over a period of a year in the Caribbean is known to have a Poisson distribution with = 13.1 Determine the probability function of X. Compute the probability that X is at most 8. Compute the probability that X is at least 10. Given that at least 10 hurricanes occur, what is the probability that X is at most 15? Solution: p e 0,1,,3,4,! 13.1 13.1 e 0,1,,3, 4,! 30
RS Chapter Random Variables 9/8/017 Poisson Distribution - Eample: Hurricanes Table of p() p ( ) p ( ) 0 0.00000 10 0.083887 1 0.00007 11 0.099901 0.000175 1 0.109059 3 0.000766 13 0.109898 4 0.00510 14 0.10833 5 0.006575 15 0.089807 6 0.014356 16 0.073530 7 0.06866 17 0.056661 8 0.043994 18 0.04137 9 0.064036 19 0.0843 Poisson Distribution - Eample: Hurricanes at most 8 8 p p p P P X 0 1 8.0957 at least 10 10 1 9 P P X P X 1 p 0 p 1 p 9.8400 P at most 15 at least 10 P X 15 X 10 P X 15 X 10 P 10 X 15 P X 10 PX 10 p10 p11 p15 0.708.8400 31