Random Variables LECTURE DEFINITION

Transcription

1 LECTURE 3 Random Variables 3. DEFINITION It is often convenient to represent the outcome of a random eperiment by a number. A random variable(r.v.) is such a representation. To be more precise, let (Ω,F, P) be a probability space. Thenarandomvariable X : Ω Risamappingoftheoutcome. Ω ω X(ω) Figure 3.. Random variable as a mapping. Eample3.. Let the random variable X be the number of heads in n coin flips. The samplespaceisω = {H,T} n,thepossibleoutcomesofncoinflips;then X {0,,2,...,n} Eample3.2. Consider packet arrival times t,t 2,... in the interval (0,T]. The sample space Ω consistsof theempty string (nopacket) and all finitelengthstrings of theform (t,t 2,...,t n ) such that 0 < t t 2 t n T. Define the random variable X to be thelengthofthestring;thenx {0,,2,3,...}. Eample 3.3. Consider the voltage across a capacitor. The sample space Ω = R. Define the random variables X(ω) = ω, Y(ω) = +, ω 0,, otherwise.

2 2 Random Variables Eample3.4. Let (Ω,F, P) be a probability space. For a given event A F, define the indicator random variable, ω A, X(ω) = 0, otherwise. Weusethenotation A (ω)or χ A (ω)todenotetheindicatorrandomvariable fora. Throughout thecourse, we use uppercase letters, say, X, Y, Z, Φ, Θ, todenote random variables, and lowercase letters to denote the values taken by the random variables. Thus, X(ω) = meansthattherandomvariable Xtakesonthevaluewhentheoutcome isω. As a representation of a random eperiment in the probability space (Ω,F, P), the randomvariable X canbeviewedasanoutcomeofarandomeperimentonitsown. The samplespaceisrandthesetofeventsistheborelσ-algebrab. AneventA B occursif X Aanditsprobability is P({ω Ω: X(ω) A}), which is determined by the probability measure P of the underlying random eperiment and the inverse image of A under the mapping X : Ω R. Thus, (Ω,F, P) induces a probability space(r,b, P X ),where P X (A) = P({ω Ω: X(ω) A}), A B. AnimplicitassumptionhereisthatforeveryA B,theinverseimage{ω Ω: X(ω) A} is an event in F. A mapping X(ω) satisfying this condition is called measurable (with respecttof)andwewillalwaysassumethatagivenmappingismeasurable. Since we typically deal with multiple random variables on the same probability space, we will use thenotation P{X A} instead of themore formal notation P X (A) or P({ω Ω: X(ω) A}). R AsetA Theinverseimageof Aunder X(ω),i.e.,{ω : X(ω) A}

3 3.2 Cumulative Distribution Function CUMULATIVE DISTRIBUTION FUNCTION To determine P{X A} for any Borel set A, i.e., any set generated by open intervals via countable unions, intersections, and complements, it suffices to specify P{X (a, b)} or P{X (a,b]}forall < a < b <. Thentheprobability ofanyotherborelsetcanbe determined by the aioms of probability. Equivalently, it suffices to specify the cumulative distribution function(cdf) of the random variable X: F X () = P{X } = P{X (,]}, R. The cdf of a random variable satisfies the following properties.. F X ()isnonnegative,i.e., F X () 0, R. 2. F X ()ismonotonically nondecreasing,i.e., 3. Limits. F X (a) F X (b), a < b. lim F X() = 0 and lim F + X() =. 4. F X ()isrightcontinuous,i.e., 5. Probability of a singleton. wheref X (a ) := lim a F X (). F X (a + ) := lim a + F X() = F X (a). P{X = a} = F X (a) F X (a ), Throughout,weusethenotation X F()meansthattherandomvariable X hasthe cdff(). F X () Figure 3.2. An illustration of a cumulative distribution function (cdf).

4 4 Random Variables 3.3 PROBABILITY MASS FUNCTION (PMF) Arandomvariable X issaidtobediscreteiff X ()consistsonlyofstepsoveracountable setx asillustrated infigure3.3. Figure 3.3. The cdf of a discrete random variable. A discrete random variable X can be completely specified by its probability mass function(pmf) p X () = P{X = }, X. The set X is often referred to as the alphabet of X. Clearly, p X () 0, X p X () =, and P(X A) = p X (). A X Throughout, weuse thenotation X p() tomean that X is adiscreterandom variable X withpmf p(). We review a few famous discrete random variables. Bernoulli. X Bern(p), p [0,],hasthepmf p X () = p and p X (0) = p. Thisistheindicatorofobservingaheadfromflippingacoinwithbias p. Geometric. X Geom(p), p [0,],hasthepmf p X (k) = p( p) k, k =,2,3,... Thisisthenumberofindependentcoinflipsofbias puntilthefirsthead. Binomial. X Binom(n, p), p [0,],n =,2,..., hasthepmf p X (k) = n k pk ( p) n k, k = 0,,...,n. Thisisthenumberofheadsinnindependentcoinflipsofbias p. Poisson. X Poisson(λ), λ > 0,hasthepmf p X (k) = λk k! e λ, k = 0,,2,...

5 3.4 Probability Density Function 5 [h] Figure 3.4. The cdf of a continuous random variable. This is often used to characterize thenumber of random arrivals in aunit time interval, thenumberofrandompointsinaunitarea,andsoon. Let X Binom(n,λ/n). Then,itspmfforafied k is p X (k) = n k λ n k λ n n k = n(n ) (n k+) n k λ k k! λ n n λ n k, which converges to the Poisson(λ) pmf (λ k /k!)e λ as n. Thus, Poisson(λ) is the limit of Binom(n, λ/n). Eample 3.5. In a popular lottery called Powerball, the winning combination of numbers is selected uniformly at random among 292,20,338 possible combinations. Suppose thatαnticketsaresold. Whatistheprobability thatthereisnowinner? SincethenumberN ofwinnersisabinom(αn,/n)randomvariablewithn = verylarge, wecanusethepoissionapproimation toobtain P{N = 0} = n αn e α, asn. Ifα =,thereisnowinnerwithprobabilityof37%. Ifα = 2,thisprobabilitydecreasesto 4%. Thus, even with 600 million tickets sold, there is a significant chance that the lottery hasnowinnerandrollsovertothenetweek(withabiggerjackpot). 3.4 PROBABILITY DENSITY FUNCTION A random variable is said to be continuous if its cdf is continuous as illustrated in Figure 3.4. If F X () is continuous and differentiable (ecept possibly over a countable set), then X canbecompletelyspecifiedbyitsprobabilitydensityfunction(pdf) f X ()suchthat F X () = f X (u)du.

6 6 Random Variables IfF X ()isdifferentiable everywhere,thenbythedefinitionofderivative f X () = df X() d F( +Δ) F() = lim Δ 0 Δ P{ < X +Δ} = lim. Δ 0 Δ The pdf of a random variable satisfies the following properties.. f X ()isnonnegative,i.e., 2. Normalization. f X()d =. f X () 0, R. 3. Foranyevent A R, In particular, P{X A} = f X ()d. A b P{a < X b} = P{a < X < b} = P{a X < b} = P{a X b} = f X ()d. a Notethat f X ()shouldnot beinterpretedastheprobabilitythat X =. Infact, f X () can be greater than. It is f X ()Δ that can be interpreted as the approimation of the probability P{ < X +Δ}forΔ sufficientlysmall. Throughout, we use the notation X f() to mean that X is a continuous random variable withpdf f(). We review a few famous continuous random variables. Uniform. X Unif[a,b],a < b,hasthepdf [a,b], f X () = b a 0 otherwise. This is often used to model quantization noise. Eponential. X Ep(λ), λ > 0,hasthepdf f X () = λe λ 0, 0 otherwise. Thisisoften usedtomodeltheservicetime inaqueue orthetime betweentworandom arrivals. An eponential random variable satisfies the memoryless property P(X > + t X > t) = P{X > +t} P{X > t} = P{X > }, t, > 0.

7 3.5 Functions of a Random Variable 7 Eample3.6. Suppose that for every t > 0, the number of packet arrivals during time interval(0, t] is a Poisson(λt) random variable, i.e., p N (n) = (λt)n e λt, n = 0,,2,... n! Let X bethetimeuntilthefirstpacketarrival. Thentheevent{X > t}isequivalenttothe event{n = 0}andthus Hence, f X (t) = λe λt and X Ep(λ). Gaussian. X N(μ,σ 2 )hasthepdf F X (t) = P{X > t) = P{N = 0} = e λt. f X () = e ( μ) 2σ 2 2πσ 2 2 This characterizes many random phenomena such as thermal and shot noise, and is also called a normal random variable. The cdf of the standard normal random variable N(0, ) is Its complement is Φ() = u 2 2π e 2 du. Q() = Φ() = P{X > }. The numerical values of the Q function is often used to compute probabilities of any GaussianrandomvariableY N(μ,σ 2 )as P{Y > y} = P X > y μ σ = Q y μ. (3.) σ 3.5 FUNCTIONS OF A RANDOM VARIABLE Let X be a random variable and д : R R be a given function. Then Y = д(x) is a random variable and its probability distribution can be epressed through that of X. For eample,if X isdiscrete,theny isdiscreteand p Y (y) = P{Y = y} = P{д(X) = y} = p X (). : д()=y

8 8 Random Variables f X Q() Figure3.5. ThepdfofthestandardnormalrandomvariableandtheQfunction. In general, F Y (y) = P{Y y} which can be further simplified in many cases. = P{д(X) y}, Eample3.7(Linearfunction). Let X F X ()andy = ax +b,a 0. Ifa > 0,then F Y (y) = P{aX +b y} = P X y b a = F X y b a. Taking derivative with respect to y, we have Wecansimilarly showthatifa < 0,then and f Y (y) = a f X y b a F Y (y) = F X y b a f Y (y) = a f X y b a. y y y b a Figure 3.6. A linear function.

9 3.5 Functions of a Random Variable 9 Combining both cases, f Y (y) = a f X y b a Asaspecialcase,let X N(μ,σ 2 ),i.e., f X () = e ( μ) 2πσ 2 2 2σ 2. AgainsettingY = ax +b,wehave f Y (y) = a f X y b a = μ 2 a 2πσ 2 e 2σ 2 e (y b aμ) = 2π(aσ) 2 2 2a 2 σ 2. Therefore,Y N(aμ+b, a 2 σ 2 ). This result justifies theuse of theqfunctionin (3.) to compute probabilities for an arbitrary Gaussian random variable. Eample3.8(Quadraticfunction). LetX F X ()andy = X 2.Ify < 0,thenF Y (y) = 0. Otherwise, F Y (y) = P y X y = F X y F X ( y) If X iscontinuouswithpdf f X (),then f Y (y) = 2 y f X( y)+ f X ( y). y y y Figure 3.7. A quadratic function.

10 0 Random Variables The above two eamples can be generalized as follows. Proposition3.. Let X f X ()andy = д(x)bedifferentiable. Then f f Y (y) = X ( i ) i= д ( i ), where, 2,...are thesolutions of theequation y = д() and д ( i )is thederivative of д evaluated at i. The distribution of Y can be written eplicitly even when д is not differentiable. Eample3.9(Limiter). Let X be a r.v. with Laplacian pdf f X () = 2 e, and let Y be definedbythefunctionof X showninfigure3.8. Considerthefollowingcases. If y < a,clearlyf Y (y) = 0. If y = a, If a < y < a, F Y ( a) = F X ( ) = 2 e d = 2 e. F Y (y) = P{Y y} = P ax y = P X y a = F X y a = y/a 2 e + 2 e d. y +a a + Figure 3.8. The limiter function.

11 3.6 Generation of Random Variables If y a,f Y (y) =. Combiningthesecases,thecdfofY issketchedinfigure3.9. F Y (y) a a y Figure3.9.ThecdfoftherandomvariableY. 3.6 GENERATION OF RANDOM VARIABLES Suppose that we are given a uniform random variable X Unif[0,] and wish to generate a random variable Y with prescribed cdf F(y). If F(y) is continuous and strictly = F(y) y = F () Figure3.0. GenerationofY F(y)fromauniformrandomvariable X. increasing, set Y = F (X). Then,since X Unif[0,]and0 F(y), F Y (y) = P{Y y} = P{F (X) y} = P{X F(y)} (3.2) = F(y). Thus,YhasthedesiredcdfF(y).Foreample,togenerateY Ep(λ)fromX Unif[0,], weset Y = ln( X). λ

12 2 Random Variables More generally, for an arbitrary cdf F(y), we define F () := min{y: F(y)}, (0,]. (3.3) Since F(y) is right continuous, the above minimum is well-defined. Furthermore, since F(y)ismonotonicallynondecreasing,F () yiff F(y). WenowsetY = F (X)as before, but under this new definition of inverse. It follows immediately that the equality in (3.2) continues to hold and thaty F(y). For eample, to generatey Bern(p), we set 0 X p, Y = otherwise. In conclusion, we can generate a random variable with any desired distribution from a Unif[0, ] random variable. = F(y) p 0 y Figure 3.. Generation of a Bern(p) random variable. Conversely, a uniform random variable can be generated from any continuos random variable. Let X be a continuous random variable with cdf F() and Y = F(X). Since F() [0,],F Y (y) = P{Y y} = 0for y < 0andF Y (y) = for y >. For y [0,],let F (y)bedefinedasin(3.3). Then F Y (y) = P{Y y} = P{F(X) y} = P{X F (y)} = F(F (y)) (3.4) = y, where the equality in (3.4) follows by the definition of F (y). Hence, Y U[0,]. For eample, let X Ep(λ)and ThenY Unif[0,]. ep( λx) X 0. Y = 0 otherwise.

13 Problems 3 The eact generation of a uniform random variable, which requires an infinite number of bits to describe, is not possible in any digital computer. One can instead use the following approimation. Let X,X 2,...X n be independent and identically distributed (i.i.d.) Bern(/2) random variables, and Y =.X X 2...X n be a fraction in base 2 that lies between 0 and. Then Y is a discrete random variable uniformlydistributedovertheset{k/2 n : k = 0,,...,2 n }anditscdff(y)converges tothatofaunif[0,]randomvariableforevery yasn. Thus,byflippingmanyfair coin flips, one can simulate a uniform random variable. The fairness of coin flips is not essential to this procedure. Suppose that Z and Z 2 are i.i.d. Bern(p) random variable. The following procedure due to von Neumann can generate a single Bern(/2) random variable, even when the bias p is unknown. Let X = 0 (Z,Z 2 ) = (0,), (Z,Z 2 ) = (,0). If (Z,Z 2 ) = (0,0) or (,), then the outcome is ignored. Clearly p X (0) = p X () = /2. By repeating the same procedure, one can generate a sequence of i.i.d. Bern(/2) random variables from a sequence of i.i.d. Bern(p) random variables. PROBLEMS 3.. Probabilitiesfromacdf. Let X bearandomvariable withthecdfshownbelow. F() 2/3 / Find the probabilities of the following events. (a) {X = 2}. (b){x < 2}. (c) {X = 2} {0.5 X.5}. (d){x = 2} {0.5 X 3} Gaussian probabilities. Let X N(000, 400). Epress the following in terms of the Q function.

14 4 Random Variables (a) P{0 < X < 020}. (b) P{X < 020 X > 960} Laplacian. Let X f() = 2 e. (a) Sketchthecdfof X. (b)find P{ X 2or X 0}. (c) Find P{ X + X 3 3}. (d)find P{X 0 X } Distancetotheneareststar. Lettherandomvariable N bethenumberofstarsina regionofspaceofvolumev. AssumethatN isapoissonr.v.withpmf p N (n) = e ρv (ρv) n, forn = 0,,2,..., n! where ρ is the density of stars in space. We choose an arbitrary point in space anddefinetherandomvariable X tobethedistancefromthechosenpointtothe neareststar. Findthepdfof X (intermsof ρ) Uniform arrival. The arrival time of a professor to his office is uniformly distributed in the interval between 8 and 9 am. Find the probability that the professor will arrive during the net minute given that he has not arrived by 8:30. Repeat for 8: Lognormal distribution. Let X N(0,σ 2 ). FindthepdfofY = e X (known asthe lognormal pdf) Randomphase signal. LetY(t) = sin(ωt +Θ) beasinusoidal signal withrandom phaseθ U[ π,π]. FindthepdfoftherandomvariableY(t)(assumeherethat both t andtheradial frequencyω are constant). Comment onthedependenceof thepdfofy(t)ontime t Quantizer. Let X ep(λ), i.e., an eponential random variable with parameter λ andy = X, i.e., Y = k for k X < k +, k = 0,,2,... Find the pmf of Y. Definethequantization error Z = X Y. Findthepdfof Z Gambling. Aliceentersacasinowithoneunitofcapital. Shelooksatherwatchto generateauniformrandomvariableu unif[0,],thenbetstheamountu ona faircoinflip. Herwealthisthusgivenbyther.v. Findthecdfof X. X = +U, withprobability /2, U, withprobability / Nonlinear processing. Let X Unif[, ]. Define the random variable Y = X2 +, if X 0.5 0, otherwise.

15 FindandsketchthecdfofY. Problems 5