CSE 312, 2017 Winter, W.L. Ruzzo. 7. continuous random variables

Transcription

1 CSE 312, 2017 Winter, W.L. Ruzzo 7. continuous random variables

2 The new bit continuous random variables Discrete random variable: values in a finite or countable set, e.g. X {1,2,..., 6} with equal probability X is positive integer i with probability 2 -i probability mass function assigns probabilities to points Continuous random variable: values in an uncountable set, e.g. X is the weight of a random person (a real number) X is a randomly selected angle [0.. 2π) X is the waiting time until the next packet arrives can t put non-zero probability at points; probability density function assigns how probability mass is distributed near points; probability per unit length 2

3 pdf f(x): R R, the probability density function (or simply density ) f(x) Require: f(x) 0, and I.e., distribution is: nonnegative, and + f(x) dx = 1 normalized, - just like discrete PMF. Distributes probability across the real line 3

4 F(x): the cumulative distribution function (aka the distribution ) cdf f(x) a b a F(a) = P(X a) = f(x) dx (Area left of a) P(a < X b) = F(b) - F(a) (Area between a and b) A key relationship: f(x) = d dx a F(x), since F(a) = f(x) dx, 4

5 Densities are not probabilities; e.g. may be > 1 P(X = a) = lim ε 0 P(a-ε/2 < X a+ε/2) = F(a)-F(a) = 0 I.e., But the probability that a continuous r.v. falls at a specified point is zero. the probability that it falls near that point is proportional to the density: P(a - ε/2 < X a + ε/2) = F(a + ε/2) - F(a - ε/2) I.e., ε f(a) f(a) probability per unit length near a. in a large random sample, expect more samples where density is higher (hence the name density ). f(a) vs f(b) give relative probabilities near a vs b. a-ε/2 a a+ε/2 densities f(x) 5

6 Much of what we did with discrete r.v.s carries over almost unchanged, with (a) p.m.f. replaced by density, and E.g. (b) Σx replaced by... dx For discrete r.v. X, For continuous r.v. X, sums and integrals; expectation E[X] = Σ x x p(x) Why? (a) We define it that way (b) The probability that X falls near x, say within x±dx/2, is f(x)dx, so the average X should be Σ xf(x)dx (summed over grid points spaced dx apart on the real line) and the limit of that as dx 0 is xf(x)dx 6

7 Let 1 example f(x) F(x)

8 properties of expectation Linearity E[aX+b] = ae[x]+b E[X+Y] = E[X]+E[Y] still true, just as for discrete Functions of a random variable E[g(X)] = g(x)f(x)dx just as for discrete, but w/integral Alternatively, let Y = g(x), find the density of Y, say f Y, (see B&T 4.1; somewhat like r.v. slides 33-35) and directly compute E[Y] = yf Y (y)dy. 8

9 variance Definition is same as in the discrete case Var[X] = E[(X-μ) 2 ] where μ = E[X] Identity still holds: Var[X] = E[X 2 ] - (E[X]) 2 proof same 9

10 Let 1 example f(x) F(x)

11 continuous random variables: summary Continuous random variable X has density f(x), and

12 uniform random variables X ~ Uni(α,β) is uniform in [α,β] The Uniform Density Function Uni(0.5,1.0) f(x) (α) x 1.0 (β)

13 uniform random variables X ~ Uni(α,β) is uniform in [α,β] f(x) The Uniform Density Function Uni(0.5,1.0) x if α a b β: Yes, you should review your basic calculus; e.g., these 2 integrals would be good practice.

14 X ~ Uni(α,β) is uniform in [α,β] uniform random variable: example You want to read a disk sector from a 7200rpm disk drive. Let T be the time you wait, in milliseconds, after the disk head is positioned over the correct track, until the desired sector rotates under the head. T ~ Uni(0, 8.33) Average Wait? 4.17ms 14

15 Radioactive decay: How long until the next alpha particle? waiting for events Customers: how long until the next customer/packet arrives at the checkout stand/server? Buses: How long until the next #71 bus arrives on the Ave? Yes, they have a schedule, but given the vagaries of traffic, riders with-bikes-and-babycarriages, etc., can they stick to it? Assuming events are independent, happening at some fixed average rate of λ per unit time the waiting time until the next event is exponentially distributed (next slide) 15

16 exponential random variables X ~ Exp(λ) The Exponential Density Function f(x) λ = 2 λ = x

17 exponential random variables X ~ Exp(λ) Memorylessness: = 1-F(t) Assuming exp distr, if you ve waited s minutes, prob of waiting t more is exactly same as s = 0

18 Gambler s fallacy: I m due for a win Relation to the Poisson: same process, different measures: Poisson: how many events in a fixed time; Exponential: how long until the next event Relation to geometric: Geometric is discrete analog: How long to a Head, 1 flip per sec, prob p vs How long to a Head, 2 flips per sec, prob p/2, vs How long to a Head, 3 flips per sec, prob p/3, vs Limit is exponential with parameter 1/p examples λ is avg # per unit time; 1/λ is mean wait All have the same } mean: 1/p secs see also B&T fig 3.8, p152 18

19 Cumulative Probability Exp(λ): waiting time until next event, if events happen at average rate 1/λ 1/k geometric is discrete analog of exponential Exponential CDF 1-exp(-λt), λ=p Geometric CDF How long to a Head, if: Flips/sec p(h) E(flips) E(secs) 1 p 1/p 1/p 2 p/2 2/p 1/p cf also B&T fig 3.8, pg 152 Geom(p): number of flips to first head when p(head) = p k p/k k/p 1/p Geometric CDF 1-(1-p/k) n, n = # flips Rescaling to CDF as a function of time (n = kt): 1-(1-p/k) kt and lim k 1-(1-p/k) kt = 1-exp(-pt) I.e., limit is exponential with parameter λ = p Graphs: λ =.95, k = 5 t 19

20 normal random variables X is a normal (aka Gaussian) random variable X ~ N(μ, σ 2 ) The Standard Normal Density Function f(x) µ = 0 σ = x μ±σ

21 changing μ, σ µ = 0 σ = µ = 0 σ = μ±σ µ = 4 σ = μ±σ µ = 4 σ = μ±σ μ±σ density at μ is.399/σ 21

22 X is a normal random variable X ~ N(μ,σ 2 ) normal random variables Y = ax + b E[Y] = E[aX+b] = aμ + b Var[Y] = Var[aX+b] = a 2 σ 2 Y ~ N(aμ + b, a 2 σ 2 ) Important special case: Z = (X-μ)/σ ~ N(0,1) E[ ], Var[ ] as expected; normality is the surprise Z ~ N(0,1) standard (or unit) normal Use Φ(z) to denote CDF, i.e. no closed form L

23 ble 1: Table of the Standard Normal Cumulative Distribution Fun Table of the Standard Normal Cumulative Distribution Function Φ(z) -0.0z Φ(.46) The Standard Normal Density Function NB: by symmetry µ = Φ(-z)=1-Φ(z) σ = μ±σ x E.g., see B&T p155, p f(x)

24 The Standard Normal Density Function f(x) µ = 0 σ = 1 If Z ~N(μ,σ 2 ) what is P( μ-σ < Z < μ+σ )? P( μ - σ < Z < μ + σ ) = Φ(1) - Φ(-1) 68% P( μ - 2σ < Z < μ + 2σ ) = Φ(2) - Φ(-2) 95% P( μ - 3σ < Z < μ + 3σ ) = Φ(3) - Φ(-3) 99% Why? N(μ,σ 2 ) N(0,1) x μ - kσ < Z < μ + kσ -k < (Z-μ)/σ < +k 24 μ±σ

25 the central limit theorem (CLT) Consider i.i.d. (independent, identically distributed) random vars X 1, X 2, X 3, X i has μ = E[X i ] < and σ 2 = Var[X i ] < As n, Restated: As n, M n = 1 n nx i=1 X i! N µ, 2 n More of the theory behind this later, but first, some examples:

26 How tall are you? Why? Credit: Annie Leibovitz, 1987? Willie Shoemaker & Wilt Chamberlain 26

27 in the real world Human height is approximately normal. Why might that be true? Frequency R.A. Fisher (1918) noted it would follow from CLT if height Male Height in Inches were the sum of many independent random effects, e.g. many genetic factors (plus some environmental ones like diet). I.e., suggested part of mechanism by looking at shape of the curve. (WAY before anyone really knew what genes, DNA, etc. were...) 27

28 Meta-analysis of Dense Genecentric Association Studies Reveals Common 194 and Uncommon Variants Associated with Height, Lanktree, et ^al. The American Journal of Human Genetics 88, 6 18, January 7, 2011 Sixty-Four (and hundreds more probably exist) 28

29 Big Dog, Little Dog A Single IGF1 Allele Is a Major Determinant of Small Size in Dogs Nathan B. Sutter, et al. Science 316, 112 (2007); 29

30 in the real world 30

31 in the real world 31

32 in the real world 32

33 in the real world 33

34 pdf vs cdf continuous r.v. s: summary sums become integrals, e.g. E[X] = Σx x p(x) a f(x) = d F(x) F(a) = f(x) dx dx most familiar properties still hold, e.g. E[aX+bY+c] = ae[x]+be[y]+c Var[X] = E[X 2 ] - (E[X]) 2 34

35 Three important examples X ~ Uni(α,β) uniform in [α,β] continuous r.v. s: summary E[X] = (α+β)/2 Var[X] = (α-β) 2 /12 f(x) x X ~ Exp(λ) exponential f(x) λ = 2 λ = x X ~ N(μ, σ 2 ) normal (aka Gaussian, aka the big Kahuna) f(x) µ = 0 σ = x μ±σ 35

36 Joint, marginal and conditional distributions continuous r.v. s: more stuff Distribution of Z = g(x,y) based on distributions of X, Y Independence Joint = product of marginals, and/or Conditional = unconditional Chain rule Total probability rule Conditional expectation Law of total expectation See Text, Chapter 3! 36