Continuous Variables Chris Piech CS109, Stanford University

Size: px

Start display at page:

Download "Continuous Variables Chris Piech CS109, Stanford University"

Augustus Stevens
5 years ago
Views:

1 Continuous Variables Chris Piech CS109, Stanford University

2 1906 Earthquak Magnitude 7.8

3 Learning Goals 1. Comfort using new discrete random variables 2. Integrate a density function (PDF) to get a probability 3. Use a cumulative function (CDF) to get a probability

4 Discrete Distributions Don t have to derive all of the following distributions. We want you to get a sense of how random variables work.

5 Grid of Random Variables One trial number of successes X Ber( p) time to get successes X Geo( p) One success n = 1 r = 1 Several trials X Bin(n, p) X NegBin(r, p) Several successes Interval of time X Poi(λ ) X Ep(λ ) One success

probability of success on each trial X takes on values 1, 2, 3,,

6 Geometric Random Variable X is Geometric Random Variable: X ~ Geo(p) X is number of independent trials until first success p is probability of success on each trial X takes on values 1, 2, 3,, with probability: n-1 P( X = n) = (1 - p) p E[X] = 1/p Var(X) = (1 p)/p 2

trial X takes on values r, r + 1, r + 2, with probability: æn -1ö r n-r P( X = n) = ç p

7 Negative Binomial Random Variable X is Negative Binomial RV: X ~ NegBin(r, p) X is number of independent trials until r successes p is probability of success on each trial X takes on values r, r + 1, r + 2, with probability: æn -1ö r n-r P( X = n) = ç p (1 - p), where n = r, r + 1,... è r -1ø E[X] = r/p Var(X) = r(1 p)/p 2 Note: Geo(p) ~ NegBin(1, p)

8 Bernoulli: indicator of coin flip X ~ Ber(p) Binomial: # successes in n coin flips X ~ Bin(n, p) Poisson: # successes in n coin flips X ~ Poi(l) Geometric: # coin flips until success X ~ Geo(p) Negative Binomial: Zipf: Discrete Distributions # trials until r successes X ~ NegBin(r, p) The popularity rank of a random word, from a natural language X ~ Zipf(s)

9 Bit Coin Mining Data Salt SHA-256 Hash(, ) Fied Choice Number that looks like random bits You mine a bitcoin if, for given data D, you find a number N such that Hash(D, N) produces a string that starts with g zeroes.

10 Midterm Question: Bit Coin Mining You mine a bitcoin if, for given data D, you find a number N such that Hash(D, N) produces a string that starts with g zeroes. (a) What is the probability that the first number you try will produce a bit string which starts with g zeroes (in other words you mine a bitcoin)? (b) How many different numbers do you epect to have to try before you mine five bitcoins?

11 Dating at Stanford Each person you date has a 0.2 probability of being someone you spend your life with. What is the average number of people one will date? What is the standard deviation?

12 Equity in the Courts Berghuis v. Smith If a group is underrepresented in a jury pool, how do you tell? Article by Erin Miller January 22, 2010 Thanks to (former CS109er) Josh Falk for this article Justice Breyer [Stanford Alum] opened the questioning by invoking the binomial theorem. He hypothesized a scenario involving an urn with a thousand balls, and sity are blue, and nine hundred forty are purple, and then you select them at random twelve at a time. According to Justice Breyer and the binomial theorem, if the purple balls were under represented jurors then you would epect something like a third to a half of juries would have at least one minority person on them.

13 Justin Breyer Meets CS109 Approimation using Binomial distribution Assume P(blue ball) constant for every draw = 60/1000 X = # blue balls drawn. X ~ Bin(12, 60/1000 = 0.06) P(X 1) = 1 P(X = 0)» = In Breyer s description, should actually epect just over half of juries to have at least one non-white person on them P(X = ) Demo # Minority Jurrors

14 Four Prototypical Trajectories Big hole in our knowledge

15 Not all values are discrete

16 Four Prototypical Trajectories random()?

17 Riding the Marguerite

18 Riding the Marguerite You are running to the bus stop. You don t know eactly when the bus arrives. You have a distribution of probabilities. You show up at 2:20pm. What is P(wait < 5 minutes)? What is the probability that the bus arrives at: 2:17pm and seconds?

19 Riding the Marguerite You are running to the bus stop. You don t know eactly when the bus arrives. You have a distribution of probabilities. You show up at 2:15pm. What is P(wait < 5 minutes)? wait <5 min P(T = t) 2:00pm Bus arrives (t) mins after 2:00pm

20 Riding the Marguerite You are running to the bus stop. You don t know eactly when the bus arrives. You have a distribution of probabilities. You show up at 2:15pm. What is P(wait < 5 minutes)? P(T = t) wait <5 min P(15 < T 20) image: Haha169 2:00pm Bus arrives (t) mins after 2:00pm

21 Riding the Marguerite You are running to the bus stop. You don t know eactly when the bus arrives. You have a distribution of probabilities. You show up at 2:15pm. What is P(wait < 5 minutes)? P(T = t) wait <5 min P(15 < X 17.5) + P(17.5 < X 20) image: Haha169 2:00pm Bus arrives (t) mins after 2:00pm

22 Riding the Marguerite You are running to the bus stop. You don t know eactly when the bus arrives. You have a distribution of probabilities. You show up at 2:15pm. What is P(wait < 5 minutes)? Probability Density Function f T (t) wait <5 min P(15 < T 20) image: Haha169 2:00pm Bus arrives (t) mins after 2:00pm

23 Probability Density Function The probability density function (PDF) of a continuous random variable represents the relative likelihood of various values. Units of probability divided by units of X. Integrate it to get probabilities! P (a <X<b)= Z b f X () d =a

24 Integrals *loving, not scary

25 Properties of PDFs 0 apple Z b =a f X () d apple 1 Z 1 f X () d =1 1

26 What do you get if you integrate over a probability density function? A probability!

27 Probability Density Function Probability density functions articulate relative belief. Let X be a random variable which is the # of minutes after 2pm that the bus arrives at the stop: A C B Which of these represent that you think the arrival is more likely to be close to 3:00pm 0 60

28 Piech, CS106A, Stanford University The ratio of probability densities is meaningful

29 f() is Not a Probability Rather, it has units of: probability divided by units of X. 2 min f() 0.05 prob/min 2:00pm 2:20pm 2:30pm = 0.1 prob

30 f() is Not a Probability Note: f() can be greater than 1! f() 2.0 prob/min 2:00pm 0.5 min 2:20pm 2:30pm = 1.0 prob

Simple Eample Consider a random 50005000 matri, where each element in the matri is Uniform(0,1).

31 Simple Eample Consider a random matri, where each element in the matri is Uniform(0,1). What is the probability that a selected eigenvalue (λ) of the matri is greater than 0?* * With help from Wigner, Chris is going to rephrase this problem

32 Simple Eample from Quantum Physics Let X be a continuous random variable: Theory f() = 1 p f() p() Practice From simulations P (X >0) =?

33 Simple Eample from Quantum Physics Let X be a continuous random variable: Theory f() = 1 p f() p() Practice From simulations Approach #1: Integrate over the PDF P (X >0) = Z f X () d

34 Simple Eample from Quantum Physics Let X be a continuous random variable: Theory f() = 1 p f() p() Practice From simulations Approach #2: Discrete Approimation 100 P (X >0) X P (X = i) ii=0

35 Simple Eample from Quantum Physics Let X be a continuous random variable: Theory f() = 1 p f() p() Practice From simulations Approach #3: Know Semi-Circles P (X >0) = 1 2

36 What do you get if you integrate over a probability density function? A probability!

37 Uniform Random Variable A uniform random variable is equally likely to be any value in an interval. X Uni(α,β) Probability Density Properties f ( ) = ì ï í ïî 1 b -a 0 a b otherwise E[X] = 2 f() Var(X) = ( )2 12 β

38 Uniform Bus You are running to the bus stop. You don t know eactly when the bus arrives. You believe all times between 2 and 2:30 are equally likely. You show up at 2:15pm. What is P(wait < 5 minutes)? f T (t) wait <5 min T Uni( =0, = 30) P (Wait < 5) = Z = 1 d 20 2:00pm Bus arrives (t) mins after 2:00pm = = 5 30

39 Epectation and Variance For discrete RV X: For continuous RV X: For both discrete and continuous RVs:

40 Epectation of Uniform Z E[X] = Z = Z 1 1 X Uni(, ) f() d 1 d Z i = 1 h 1 2 2i just average the start and end! = 1 2 ( + ) h i = ( + )( )

41 Eponential Random Variable X is an Eponential RV: X ~ Ep(l) Rate: l > 0 Probability Density Function (PDF): -l ìle f ( ) = í î 0 1 E[ X ] = l 1 Var( X ) = Represents time until some event o 2 l if ³ 0 where - < < if < 0 f () Earthquake, request to web server, end cell phone contract, etc.

42 1906 Earthquak Magnitude 7.8

43 How Many Earthquakes Based on historical data, major earthquakes (magnitude 8.0+) happen at a rate of per year*. What is the probability of zero major earthquakes magnitude net year? X = Number of major earthquakes net year *In California, according to the USGS, 2015

44 How Long Until Net Earthquake Based on historical data, major earthquakes (magnitude 8.0+) happen at a rate of per year*. What is the probability of an earthquake of magnitude 8+ in the net 30 years? Y = Years until the net earthquake of magnitude 8.0+ Y Ep( =0.002) f Y (y) = e P (Y < 30) = Z e 0.002y dy y = y 0 h i 30 *In California, according to the USGS, 2015

45 Integral Review Z e c d = 1 c ec

46 How Long Until Net Earthquake Based on historical data, major earthquakes (magnitude 8.0+) happen at a rate of per year*. What is the probability of an earthquake of magnitude 8+ in the net 30 years? Y = Years until the net earthquake of magnitude 8.0+ Y Ep( =0.002) f Y (y) = e P (Y < 30) = Z Z e 0.002y dy h = e 0.002yi i30 = ( e e 0 ) y = y *In California, according to the USGS, 2015

47 Four Prototypical Trajectories Is there a way to avoid integrals?

48 Cumulative Density Function A cumulative density function (CDF) is a closed form equation for the probability that a random variable is less than a given value F () =P (X <)= If you learn how to use a cumulative density function, you can avoid integrals! F X () This is also shorthand notation for the PMF

49 Cumulative Density Function F () =P (X <)= =

50 CDF of an Eponential F X () =1 e P (X <)= Z f(y) dy y= 1 Z = e y dy = y=0 h =[ e =1 e e y i 0 ] [ e 0 ]

51 CDF: X ~ Ep(l = 1) Probability density function f() = e Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

52 CDF: X ~ Ep(l = 1) Probability density function f() = e P(X < 2) Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

53 CDF: X ~ Ep(l = 1) Probability density function f() f() = e P(X < 2) Z 2 = f() d = 1 Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

54 CDF: X ~ Ep(l = 1) Probability density function f() f() = e P(X < 2) Z 2 = f() d = 1 Cumulative density function F () =1 e = F (2) or F X () =P (X <) Z = f(y) dy y= 1 =1 e

55 CDF: X ~ Ep(l = 1) Probability density function f() = e P(X > 1) Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

56 CDF: X ~ Ep(l = 1) Probability density function f() = e = P(X > 1) Z 1 f() d =1 Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

57 CDF: X ~ Ep(l = 1) Probability density function f() = e = P(X > 1) Z 1 f() d =1 Cumulative density function F () =1 e or =1 F (1) F X () =P (X <) Z = f(y) dy y= 1 = e

58 CDF: X ~ Ep(l = 1) Probability density function f() = e P(1 < X < 2) Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

59 CDF: X ~ Ep(l = 1) Probability density function f() = e P(1 < X < 2) Z 2 = f() d =1 Cumulative density function F () =1 e F X () =P (X <) Z = f(y) dy y= 1

60 CDF: X ~ Ep(l = 1) Probability density function f() f() = e P(1 < X < 2) Z 2 = f() d =1 Cumulative density function F () =1 e or = F (2) F (1) F X () =P (X <) Z = f(y) dy y= 1 =(1 e 2 ) 0.23 (1 e 1 )

61 Probability of Earthquake in Net 4 Years? Based on historical data, earthquakes of magnitude 8.0+ happen at a rate of per year*. What is the probability of an earthquake of magnitude 8+ in the net 4 years? Y = Years until the net earthquake of magnitude 8.0+ Y Ep( =0.002) F (y) =1 e 0.002y P (Y < 4) = F (4) =1 e Feeling lucky? *According to USGS, 2015

62 Properties for Continuous Random Variables Semantic Meaning f X () F() Continuous Random Variable E[X] Var(X) E[X 2 ] Std(X)

63 Four Prototypical Trajectories Etra Problems

64 Visits to a Website Say visitor to your web site leaves after X minutes On average, visitors leave site after 5 minutes Assume length of stay is Eponentially distributed X ~ Ep(l = 1/5), since E[X] = 1/l = 5 What is P(X > 10)? P( X > 10) = 1- F(10) = 1- (1 - e -l10 ) = e -2» What is P(10 < X < 20)? P(10 < X < 20) = F(20) - F(10) = (1 - e -4 ) - (1 - e -2 )»

65 Replacing Your Laptop X = # hours of use until your laptop dies P( X On average, laptops die after 5000 hours of use X ~ Ep(l = 1/5000), since E[X] = 1/l = 5000 You use your laptop 5 hours/day. What is P(your laptop lasts 4 years)? That is: P(X > (5)(365)(4) = 7300) > 7300) = 1- F(7300) = 1- (1 - e -7300/ 5000 ) = e -1.46» Better plan ahead... especially if you are coterming: P( X > 9125) = 1- F(9125) = e » (5year plan) P( X > 10950) = 1- F(10950) = e -2.19» (6year plan)

66 X = time until some event occurs X ~ Ep(l) What is P(X > s + t X > s)? After initial period of time s, P(X > t ) for waiting another t units of time until event is same as at start Memoryless = no impact from preceding period s ) ( ) ( ) ( ) and ( ) ( s X P t s X P s X P s X t s X P s X t s X P > + > = > > + > = > + > ) ( ) ( 1 ) ( 1 ) ( 1 ) ( ) ( ) ( t X P t F e e e s F t s F s X P t s X P t s t s > = - = = = = > + > l l l ) ( ) ( So, t X P s X t s X P > = > + > Eponential is Memoryless

67 Disk Crashes X = days of use before your disk crashes 1= First, determine l to have actual PDF u u o Good integral to know: ò e du = e ò le -/100 d = -100l What is P(50 < X < 150)? F(150) - F(50) = What is P(X < 10)? F(10) = -/100 ì le ³ 0 f ( ) = í î 0 otherwise 150 ò ò 0 ò e e e -/100 -/100 -/100 d = -100le d = -e d = -e -/ / /100 = -e = -e 0-3/ 2-1/10 = 100l Þ + e -1/ 2 + 1» l =»

68 Zipf Random Variable X is Zipf RV: X ~ Zipf(s) X is the rank inde of a chosen word P (X = k) =C 1 k s

Poisson Chris Piech CS109, Stanford University. Piech, CS106A, Stanford University

Poisson Chris Piech CS109, Stanford University Piech, CS106A, Stanford University Probability for Extreme Weather? Piech, CS106A, Stanford University Four Prototypical Trajectories Review Binomial Random