The Normal Distribution

Transcription

1 The Normal Distribution image: Etsy Will Monroe July 19, 017 with materials by Mehran Sahami and Chris Piech

2 Announcements: Midterm A week from yesterday: Tuesday, July 5, 7:00-9:00pm Building One page (both sides) of notes Material through today s lecture Review session: Tomorrow, July 0, :30-3:0pm in Gates B01

3 Review: A grid of random variables number of successes One trial X Ber( p) n=1 Several trials Interval of time X Bin(n, p) X Poi(λ) time to get successes X Geo( p) One success r=1 X NegBin (r, p) X Exp(λ) (continuous!) Several successes One success after interval of time

4 Review: Continuous distributions A continuous random variable has a value that s a real number (not necessarily an integer). Replace sums with integrals! P (a< X b)=f X (b) F X (a) a F X (a)= x= dx f X (x)

5 Review: Probability density function The probability density function (PDF) of a continuous random variable represents the relative likelihood of various values. Units of probability divided by units of X. Integrate it to get probabilities! b P (a< X b)= dx f X (x) x=a

6 Continuous expectation and variance Remember: replace sums with integrals! E [ X ]= x p X (x) E [ X ]= x= E [ X ]= x p X ( x) x= E [ X ]= x= dx x f X ( x) dx x f X ( x) x= Var( X )=E [( X E [ X ]) ]=E [ X ] (E [ X ]) (still!)

7 Review: Uniform random variable A uniform random variable is equally likely to be any value in a single real number interval. X Uni(α,β) 1 f X (x)= β α 0 { if x [α,β] otherwise

8 Uniform: Fact sheet minimum value X Uni(α,β) maximum value PDF: CDF: 1 f X ( x)= β α 0 { x α β α F X ( x)= 1 0 { if x [α,β] otherwise if x [α,β] if x>β expectation: otherwise variance: image: Haha169 α+β E[ X ]= (β α) Var( X )= 1

9 Review: Exponential random variable An exponential random variable is the amount of time until the first event when events occur as in the Poisson distribution. X Exp(λ) λ x λe f X (x)= 0 { image: Adrian Sampson if x 0 otherwise

10 Exponential: Fact sheet rate of events per unit time X Exp(λ) time until first event PDF: CDF: expectation: variance: image: Adrian Sampson λ x λe f X ( x)= 0 { { 1 e F X ( x)= 0 1 E [ X ]= λ 1 Var( X )= λ λ x if x 0 otherwise if x 0 otherwise

11 Normal random variable An normal (= Gaussian) random variable is a good approximation to many other distributions. It often results from sums or averages of independent random variables. X N (μ, σ ) 1 x μ ( 1 σ ) f X ( x)= e σ π

12 Déjà vu?

13 Déjà vu? P( X =k ) k

14 Déjà vu? f X ( x) x X = sum of n independent Uni(0, 1) variables image: Thomasda

15 The normal distribution Also known as: Gaussian distribution Shape: bell curve Personality: easygoing

16 What is normally distributed? Natural phenomena: heights, weights (approximately) Noise in measurements Sums/averages of many random variables (caveats: independence, equal weighting, continuity...) Averages of samples from a population (with sufficient sample sizes)

17 The Know-Nothing Distribution maximum entropy The normal is the most spread-out distribution with a fixed expectation and variance. If you know E[X] and Var(X) but nothing else, a normal is probably a good starting point!

18 Normal: Fact sheet mean X N (μ, σ ) variance (σ = standard deviation) PDF: 1 x μ σ 1 f X ( x)= e σ π ( )

19 The Standard Normal Z N (0,1) μ X N (μ, σ ) σ² X =σ Z +μ X μ Z= σ

20 De-scarifying the normal PDF 1 x μ σ 1 f X ( x)= e σ π ( )

21 De-scarifying the normal PDF 1 z f Z ( z)= e 1 π ( )

22 De-scarifying the normal PDF 1 z 1 f Z ( z)= e π

23 De-scarifying the normal PDF 1 z f Z ( z)=c e

24 De-scarifying the normal PDF 1 z f Z ( z)=c e 1 z

25 De-scarifying the normal PDF 1 z f Z ( z)=c e 1 z

26 De-scarifying the normal PDF 1 x μ σ 1 f X ( x)= e σ π normalizing constant ( ) X μ Z= σ

27 Normal: Fact sheet mean X N (μ, σ ) variance (σ = standard deviation) PDF: CDF: 1 x μ σ ( ) 1 f X ( x)= e σ π x x μ F X ( x)=φ σ = dx f X ( x) ( ) (no closed form)

28 The Standard Normal Z N (0,1) μ X N (μ, σ ) σ² X =σ Z +μ X μ Z= σ Φ(z)=F Z ( z)=p(z z)

29 Symmetry of the normal P( X μ x)=p( X μ+ x) and don t forget: P( X > x)=1 P( X x)

30 Symmetry of the normal P(Z z)=p(z z) and don t forget: P(Z > z)=1 P(Z z)

31 Symmetry of the normal Φ( z)=p(z z) and don t forget: P(Z > z)=1 Φ( z)

32 The standard normal table Φ(0.54)=P(Z 0.54)=0.7054

33 With today s technology scipy.stats.norm(mean, std).cdf(x) standard deviation! not variance. you might need math.sqrt here.

34 Break time!

35 Practice with the Gaussian X ~ N(3, 16) μ=3 σ² = 16 σ=4 X > =P Z > =1 P Z =1 Φ( ) =1 (1 Φ( )) 4 3 =Φ( ) P( X >0)=P ( ( ) ) ( )

36 Practice with the Gaussian X ~ N(3, 16) μ=3 σ² = 16 σ=4 P( X 3 > 4)=P ( X < 1)+ P( X >7) X X =P < +P > =P (Z < 1)+ P( Z >1) =Φ( 1)+(1 Φ(1)) =(1 Φ(1))+(1 Φ(1)) ( ) = ( ) ( )

37 Practice with the Gaussian X ~ N(3, 16) μ=3 σ² = 16 σ=4 P( X μ >σ)=p( X <μ σ)+ P( X >μ+σ) X μ μ σ μ X μ μ+σ μ =P σ < +P σ > σ σ ( ) ( =P (Z < 1)+ P( Z >1) =Φ( 1)+(1 Φ(1)) =(1 Φ(1))+(1 Φ(1)) ( ) = )

38 Normal: Fact sheet mean X N (μ, σ ) variance (σ = standard deviation) PDF: CDF: 1 x μ σ ( ) 1 f X ( x)= e σ π x x μ F X ( x)=φ σ = dx f X ( x) ( ) (no closed form) expectation: variance: E[ X ]=μ Var( X )=σ

39 Carl Friedrich Gauss ( ) remarkably influential German mathematician Started doing groundbreaking math as a teenager Didn t invent the normal distribution (but popularized it)

40 Noisy wires Send a voltage of X = or - on a wire. + represents 1, - represents 0. Receive voltage of X + Y on other end, where Y ~ N(0, 1). If X + Y 0.5, then output 1, else 0. P(incorrect output original bit = 1) = P(+Y <0.5)=P (Y < 1.5) =Φ( 1.5) =1 Φ(1.5)

41 Noisy wires Send a voltage of X = or - on a wire. + represents 1, - represents 0. Receive voltage of X + Y on other end, where Y ~ N(0, 1). If X + Y 0.5, then output 1, else 0. P(incorrect output original bit = 0) = P( +Y 0.5)=P(Y.5) =1 P(Y <.5) =1 Φ(.5) 0.006

42 Poisson approximation to binomial large n, small p P( X =k ) Bin (n, p) Poi (λ) k

43 Normal approximation to binomial large n, medium p P( X =k ) Bin (n, p) N (μ, σ ) k

44 Something is strange...

45 Continuity correction X Bin (n, p) Y N (np, np(1 p)) P ( X 55) P (Y >54.5) When approximating a discrete distribution with a continuous distribution, adjust the bounds by 0.5 to account for the missing half-bar.

46 Miracle diets 100 people placed on a special diet. Doctor will endorse diet if 65 people have cholesterol levels decrease. What is P(doctor endorses diet has no effect)? X: # people whose cholesterol decreases X ~ Bin(100, 0.5) np = 50 np(1 p) = 50(1 0.5) = 5 Y ~ N(50, 5) Y P (Y >64.5)=P > 5 5 =P(Z >.9)=1 Φ(.9) ( )

47 Stanford admissions Stanford accepts 480 students. Each student independently decides to attend with p = What is P(at least 1750 students attend)? X: # of students who will attend. X ~ Bin(480, 0.68) np = σ² = np(1 p) Y ~ N(1686.4, ) Y P (Y >1749.5)=P > P (Z >.54)=1 Φ(.54) ( image: Victor Gane )

48 Stanford admissions changes