Topic 2: Probability & Distributions. Road Map Probability & Distributions. ECO220Y5Y: Quantitative Methods in Economics. Dr.

Similar documents
Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing. Road Map Sampling Distributions, Confidence Intervals & Hypothesis Testing

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems

Econ 325: Introduction to Empirical Economics

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

A Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.

Learning Objectives for Stat 225

Bivariate distributions

Statistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions

Probability and Distributions

Probability Theory and Statistics. Peter Jochumzen

Statistics for Managers Using Microsoft Excel (3 rd Edition)

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables

CME 106: Review Probability theory

Recitation 2: Probability

Probability- describes the pattern of chance outcomes

The enumeration of all possible outcomes of an experiment is called the sample space, denoted S. E.g.: S={head, tail}

Lecture 6. Probability events. Definition 1. The sample space, S, of a. probability experiment is the collection of all

1 Presessional Probability

1 INFO Sep 05

Probability Dr. Manjula Gunarathna 1

Single Maths B: Introduction to Probability

Examples of random experiment (a) Random experiment BASIC EXPERIMENT

Lecture 2: Probability and Distributions

Relationship between probability set function and random variable - 2 -

QUANTITATIVE TECHNIQUES

STAT 302 Introduction to Probability Learning Outcomes. Textbook: A First Course in Probability by Sheldon Ross, 8 th ed.

Review of Statistics

1 Review of Probability and Distributions

Elements of probability theory

Probability: Why do we care? Lecture 2: Probability and Distributions. Classical Definition. What is Probability?

Dept. of Linguistics, Indiana University Fall 2015

Stochastic Models of Manufacturing Systems

The Binomial distribution. Probability theory 2. Example. The Binomial distribution

Lecture 2: Review of Probability

01 Probability Theory and Statistics Review

Statistics for Business and Economics

Discrete Probability distribution Discrete Probability distribution

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities

Probability and Applications

Chapter 2. Probability

Chapter 5. Chapter 5 sections

FINAL EXAM: Monday 8-10am

2 (Statistics) Random variables

Bayesian Statistics Part III: Building Bayes Theorem Part IV: Prior Specification

Lecture 2: Repetition of probability theory and statistics

Expectation MATH Expectation. Benjamin V.C. Collins, James A. Swenson MATH 2730

Random Variables and Their Distributions

Properties of Summation Operator

Practice Examination # 3

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

LIST OF FORMULAS FOR STK1100 AND STK1110

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

2011 Pearson Education, Inc

STAT 4385 Topic 01: Introduction & Review

Copyright c 2006 Jason Underdown Some rights reserved. choose notation. n distinct items divided into r distinct groups.

Probability and Probability Distributions. Dr. Mohammed Alahmed

Advanced Herd Management Probabilities and distributions

STAT 430/510: Lecture 16

Math Review Sheet, Fall 2008

Chapter 4. Probability-The Study of Randomness

Final Exam # 3. Sta 230: Probability. December 16, 2012

Topic 2 Probability. Basic probability Conditional probability and independence Bayes rule Basic reliability

Lecture 4: Probability and Discrete Random Variables

Chapter 1. Sets and probability. 1.3 Probability space

ECE 302 Division 2 Exam 2 Solutions, 11/4/2009.

Quick Tour of Basic Probability Theory and Linear Algebra

Chapter 4 Multiple Random Variables

Bivariate scatter plots and densities

Week 2. Review of Probability, Random Variables and Univariate Distributions

THE QUEEN S UNIVERSITY OF BELFAST

Introduction to Probability and Stocastic Processes - Part I

15 Discrete Distributions

Statistics for Economists Lectures 6 & 7. Asrat Temesgen Stockholm University

ACE 562 Fall Lecture 2: Probability, Random Variables and Distributions. by Professor Scott H. Irwin

Counting principles, including permutations and combinations.

b. ( ) ( ) ( ) ( ) ( ) 5. Independence: Two events (A & B) are independent if one of the conditions listed below is satisfied; ( ) ( ) ( )

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com

Classical Probability

Probability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014

What is Probability? Probability. Sample Spaces and Events. Simple Event

Chapter 2. Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)

Multiple Random Variables

CS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro

STA301- Statistics and Probability Solved Subjective From Final term Papers. STA301- Statistics and Probability Final Term Examination - Spring 2012

Algorithms for Uncertainty Quantification

Random Variables. P(x) = P[X(e)] = P(e). (1)

STAT Chapter 5 Continuous Distributions

SOR201 Solutions to Examples 3

Math 416 Lecture 2 DEFINITION. Here are the multivariate versions: X, Y, Z iff P(X = x, Y = y, Z =z) = p(x, y, z) of X, Y, Z iff for all sets A, B, C,

Standard & Conditional Probability

M378K In-Class Assignment #1

EE4601 Communication Systems

Deep Learning for Computer Vision

1.1 Review of Probability Theory

Discrete Random Variables

Appendix A : Introduction to Probability and stochastic processes

SDS 321: Introduction to Probability and Statistics

Probability Theory. Patrick Lam

Transcription:

Topic 2: Probability & Distributions ECO220Y5Y: Quantitative Methods in Economics Dr. Nick Zammit University of Toronto Department of Economics Room KN3272 n.zammit utoronto.ca November 21, 2017 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 1 / 47 Road Map Probability & Distributions Key Concepts: 1 Definitions (Set Theory & Rules, Marginal Probability, Joint Probability, Conditional Probability, Expectation Operator) 2 Tables/Plots (Contingency Tables, Probability Trees, Normal Probability, Area of Standard Normal, Area of Student s t) 3 Probability Distributions (Uniform, Geometric, Binomial, Poisson, Student-t, χ 2, F-dist.) 4 Ideas (Law of Large Numbers, Independence vs. Mutual Exclusivity, Bayes Rule, Chebychev s Formula & 68-95-99.7 Rule, Normal Approximations) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 2 / 47

Basic Definitions Probability P(A) is the probability of an event A occuring. Sample Space S is the set of all possible outcomes Event A is an event made up of a subset of outcomes from the sample space Outcome C i is the value of a specific outcome i, i ɛ (1,.., N) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 3 / 47 Set Theory Union A B is the set of all elements which are in A or B or both Intersection A B is the set of all elements which are in both A and B Complement A c is the set of all elements which are not in A Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 4 / 47

Set Rules 1 A B = B A 2 A B = B A 3 A (B C) = (A B) C 4 A (B C) = (A B) C 5 A = 6 A = A 7 (A c ) c = A 8 A (B C) = (A B) (A C) 9 A (B C) = (A B) (A C) 10 (A B) c = A c B c 11 (A B) c = A c B c Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 5 / 47 Empirical vs. Theoretical Probability Theoretical (Classical) Probability The mathmatical/model based likelihood of an event If N outcomes are equally likely and N A is the number of outcomes in A, the theoretical probability of event A is P(A) = N A N Empirical (Frequentist) Probability The long-run relative frequency of an event. Guaranteed by Law of Large Numbers If n A is the number of times the event A occurred in n trials, the empirical probability of event A is P(A) n A n Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 6 / 47

Empirical Probability and Law of Large Numbers Law of Large Numbers The long-run relative frequency of repeated, independent events homes in on empirical probability as number of trials increases Proves that empirical probability is the same as theoretical probability as n This is easy to see with a simple example! Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 7 / 47 Empirical Probability and Law of Large Numbers 010 20 30 Percent 12 23 34 45 56 6Dice Roll Random Dice Rolls (10000 Obs) Dice Rolls (10 Obs) Random Random Dice Rolls (10 Obs) 30 30 Random Dice Rolls (100 Obs) 0 10 Percent 20 0 1 2 3 4 Dice Roll 5 6 0 10 Percent 20 0 1 2 3 4 Dice Roll 5 6 30 Random Dice Rolls (1000 Obs) 0 10 30 Percent 20 0 1 2 3 4 Dice Roll 5 6 0 10 Percent 20 Random Dice Rolls (10000 Obs) 0 1 2 3 4 Dice Roll 5 6 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 8 / 47

Basic Probability Rules Range Rule (Rule 1) A probability is a number between 0 and 1 For any event A, 0 P(A) 1 Probability Assignment Rule (Rule 2) The probability of the set of all possible outcomes must be 1 P(S) = 1 Complement Rule (Rule 3) The probability of an event A occurring is 1 minus the probability that it does not occur (A complement) P(A) = 1 P(A c ) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 9 / 47 Basic Probability Concepts Marginal Probability For two events A and B, the marginal probability is the empirical probability based on total frequency of one event. Joint Probability For two events A and B, the joint probability is the empirical probability based on frequency of both events occurring together. Conditional Probability For two events A and B, the conditional probability is the empirical probability based on frequency of one event given a condition on the other event. Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 10 / 47

Advanced Probability Rules Multiplication Rule (Rule 4) For two independent events A and B, the probability that both A and B occur is the product of the individual probabilities. P(A B) = P(A) x P(B) General Multiplication Rule (Rule 7) For two events A and B, the probability that both A and B occur is the probability of one event occuring times the probability of the other occurring conditional on the first. P(A B) = P(A) x P(B A) = P(B) x P(A B) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 11 / 47 Advanced Probability Concepts Formal Independence Events A and B are independent when the probability of either event is the same as the probability of that event conditional on the other event. P(A B) = P(A) and P(B A) = P(B) This can be extended to 3 or more events P(A B, D) = P(A) and P(B A, D) = P(B) and P(D A, B) = P(D) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 12 / 47

Advanced Probability Rules Addition Rule (Rule 5) For two disjoint events A and B, the probability that either A or B occurs is the sum of the individual probabilities. P(A B) = P(A) + P(B) General Addition Rule (Rule 6) For two events A and B, the probability that either A or B occurs is the sum of the probability each event occurs less the probability that both events occur. P(A B) = P(A) + P(B) P(A B) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 13 / 47 Advanced Probability Concepts Mutual Exclusivity Generally the events A and B are said to be mutually exclusive events if they share no common elements P(A B) = Independent vs. Disjoint If two events are disjoint (mutually exclusive) they cannot be independent P(A B) = P(A) x P(B) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 14 / 47

Advanced Probability Concepts Bayes Rule Consider an event B and i mutually exclusive events A i : P(A i B) = P(B A i)p(a i ) j P(B A j)p(a j ) (1) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 15 / 47 What makes a valid probability distribution? For a density function p(x): 1 p(x) is non-negative for all values in the range of x-axis values 2 The total area under the density function p(x) in the range of x-axis values is equal to 1 S p(x)dx = 1 Note for a valid density function: If p(x) is the p.d.f. of a continuous random variable x, then the probability that x belongs to A, where A is some interval, is given by the integral of p(x) over that interval P(xɛA) = A p(x)dx If p(x) is the p.d.f. of a discrete random variable x P(xɛA) = A p(x) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 16 / 47

Probability Distributions Example: Univariate Distribution Highest Education Obtained no quals high school high school associate degree professional masters phd Total drop-out diploma diploma 65 257 1431 217 529 27 176 12 2714 2.39 9.47 52.73 8.00 19.49 0.99 6.48 0.44 100.00 Notice this is a valid probability density function: p(x) is non-negative for all possible categories All the area in the density function sums to one p(x) = 0.0239+0.0947+0.5273+0.08+0.1949+0.0099+0.0648+0.0044 = 1 So if we have A = {no quals, high school drop-out} then we can calculate P(A) = 65+257 2714 = 0.1186 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 17 / 47 Probability Distributions Example: Bivariate Distribution Highest Education Obtained Gender no high school high school associate degree professional masters phd Total quals drop-out diploma diploma Female 38 114 724 127 250 8 95 3 1359 1.40 4.20 26.68 4.68 9.21 0.29 3.50 0.11 50.07 Male 27 143 707 90 279 19 81 9 1355 0.99 5.27 26.05 3.32 10.28 0.70 2.98 0.33 49.93 Total 65 257 1431 217 529 27 176 12 2714 2.39 9.47 52.73 8.00 19.49 0.99 6.48 0.44 100.00 So now if we have A = {no quals, high school drop-out} and B = {male} then we can calculate P(A B) = 27+143 1355 = 0.125 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 18 / 47

Probability Trees Tree Diagrams A tree shows sequences of events as paths that look like branches of a tree Allows comparison of multiple scenarios based on multivariate probability distributions Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 19 / 47 The Expectation Operator Expectations for Distributions 1 E[X ] = x xp(x) = µ or E[X ] = x xp(x)dx 2 E[(X E[X ]) 2 ] = V [X ] = x (x µ)2 p(x) or E[(X E[X ]) 2 ] = x (x µ)2 p(x)dx 3 E[(X E[X ])(Y E[Y ])] = COV [X, Y ] = x y (x µ x)(y µ y )p(x, y) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 20 / 47

The Expectation Operator Laws of Expected Values 1 E[c] = c 2 E[X + c] = E[X ] + c 3 E[cX ] = ce[x ] 4 E[X + Y ] = E[X ] + E[Y ] 5 E[a + bx + cy ] = a + be[x ] + ce[y ] Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 21 / 47 The Expectation Operator Laws of Variance 1 V [c] = 0 2 V [X + c] = V [X ] 3 V [cx ] = c 2 V [X ] 4 V [a + bx + cy ] = b 2 V [X ] + c 2 V [Y ] + 2(b)(c)COV [X, Y ] Laws of Covariance 1 COV [X, c] = 0 2 COV [a + bx, c + dy ] = (b)(d)cov [X, Y ] Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 22 / 47

Special Probability Distributions The Bernoulli Distribution Generated by an experiment called a Bernoulli Trial leading to only two outcomes (success, failure) Bernoulli Trial has probability of success equal to p P(X = 1) = p Bernoulli Trial has probability of failure equal to 1 p P(X = 0) = 1 p = q Distribution has the following central tendencies: E[X ] = 0(q) + 1(p) = p V [X ] = E[X 2 ] E[X ] 2 = p p 2 = p(1 p) = pq Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 23 / 47 Special Probability Distributions The Geometric Distribution Generated by having independent Bernoulli Trials until success The probability of the first success in x Bernoulli Trials has probability P(X = x) = p(q) x 1 Distribution has the following central tendencies: E[X ] = xp(x) = 1p + 2pq + 3pq 2 +... = 1 p i=0 V [X ] = E[X 2 ] E[X ] 2 = q+1 p 2 1 p 2 = q p 2 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 24 / 47

Permutations & Combinations If there are n objects to be arranged in order then how many ways can you do this? np n = n(n 1)(n 2)...1 = n! (2) If there are n objects to arrange and you must choose and order r of them how many ways can you do this? np r = n(n 1)(n 2)...(n r + 1) = n! (n r)! (3) If there are n objects to arrange and you must simply choose r of them how many ways can you do this? nc r = n P r r! = n! r!(n r)! (4) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 25 / 47 Special Probability Distributions The Binomial Distribution Generated by having n independent Bernoulli Trials The probability of x successes in n Bernoulli Trials has probability P(X = x) = ( n C x ) p x q n x Distribution has the following central tendencies: E[X ] = n E[X i ] = p + p +... + p = np i=0 V [X ] = n V [X i ] = p = pq + pq +... + pq = npq i=0 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 26 / 47

Finding Probability of Binomial Distributions Using the equation: P(X = x) = ( n C x ) p x q n x Assume n = 5 and p = 0.5 P(X = 0) = ( 5 C 0 ) 0.5 0 0.5 5 = 0.03125 P(X = 1) = ( 5 C 1 ) 0.5 1 0.5 4 = 0.15625 P(X = 2) = ( 5 C 2 ) 0.5 2 0.5 3 = 0.3125 P(X = 3) = ( 5 C 3 ) 0.5 3 0.5 2 = 0.3125 P(X = 4) = ( 5 C 4 ) 0.5 4 0.5 1 = 0.15625 P(X = 5) = ( 5 C 5 ) 0.5 5 0.5 0 = 0.03125 P(X 1) = P(X = 0) + P(X = 1) = 0.031 + 0.156 = 0.187 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 27 / 47 Finding Probability of Binomial Distributions Using a Binomial table: n = 4, X n = 5, X p 0 1 2 3 0 1 2 3 4 0.01 0.961 0.999 1.000 1.000 0.951 0.999 1.000 1.000 1.000 0.02 0.922 0.998 1.000 1.000 0.904 0.996 1.000 1.000 1.000 0.03 0.885 0.995 1.000 1.000 0.859 0.992 1.000 1.000 1.000.............................. 0.49 0.068 0.328 0.702 0.942 0.035 0.200 0.519 0.825 0.972 0.50 0.062 0.312 0.688 0.938 0.031 0.187 0.500 0.813 0.969 Assume n = 5 and p = 0.5 P(X 1) = 0.187 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 28 / 47

Special Probability Distributions The Poisson Distribution Generated given we have time divided into n unit time intervals where independent events occur (arrive) with a mean rate of occurance The probability x independent events occur given a mean rate of occurrence λ during n is P(X = x) = e λ λ x x! Distribution has the following central tendencies: E[X ] = ( ) xp(x) = e λ 1λ 1! + 2λ2 2! + 3λ3 3! +... = e λ λe λ = λ x V [X ] = E[X 2 ] E[X ] 2 = (λ 2 + λ) λ 2 = λ Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 29 / 47 Finding Probability with Poisson Distributions Using the equation: P(X = x) = e λ λ x x! Assume λ = 6 for our chosen n time interval P(X = 0) = e 6 6 0 0! = 0.0025 P(X = 1) = e 6 6 1 1! = 0.0149 P(X = 2) = e 6 6 2 2! = 0.0446 P(X = 6) = e 6 6 6 6! = 0.1606 P(X = 8) = e 6 6 8 8! = 0.1033 P(X = 12) = e 6 6 12 12! = 0.0113 P(X 1) = P(X = 0) + P(X = 1) 0.002 + 0.015 0.017 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 30 / 47

Finding Probability of Poisson Distributions Using a Poisson table: X λ 0 1 2 3 4 5 6 7 8 9 10 0.1 0.905 0.995 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.2 0.819 0.982 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.3 0.741 0.963 0.996 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000.................................... 5.9 0.003 0.019 0.067 0.160 0.299 0.462 0.622 0.758 0.857 0.923 0.961 6.0 0.002 0.017 0.062 0.151 0.285 0.446 0.606 0.744 0.847 0.916 0.957 Assume λ = 6 P(X 1) = 0.017 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 31 / 47 Special Probability Distributions The Uniform Distribution (Discrete) Generated over n equally likely outcomes The probability outcome x occurs given n possible outcomes is P(X = x) = 1 n Distribution has the following central tendencies: E[X ] = x xp(x) = 1 1 n + 2 1 n +... + n 1 n = 1 n V [X ] = E[X 2 ] E[X ] 2 = 1 n n(n+1) 2 = n+1 2 n(n+1)(2n+1) 6 (n+1)2 4 = N2 1 12 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 32 / 47

Special Probability Distributions The Uniform Distribution (Continuous) Generated over an equally likely interval of outcomes (a to b) The probability density function p(x) is: a x b 1 b a 0 otherwise The probability x is in the interval c and d is P(c x d) = d c b a Distribution has the following central tendencies: E[X ] = b a xp(x)dx = 1 b a b a xdx = 1 b a [ ] x 2 b 2 = b+a a 2 V [X ] = E[X 2 ] E[X ] 2 = (b2 +a 2 )+ab 3 (b+a)2 4 = (b a)2 12 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 33 / 47 Finding Probability with Uniform Distributions Using the equation: P(c x d) = d c b a Assume a = 0 and b = 6 P(2 x 3) = 3 2 6 0 = 1 6 P(2 x 4) = 4 2 6 0 = 2 6 P(0 x 6) = 6 0 6 0 = 1 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 34 / 47

Special Probability Distributions The Normal Distribution The probability density function p(x) is ( ) p(x) = (2πσ 2 ) 1/2 exp (x µ) 2 where < x < 2σ 2 The probability x is in the interval c and d is d ( ) P(c X d) = (2πσ 2 ) 1/2 exp (x µ) 2 dx 2σ 2 c Distribution has the following central tendencies: ( ) E[X ] = (2πσ 2 ) 1/2 exp (x µ) 2 dx = µ 2σ 2 V [X ] = E[X 2 ] E[X ] 2 = σ 2 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 35 / 47 Special Probability Distributions The Standard Normal Distribution Transform the normal distribution by standardizing: Z = X µ σ The probability density function p(z) is ( ) p(z) = (2π) 1/2 exp (x) 2 where < x < The probability x is in the interval c and d is P(c X d) = P( c µ σ X µ σ d µ σ ) 2 Distribution has the following central tendencies: E[Z] = 0 V [Z] = E[Z 2 ] E[Z] 2 = 1 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 36 / 47

Empirical Rules Chebychev s Rule For any empirical distribution with mean x and standard deviation s, at least 100(1 1 2k )% of the data lie within k standard deviations around the mean, for k > 1 68-95-99.7 Rule (The Empirical Rule) If a data set has an approximately bell-shaped relative frequency histogram (is approximately normally distributed): 1 Approximately 68% of the data lie within one standard deviation of the mean ( x ± s) 2 Approximately 95% of the data lie within two standard deviations of the mean ( x ± 2s) 3 Approximately 99.7% of the data lie within three standard deviations of the mean ( x ± 3s) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 37 / 47 Finding Probability with Normal Distributions Using the equation: P(Z c) = (2π) 1/2 c exp ( (z) 2 2 ) dz P(c Z d) = P(Z d) P(Z c) = d ( ) exp (z) 2 c dz (2π) 1/2 exp (2π) 1/2 P(c X d) = P( X µ σ 2 ( (z) 2 d µ X µ σ ) P( σ c µ σ ) 2 ) dz This requires software or a very non-trivial calculation by hand! Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 38 / 47

Finding Probability with Standard Normal Distributions Using a standard normal table: Area to left of z z.00.01.02.03.04.05.06.07.08.09-3.4.0003.0003.0003.0003.0003.0003.0003.0003.0003.0002-3.3.0005.0005.0005.0004.0004.0004.0004.0004.0004.0003-3.2.0007.0007.0006.0006.0006.0006.0006.0005.0005.0005................................. -0.7.2420.2389.2358.2327.2296.2266.2236.2206.2177.2148-0.6.2743.2709.2676.2643.2611.2578.2546.2514.2483.2451-0.5.3085.3050.3015.2981.2946.2912.2877.2843.2810.2776 Assume X N(3, 16) Z N(0, 1) P(X 0) = P(Z 0 3 4 ) = P(Z 0.75) = 0.2266 P(X 1) = P(Z 1 3 4 ) = P(Z 0.5) = 0.3085 P(0 X 1) = P(Z 0.5) P(Z 0.75) = 0.309 0.227 = 0.082 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 39 / 47 Special Probability Distributions The Chi-Squared Distribution The probability density function is the squared standard normal χ 2 1 N(0, 1)2 The sum of independent χ 2 1 distributions can be generalised If W i χ 2 1 and W i are independent then where n are the degrees of freedom of χ 2 n χ 2 n has the following central tendencies: E[X ] = n V [X ] = E[X 2 ] E[X ] 2 = 2n n i=1 W i = χ 2 n Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 40 / 47

Finding Probability with Chi-Squared Distributions Using a Chi-Squared Density table: n = 1 (χ 2 1 ) Area to left of x x.00.01.02.03.04.05.06.07.08.09 0.0.0000.0797.1125.1375.1585.1769.1935.2087.2227.2358 0.1.2482.2599.2710.2816.2917.3018.3108.3199.3286.3371 0.2.3453.3532.3610.3685.3758.3829.3899.3967.4033.4098................................. 2.6.8931.8938.8945.8951.8958.8965.8971.8978.8984.8990 2.7.8997.9003.9009.9015.9021.9027.9034.9040.9046.9051 Assume n = 1 χ 2 1 P(X 2.71) =.90 P(X > 2.71) = 1 P(X 2.71) = 1 0.9 = 0.1 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 41 / 47 Special Probability Distributions Student s-t Distributions t-distribution is ratio of a standard normal to a chi-squared t n N(0,1) χ 2 n /n where n are the degrees of freedom of χ 2 n As the sample size becomes large enough t becomes normal As n we get t n N(0, 1) t n has the following central tendencies: E[X ] = 0 V [X ] = E[X 2 ] E[X ] 2 = n (n 2) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 42 / 47

Finding Probability with Student s-t Distributions Using a Student-t Density table: n = 10 (t 10 ) Area to left of t t.00.01.02.03.04.05.06.07.08.09-3.4.0034.0033.0033.0032.0032.0031.0031.0030.0030.0029-3.3.0040.0039.0039.0038.0037.0037.0036.0036.0035.0034-3.2.0047.0047.0046.0045.0044.0044.0043.0042.0041.0041................................. -0.7.2499.2470.2440.2411.2382.2353.2324.2296.2267.2239-0.6.2809.2777.2746.2714.2683.2652.2621.2590.2560.2529-0.5.3139.3106.3072.3038.3005.2972.2939.2906.2874.2841 Assume n = 10 t 10 P(T 0.75) = 0.2353 P(T 0.5) = 0.3139 N(0,1) χ 2 10 /10 P( 0.75 T 0.5) = 0.314 0.235 = 0.079 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 43 / 47 Special Probability Distributions F-Distributions F-Distribution is the ratio of two independent chi-squared F m,n χ2 m /m χ 2 n /n where m & n are degrees of freedom of the numerator and denominator respectively As the sample size becomes large enough F becomes χ 2 m As n we get F m,n = χ2 m /m χ 2 / χ2 m/m F m,n has the following central tendencies: E[X ] = n (n 2) for n > 2 V [X ] = E[X 2 ] E[X ] 2 = 2n2 (m+n 2) m(n 2) 2 (n 4) Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 44 / 47

Approximations to the Binomial Poisson Approximation to the Binomial If a binomial distribution has n > 50 and p < 0.1 then we can approximate it with a poison distribution where λ = np Assume we have a binomial distribution where n = 400 and p = 0.03 P(X = 5) = ( n C x ) p x q n x = ( 400 C 5 ) 0.03 5 0.97 395 = 0.012 We could approximate with a poisson where λ = np = 400(0.03) = 12 P(X = 5) = e 12 12 5 5! = 0.013 Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 45 / 47 Approximations to the Binomial Normal Approximation to the Binomial If a binomial distribution has np 10 and nq 10 we can approximate it with a normal distribution where µ = np and σ = npq adjusted with the continuity correction The Continuity Correction To approximate binomial with normal make the adjustments: P(X x) P(X (x + 0.5)) P(X > x) P(X > (x 0.5)) P(X = x) P((x 0.5) X (x + 0.5)) We could approximate with a normal where µ = np = 12 and σ = (400)(0.03)(0.97) = 3.412 P(X = 5) = P(4.5 X 5.5) = P( 2.20 Z 1.91) = 0.028 Dr. Nick 0.014 Zammit (UofT) = 0.014 Topic 2 November 21, 2017 46 / 47

Supplementary References Dr. Nick Zammit (UofT) Topic 2 November 21, 2017 47 / 47