Basic probability. Inferential statistics is based on probability theory (we do not have certainty, but only confidence).

Transcription

1 Basic probability Inferential statistics is based on probability theory (we do not have certainty, but only confidence). I Events: something that may or may not happen: A; P(A)= probability that A happens; For instance P(it rains tomorrow in Trento); P(there is at least one son in a family with three children); P(the ball number 90 is extracted at lotto ).

2 Basic probability Inferential statistics is based on probability theory (we do not have certainty, but only confidence). I Events: something that may or may not happen: A; P(A)= probability that A happens; For instance P(it rains tomorrow in Trento); P(there is at least one son in a family with three children); P(the ball number 90 is extracted at lotto ). Formally, A, the sample space (all possible occurrence). We consider A \ B (both A and B occur), A [ B (A orboccurs, or both)...

3 Computing probabilities How do we assign probabilities? We generally use models based on experience and intuition. After seeing data, statistics helps in deciding whether the model used was correct. Often, it is assumed that all elementary events are equally likely (classical probability). Examples... I Sequences of heads and tails I Drawing balls from an urn

4 Random variables Often, we are more interested in events that concern a quantitative measure: I Random variable: something that takes an unpredictable numerical value: X P(X = k) =probability that X takes value k. For instance, X is the number of tails when tossing a coin 10 times.

5 Random variables Often, we are more interested in events that concern a quantitative measure: I Random variable: something that takes an unpredictable numerical value: X P(X = k) =probability that X takes value k. For instance, X is the number of tails when tossing a coin 10 times. Formally, X :! R, P(X = k) =P X 1 ({k}).

6 Binomial distribution Assumptions: I X represents the number of successes in n trials; I Trials can result only in success or failure ; I Trials are independent; I The probability of success is the same p in all trials. Then = where n! k!(n k)! n P(X = k) = p k (1 p) n k k n n (n 1) (n k + 1) [binomial coe cient] = k 1 2 k with n! [nfactorial]= n (n 1) 2 1.

7 Graphical illustration of binomial distributions Binomial distribution with n = 20, p = 0.1 Binomial distribution with n = 20, p = 0.45 probability probability # successes # successes Binomial distribution with n = 200, p = 0.1 Binomial distribution with n = 200, p = 0.45 probability probability

8 Problem: are data consistent with the assumption of a binomial distribution? A classical case study are the sex ratios obtained by Geissler (1889) on the sex of 6115 sibships, each of 12 children.

9 Problem: are data consistent with the assumption of a binomial distribution? A classical case study are the sex ratios obtained by Geissler (1889) on the sex of 6115 sibships, each of 12 children. #females #sibships

10 Problem: are data consistent with the assumption of a binomial distribution? A classical case study are the sex ratios obtained by Geissler (1889) on the sex of 6115 sibships, each of 12 children. #females #sibships 0 7 Sibships with 12 children (Saxony, 1889) Number of females 12 3 frequency

11 Fitting a binomial p = frequency of female newborns = total#females total#children P(# females in a sibship) = k) = p k (1 p) 12 k k

12 Fitting a binomial p = frequency of female newborns = total#females P(# females in a sibship) = k) = # obs. exp total#children p k (1 p) 12 k k

13 Fitting a binomial p = frequency of female newborns = total#females P(# females in a sibship) = k) = # obs. exp frequency total#children p k (1 p) 12 k k Sibships with 12 children (Saxony, 1889) observed expected Number of females

14 probability probability Poisson distribution Another discrete distribution often used is the Poisson distribution, used for the occurrence of rare events: P(X = k) = k e, k =0, 1, 2,... k! =1 2 k. k! is the only parameter of the Poisson [relations with binomial]. Poisson distribution with lambda = 1.5 Poisson distribution with lambda = # events # events

15 Poisson approximation Poisson can be viewed as a limiting case of binomial (law of small numbers. The figure shows how binomials with larger n and the same value for np can be approximated by a Poisson with parameter = np

16 Poisson fit of a distribution Deaths of Prussian kicked by horses Deaths of Prussian kicked by horses Army corps years Army corps years observed expected Number of deaths Number of deaths A Poisson distribution fits a famous dataset by von Bortkiewicz (1898) on the number of soldiers killed by being kicked by a horse each year in each of 14 cavalry corps over a 20-year period.

17 Mean and variance of a random variable In general, the distribution of a discrete random variable is given by I the list of possible values {x 1,...,x n }; I the respective probabilities {p 1,...,p n },i.e.p k = P(X = x k ) For a random variable, one can compute its expected value or mean: E(X )= nx x i p i will be denoted also as µ X. i=1 To describe its spread, one uses the variance, i.e. the expected value of the squared deviations from the mean: V(X )=E((X µ X ) 2 )= nx (x i µ X ) 2 p i = i=1 nx xi 2 p i µ 2 X. i=1

18 Mean and variance of some distributions If X Bin(n, p) [binomial of parameters n and p] E(X )= nx n k p k (1 p) n k = n p. [# trials prob. success] k k=0 V(X )= nx n (k np) 2 p k (1 p) n k = n p (1 p). k k=0 If X P( ) [Poisson of parameters ] E(X )= V(X )= 1X k k k! e =. k=0 nx (k ) 2 k k! e =. k=0

19 Limit theorems of probability. I. The law of large numbers Istogramma della media dopo lanci Istogramma della media dopo lanci Density Density media media

20 Limit theorems of probability. II. Summing variables Istogramma del punteggio totale dopo lanci Istogramma del punteggio totale dopo lanci Density Density punteggio totale punteggio totale

21 Limit theorems of probability. III. Central limit theorem istogramma della media dopo lanci istogramma della media dopo lanci Density Density media media With an appropriate scaling, the deviations from the mean follow a universal distribution, the normal or Gaussian.

22 density Normal distribution Standard normal density P(a<X<b) a b x Standard normal: P(a < X < b) =P(a apple X apple b) = 1 p 2 R b a e x2 /2 dx.

23 More on normal distribution Generic normal: X N(µ, 2 ), density p(x) = 1 p 2 2 e (x µ)2 /(2 2) : P(a < X < b) =P(a apple X apple b) = Z b a p(x) dx E(X )= Z +1 1 xp(x) dx = µ, V(X )= Z +1 1 (x µ) 2 p(x) dx = 2. If X N(µ, 2 ), Z = X µ N(0, 1), i.e. standard normal.

24 Normal approximation to the binomial If X Bin(n, p), # successes after n trials E(X )=np V(X )=np(1 p) for n large [say n 25, np, n(1 p) 10] approximate X N (np, np(1 p)). i.e. P(a apple Bin(n, p) apple b) P(a apple N(np, np(1 p)) apple b). Continuity approximation True value: P(40 apple Bin(100, 0.42) apple 48) = P(40 apple Bin(100, 0.42) apple 48) = P(39.5 apple Bin(100, 0.42) apple 48.5) P(39.5 apple N(42, 24.36) apple 48.5) while P(40 apple N(42, 24.36) apple 48) 0.545

25 Conditional probability All probability judgements depend on the available information. We discuss the conditional probability P(E F ), i.e. the probability of an event E given that we know that an event F has occurred. After examine some examples, we arrive at. Definition Let E and F two events in a sample space with P(F ) > 0. P(E F ) is defined as: P(E F )= P(E \ F ). (1) P(F )

26 Conditional probability All probability judgements depend on the available information. We discuss the conditional probability P(E F ), i.e. the probability of an event E given that we know that an event F has occurred. After examine some examples, we arrive at. Definition Let E and F two events in a sample space with P(F ) > 0. P(E F ) is defined as: P(E F )= P(E \ F ). (1) P(F ) Note: P(E F ) is a new probability of the same event E.

27 Product rule and independence Multiplying both sides of (1) by P(F ), we arrive at the product rule: P(E \ F )=P(F ) P(E F ) (2)

28 Product rule and independence Multiplying both sides of (1) by P(F ), we arrive at the product rule: P(E \ F )=P(F ) P(E F ) (2) Note: We may use these relations either way. Sometimes we know P(E \ F ) and use (1) to compute P(E F ). In other cases, we know P(E F ) and use (2) to obtain P(E \ F )

29 Product rule and independence Multiplying both sides of (1) by P(F ), we arrive at the product rule: P(E \ F )=P(F ) P(E F ) (2) Note: We may use these relations either way. Sometimes we know P(E \ F ) and use (1) to compute P(E F ). In other cases, we know P(E F ) and use (2) to obtain P(E \ F ) We consider E and Findependentif knowledge of F does not change the probability of E, i.e.ifp(e F )=P(E).

30 Product rule and independence Multiplying both sides of (1) by P(F ), we arrive at the product rule: P(E \ F )=P(F ) P(E F ) (2) Note: We may use these relations either way. Sometimes we know P(E \ F ) and use (1) to compute P(E F ). In other cases, we know P(E F ) and use (2) to obtain P(E \ F ) We consider E and Findependentifknowledge of F does not change the probability of E, i.e.ifp(e F )=P(E). Inserting this in (2), we obtain E and Findependentif P(E \ F )=P(E) P(F ). (3)

31 Tree diagram A tree diagram is a graphical tool to represent chains of events. Suppose I draw 2 cards from a deck of 52. Then first card red = 1 Ē : diamonds 4 ``` ` black XX XX 39 XX 52 = 3 red 4 E : non-diamonds ``` ` black Product and sum rules translate into visual rules: second card I the probability of a chain (say 1st Diamonds-2nd Black) is obtained by multiplying the probabilities of each link; I the probability of an event (say 2nd Black) is obtained by summing the probabilities of all chains leading to the event.