Probability. 25 th September lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.)

Transcription

1 Probability 25 th September 2017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.)

2 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

3 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

4 what statisticians do: problem/situation that needs to be considered (measurement problem) collect data through observations summarize the results (descriptive statistics, graphical methods) analyze the situation (statistical inferences) report with recommendations

5 statistics deals with the data collection and analysis of data variability in results statisticians try to find a pattern in data and deal with errors decision makers can decide upon confidence in the analyzed data

6 variability is a fact of life (variability is inherent) decisions have to involve uncertainties examples meteorology seismology medical research state legislature economy (weather forecast) (seismic hazard analysis) (drug/vaccine testing) (speed limit vs. no. of accidents) (estimation of unemployment rate)

7 how to make sense to the observations? sometimes the answers are obvious however, in many investigations appropriate probability and statistical models are needed

8 random experiments experiments for which the outcome cannot be predicted with certainty although the specific outcome of a r.e. cannot be predicted before the experiment is performed, the collection of all possible outcomes is known (can be described) outcome space S the collection of all possible outcomes event A A is a part of the collection of outcomes in S, A S

9 words set and event are interchangeable algebra of sets A B A B A B null or empty set A is subset of B union of A and B intersection of A and B A complement of A (all elements in S that are not in A) Venn diagrams

10 A 1, A 2, A k are mutually exclusive events A i A j =, i j that is, A 1, A 2, A k are disjoint sets A 1, A 2, A k are exhaustive events A 1 A 2 A k = S if A 1, A 2, A k are mutually exclusive and exhaustive events A i A j =, i j and A 1 A 2 A k = S

11 Commutative Laws Associative Laws Distributive Laws De Morgan s Laws A B = B A A B = B A (A B) C = A (B C ) (A B) C = A (B C ) A (B C) = (A B) (A C) A (B C) = (A B) (A C) (A B) = A B (A B) = A B

12 P(A) probability of event A (the chance of A occuring) consider repeating the experiment n times N(A) number of times that event A occurred throughout n performances ratio N(A) n is the relative frequency of event A in n repetitions relative frequency is usually very unstable for small values of n

13 number p number about which the relative frequency tends to stabilize (number that the relative frequency will be in future performances) although we cannot predict the outcome of a random experiment with certainty, if we know p (for a large number of n), we can predict fairly accurately the relative frequency of event A p is called probability of event A P(A) proportion of outcomes of a random experiment that terminate in the event A

14 set function a function such as P(A) that is evaluated for a set A PROPERTIES N(A)/n is always nonnegative if A = S, the outcome of the experiment will always belong to S N(A)/n = 1 if A and B are mutually exclusive events N(A B)/n = N(A)/n + N(B)/n

15 DEFINITION Probability is a real-valued set function P that assign, to each event A in the sample space S, a number P(A) called the probability of the event A, such that the following properties are satisfied (a) P(A) 0 (b) P(S) = 1 (c) A 1, A 2, A k are events and A i A j =, i j, then P(A 1 A 2 A k ) = P(A 1 ) + P(A 2 ) + + P(A k ) for each positive integer k P(A 1 A 2 ) = P(A 1 ) + P(A 2 ) + for an infinite, but countable, number of events

16 THEOREMS 1. For each event A, 2. P( ) = 0. P(A) = 1 P(A ). 3. If events A and B are such that A B, then P(A) P(B). 4. For each event A, P(A) If events A and B are two events, then P(A B) = P(A) + P(B) - P(A B). 6. If A, B and C are any three events, then P(A B C) = P(A) + P(B) + P(C) - P(A B) - P(A C) - P(B C) + P(A B C).

17 PROOF 1. For each event A, P(A) = 1 P(A ). from Venn diagrams, we have S = A A and A A = from properties (b) and (c) for probability, it follows than 1 = P(A) + P(A ) then P(A) = 1 P(A )

18 PROOF 2. P( ) = 0. in the previous theorem P(A) = 1 P(A ), take A = so that A = S then P( ) = 1 P(S) = 1-1 = 0

19 PROOF 3. If events A and B are such that A B, then P(A) P(B). we have B = A (B A ) and A (B A ) = from properties (b) and (c) for probability, it follows than P(B) = P(A) + P(B A ) P(A) because, from property (a) for probability, we have P(B A ) 0

20 PROOF 4. For each event A, P(A) 1. since A S, we have by theorem (1) and property (b) for probability P(A) P(S) = 1 which gives the desired result

21 PROOF 5. If events A and B are two events, then P(A B) = P(A) + P(B) - P(A B). from Venn diagrams, the event A and B can be represented as a union of mutually exclusive events, A A = A (A B) from property (c) for probability P(A B) = P(A) + P(A B) however B = (A B) (A B), which is a union of mutually exclusive events thus P(B) = P(A B) + P(A B) if we substitute P(A B) to the right-hand side of the second equation, we obtain the desired result

22 let S = e 1, e 2,, e m, e i is a possible outcome of the experiment integer m is called the total number of ways in which the random experiment can terminate if each of these outcomes has the same probability of occurring, we say that the m outcomes are equally likely P ( e i ) = 1/m, i = 1, 2, m if the number of outcomes in an event A is h, then the integer h is called the number of ways that are favorable to the event A P A = h m = N(A) N(S)

23 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

24 Multiplication principle Suppose that the experiment E i has n i, i = 1,2, m possible outcomes after previous experiments have been performed. Then the composite experiment E 1 E 2 E m that consists of performing E 1, then E 2 and finally E m has n 1 n 2 n m possible outcomes.

25 Permutation Suppose that n positions are to be filled with n different objects. There are n choices for filling the first position, n - 1 for the second, and 1 choice for the last position. By multiplication principle, the are n (n - 1) (2) (1) = n! possible arrangements Each of the n! arrangements (in a row) of n different objects is called a permutation of n objects.

26 Permutation If only r positions are to be filled with objects selected from n different objects, r n, then the number of possible ordered arrangements is np r = n (n - 1) (n - 2) (n - r + 1) Each of the n P r arrangements is called a permutation of n objects taken r at a time. np r = n n 1 n r+1 n r 3 (2)(1) n r (3)(2)(1) = n! n r!

27 a set contains n objects consider a problem drawing r objects from the set the order in which the objects are drawn may/may not be important a drawn object is/is not replaced before the next object is drawn If r objects are selected from a set of n objects, and if the order of selection is noted, then the selected set of r objects is called an order sample of size r. Sampling with replacement occurs when an object is selected and then replaced before the next object is selected. The number of possible ordered samples of size r taken from a set of n objects is n r. (multiplication principle)

28 sampling without replacement occurs when an object is not replaced after it has been selected By the multiplication principle, the number of possible ordered samples of size r taken from a set of n objects without replacement is n n 1 (n r + 1) = n! n r! = np r

29 often the order of selection is not important and interest centers only on the selected set of r objects we are interested in a number of subsets of size r that can be selected from a set of n different objects each of the n C r unordered subsets is called a combination of n objects taken r at a time, where nc r = n r = n! r! n r! numbers n r are called binomial coefficients (a + b) n = n r=0 n r br a n r

30 suppose that in a set of n objects, n 1 are similar, n 2 are similar, n s are similar, where n 1 + n n s = n the number of distinguishable permutations of the n objects is n n! n 1, n 2, n = s n 1! n 2! n s! the coefficients are sometimes called multinomial coefficients expansion (a 1 + a a s ) n

31 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

32 we are interested only in those outcomes which are elements of a subset B of the sample space S we are confronted with the problem of defining a probability set function with B as the new sample space for a given event A we want to define P(A B) the probability of A, considering only those outcomes of the random experiment that are elements of sample space B P A B = N(A B) N(B) = N(A B) N(S) N(B) N(S) = P(A B) P(B)

33 DEFINITION The conditional probability of an event A, given that event B has occurred, is defined by P A B = P(A B) P(B), provided that P(B) 0.

34 Conditional probability satisfies the axioms for a probability function, with P(B) 0 : 1. P(A B) 0 2. P(B B) = 1 3. if A 1, A 2, A k are mutually exclusive events, then P(A 1 A 2 A k B) = P(A 1 B) + P(A 2 B) + + P(A k B) for each positive integer k P(A 1 A 2 B) = P(A 1 B) + P(A 2 B) + for an infinite, but countable, number of events

35 once we know the conditional probability P(B A), we can determine the probability of the intersection of two events DEFINITION The probability that two events, A and B, both occur is given by the multiplication rule, P(A B) = P(A) P(B A), provided P(A) > 0 or by P(A B) = P(B) P(A B), provided P(B) > 0

36 the multiplication rule can be extended to three or more events P(A B C) = P[(A B) C] = P(A B) P(C A B) P(A B) = P(A) P(B A) P(A B C) = P(A) P(B A) P(C A B)

37 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

38 for certain pair of events, the occurrence of one of them may/may not change the probability of the occurrence of the other DEFINITION Events A and B are independent if and only if P(A B) = P(A) P(B). Otherwise, A and B are called dependent events. statistically independent stochastically independent independent in a probabilistic sense

39 THEOREM DEFINITION If A and B are independent events, then the following pairs of events are also independent: A and B A and B A and B Events A, B and C are mutually independent if and only if the following two conditions hold: 1. A, B, and C are pairwise independent, that is P(A B) = P(A) P(B), P(A C) = P(A) P(C), P(B C) = P(B) P(C) 2. P(A B C) = P(A) P(B) P(C)

40 Properties of Probability Methods of Enumeration Conditional Probability Independent Events Bayes Theorem

41 let B 1, B 2 B m constitute a partition of the sample space S S = B 1 B 2 B m and B i B j =, i j events B 1, B 2, B m are mutually exclusive and exhaustive suppose that the prior probability of the event B i is positive, P(B i ) > 0, i = 1, m if A is an event, then A is the union of m mutually exclusive events A = (B 1 A) (B 2 A) (B m A) P A = m i=1 P B i A = m i=1 P B i P A B i law of total probability

42 If P(A) > 0, then P B k A = P(B k A) P(A), k = 1,2, m Bayes theorem P B k A = P B k P A B k m, k = 1,2,... m P B i P A B i i=1 the conditional probability P B k A is often called the posterior probability of B k

43 EXAMPLE Compute the probability that building X collapses during the next earthquake in the region. We do not know exactly if the next earthquake will be strong, medium or weak, but we estimated the following probabilities: P(strong) = 0,01, P(medium) = 0,1, P(weak) = 0,89 Additionally, structural engineers have performed analyses and estimated following: P(collapse strong) = 0,9, P(collapse medium) = 0,2, P(collapse weak) = 0,01 law of total probability P(collapse) = P(collapse strong) P(strong) + P(collapse medium) P(medium) + P(collapse weak) P(weak) P(collapse) = 0,9. 0,01 + 0, ,01. 0,89 = 0,0379 l.o.t.p. allows us to break the problem into 2 parts (size of an earthquake, capacity of the building)

44 EXAMPLE Suppose that an earthquake occurred and building X collapsed. Bayes theorem P(strong collapse) = P(collapse strong)p(strong) P(collapse) = 0,9.0,01 0,0379 = 0.24 B.t. provides a valuable calculation approach for combining pieces of information to compute a probability that may be difficult to determine directly