Statistical Theory 1

Transcription

1 Statistical Theory 1 Set Theory and Probability Paolo Bautista September 12, 2017

2 Set Theory We start by defining terms in Set Theory which will be used in the following sections. Definition 1 A set is a collection of objects. These objects are called points or elements of the set. Sets are denoted by capital latin letters. If x is an element in a set A, then we write x A. Definition 2 A set A is a subset of a set B (A B) if every element of A belongs to B. Definition 3 Two sets A and B are equal, (denoted A = B) if every element of A belongs to B and every element of B belongs to A.

3 Set Theory Definition 4 A set which has no elements is called the empty set or null set, denoted by {} or. Definition 5 The compliment of a set A (denoted A C ), is the set containing all elements in a pre-defined universal set (denoted U), which are not in A. The following are operations defined for sets. Definition 6 The union of A and B (denoted A B), is the set containing all elements belonging to either A or B, i.e. A B = {x x A or x B}.

4 Definition 7 Set Theory The intersection of A and B (denoted A B), is the set containing all elements belonging to both A and B, i.e. A B = {x x A and x B}. Example 8 Let A = {2, 4, 6, 8, 10} and B = {1, 3, 5, 7, 8}. Then A B = {1, 2, 3, 4, 5, 6, 7, 8, 10} while A B = {8}. Remarks: i) If two sets A and B have an intersection of an empty set, i.e., A B =, then the sets are called disjoint sets. ii) A A B and B A B. iii) A B A and A B B.

5 Definition 9 Set Theory The set difference between A and B (denoted A B), is the set of all elements of A that do not belong to B, i.e. A B = {x x A and x B}. Example 10 Using the sets in Example 8, we can see that A B = {2, 4, 6, 10}.

6 Set Theory Theorem 11 The following are properties of any sets A, B, or C. 1 Commutative Laws: A B = B A and A B = B A 2 Associative Laws i) A (B C) = (A B) C ii) A (B C) = (A B) C 3 Distributive Laws i) A (B C) = (A B) (A C) ii) A (B C) = (A B) (A C) 4 (A C ) C = A. 5 A U = A and A U = U. Similarly, A = and A = A. 6 A A C = and A A C = U. Also, A A = A and A A = A. 7 De Morgan s Law i) (A B) C = A C B C ii) (A B) C = A C B C 8 A B = A B C.

7 Definition 12 Set Theory Let I be an index set and let {A i i I } be a collection of subsets of U indexed by I. We define the following sets: i) i I A i = the set of all points that belong to A i for at least one i, and ii) i I A i = the set of all points that belong to A i for every i. Theorem 13 ( De Morgan s Theorem) The following is a generalization of De Morgan s Law. ( i) ii) A i)c = i I i I ( A i)c = i I i I A C i A C i

8 Theorem 14 If A B, then i) A B = A, and ii) A B = B. Set Theory Theorem 15 If A U and B U, then i) A = (A B) (A B C ), and ii) (A B) (A B C ) =, i.e., A = (A B) + (A B C ). Next we define common terms used in the study of Statistics. Later on, we will compare the parallelism between the notions in Set Theory and Statistics.

9 Set Theory Definition 16 A statistical experiment refers to any activity which generates chance outcome. Definition 17 The sample space (denoted Ω), is the set of all possible outcomes of an experiment. Any element in the sample space is called a sample point (denoted ω).

10 Example 18 Set Theory Suppose we have the experiment of tossing a fair die. Then the sample space would be Ω 1 = {1, 2, 3, 4, 5, 6}. If we have the experiment of tossing two coins, then the sample space would be Ω 2 = {HH, HT, TH, TT }. Definition 19 An event (denoted by capital latin letters), is any subset of the sample space. The null event refers to an event containing no elements. Also, an event E occurs when the outcome of the experiment is a sample point contained in E. Thus, the null event is the event which is not possible to occur.

11 Example 20 Set Theory Refer to the first experiment and sample space Ω 1 defined above. Furthermore, let A denote the event of getting a prime number, B the event of getting a number between 1 and 2, and C the event of getting an even number. Then i) A = {2, 3, 5} ii) B = {} iii) C = {2, 4, 6} iv) A C =the event of getting an even prime number = {2} Definition 21 Two events A and B are called mutually exclusive events (MEE) if the two events cannot occur at the same time, i.e., they have an empty intersection.

12 Definition 22 Set Theory An event space is a collection of subsets of the sample space (denoted by script latin letters), satisfying the following properties (suppose A is an event space): i) Ω A, ii) If A A, then A C A. iii) If A 1 A and A 2 A, then A 1 A 2 A. Remarks: 1 Properties (i) to (iii) are the properties of an algebra or field. Hence, an event space is an algebra. 2 If Property (iii) is replaced by: If A 1, A 2,... A, then A i A, then A is a sigma-algebra. 3 If A is a sigma-algebra, then A is an algebra. i=1

13 Example 23 Set Theory The following are examples of event spaces: 1 A 1 = {, Ω} This is called the trivial event space, and is the smallest possible event space. 2 P(Ω) or 2 Ω This is the power set of the sample space, and is the largest possible event space. 3 A 2 = {, Ω, A, A C }, where A A. Theorem 24 The null set is in an event space, i.e., A. Theorem 25 If A 1, A 2 A, then A 1 A 2 A.

14 Theorem 26 If A 1, A 2,..., A n A, then n i) A i A, and ii) i=1 n A i A. i=1 Set Theory The following table now shows the parallelism between Set Theory and Probability Theory.

15 Set Theory Set Theory Probability Theory Universal Set, U Sample Space, Ω Set E Occurrence of an Event E Compliment E C Non-occurrence of an Event E Element, x Sample Point, ω Union, A B Occurrence of at least one of A or B Intersection, A B Occurrence of both A and B Disjoint Sets Mutually Exclusive Events Difference A B Occurrence of A but not B Null Set, Null Event, (impossible event) Subset, A B Occurrence of A implies the occurrence of B De Morgan s Law (i), (A B) C = A C B C Occurrence of at most one of A and B De Morgan s Law (ii), (A B) C = A C B C Event where neither A nor B occur (A B) (B A) Occurrence of exactly one of A and B

16 Definition 27 Set Theory A function f ( ) with domain A and codomain B is a collection of ordered pairs (a, b) satisfying the following: i) a A and b B, and ii) Each a A occurs as the first element of some ordered pairs in the collection. iii) No two distinct ordered pairs in the collection have the same first element. Remarks: If (a, b) f, or f (a) = b, then 1 b is the value of f at a, or the image of a under f, 2 a is the pre-image of b, and 3 the set of all values of f ( ) is called the range of f, and the range is a subset of the codomain B.

17 Set Theory Definition 28 Let Ω be any space with points ω and A any event of Ω. The indicator function of A, denoted by I A ( ), is the function with domain Ω and codomain {0, 1} defined by: { 1, if ω A I A (ω) = 0, if ω A Example 29 Let f (x) = 1 x, if x < 0 3, if 0 < x 1 x 2, if x > 1 Then using indicator functions, f (x) = (1 x)i (,0) (x) + 3I (0,1] (x) + x 2 I (1,+ ) (x).

18 Properties of Probability Theorem 30 Let Ω be any space with points ω and A an event space with elements A i, where i = 1, 2,..., n. Then, i) I A (ω) = 1 I A C (ω), n ii) I n i=1 A (ω) = I Ai (ω), i i=1 iii) I n i=1 A i (ω) = max{i A1 (ω), I A2 (ω),..., I An (ω)}, and iv) (I A (ω)) 2 = I A (ω), A A. Probability is generally known as the chance or likelihood that a certain event will occur. There are three ways on how to compute probabilities, and these are defined below.

19 Definition 31 Properties of Probability ( Classical Approach) If a random experiment can result in n mutually exclusive and equally likely outcomes and if n A of these outcomes have an attribute A, then the probability of A is given by: P C (A) = n A n. Remark: For any event A, 0 P(A) 1. Example 32 In the experiment of tossing a fair die, there are n = 6 equally likely outcomes. If A = prime number, then n A = 3 and P(A) = 3/6 = 1/2. It is important to note that the classical approach is not applicable when n is infinite or when the possible outcomes of the experiment are not equally likely.

20 Definition 33 Properties of Probability ( Frequency Approach) If an event is performed n times under similar conditions, and when n is large and n A of the outcomes have the attribute A, then Remark: P C (A) = lim n P F (A) Example 34 P F (A) = n A n. If a defective coin was tossed 100 times under similar conditions, and a tail occured 63 times, then P(T ) = 63/100 = Definition 35 ( Subjective Approach) This approach refers to the use of intuition, personal beliefs or expert s opinion and other indirect information in assigning probability.

21 Properties of Probability Given our knowledge of functions, we can now define a probability function, which will help us compute probabilities for statistical experiments. Definition 36 Let Ω denote the sample space and A denote an event space (assumed to be a sigma-algebra) for some random or statistical experiment. A probability function P[ ] is a set of functions with domain A and codomain [0, 1] which satisfy the following axioms: i) P[A] 0, A A ii) P[Ω] = 1 iii) If A 1, A 2,... is a sequence [ of mutually exclusive events in A, and if ] A i A, then P A i = P[A i ]. i=1 i=1 i=1

22 Example 37 Properties of Probability Consider the experiment of tossing two fair coins. Then Ω = {(H, H), (H, T ), (T, H), (T, T )} and P[(H, H)] = P[(H, T )] = P[(T, H)] = P[(T, T )] = 1/4. Show that the function P[ ] for this experiment is a probability function. Example 38 Let Ω be the set of positive integers, and A the collection of all subsets of Ω. Define P[.] on A as P[i] = 1, i = 1, 2, 3,... 2i Show that P[.] is a probability function.

23 Definition 39 Properties of Probability A probability space is the triplet (Ω, A, P[ ]) where Ω is a sample space, A is an event space (assumed to be an algebra), and P[ ] is a probability function with domain A. The probability space is a term that will make it convenient to note the presence of its three components. The three components are all related to each other, and the probability space ties these notions together. Next we show some important properties of the probability function P[ ]. For each of the following theorems, we assume the presence of a probability space (Ω, A, P[ ]). Theorem 40 P[ ] = 0.

24 Properties of Probability Theorem 41 If A 1, A 2,..., A n are mutually exclusive events in A, then [ n ] n P A i = P[A i ]. i=1 i=1 Theorem 42 If A A, then P[A C ] = 1 P[A]. Theorem 43 If A, B A, then P[A] = P[A B] + P[A B C ]. Theorem 44 For any two events A, B A, P[A B] = P[A] + P[B] P[A B]. Similarly, if we have three sets A, B, and C, we have P[A B C] = P[A] + P[B] + P[C] (P[A B] + P[A C] + P[B C]) + P[A B C].

25 In general, we have Conditional Probability and Bayes Theorem [ n n i=1 P[A i ] i<j ] P A i = i=1 P[A i A j ]+ [ n P[A i A j A k ]...+( 1) n+1 P A i ]. i<j<k i=1 Theorem 45 If A, B A and A B, then P[A] P[B]. Theorem 46 ( Boole s Inequality) If A 1, A 2 A, then P[A 1 A 2 ] P[A 1 ] + P[A 2 ].

26 Definition 47 Conditional Probability and Bayes Theorem Let (Ω, A, P[ ]) be a probability space, and A, B A. The conditional probability of A given B has occurred is given by P(A B) = P[A B], P[B] > 0. P[B] From this definition we can derive the Multiplication Rule: P[A B] = P[A]P[A B], P[B] > 0. In general, we have P[(A 1 A 2... A n )] = P[A 1 ]P[A 2 A 1 ]P[A 3 (A 1 A 2 )]... P[A n (A 1 A 2... A n 1 )]. Note that when we compute the probability P[A B], we assume that the experiment has already resulted in the event B. Thus B becomes our reduced sample space.

27 Example 48 Conditional Probability and Bayes Theorem Suppose we have the experiment of tossing two six-sided dice. Let A be the event of getting a 3 on the first die, and B the event of getting a sum of 8 from the two dice. Compute the ff: 1 P[A] 2 P[B] 3 P[B A] Example 49 Show that P[ B] is a probability function in A. Next we show some properties of P[ B]. These theorems are easy to show if we think of B as our smaller or reduced sample space. Suppose we have a probability space (Ω, A, P[ ]) and B A such that P[B] > 0. Then we have the following:

28 Theorem 50 P[ B] = 0 Conditional Probability and Bayes Theorem Theorem 51 If [ A 1, A 2,..., A n A is a series of mutually exclusive events, then n ] n P A i B = P[A i B]. i=1 i=1 Theorem 52 If A A, then P[A C B] = 1 P[A B]. Theorem 53 If A 1, A 2 A, then P[A 1 B] = P[(A 1 A 2 ) B] + P[(A 1 A C 2 ) B].

29 Conditional Probability and Bayes Theorem Theorem 54 If A 1, A 2 A, then P[(A 1 A 2 ) B] = P[A 1 B] + P[A 2 B] P[(A 1 A 2 ) B]. Theorem 55 If A 1, A 2 A and A 1 A 2, then P[A 1 B] P[A 2 B]. Theorem 56 [ n ] If A 1, A 2,..., A n A, then P A i B i=1 n P[A i B]. i=1

30 Example 57 Conditional Probability and Bayes Theorem A student is randomly selected from a group of 200 classified according to course and year level as follows: Freshmen Sophomore Junior Total Math Physics Total Assuming the 200 students have an equal chance of being selected, compute the ff: 1 P[S] 2 P[M] 3 P[F C ] 4 P[J C ] 5 P[(F S)] 6 P[(F S)] 7 P[(F C S)] 8 P[(S M)] 9 P[(F C S)] 10 P[(S M)] 11 P[(M S)] 12 P[(M C F )] 13 P[(P J C )] 14 P[(F J) M] 15 P[(F S) M C ]

31 Theorem 58 Conditional Probability and Bayes Theorem ( Theorem of Total Probability) Let (Ω, A, P[ ]) be a probability space. If B 1, B 2,..., B n A is a sequence of mutually exclusive events satisfying n i=1 B i = Ω and P[B i ] > 0 for i = 1, 2,..., n, then for any event A A, [ n ] n n P[A] = P (A B i ) = P[A B i ] = P[B i ]P[A B i ]. i=1 i=1 i=1 Corollary 59 Let (Ω, A, P[ ]) be a probability space. If B A such that 0 < P[B] < 1, then for every A A, P[A] = P[(A B) (A B C )] = P[(A B)] + P[(A B C )]

32 Theorem 60 Conditional Probability and Bayes Theorem ( Bayes Theorem) Let (Ω, A, P[ ]) be a probability space. If B 1, B 2,..., B n A is a sequence of mutually exclusive events satisfying n i=1 B i = Ω and P[B i ] > 0 for i = 1, 2,..., n, then for any event A A such that P[A] > 0, P[B k A] = P[A B k] P[A] = P[B k]p[a B k ] n P[B i ]P[A B i ] i=1 Corollary 61 Let (Ω, A, P[ ]) be a probability space. If A and B are events such that A, B A and P[A] > 0 and 0 < P[B] < 1, then P[B A] = P[A B] P[A] = P[B]P[A B] P[B]P[A B] + P[B C ]P[A B C ]

33 Example 62 Independence of Events (p.37 #25 MGB) There are five urns labeled 1 to 5, and each urn has 10 balls. Urn i has i white balls and 10 i red balls. Consider the following random experiment: Pick an urn at random and a ball is picked at random from that urn. 1 What is the probability that the chosen ball is white? 2 Given that the ball picked is white, what is the probability that it came from urn no. 5? Definition 63 Let (Ω, A, P[ ]) be a probability space. Two events A and B are said to be independent if any of the following holds: i) P[A B] = P[A]P[B] ii) P[A B] = P[A], if P[B] > 0, or iii) P[B A] = P[B], if P[A] > 0.

34 Theorem 64 Independence of Events Let (Ω, A, P[ ]) be a probability space, and A and B be two independent events. Then i) A and B C are independent ii) A C and B are independent ii) A C and B C are independent Definition 65 Let (Ω, A, P[ ]) be a probability space. Events A 1, A 2,..., A n A are said to be independent if P[(A 1 A j )] = P[A i ]P[A j ], i j P[(A i A j A k )] = P[A i ]P[A j ]P[A k ], i j k [ n. P i=1 A i ] = Π n i=1p[a i ].

35 Independence of Events All of these assumptions must be true in order for the n events to be independent. Note also that pairwise independence does not imply independence for the whole group, and independence by groups of three does not imply pairwise independence. Example 66 Consider the experiment of tossing two dice. Let A be the event of getting an odd sum, B the event of getting a 1 on the second die, and C the event of getting a sum of seven. Which of the following are independent: A and B, A and C, or B and C? Exercises: 1 An urn contains three red balls, two white balls, and one blue ball. A second urn contains one red ball, two white balls, and three blue balls. A ball is then selected from each urn. a. Describe a sample space for the experiment. b. Find the probability that both balls are of the same color. c. Is the probability that both balls will be red greater than the probability that both will be white?

36 Independence of Events 2 Given P[A] = 0.5 and P[(A B)] = 0.6, find P[B] if a. A and B are mutually exclusive. b. A and B are independent. c. P[A B] = Four drinkers, (say I, II, III, and IV) are to rank three different brands of beer (say A, B, and C) in a blindfold test. Each drinker ranks the three beers 1 (for the beer he likes best), 2 or 3, and then the assigned ranks of each brand of beer are summed. Assume that the drinkers really cannot discriminate between beers so that each is assigning his rankings at random. a. What is the probability that beer A will receive a total score of 4? b. What is the probability that some beer will receive a total score of 4? c. What is the probability that some beer will receive a total score of 5 or less? 4 A certain computer program will operate using either of two subroutines, say A and B, depending on the problem; experience has shown that subroutine A will be used 40 percent of the time and B will be used 60 percent of the time. IF A is used, then there is a 0.75 probability that the program will run before its time limit is exceeded,

37 Independence of Events and if B is used, there is a 0.50 probability that it will do so. What is the probability that the program will run without exceeding the time limit? 5 Suppose it is known that a fraction of the people in a town have tuberculosis. A tuberculosis test is given with the following properties: If the person does have TB, the test will indicate it with a probability If he does not have TB, then there is a probability that the test will erroneously indicate that he does. For one randomly selected person, the test shows that he has TB. What is the probability that he really does? 6 Five percent of the people in a city have high blood pressure. Of the people with high blood pressure, 75% drink alcohol whereas only 50% of the people without high blood pressure drink alcohol. If one drinker is selected at random, what is the probability that he has high blood pressure? 7 Assume that the conditional probability that a child born to a couple will be male is 1/2 + mε 1 f ε 2, where ε 1 and ε 2 are small constants, m is the number of male children already born to the couple, and f is the number of female children already born to the couple.

38 Independence of Events a. What is the probability that the third child will be a boy given that the first two are girls? b. Find the probability that the first three children will be all boys. c. Find the probability of at least one boy in the first three children. Express your answers in terms of ε 1 and ε 2. 8 Prove the following: a. If P[A B] > P[A], then P[B A] > P[B]. b. If P[A C ] = α, and P[B C ] = β, then P[A B] 1 α β.