Set Notation and Axioms of Probability Memory Hints: Intersection I AND I looks like A for And Union U OR + U looks like U for Union Complement NOT X = X = X' NOT NOT X = X = X'' = X Commutative Law A AND B = B AND A A I B = B I A A B = B A A OR B = B OR A A U B = B U A A + B = B + A Distributive Law (A AND B) OR C = (A OR C) AND (B OR C) (A I B) U C = (A U C) I (B U C) (A B) + C = (A + C) (B + C) (A OR B) AND C = (A AND C) OR (B AND C) (A U B) I C = (A I C) U (B I C) (A + B) C = (A C) + (B C) A = (A AND B) OR (A AND B) B = (A AND B) OR (A AND B) A = (A I B) U (A I B) B = (A I B) U (A I B) A = (A B) + (A B) B = (A B) + (A B) DeMorgan s Law _ (A AND B) = A OR B (A OR B) = A AND B _ (A I B) = A U B (A U B) = A I B _ (A B) = A + B (A + B) = A B Axioms of Probability P(A) + P(A) = 1 P(A) = 1 - P(A) 0 < P(A) < 1 Σ P(A i ) = 1
Joint Probabilities Addition Rule P(A OR B) = P(A) + P(B) - P(A AND B) Mutually Exclusive If A and B are mutually exclusive then A AND B = P(A AND B) = 0 P(A OR B) = P(A) + P(B) Conditional Probability The conditional probability of an event A given an event B, denoted P(A B) is P(A B) = P(A AND B) / P(B) The conditional probability of an event B given an event A, denoted P(B A) is P(B A) = P(B AND A) / P(A) Multiplication Rule P(A AND B) = P(B AND A) P(A AND B) = P(A B) * P(B) P(B AND A) = P(B A) * P(A) P(A B) * P(B) = P(B A) * P(A) Bayes Theorem P(A B) = P(B A) * P(A) / P(B) P(A) = P(B) * [ P(A B) / P(B A) ] P(B A) = P(A B) * P(B) / P(A) P(B) = P(A) * [ P(B A) / P(A B) ] Total Probability Rule P(A) = P(A AND B) + P(A AND B) = P(A B) * P(B) + P(A B) * P(B) P(B) = P(B AND A) + P(B AND A) = P(B A) * P(A) + P(B A) * P(A) Joint Probabilities Dependent (Without Replacement) P(A AND B) = P(A) * P(B A) = P(B) * P(A B) Independent (With Replacement) P(A AND B) = P(A) * P(B) Independence Two events are independent if any one of the following equivalent statements is true. P(A B) = P(A) P(B A) = P(B) P(A AND B) = P(A) * P(B) If any one of the above statements is true, then all of them are true.
Probability Distribution Functions Discrete Probability Functions Continuous Probability Functions Probability Mass Function f(x) Probability Density Function f(x) f(x i ) 0 f(x) 0 Σ f(x i ) = 1 f(x) dx = 1 f(x i ) = Prob (X = x i ) Prob (a X b) = f(x) dx Prob (X = x) = 0 Cumulative Distribution Function F(x) Cumulative Distribution Function F(x) F(x) = P(X x) = Σ f(x i ) F(x) = P(X x) = f(x) dx 0 F(x) 1 0 F(x) 1 Mean & Variance Mean & Variance μ = E(x) = Σ x f(x) μ = E(x) = x f(x) dx σ 2 = V(x) = E(x - μ) 2 σ 2 = V(x) = E(x - μ) 2 σ 2 = Σ [(x - μ) 2 f(x)] = Σ x 2 f(x) - μ 2 σ 2 = (x - μ) 2 f(x) dx = x 2 f(x) dx - μ 2
Methods of Counting Multiplication Rule For a process or operation with a sequence of k steps or alternatives, the total number of ways to complete the process is n 1 x n 2 x n 3 x... x n k Permutations A permutation is an ordered sequence of the elements of a set. The number of permutations of n different elements is n! = n (n - 1) (n -2) (n -3)... (3) (2) (1) Example for the elements a, b, c, d: abcd, abdc, acbd, acdb, adbc, adcb, bacd, badc, bcad, bcda, bdac, bdca, cabd, cadb, cbad, cbda, cdab, cdba, dabc, dacb, dbac, dbca, dcab, dcba For n = 4 n! = 4 x 3 x 2 x 1 = 24 The number of permutations of subsets of r elements from a set of n different elements is n P = r n! (n r)! Example the four letters a, b, c, d taken two at a time, the number of permutations is 4! 4! 4x3x2x1 12 (4 2)! (2)! 2 x1 4 P = = = = 2 ab, ac, ad, ba, bc, bd, ca, cb, cd, da, db, dc The number of permutations of n = n 1 + n 2 +... + n r objects of which n 1 are of one type, n 2 are of a second type,..., and n r of are an r th type is n! n 1!n 2!n 3!... n r! Example: three O's and two X's (n = 3 + 2 = 5) Permutations = 5! 120 = = 10 (3!) (2!) (6)(2) OOOXX, OOXOX, OOXXO, OXOOX, OXOXO, OXXOO, XXOOO, XOXOO, XOOXO, XOOOX Combinations A combination is an unordered subset of r elements from a set of n elements. The number of combinations, subsets of size r that can be selected from a set of n elements, is n C = r n! r!(n r)! Example the four letters a, b, c, d taken two at a time, the number of combinations is 4! 4! 2!(4 2)! (2)!(2)! 4 C = = = 2 6 ab, ac, ad, bc, bd, cd
Combinations & Permutations Permutations - "Rearranging" Examples: 1234 is different than 4312 ABC is different than CAB For a set of n items taken all at once, the number of permutations = n! For a set of n items taken r at a time (without repetitions) i.e., 11123 not allowed npr = n! / (n - r)! Examples: For A, B C, D taken three at a time ABC, ACB, ADE, BAC, BCA, DEA, etc, etc, etc. Combinations - "Group" where "Order" is not important; i.e., ABC is the same as BCA For a set of n items taken all at once, the number of combinations is 1. For a set of n items taken one at a time, the number of combinations is n. For a set of n items taken r at a time, the number of combinations is ncr = n! / [(r!)(n - r)!]
Bernoulli Processes and Binomial Distributions Conditions: 1. Only two possible outcomes per trial ("success" or "failure") 2. n independent trials 3. Probability of success on any one trial is constant = p 4. The random variable X is the number of successes in n trials, such that Probability Mass Function n n! = = C C = x x!(n x)! n x (n x) p(x x) p (1 p) where x Mean: μ = E(X) = np Variance: σ 2 = V(X) = np(1 - p)