MS-A0504 First course in probability and statistics

Transcription

1 MS-A0504 First course in probability and statistics Heikki Seppälä Department of Mathematics and System Analysis School of Science Aalto University Spring 2016

2 Probability is a field of mathematics, which investigates the behaviour of mathematically defined random phenomena. Statistics attempts to describe, model and interpret the behaviour of observed random phenomena.

3 Learning Outcomes After passing the course the student knows 1. the basic concepts and rules of probability 2. the basic properties of one- and two-dimensional discrete and continuous probability distributions 3. common one- and two-dimensional discrete and continuous probability distributions and knows how to apply them to simple random phenomena 4. the basic properties of the bivariate normal distribution 5. the basic methods for collecting and describing statistical data 6. how to apply basic methods of estimation and testing in simple problems of statistical inference 7. the basic concepts of statistical dependence, correlation and linear regression.

4 Content I Probability Week 1 Concept of probability and basic rules. Statistical independence. Conditional probability. Discrete random variabled Week 2 Continuous random variables, distributions and statistics. Generating functions. One dimensional distributions. Week 3 Random vectors and their distributions. Multidimensional distributions. II Statistics Week 4 Measurement and description of statistical data. Sample and sample distribution. Statistical estimation. Week 5 Statistical significance. Statistical testing. Week 6 Statistical dependence and correlation. Linear regression with one explanatory variable.

5 Course arrangements Lecturer: Heikki Seppälä Assistants: Hoa Ngo Zhe Chen Lectures : Mondays at 10 12, lecture hall M1 Fridays at 10 12, class Y405 Exercises: weekly 2 x 2h Completion: Exercises and 2 mid-term exams or the final exam. More information:

6 Course material on the web Main material Lecture slides Exercises Statistical tables Additional material C M Grinstead & J L Snell Introduction to Probability and Statistics. MIT lecture notes. DeGroot & Schervish Probability and Statistics

7 Any questions about the course? Lecturer: Heikki Seppälä Main assistant: Hoa Ngo MyCourses:

8 MS-A0504 First course in probability and statistics Week 1: Basics of probability Heikki Seppälä Aalto University

9 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

10 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

11 Random phenomena Random phenomenon is a phenomenon, realizations of which are uncertain Realization is an observable outcome of the random phenomenon. Sample space S is the set of possible realizations of the random phenomenon. Events or subsets of the sample space, A S, correspond to observations of random phenomenon. Interpretation Event A occurs, when the realization s A. Full subset of S is certain event. Empty set is impossible event.

12 Example. Dice Realization i = outcome of a roll of a die Sample space S = {1, 2,..., 6} Events are the subsets of S, e.g., A = outcome is even = {2, 4, 6}. B = outcome is > 4 = {5, 6}.

13 Example. Rolling two dice Realization (i, j), where i and j are outcomes of 1. and 2. roll respectively. Sample space is S = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6), (5, 1), (5, 2), (5, 3), (5, 4), (5, 5), (5, 6), (6, 1), (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}. Events are all subsets of S, e.g., A = outcomes are the same = {(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)}. B = outcome of the 1. roll is 1 = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6)}.

14 Example. Rainfall tomorrow in Espoo (mm) Realizations are real real numbers x 0. Sample space S = {x R : x 0}. Events are e.g. A = rainfall tomorrow is more than 10 mm = (10, ) B = no rain tomorrow = {0}

15 Combining events We can form new events from events by using logical rules: A and B occur A or B occur A does not occur B occurs, but A does not In order to determine the probabilities, we must express the events using set theory.

16 Intersection of events Event A and B occur includes those realizations that belong to sets A and B: A B = {s S : s A and s B}. Example (Dice) A = Outcome is > 3 = {4, 5, 6} B = Outcome is even = {2, 4, 6} A B = Outcome is > 3 and even = {4, 6}

17 Union of events Event A or B occurs includes those realizations that belong to set A or B: A B = {s S : s A or s B}. Example (Dice) A = Outcome is > 3 = {4, 5, 6} B = Outcome is even = {2, 4, 6} A B = Outcome is > 3 or even = {2, 4, 5, 6}

18 Complement of event Event A does not occur includes those realizations that do not belong to set A: A c = {s S : s A}. Example (Dice) A = Outcome is > 3 = {4, 5, 6} A c = Outcome is 3 = {1, 2, 3}

19 Difference of events Event B occurs but A does not includes those realizations that belong to set B but not to set A: B \ A = {s S : s B ja s A}. Example (Dice) A = Outcome is > 3 = {4, 5, 6} B = Outcome is even = {2, 4, 6} B \ A = Outcome is even and 3 = {2}

20 Mutually exclusive events Events A and B are mutually exclusive, if only one of them can occur, that is, A B =. Events A 1, A 2,... are mutually exclusive, if only one of them can occur A i A j = whenever i j. Example (Dice) A = Outcome is even. Events A and A c are mutually exclusive. A i = Outcome is i. Events A 1, A 2,..., A 6 are mutually exclusive.

21 Combining events Summary Interpretation Certain event Impossible event A occurs A and B occur A or B occur A does not occur B occurs but A does not A and B are mutually exclusive Set theory expression S A A B A B A c B \ A A B =

22 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

23 Concept of probability The phrase probability is widely used the real world: Probability of rain tomorrow is 5 %. A new elementary particle has been found in CERN with probability %. A criminal was jailed with probable causes. It is very difficult to give a generally satisfactory scientific definition for probability. David Aldous: Annotated list of contexts where we perceive chance

24 Empirical probability Suppose we have n independent observations of a random phenomenon (on stable conditions). The relative frequency of event A is the ratio f A n, where f A is the number of observations in which A occurs In certain contexts we can assume that the relative frequency approaches the limit f A n p A, as n. If this limit p A exists, we call it empirical probability of A and in this case p A f A n.

25 Example: Coin flip The relative frequency of heads as the number of flip increases. Relative frequencies of heads and tails from 1000 flips.

26 Example: Dice The relative frequency of outcome 6 as the number of flips increases. Relative frequencies of all outcomes from 1000 flips.

27 Empirical probability Restrictions Determining the relative frequency requires repeated empirical experiments Nothing guarantees that the limit of relative frequency exists We can not deal with the situations where observations are unavailable Example Probability that a nuclear plant functions without problems for ten years. Probability that the sequence of bits transmitted by wireless base station arrives at terminal without errors. Probability of passing this course on the first attempt.

28 Symmetric probability If each realization of finite sample space is equally probable, then the probability of event A S can be naturally defined via Pr(A) = n(a) n(s), where n(a) is the number of elements in set A. Symmetric random phenomena Coin flip Dice National lottery Non-symmetric random phenomena Falling pin Darts Sports betting

29 Discrete uniform distribution Mapping A Pr(A) = n(a) n(s) It satisfies the following conditions: is the uniform distribution of set S. (i) Probability of the certain event is Pr(S) = 1. (ii) For each event A it holds that 0 Pr(A) 1. (iii) For mutually exclusive events A 1,..., A k it holds that Pr(A 1 A k ) = Pr(A 1 ) + + Pr(A k ). Remark In the definition above the sample space S must be finite.

30 Combinatorial probability If a random phenomenon on finite state space S is symmetric (i.e. uniformly distributed), we can in principle calculate all the probabilities directly using formula Pr(A) = n(a) n(s). When A (or S) is large it can be very difficult or even impossible to calculate the number of observations n(a). Combinatorics is the field of mathematics that concentrates on this sort of problems. Example (Difficult combinatorial problem) What is the probability that you find a 10 steps long path from perfectly random net that consists of 10 9 knots and links?

31 Two basic results of combinatorics Fact (Ordered lists) We can pick k elements into an ordered list from a set of n elements in n(n 1) (n k + 1) different ways. In particular, elements from a set of n elements can be ordered in n! = n(n 1) 1 different ways. Fact (Unordered subsets) We can form an unordered subset of k elements from a set of n elements in ( ) n n! = k k!(n k)! different ways.

32 Example: National lottery What is the probability to get 7 right with one coupon? Sample space of Finnish national lottery is S = subsets of 7 elements from the set {1,..., 39} and its size is n(s) = ( 39 7 ). Event A = chosen lottery coupon has 7 numbers right includes exactly one realization so that n(a) = 1. National lottery is uniformly distributed due to symmetry and hence Pr(A) = n(a) n(s) = ( ) =

33 General probability Probability distribution, i.e., probability measure on sample space S is a mapping, that attaches the value Pr(A) to each event A S, and satisfies: (i) Probability of a certain event is Pr(S) = 1. (ii) For each event A it holds that 0 Pr(A) 1. (iii) For any finite or infinite sequence of mutually exclusive events A 1, A 2,... it holds Pr(A 1 A 2 ) = Pr(A 1 ) + Pr(A 2 ) + Remark Property (iii) is the sum rule of mutually exclusive events. Discrete uniform distribution satisfies conditions (i) (iii) so it is a probability distribution.

34 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

35 The sum rule of mutually exclusive events Calculation rule For mutually exclusive events A 1, A 2,..., A k it holds that Pr(A 1 A 2 A k ) = Pr(A 1 ) + + Pr(A k ). Proof. The claim is same as property (iii) of probability distribution. Example There are n = 200 members of parliament in Finland, n SDP = 42 and n LA = 12. Randomly chosen member of parliament is a member of SDP or Left Alliance (Vasemmistoliitto) with probability Pr( SDP or LA ) = Pr( SDP ) + Pr( LA ) = =

36 Probability of complement Calculation rule The probability of complement of A is Pr(A c ) = 1 Pr(A). Proof. We have that A A c = S, and since A and A c are mutually exclusive, we have 1 (i) = Pr(S) = Pr(A A c ) (iii) = Pr(A) + Pr(A c ). Example Probability that we get at least one heads with two coin flips is Pr( at least 1 heads ) = 1 Pr( two tails ) = = 3 4.

37 Probability of impossible event Calculation rule The probability of an impossible event is Pr( ) = 0. Proof. Since is the complement of the certain event S, we see by applying calculation rule for complement and property (i) that Pr( ) = 1 Pr(S) (i) = 1 1 = 0. Example Randomly chosen member of parliament is a member of SDP and NCP (Kokoomus) with probability Pr( SDP and NCP ) = Pr( ) = 0.

38 Probability of difference Calculation rule Probability of event B occurs but A does not is Pr(B \ A) = Pr(B) Pr(A B). Proof. By writing B as a union of mutually exclusive events B \ A and A B we have, Pr(B) = Pr((B \ A) (A B)) (iii) = Pr(B \ A) + Pr(A B). Example Let us choose randomly a card from the deck: Pr( card is spade but not face card ) = Pr( spade ) Pr( spade and face ) = =

39 General sum rule Calculation rule For the union of events A and B it holds that Pr(A B) = Pr(A) + Pr(B) Pr(A B). Proof. Writing A B = (A \ B) (A B) (B \ A), we have Pr(A B) (iii) = Pr(A \ B) + Pr(A B) + Pr(B \ A). By calculation rule for difference event Pr(A \ B) = Pr(A) Pr(A B), Pr(B \ A) = Pr(B) Pr(A B). The claim follows by adding and simplifying these.

40 Monotonicity of probability Calculation rule If A B, then Pr(A) Pr(B). Proof. In this case A B = A and by the calculation rule for difference 0 Pr(B \ A) = Pr(B) Pr(A B) = Pr(B) Pr(A). Example (Random card from the deck) If the card is spade, it is also black. Therefore = Pr( spade ) Pr( black ) =

41 Basic rules of probability Summary General sum rule: Pr(A B) = Pr(A) + Pr(B) Pr(A B). Sum rule of mutually exclusive events: Pr(A B) = Pr(A) + Pr(B), kun A B =. Probabilities of complement and difference: Pr(A c ) = 1 Pr(A), Pr(B \ A) = Pr(B) Pr(A B). Monotonicity: Pr(A) Pr(B), if A B.

42 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

43 Conditional probability Conditional probability of event A on condition that B occurs is defined via formula Pr(A B) = Pr(A B), Pr(B) 0. Pr(B) If Pr(B) = 0, then Pr(A B) is not defined.

44 Example. Parliament 83 of 200 members of the parliament are women. SDP has 34 members in the parliament, 21 of which are women. Randomly chosen member of the parliament is a member of SDP with probability Pr( SDP ) = = What is the probability that a randomly chosen female member of the parliament is a member of SDP? Pr( SDP and female ) Pr( SDP female ) = Pr( female ) = 21/200 83/

45 General product rule Calculation rule Whenever Pr(A) 0, the general product rule holds Pr(A B) = Pr(A) Pr(B A). Interpretation The probability of the joint event A and B occur can be calculated by multiplying the probability of event A by the conditional probability of event B on condition that event A occurs. Proof. By the definition of conditional probability Pr(A B) Pr(A B) = Pr(A) = Pr(A) Pr(B A). Pr(A)

46 Product rule of multiple events Calculation rule Whenever Pr(A 1 A k 1 ) 0, the general product rule holds: Pr(A 1 A k ) = Pr(A 1 ) Pr(A 2 A 1 ) Pr(A 3 A 1 A 2 ) Pr(A k A 1 A k 1 ). Interpretation The probability of joint event each of events A 1,..., A k occurs can be calculated by multiplying: the probability of A 1, the conditional probability of A 2 on condition A 1, the conditional probability of A 3 on condition A 1 and A 2,... the conditional probability of A k on condition A 1, A 2,..., A k 1 occurs.

47 Product rule Example Let us pick 3 cards from the deck without replacement. What is the probability that all the cards are spades? A i = card i is spade A = A 1 A 2 A 3 By the general product rule Pr(A) = Pr(A 1 ) Pr(A 2 A 1 ) Pr(A 3 A 1 A 2 ) = Alternative combinatorial way: S = unordered subsets of 3 cards, n(s) = ( ) Realizations of event A corresponds to 3 card subsets from set of spades. Number of these is n(a) = ( ) 13 3 kpl. The random phenomena is uniformly distributed based on symmetry so that Pr(A) = n(a) ( 13 ) n(s) = 3 ( 52 3 ) =

48 Statistical dependence and independence Events A and B are independent if Pr(A B) = Pr(A) Pr(B). Collection of events {A i, i I } is independent if for all i 1, i 2,..., i k I. Pr(A i1 A ik ) = Pr(A i1 ) Pr(A ik ) Example Situations where independence is intuitively clear: Consecutive coin flips as long as flips are high enough. Sampling with replacement: pick coupons from an urn such that the coupon is returned to the urn and mixed before the next pick.

49 Independence and conditional probability Fact If Pr(A) 0 and Pr(B) 0, the following assertions are equivalent: A and B are independent. Pr(A B) = Pr(A). Pr(B A) = Pr(B). Interpretation Pr(A B) Pr(A) means that information about occurrence of B contains information that can be used for determining the probability of A. Proof. Nice exercise.

50 Example: Deck of cards Let us pick a card randomly. A = card is spade B = card is ace Are A and B dependent or independent? Let s check if Pr(A B) = Pr(A) Pr(B). Pr(A) = = 1 4. Pr(B) = 4 52 = Pr(A B) = Pr( card is ace of spades ) = Since Pr(A B) = Pr(A) Pr(B), we see that A and B are independent.

51 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

52 Formula of total probability Collection of mutually exclusive events, union of which is S, is called decomposition of sample space S. Calculation rule If B 1,..., B n form a decomposition and Pr(B i ) 0 for all i, then Pr(A) = n Pr(B i ) Pr(A B i ). i=1

53 Proof. Events C i = A B i are mutually exclusive and their union is A. Sum rule of mutually exclusive events and the product rule Pr(A B i ) = Pr(B i ) Pr(A B i ) imply ( n ) Pr(A) = Pr C i = i=1 n Pr(C i ) = i=1 = n Pr(A B i ) i=1 n Pr(B i ) Pr(A B i ). i=1

54 Formula of total probability: Example Suppose we know that 75% of women and 15% of men have long hair. Approximately 27 % of engineering students are women. What is the probability that an engineering student passing by has long hair? H = { student has long hair } N = { student is woman } M = { student is man } Since N and M are complements of each other, they decompose the sample space. Formula of total probability yields Pr(H) = Pr(N) Pr(H N) + Pr(M) Pr(H M) = (1 0.27) 0.15 =

55 Bayes formula Can we determine the conditional expectation Pr(B A) if we know Pr(A B), Pr(A) 0, and Pr(B) 0? Calculation rule (Bayes formula) Pr(B A) = Pr(A B) Pr(B). Pr(A) Proof. Definition of conditional probability implies that Pr(B A) = Pr(A B) Pr(A) = Pr(A B) Pr(B) Pr(B) Pr(A) = Pr(A B)Pr(B) Pr(A).

56 Bayes formula: Example Suppose we know that 75% of women and 15% of men have long hair. Approximately 27 % of engineering students are women. What is the probability that an engineering student with long hair is woman? H = { student has long hair } N = { student is woman } M = { student is man } We know that Pr(H N) = 0.75 Pr(N) = 0.27 Pr(H) = (previous example) Applying Bayes formula Pr(N H) = Pr(H N) Pr(N) Pr(H) = %.

57 Extended Bayes formula Suppose that B 1,..., B n form a decomposition of the sample space and that probabilities Pr(A B i ) and Pr(B i ) 0 are known. Can we determine inverse conditional probabilities Pr(B i A) from these? Fact (Extended Bayes formula) If Pr(A) 0, then Pr(B i A) = Pr(A B i ) Pr(B i ) n j=1 Pr(A B, i = 1,..., n. j) Pr(B j ) Proof. Recall formula of total probability: Pr(A) = n j=1 Pr(A B j) Pr(B j ). Now the Bayes formula proved earlier implies Pr(B i A) = Pr(A B i) Pr(B i ) Pr(A) = Pr(A B i ) Pr(B i ) n j=1 Pr(A B j) Pr(B j ).

58 Interpretation of Bayes formula Pr(B i A) = Pr(A B i ) Pr(B i ) n j=1 Pr(A B, i = 1,..., n. j) Pr(B j ) Numbers Pr(B i ) are called prior probabilities prior (latin) previous, earlier. Our understanding of the probability of event B i before we know if A occurs or not. Numbers Pr(B i A) are called posterior probabilities posterior (latin) following, later. Updated understanding of probability of B i after we know the occurrence of A.

59 Example: Quality control of factory Factory manufactures same product in two product lines. Finished products are mixed and packed into boxes. Line 1 manufactures 3 products/min, 5 % of which are faulty. Line 2 manufactures 5 products/min, 8 % of which are faulty. We randomly inspect a product from a randomly selected box. What is the probability that the product is from line 1? What is the probability that the product is from line 1, given that it is faulty?

60 Example: Quality control of factory Solution Line 1 manufactures 3 products/min, 5 % of which are faulty. Line 2 manufactures 5 products/min, 8 % of which are faulty. Known probabilities: B 1 = Product is from line 1, Pr(B 1 ) = 3/8 B 2 = Product is from line 2, Pr(B 2 ) = 5/8 A = Product is faulty, Pr(A B 1 ) = 0.05, Pr(A B 2 ) = 0.08 Events B 1 and B 2 form a decomposition of the sample space so that extended Bayes formula yields Pr(A B 1 ) Pr(B 1 ) Pr(B 1 A) = Pr(A B 1 ) Pr(B 1 ) + Pr(A B 2 ) Pr(B 2 ) /8 = / /

61 Example: Quality control of factory Summary Factory manufactures same product in two product lines. Finished products are mixed and packed into boxes. Line 1 manufactures 3 products/min, 5 % of which are faulty. Line 2 manufactures 5 products/min, 8 % of which are faulty. Prior probabilities of the product under inspection are: Product is from line 1 with probability 3/8 = 37.5 % Product is from line 2 with probability 5/8 = 62.5 % Posterior probabilities of the product under inspection (after observation that the product is faulty) are: Product is from line 1 with probability 27.3 % Product is from line 2 with probability 72.7 %

62 Calculation rules of probability Summary Sum rule Pr(A B) = Pr(A) + Pr(B) Pr(A B) Product rule = Pr(A) + Pr(B) (if A and B are mutually exclusive) Pr(A B) = Pr(A) Pr(B A) Total probability = Pr(A) Pr(B) (if A and B are independent) Pr(A) = i Pr(B i ) Pr(A B i ) (if B i s form a decomposition) Bayes formula Extended Bayes formula Pr(B A) = Pr(B i A) = Pr(A B i) Pr(B i ) j Pr(A B j) Pr(B j ) Pr(A B) Pr(B) Pr(A) (if B i s form a decomposition)

63 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

64 Random variable Random variable is a measurable 1 mapping X : S S, that attaches the value X (s) S to each realization s S of the random phenomenon. Interpretation Realization s S is randomly determined. Realization s determines the value X (s) of the random variable. Value of X is a is the event {X = a} := {s S : X (s) = a}. X belongs to set A is the event {X A} := {s S : X (s) A}. 1 X is measurable if we can define a probability for event {X A} whenever the set A S is regular enough.

65 Different types of random variables Random variable X : S S can be called random number, when S R random vector, when S R n random matrix, when S R m n random net, when S {nets with n knots} stochastic process, when S {functions f : R R} In this course we concentrate on random numbers (i.e. real valued random variables) and random vectors on R 2.

66 Example: Dice, three rolls Let us roll a die three times on row and denote: X = outcome of roll 1 Y = sum of outcomes Z = largest outcome Realizations of random phenomenon are the ordered sequences of three elements s = (s 1, s 2, s 3 ), where s i {1,..., 6}, and the sample space S is the collection of these sequences. X, Y, Z are random variables defined on sample space S : X (s) = s 1, Y (s) = s 1 + s 2 + s 3, Z(s) = max{s 1, s 2, s 3 }. Remark If we know the realization of a random phenomenon, then we know values of all the related random variables.

67 Random variable: Interpretation Random variable is a measurable function X : S S, that attaches the value X (s) S to each realization s S. Random variables are observations of random phenomenon. If we know the realization s S of a random phenomenon, then we know the values of all related random variables. Probability studies the probabilities of values of random variables under the assumption that the probability distribution Pr of the sample space S is known. In statistics we try to draw conclusions about the unknown probability distribution Pr of the sample space S based on observed values of random variables.

68 Distribution of random variable The distribution of a random variable X P X (A) := Pr(X A) tells what is the probability that X gets values in A. Fact Distribution P X of random variable X is a probability distribution on the range of X. Hence we can apply general rules of probability to distribution P X, e.g., P X (A c ) = Pr(X A c ) = Pr(X / A) = 1 Pr(X A) = 1 P X (A).

69 Contents Random phenomena, realizations and events Empirical, symmetric and general probability Basic rules of probability Conditional probability and independence Total probability and Bayes formula Concept of random variable Discrete random variables

70 Discrete random variable Random variable is discrete, if its range can be expressed as S = {x 1,..., x n } or S = {x 1, x 2, x 3,... }. The probability mass function of discrete random variable X f (x i ) = Pr(X = x i ) tells what is the probability that the value of X equals x i. Probability mass function determines the distribution of random variable, that is, we can calculate the probability of event {X A} using the formula Pr(X A) = f (x i ). i:x i A

71 Discrete uniform distribution Discrete random variable X has the uniform distribution on set {x 1,..., x n } if its mass probability function is given by f (x i ) = 1, i = 1,..., n. n Example The probability mass function of the random variable X, that indicates the outcome of a symmetric die, is f (k) = Pr(X = k) = 1, k = 1,..., 6. 6 Discrete random variable X has the uniform distribution on set{1,..., 6}.

72 Binomial distribution Discrete random variable X has the binomial distribution with parameters n and p if the range of X is {0, 1,..., n} and probability mass function is ( ) n f (k) = p k (1 p) n k, k = 0, 1,..., n. k Example If X is the number of sixes in three consecutive rolls of a die, then f (k) = Pr(X = k) = ( 3 k ) ( 1 6 ) k ( 1 1 6) 3 k, k = 0, 1, 2, 3. Discrete random variable X has the binomial distribution with parameters n = 3 and p = 1 6.

73 Multinomial distribution Discrete random vector (X 1,..., X k ) has the multinomial distribution with parameters (p 1,..., p k ) and n if its range is { x {0,..., n} k : x x k = n } and the probability mass function is f (x) = n! x 1! x k! px 1 1 px k k. Example What is the probability that n = 10 rolls of a die gives exactly 2 ones and 3 sixes? X 1 = number of ones, X 2 = number of sixes, X 3 = n X 1 X 2. It can be shown that (X 1, X 2, X 3 ) has multinomial distribution with parameters (p 1, p 2, p 3 ) = (1/6, 1/6, 4/6) and n = 10. Asked prob. = n! x 1!x 2!x 3! px1 1 px2 2 px2 2 = 10! 2!3!6! (1 6 ) 2 (1 6 ) 3 (4) If X 1 has binomial distribution with parameters n and p, then (X 1, n X 1 ) has multinomial distribution with parameters (p 1, p 2 ) = (p, 1 p) and n.

74 Next week: continuous random variables, their distributions and generating functions.....

75 Literature Slides are based on the slides from previous years (Ilkka Mellin, Milla Kibble, Juuso Liesiö, Lasse Leskelä, Kalle Kytölä).