Probability and Statistics

Transcription

1 RAMK fall Probability and Statistics with computer applications Contents Part A: PROBABILITY 1. Basics of Probability Classical probability Statistical probability Probability axioms Complement Multiplication rules Addition rules Law of total probability Bayes rule. Combinatorics Permutations k -permutations k- combinations Binomial coefficient 3. Discrete probability distributions probability mass function cumulative distribution function (cdf) Expected value Binomial distribution Poisson distribution 4. Continuous probability distributions probability density function PDF cumulative density function CDF expected value E(x) uniform distribution normal distribution N(, ) PART B: STATISTICS 5. Basic Concepts and Descriptive Statistics Population Sample Variables Variable types Statistical parameters Levels of measurement Measures of central tendency Measures of variation Graphical presentation 6. Grouped variables Frequency table Histogram Mean, median and mode Cumulative distribution function 7. Confidence intervals Confidence intervals : Normal distribution Confidence intervals : Students distribution Confidence intervals : relative proportions 8. Hypothesis testing One sample t- test Two sample t- test Chi square test 9. Statistical dependence Linear regression Non-linear regression Correlation coefficient 1

2 Evaluation Returned exercises 50 % Exams 50 % Scale: failed = 0, passed 1-5 A. Classical definition of probability Classical theory of probability applies to equally probable events Classical probability is the frequency of an event occurrences divided by total number of possible outcomes. This can only be used when all events are equally likely. P(A) = f / n A = event, n = number of equally likely outcomes, f = number of favorable (successful) outcomes = the size of the event Example 1: A box contains 4 blue balls, one yellow ball and 3 red balls. One ball is picked randomly from the box. What is the probability that the ball is a) blue b) yellow c) red? P(blue) = 4/8 P(yellow) = 1/8 P(red) = 3/8

3 Example : Matias flips two dice. What is the probability that the sum of the pair of dice is greater or equal to 10? Solution: The possible outcomes are all pairs of dice. 1+1 = 1+ = = = = = 7 +1 =3 + =4 +3 = 5 +4 = 6 +5 =7 +6 =8 3+1=4 3+=5 3+3=6 3+4=7 3+5=8 3+6=9 4+1=5 4+=6 4+3=7 4+4=8 4+5=9 4+6=10 5+1=6 5+=7 5+3=8 5+4=9 5+5=10 5+6=11 6+1=7 6+=8 6+3=9 6+4=10 6+5=11 6+6=1 The event sum >= 10 consist of six outcomes: P(sum of dice >=1) = 6/36 = 1/6 B. Statistical definition of probability Relative frequency = another term for proportion; it is the value calculated by dividing the number of times an event occurs by the total number of times an experiment is carried out. P(event) = relative frequency = f / n n = number of repetitions of an experiment f = number of occurrences of event E Example3. In a digital transmission of 10 MB file, 15 bits were transmitted incorrectly. What was the probability of bit error in transmission? Answer: P = 15/(10*10 6 *8) = 1.9*10-7 Example4. An engine failure occurs in a certain aeroplane model twice per 1000 flights. What is the probability of engine failure during a flight Answer: P(engine failure) = /1000 =

4 C. Kolmogorov s Probability axioms Let E = event and = set of all possible outcomes Probability P of event E satisfies following axioms: P(E) 0 P()= 1 P(E 1 E ) = P(E 1 ) + P(E ) +, when E 1 E = (Addition rule for exclusive events) Consequences: C1) If A B, then P(A) P(B) C) P() = 0 C3) 0 P(E) 1 for all events E COMPLEMENT RULE In probability theory the complement of event A is not A : i.e that the event A does not occur. Notation for complement of A ~A, A, A c or Complement rule: P(A c ) = 1 P(A) Proof: Because A and A c are distinct, P(A) + P(Ac) = 1 (Kolmogorov) => complement rule Example5: A person flips two coins. What is the probability of getting at least one head? Answer: if event A = at least one head Complement Ac = both are tails, with probability ½ * ½ = ¼. => P(at least one head) = P(A) = 1 P(A c ) = 1 ¼ = ¾. 4

5 MULTIPLICATION RULE FOR INDEPENDENT EVENTS If A and B are two independent event, the probability of the joint event (A B) is P(A B) = P(A)*P(B) Example 6: A school has 3 copying machines: A, B and C. The probabilities that the machines functions normally at a random time are A: 0.95 B: 0.85 C: 0.80 Calculate a) the probability that all machines function normally b) at least two of three machines function normally c) all machines are broken Each path presents one possible outcome or state. a) P(all ok) = 0.95*0.75*0.8 = = 65 % b) P( ok) = P(ABC)+P(A c BC)+P(AB c C)+P(ABC c ) = 0.95*0.85* *0.85* *0.15* *0.85*0. = = 96 % c) P(all fail) = 0.05*0.15*0. = = 0.15 % MULTIPLICATION RULE / CONDITIONAL PROBABILITY If probability of event A depends on event B, then P(A B) = P(A B)*P(B) where P(A B) = probability of A under condition that B is true Basicly this means that multiplication can be used as earlier. Only change is that the in the notation P(A) is changed to P(A B) which expresses dependence. Example 7: There are 3 blue and red balls in a box. A person takes randomly two balls from the box. Calculate a) the probability that both balls are blue. b) the probability that the balls are of different color Answers: a) P(both blue)=3/5*/4=0.3 = 30% b) P(diff.color) = /5*3/4 + 3/5*/4 = 3/10 + 3/10 = 0.6 = 60% 5

6 ADDITION RULE GENERAL CASE P(AB) = P(A) + P(B) P(AB) ADDITION RULE - EXCLUSIVE EVENTS P(AB) = P(A) + P(B) Exclusive events: A and B cannot be true simultaneously P(AB) = 0 Example 8 random cards are picked from deck of cards. What is the probability to get spade or picture card (ace included in pictures. spade Picture cards: jack, queen, king Answer: P(spade or picture ) = P(spade) + P(picture) P(spade and picture) = 13/5 + 1/5 3 / 5 = /5 LAW OF TOTAL PROBABILITY P(A) = P(A B)*P(B) + P(A ~B)*P(~B) Example 9: Assume that in EUROCUP match Real Madrid Chelsea winner is Real at probability 0.68 if Ronaldo plays, 0.43 if Ronaldo does not play. (even result is not possible in Cup). Ronaldo has been injured and he plays at 40 % probability. Calculate the winning probability of Real Madrid. P(Real wins) = 0.4* *0.43 = 0.53 = 53 % 6

7 BAYES THEOREM (Looks complicated but isn t) Example 10: A factory produces two kinds of light bulbs A and B, the shares of production are A: 70 % and B: 30 %. The percent of faulty bulbs are 4 % for A and % for B. Calculate probability that a bulb is type A if it is faulty. Tree Answer: P(type = A if bulb is faulty) = 0.08 / ( ) = 0.8 = 8 % 7

8 Combinatorics Concepts: Permutations k permutations k -combinations Permutation = an arrangement of elements of a set in a particular order A set of n distinct elements has n! permutations Example: Permutations of words are called anagrams. How many permutations has the word RAMK? 1st letter can be chosen in 4 ways, second in 3 ways, third in ways and last in 1 way =>The number of anagrams = 4*3**1 = 4 1

9 K - Permutation = an arrangement of k elements of a larger set of n distinct elements A set of size n has n(n-1) (n-k+1) = n!/(n-k)! k-permutations Example: In a classroom we have 30 students. How many queue s of length 4 can be formed. 1st student can be chosen in 30 ways nd student can be chosen in 9 ways 3rd student can be chosen in 8 ways 4th student can be chosen in 7 ways Number of different queues is 30*9*8*7 = n( n 1) ( n k 1) K - combination = a subset of size k of a set of n distinct elements A set of size n has n! k! n k! k -combinations Example: In Finnish LOTTO 7 randomly chosen numbers are drawn from 39. This 7 number subset is called LOTTO ROW. a) How many different lotto rows does exist? b) What is a probability of winning the JACKPOT (= 7 right) with one played row? a) Number of different LOTTO rows n n! 39*38*37*36*35*34*33 Binomial[ n, k] ( ) k k!( n k)! 1**3*4*5*6*7 b)winning probability P( 7 right ) =

10 Pascal s triangle binomial(3,)=3 binomial(5,) = 10 The n th row of Pascal s triangle contains numbers, which are the same as binomial coefficients ( n 0 ), (n 1 ), (n ), (n n ) Binomial(8,4) = 70 Binomial(n,k) in calculators ncr(n,k) TI-89 COMBIN(k;n) Excel binomial(n,k) wolframalpha.com Online calculator Product rule in combinatorics Example: If a person has pairs of shoes, 3 trousers, jackets, he can choose his /her clothing in *3* = 1 ways Example: There are 5 blue balls and 3 red balls in a box. 4 balls are picked randomly from the box. What is the probability of getting red and blue balls? nr.combination with red and blue balls Solution: P( red and blue) = nr of all combinations of 4 balls P = (5 )(3 ) ( 8 4 ) = =3 7 Other possible solution is to draw a 4 level tree, where the paths from top to down represent different outcomes. 3

11 Example: Calculate the probabilities for minor winnings: 6, 5 and 4 right numbers in LOTTO: P(6 right) = (7 6 )(3 1 ) ( 39 7 ) = P(5 right) = (7 5 )(3 ) ( 39 7 ) = P(4 right) = (7 4 )(3 3 ) ( 39 7 ) = Sums given by winnings right Euro 6 right 1999 Euro 5 right 5 Euro 4 right 10 Euro Example: Five cards ( a hand ) is drawn randomly from a deck of playing cards. Calculate the probabilities of a) flush b) four of a kind c) straight flush Total nr of hands: ( 5 5 )=.6 M a) P(royal flash)= 4: ( 5 5 ) = b) P(flash)= 4 ( 13 5 )/(5 5 ) = c) P(four of a kind) = 13 4/( 5 5 ) = d) P(straight flush) = 9*4/ ( 5 5 ) =

12 Discrete probability distributions Let x be a discrete random variable with possible values (x 1, x,, x n ). Let (p 1, p,, p n ) be the corresponding probabilities. The possible values of x together with their probabilities p define a discrete probability distribution. FUNCTIONS OF PROBABILITY DISTRIBUTION 1) Probability mass function p(x) The function which maps each variable value x onto its probability is called probability mass function p(x) Example: Two dice is tossed. Let x = sum of two dice. The possible outcomes are in the table on the right: Probability mass function : p() = 1/36, p(3) = /36,. is presented as bar graph 1

13 Cumulative distribution function F(x), or CDF(x) F(x) = P(x i x) F(x) = probability that variable value is less or equal to x Below CDF of sum of two dice Table of p(x) and cdf(x) Look from the table of cdf(x) what is the probability that a) sum of two dice 7 b) Sum of two dice > 9 P(x 7)= cdf(7) = 1/36 P(x>9)=1 cdf(9) = 1-30/36 = 6/36 DISTRIBUTION PARAMETERS A. Expected value = mean value of the random variable if one would repeat the random variable process infinitely. E(x) = m = Sp i x i In our example the expected value of the sum of two dice ( which means long term average of the sum) is following Expected value m = 1/36* + /36*3 + 3/36*4+4/36*5+5/36*6+6/36*7 +5/36*8+4/36*9+3/36*10+/36*11+1/36*1 = 7 Result: Expected Value of sum of two dice is 7, which is an obvious result, because the probability distribution is completely symmetric. B. Variance and standard deviation Var(x) = Sp i (x i -m) Standard deviation s = () Var and stdev measure the width of distribution

14 X P prob 0,4 0,35 0,3 0,5 0, 0,15 0,1 0,05 0 Probability distribution variable value Expected value of Bob s grade = E x) = p i x = =.35 ( i Example: Music festival on Midsummer Day is estimated to give 5000 Euro profit, if it does not rain, but Euro loss, if it rains. According to meteorological statistics in the festival area the probability of rain during Midsummer Day is 35 %. Is organizing the festival profitable in the long run or not? Reason your answer with calculation of Expected Return ER. Expected Return E(x) = p1*x1 + p*x = 0.65* *(-35000) = 4000 Euro 3

15 Business rule: Choose that option, which maximizes expected value of profit customers P How many turkeys he should buy weekly to maximize his profit? Solution: Calculate the expected profit for each option and choose the best customers P Buys 4 turkeys 4 sold P=100% Expected return ER = 4*30-4*15 = 60 Buys 5 turkeys Buys 6 turkeys 4 sold 15 % 4*30-5*15=45 5 sold 85% 5*30-5*15=75 4 sold 15 % 4*30-6*15=30 5 sold 40% 5*30-6*15=60 6 sold 45% 6*30-6*15=90 ER = 0.15* *75 = 70,5 ER = 0.15* * *90 = 69 Buys 7 turkeys 4 sold 15 % 4*30-7*15=15 5 sold 40 % 5*30-7*15=45 6 sold 30 % 6*30-7*15=75 7 sold 15% 7*30-6*15=105 ER = 0.15* * * *105 = 58.5 Answer: 5 turkeys per week gives best profit : average 70,50 per week 4

16 BINOMIAL DISTRIBUTION REPEATED EXPERIMENT Assume werepeat n times an experiment, which has a fixed probability of success p of failure it is 1-p). Each trial is independent of each other. Let x be the number of succeeded experiments in n trials. Then the probability that x has value k is P( x Example1: Assume that a family has 5 children. Calculate the probability that a) all are girls b) there are 3 boys and girls ( Probability of having a boy or girls are assumed to be equal: 50%) P(5 girls) = = = 3.1 % n p k P( girls) =( )*0.5 *0.5 3 = = 31.3 % k n k = ) = (1 ) k p Notation x ~BIN(n,p) means that x follows binomial distribution, where n = number of trials and p = probability of success Expected value and standard deviation It can be shown, that for binomial distribution expected value m and standard deviation can be calculated with following formulas If x ~ bin(n,p), then Expected value µ = n p Standard deviation s = n p( 1 p) 5

17 Law of big numbers Individual test outcome can be any of the possible outcomes. When the number of tests n grows, the relative proportions of different outcomes get closer and closer to the theoretical probabilities Example: If you throw 5 coins long enough the relative frequencies of different outcomes approach theoretical probabilities. Heads P In biathlonthe shooter must hit 5 targets. Biathlon example Kaisa M s probability to hit a single target is 80% Calculate a) the probabilities of all possible outcomes from 0 to 5 hits. b) Calculate also her expected number of hits Solution: Fill in probabilities into the table below Nr of hits Probability =0,0003 = 0.03 % 1 ( )0.8*0.4 = 0,0064 = 0.64% ( )0.8 *0. 3 = 0,051 = 5.1 % 3 ( )0.83 *0. = 0,05 = 0.5 % 4 ( )0.84 *0. = 0,410 = 41.0 % =0,38 = 3.8 % 6

18 EXPECTED VALUE m of binomial distribution variable is np If x ~BIN(n,p), then E(x) = n p In the example expected nr of girls in a 5 child family would be 0.5*5 =.5 Example: In football betting there are each week 13 matches, where one has to choose 1, x or 1 = home team wins, x = even, = visiting team wins Assume that you make a row randomly in such a way that 1, x and are equally probable. What is the probability to bet a) all 13 matches right b) 10 right. c) What is the expected result? a) P(13 right) = (1/3) 13 = 1 / b) P(10 right) = ( )(1/3)10 (/3) 3 = 88 / c) Expected result = n p = 13*(1/3) = 4 right POISSON DISTRIBUTION Poisson distribution gives the probability of a given number of events in a fixed time interval, if the average rate is known. Examples of usage: * number of phone calls coming to a call center * number of earthquakes * number of customers in the queue k = number of events in a fixed interval l= expected value of number of events Example: In a post office desk the average length of the queue during the day is.6 persons. Calculate the probability that a) there is no one in the queue b) the length of the queue is 1 person c) the length of the queue is 6 persons a) P(0) = m m! a) P(1) = m m! =..! =..! =0,074=7,4% =0.193=19,3% a queue P( 6) =1 P(0) P(1)-P()-P(3)-P(4)-P(5) = = = 5.8 % 7

19 Solution: Calculate the expected value for the period of time of the example: Expected number of eruptions in 10 years m= 0. (1/50 = 0./10) a) P(0) = m m! b) P(1) = m m! =..! =..! = 0,8 = 8% = 0,16 = 16% c) P( 1) = 1-P(0) = 1-0,8 = 18% Example: Adestructive tsunami (like 004) appears in average once in 500 years. Calculate the probability that there will be no such a tsunami during next 100 years. Solution: Expected number of tsunamis per 100 years l= 0. (One in 500 years means 0. in 100 years) k = 0 gives P(0) = 0. 0 *e -0. /0! = 0.8 = 8 % Below the probability mass function values and graph 8

20 Continuous distributions x = continuous variable P(x) = probability density function Probability density function p(x) Notation: pdf(x) or most often p(x) Continuous probability distribution is defined by probability density function p(x), which is proportional to the probability of value x. Probability density function fulfills requirements: p(x) 0 for all x R p( x) dx 1 1

21 Cumulative distribution function F(x) also CDF(x) CDF(X) gives the probability, that variable value is less or equal to X Z P( x Z) CDF( Z) p( t) dt CDF(Z) is equal to area below p(x), where x Z Graph of cumulative distribution function CDF values increase monotonically from 0 to 1 CDF(x) is used to calculate probabilities of x being within given intervals P(x b) = CDF(b) In the picture curve is P(x) and area F(x) = CDF(x) P(x b) = 1 - CDF(b) P(a x b) = CDF(b) CDF(a)

22 Calculation of parameters Expected value x p( x) dx Standard deviation ( x ) p( x) dx Normal ( Gaussian ) distribution N(,) p ( x) 1 ( x) e = expected value = standard deviation CDF( x) 1 x 0 e ( t ) 1 x dt (1 erf ( ) Function erf exists in only good calculators erf ( x) 1 x x e t dt p(x) is called bell shaped curve All gaussian curves are congruent with pdf of N(0,1) 3

23 Practical instructions: How to calculate pdf(x) or cdf(x) of normal distribution with given and? Option 1 : Online calculator Example : The length of male university students follows Normal Distribution x ~ N(18cm, 9 cm), (mean = 18 cm and standard deviation = 9 cm). Calculate a) Probability that the length of a male student is at least 00 cm b) Calculate such a length, that only 1% of students it taller than it? Solution: a) P(x00) = 1 cdf(00) =1-0,977=0,03 =.3% b) Solve Z from cdf(z) = 0.99 => Z = cdf -1 (0.99) = 03 cm Normal distribution is most well-known uniform distribution 4

24 Example using Excel: The length of male university students follows Normal Distribution x ~ N(18cm, 9 cm), (mean = 18 cm and standard deviation = 9 cm). Calculate a) Probability that the length of a male student is at least 00 cm b) Calculate such a length, that only 1% of students it taller than it? Solution: a) P(x00) = 1 cdf(00) cdf (also pdf) in Excel is NORM.DIST =1-0,977=0,03 =.3% b) Solve Z from cdf(z) = 0.99 => Z = cdf -1 (0.99) = 03 cm cdf -1 in Excel is NORM.INV Uniform distribution Also other continuous distributions exist Metro train passes a metro station at 5 min intervals. A passenger comes to the station at random time. Find the probability density function and cumulative density function p(x) and F(x) for the variable x = waiting time. Calculate also the expected value (average) of the waiting time Solution: Any waiting time between 0 5 min is equally probable. Thus p(x)= 1/5, when x [ 0, 5] and p(x) = 0 elsewhere. Expected value of waiting time p( x) xdx x xdx / min 5 end of probability 5