To understand and analyze this test, we need to have the right model for the events. We need to identify an event and its probability.

Transcription

1 Probabilistic Models Example #1 A production lot of 10,000 parts is tested for defects. It is expected that a defective part occurs once in every 1,000 parts. A sample of 500 is tested, with 2 defective parts found. Should you conclude that this is a bad lot? Is the number of defective parts within the tolerance? To understand and analyze this test, we need to have the right model for the events. We need to identify an event and its probability. Basic probability properties: 0 event cannot occur P (event) = 1 event must occur 0 < p(e) < 1 otherwise. If A is set of all possible events = P (A) = 1 If distinct events are disjoint = P (A 1 or A 2 or A 3 ) = P (A 1 ) + P (A 2 ) + P (A 3 ) P (A j ) = 1 j= all possible A1 A2 A3 A If all events are equally likely = P (A j ) = 1/n for n events. For example we can consider: A 1 = the first part defective A 2 = the second part defective 62

2 A 3 = only the third part defective Then A 1 and A 3 are disjoint = P (A 1 or A 3 ) = P (A 1 ) + P (A 3 ) Note: If A 1 and A 2 not disjoint = P (A 1 or A 2 ) P (A 1 ) + P (A 2 ) A1 A2 A3 A In this case we have P (A 1 or A 2 ) = P (A 1 ) + P (A 2 ) P (A 1 and A 2 ) In our example: The event space is all possible outcomes of the test - large and complicated! So we would like to break it down to smaller tests: testing each part individually. Defective/ Not Defective is similar to a coin flip: Fair Coin P (H) = 1/2 P (T ) = 1/2 Biased Coin P (T ) = p P (H) = 1 p = q It is useful to designate H, T (Success or Failure) with 1 and 0. Each test is then a Bernoulli trial with parameter p: P (1)p. Now we can calculate the probability of events involving more than one test of a part, by noting that the tests of the parts are assumed to be independent. Conditional Probability P (A B) = probability that event A occurs given B P (A, B) P (A B) = where P(A,B) = probability that both occur, and P(B) = probability only B P (B) occurs. 63

3 Simple example showing dependence: Flip a coin three times. Find the P(observe 2 heads second is a head). If we view all possible outcome ( for a fair coin ) we have: HHH HHT THH THT HTH HTT TTH TTT. The first four events are those with H as second outcome. Then P(observe 2 heads second is a head) = P(2 out of 3 second is head) = 2/4 = 1/2. Note: P(2 out of 3) is unconditional probability, i.e the probability of 2 successes in 3 Bernoulli trials. ( ) (1 3 2 ( 1 1 P(2 out of 3) = = 2 2) 2) 3 8 We can also calculate P(observe 2 heads second is a head) using P (2 out of 3 and second is H) P (second is H) = 2/8 1/2 = 1 2 Simple example showing independence: Successive outcomes of the coin flip (Bernoulli trial) are independent. So P (X 1 = H X 2 = T ) = P (X 1 = H) Then, it must be that P (X 1 = H X 2 = T ) = P (X 1 = H, X 2 = T ) P (X 2 = T ) = P (X 1 = H, X 2 = T ) = P (X 1 = H) P (X 2 = T ) In general the joint probability density for two independent random variables X 1, X 2 is written as P (X 1, X 2 ) = f(x 1, x 2 ) = f(x 1 )f(x 2 ) that is the density for the joint distribution is just the product of the individual densities. The statement of a problem can make a big difference: Example Given a room of n people, what is the probability that at least 2 have the same birthday? So we have to find P (X 2) where X is the number of people with same birthday. Stated in this way it is difficult because we have to consider all possible combinations. 64

4 But we can also state as 1 P (X = 1). So P (X 2) = 1 P (X = 1) where P (1) = P (the 2 nd person is different from the 1 st ) P (the 3 rd person is different from the 2 nd and 1 st ) P (the n th person is different from all the rest) ( = 1 1 ) ( 1 2 ) ( 1 n 1 ) P (X 2) n Recall our problem. Define X j = outcome of j th test. P (X j = 1) = p where p is the probability of defective part. Here success is the event of finding a defective part. Then if the individual tests are independent: P (X 1 = 1 and X 2 = 0) = P (X 1 = 1) P (X 2 = 0) Again, the treatment of and and or in probability of events can be viewed from the set point of view. P (X 1 = 1 or X 2 = 0) = P (X 1 = 1) + P (X 2 = 0) P (X 1 = 1 and X 2 = 0) Let s write the event space for the first two tests: P (X 2 = 0) }{{} P (X 1 = 1) P (X 1 = 1) = p P (X 2 = 0) = 1 p P (X 1 = 1 and X 2 = 0) = p(1 p) using independence P (X 1 = 1 or X 2 = 0) = p + (1 p) p(1 p) = 1 p(1 p) Note: we also have to be careful about specifying order: P (X 1 = 1 and X 2 = 0) P(1 success and 1 failure) 65

5 P(1 success and 1 failure) = P (X 1 = 1, X 2 = 0) + P (X 1 = 0, X 2 = 1) = 2p(1 p) So if we ask what is the probability of a defective part in first n trials, we need to take into account the number of ways that could occur: and so on Similarly, if we ask for P(two defective parts in n trials): and so on For any one of these sequences of n trials with j defective (successes) we have p j (1 p) n j for the probability for that particular sequence. Since the order does not affect this probability, and the trials are identical, we look at the number of combinations/permtations ( ) we can have in these sequences: n n! choose j from n = = j j!(n j)! So the probability for j defective in n trials is: P (j out of n) = ( n j ) p j (1 p) n j This is called the binomial distribution B(n, p) with parameters n and p, where n is the number of Bernoulli trials and p is the success for any 1 trail. Note: to use the binomial distribution we must have a sequence of tests which are: 1. Identical Bernoulli trials (only two possible outcomes) 2. Independent So if Y is the number of defectives found in a batch of size n, P (Y = j) = ( ) n P(all possible sequences with j successes) = p j (1 p) n j. j Back to our example: We have a test of 500 parts with 2 defectives, and we have to determine if that is enough to decide that we have a bad lot of parts. Then, not only do we have to calculate the probability of this event, but also give a tolerance for the number of defectives observed. Tolerance is topically defined in terms of 90%, or 95%, or 99% chance of observing some event, under the assumption 66

6 that the lot is good. First we calculate the probability of the obsserved event P(Y = 2) for Y B(500, 0.001). So P (Y = 2) = ( ) (0.001) 2 (0.999) How do we compare this to the tolerance? If the lot is good, then we would expect that the probability of observing k or more defectives is ( ) n P (Y k) = p j (1 p) n j j j=k Graphically, our probability mass function for the discrete binomial variable Y, P(Y = j) looks like: P(Y = j) j So P (Y k) is adding up the mass for all values j k Here we see: P (Y 1) = 1 P (Y = 0) = = P (Y 2) = 1 [P (Y = 0) + P (Y = 1)] = 1 ( ) = P (Y 3) = 1 [P (Y = 0)+P (Y = 1)+P (Y = 2)] = 1 ( ) = So if we define a 90 % tolerance - usually called a confidence interval - then this defines a rare event as an event that occurs with 10% probability. Similarly an event which occurs with 5% probability, defined as rare, corresponds to defining a 95% confidence interval. Thus for 90% confidence interval, we would conclude that our observation of 2 defectives is a rare event for a good batch. 67

7 For a 95% confidence interval, we would conclude that observing 2 defectives is not rare.. Thus the choice of the confidence interval influences how many false positives or false negatives we obtain in our testing. The choice depends on our tolerance of either one. So far our calculation of probability/confidence interval is based on discrete probability mass functions. We can generalize to other approximations by noting that we are calculating P(Y) where Y is a sum of identically distributed, independent random variables = X j = 0 or 1 with probability p. Y = X j = the number of successes, where the outcome for each trial is independent. There are many results for sums of random variables, the most famous being the Central Limit Theorem. This let us approximate the distribution of the sum with a norma distribution: [ ] [ ] Y N(µ, σ 2 ) where µ = E Y and σ 2 = V ar Y Let s first review Expected Value. Expected value is by definition the weighted average, where the weight is given by the probability mass function: P (X = x i ) = f(x i ) for discrete values X i of the random variables X. E[X] = all i x i f(x i ) E[g(X)] = all i g(x i ) f(x i ) for g a function For X a Bernoulli random variable, f(1) = p, f(0)= 1-p, so E[X] = 1 p+0 (1 p) = p For a binomial random variable ( ) n P (Y = j) = p j (1 p) n j j ( ) n E[Y ] = j p j (1 p) n j j j=0 This can be calculated by rearranging terms in the sum. We also note that since Y = n X k i.e. Y is the sum of the outcomes of n Bernoulli trials, then E[Y ] = E [ n X k ] 68

8 Since X k are identical, and we can commute the sums, [ n ] E X k = E[X k ] = ne[x k ] = np Similarly we can use this idea to calculate variance V ar[x] = E[(X E(X)) 2 ] = E[X 2 2XE[X] + (E[X]) 2 ] = E[X 2 ] (E[X]) 2 (since E[X] is just a number not a random variable). For the Bernoulli trials: E[X] = p V ar[x] = E[X 2 ] p 2 E[X 2 ] = 1 2 p (1 p) = p V ar[x] = p p 2 = p(1 p) So E[Y ] = E[X k ] since Y = X k Now we can calculate: [ n ] ( n V ar[y ] = V ar X k = E X k E Recall that [ n ] E X k = ne[x k ] = np [ n ]) 2 X k So we can write the variation as: ( n ) 2 V ar[y ] = E (X k p) = E k p) (X (X k p)(x j p) j k<j [ n ] = E (X k p) 2 + 2E (X k p)(x j p) j Note that = E[(X k p) 2 ] + 2 j k<j E[(X k p)(x j p)] k<j E[(X k p)(x j p)] = (x k p)(x j p)f(x j, x k ) all x k, x j = (x k p)(x j p)f(x j )f(x k ) all x k, x j 69

9 since x j, x k are independent (Bernoulli trials). So, E[(X k p)(x j p)] = E[X k p]e[x j p] = 0 since E[X k ] = p So, V ar[y ] = n E[(X k p) 2 ] = n V ar[x k ] = np(1 p) Thus the sum of independent random variables X i has expected value ne[x i ] and variance nv ar[x i ]. In addition the Central Limit Theorem gives an approximation of the density of sum of i.i.d. random variables. n lim = X i ne[x i ] N(0, 1) n n V ar[xi ] This says that the sum of n i.i.d. random variables X i approaches a normal distribution. A normal distribution with mean µ and variance σ 2 has probability density function of the form (y µ) 2 p(y) e 2σ 2 2πσ 2 Note: this is a continuous random variable. You can verify that : + p(y)dy = 1, E[Y ] = + By the definition of E[Y], E[Y µ] = 0, E[cY ] = ce[y ] = cµ V ar[cy ] = E[(cY cµ) 2 ] = c 2 V ar[y ] - < y < + yp(y)dy = µ, V ar[y ] = + (y µ) 2 p(y)dy = σ 2 So in the limit above, we subtract off the mean and divide by the standard deviation, leaving us with a random variable with mean 0, and variance 1. The proof that the density tends towards a normal distribution is not covered here, but the implications are significant: The Central Limit Theorem (CLT) says that we can take any random variable which has bounded mean and variance (the random variables can be discrete or continuous) and if we take a sum of these random variables, as n the density of the sum will be normal. 70

10 Then we can approximate the probability of observations for the sums by using the normal distribution. For example, in our previous example, we considered the probability that P ( n X i > k) for some k. The CLT says we can consider instead: Xi ne[x i ] P (Z) = n V arxi P(z) 0 z Then = P P ( Xi ne[x i ] > k ne[x ) i] n V arxi n V arxi ( Z > k ne[x i] n V arxi ) = P (Z > z) Notice that in this case the comparison is between n X i a sum of discrete random variables which take only positive values, and Z, a continuous random variable with range over all reals. So we would expect that this approximation may not be valid for all values of Y = n X i for finite n. Then P (Z > z) we can identify using p(z) Typically we call the range of likely variation a confidence interval, which is then defined in terms of values of Z. The confidence interval could be one-sided or two-sided, depending on the application. We can compare our previous results to approximation with the normal distribution: P (Y 1) = (binomial) P (Y 1) = (normal) FIGURE 71

11 P(z) P(Z > z*) 0 z* z Example #2 Suppose a bus can arrive at any time between 11:00 am and 11:15 am, with equal probability. If you arrive at the bus stop at 11:00 am, with what probability will you wait for 10 minutes or more for the bus? With what probability will you wait a total of 300 minutes over the whole month (assuming you arrive at 11:00 am every day)? For which value of total minutes would you question the validity of the bus schedule? Let X = the waiting time of the bus in one day Then P (X > 10) = 1 P (X 10) = 1 Where f(x) is a uniform probability density given by 1 0 x 15 f(x) = 15 0 otherwise. 10 Let Y = total amount of time waiting = 30 X i. Then Where P (Y > 300) = P ( E[X i ] = V ar[x i ] = X i > 300) = P x 15 dx = 15 2 (x 15/2) 2 15 and 0 f(x)dx ( Xi ne[x i ] > 300 ne[x ) i] n V arxi n V arxi dx = 1 15 [x3 3 15x x 2 4 ] =

12 Then P (Y > 300) = P ( X i > 300) = P Z > = P (Z > ) < 0.01 So if we could choose a 90% or 95% confidence level, that would be P (Z > 1.28) 0.90 P (Z > 1.6) 0.95 Note that is much larger that these levels. Then we could conclude that we have observed rare event, given the assumption about the bus arrivals, so we could question this assumption. Example #3 Test a drug on 100 patients, with probability p of benefit from the drug. How many patients would we expect to observe receiving benefits from the drug in order to accept the assumed effectiveness of it? In this case we have to choose the confidence level and determine the number of patients that satisfy that confidence level. If Y is the number of patients receiving a benefit, this says we want to find y such that P (Y y) = 0.95 Then we need to identify E[Y] and Var[Y], and in particular we would like to identify Y as a sum of random variables. Here Y = X i wherex i is the Bernoulli trial for each individual patient. So E[X i ] = p = E[Y ] = 100p and V ar[y ] = 100p(1 p). = ( ) Y 90 P 1.65 = = Y So, in order to accept the drugs effectiveness of.9, we would want to see more than 85 patients receiving benefits from the drug. Least Squares - Linear Regression Data:- view as a random variable or as a function + random variable at each data point. 73

13 Let Y i = the data points Y i = f(t) + ε i where ε i = error (usual assumption ε i N(0,σ)) What is σ. It depends on f(t). The data points fit a linear function f(t) = a + bt Y i t In general, we would like to minimize errors ε i. In fact, we will minimize variance about a + bt ( like the mean ). = F (a, b) = (y i (a + bt i ) 2 ) F n a = 2(y i (a + bt i )( 1)) = 0 F n b = 2(y i (a + bt i )( t i )) = 0 y i = na + b t i t i y i = a t i b t 2 i Solve for a and b: n y i b n a = t i n b = n n y i t i n y i n t i n( n t 2 i ) ( n t i ) 2 74

14 Note ŷ i = a + bt i is the estimate of y i Then R 2 variation of estimate n = variation of actual data = (ŷ i ȳ) 2 n (y i ȳ) 2 We can apply the Least square fit to the following data: x i = y i = Then a = , b = 3.065, and R 2 = x 2 i = 3671 x i y i = Example(From Larsen&Marx) Crickets make their chirping sound by sliding one wing cover very rapidly back and forth over the other. Biologists have long been aware that there is a linear relationship between temperature and frequency with which a cricket chirps, although the slope and y-intercept of the relationship varies from species to species. Listed below are 15 frequency-temperature observations recorded for the striped ground cricket, Nemobius fasciatus fasciatus. Plot these data and find the equation of the least-square line, y= a+bx. Suppose a cricket of this species is observed to chirp 18 times per second. What would be the estimated temperature? Observation number Chirps per second (x i ) Temperature,y i ( 0 F )

15 Note that X and Y do not have to be related linearly in order to use linear regression. Example If y Ae bx, then we can take logarithms of both sides to obtain linear problem. = ln y = ln(ae bx ) = ln A + Bx. So, W = ln A + Bx Then we can apply the linear regression formulae to W, and x ln a = Wi b x i n b = Wi x i y i xi n( x 2 i ) ( x i ) 2 Example(From Larsen&Marx) Mistletoe is a plant that grows parasitically in the upper branches of large trees. Left unchecked, it can seriously stunt a tree s growth. Recently an experiment was done to test a theory that older trees are less susceptible to mistletoe growth than younger trees. A number of shoots were cut from 3-,4-,9-,15-, and 40-yearold Ponderosa pines. These were then side-grafted to 3-year-old nursery stock and planted in a preserve. Each tree was inoculated with mistletoe seeds. Five years later, a count was made of the number of mistletoe plants in each stand of trees. (A stand consisted of approximately ten trees; there were three stands of each of the four youngest age groups and two stands of the oldest.)the results are shown below: Age of Trees x (years) Number of Mistletoe Plants,y So if we try to approximate mistletoe data using y = a + bx we get a = , b = , and R 2 = For y = ax b, we use z = ln y, v = ln x, α = ln a = z = ln a + b ln x = ln a + bv = α + bv. Then we get α = 4.9, b= , and r 2 = Note: Here r 2 was calculated using the linear formulation. n r 2 (ẑ i z) 2 = n (z i z) 2 where ẑ i = α + bv i and ŷ i = e α x b i For y = ae bx, w = lny = w = ln a + bx 76

16 Then we get ln a = 3.56, b= r 2 = n (ŵ i w) 2 n (w i w) 2 Again r 2 was calculated using the linear formulation where w i = ln a + bx i, and y i = e (ln a+bx i) 77