Probability calculus and statistics

Transcription

1 A Probability calculus and statistics A.1 The meaning of a probability A probability can be interpreted in different ways. In this book, we understand a probability to be an expression of how likely it is that an event will occur. Let us look at an example. Let A represent the event that a patient develops an illness, S, over the next year, when the patient shows symptoms, V. We do not know if A will occur or not there is uncertainty associated with the outcome. However, we can have an opinion on how likely it is that the patient will develop the illness. Statistics show that about 5 out of 100 patients develop this illness over the course of 1 year if they show the symptoms V. Is it then reasonable to say that the probability that A will occur is equal to 5%? Yes, if this is all the information that we have available, then it is reasonable to say that the probability that the patient will become ill next year is 0.05 if the symptoms V are present. If we have other relevant information about the patient, our probability can be entirely different. Imagine, for example, that the particular patient also has an illness B and that his/her general condition is somewhat weakened. Then it would be far more likely that the patient will develop illness S. The physician who analyses the patient may, perhaps, assign a probability of 75% for this case: For four such cases that are relatively similar, he/she predicts that three out of four cases will develop the illness. To make this a bit more formalised, we let P(A K) represent our probability that event A will occur, based on the background knowledge K. Often, we simplify the formula and write only P(A). It is then implicit that the probability is based on the background knowledge K. If we say that the probability is 75%, then we Risk Analy sis: A sse ssing Unc e rtaintie s be y ond Ex pe c te d Value s and Probabilitie s 2008 John Wiley & Sons, Ltd. ISBN: T. Aven

2 168 PROBABILITY CALCULUS AND STATISTICS mean that it is just as likely for event A to occur as it is to draw a red ball out of an urn that contains three red balls and one white ball. The uncertainty is the same. We see that we can understand a probability also as being an expression of the uncertainty about what the outcome will be. It is, however, easier to think of probability as an expression of how likely it is that the event will occur. Based on this line of thought, a correct or true probability does not exist. Even if one throws a die, there is no correct probability. This may seem strange, but one must differentiate between proportions, observed or imaginary, and probability in the meaning in which we use the term here. Imagine throwing a dice a great many times say, 6000 times. We would then obtain (if the dice is normal ) about 1000 showing a 1, about 1000 showing a 2, and so on. In the population of 6000 throws, the distribution will be rather similar to 1/6 for the various numbers. But, imagine that we did an infinite number of trials. Then the theory says that we would obtain exactly 1/6. However, these are proportions, observed and resulting from imaginary experiments. They are not probabilities in our way of thinking. A probability applies to a defined event that we do not know will occur or not, and which is normally associated with the future. We will throw a dice. The dice can show a 4, or it can show a different number. Prior to throwing the dice, one can express one s belief that the dice will show a 4. As a rule, this probability is set at 1/6, because it will yield the best prediction of the number of fours if we make many throws. Using a normal (fair) dice, we will calculate that four will be the outcome in about 1/6 of cases in the long run. However, there is nothing automatic in our assignment of the probability 1/6. We have to make a choice. We are the ones who must express how likely it is to obtain a four, given our background knowledge. If we know that the dice is fair, then 1/6 is the natural choice. However, it is possible that one is convinced that the dice is not fair, and that it will give many more fours than usual. Then we may, for example, assign a probability P( four ) = 0.2. No one can say that this is wrong, even though, one can check the proportion of fours for this dice in retrospect and verify its normality. When one originally assigned the probability, the background knowledge was different. Probability must always be seen in relation to the background knowledge. Classical statistics builds on an entirely different understanding of what probability is. Here, a probability is defined as a limit of a relative frequency, meaning the proportion given above when the number of trials become infinitely large. In this manner true probabilities are established. These are then estimated using experiments and analyses. The reader is referred to Aven (2003) for a discussion of this approach and the problems associated with it (see also Section 13.7). A.2 Probability calculus The rules of probability are widely known. We will not repeat them all here, but will only briefly summarise some of the most important ones. The reader is referred to textbooks on probability theory.

3 PROBABILITY CALCULUS AND STATISTICS 169 Probabilities are numbers between 0 and 1. If the event A cannot occur, then P(A) = 0, and if A will occur for certain, then P(A)= 1. If the probability of an event is p, the probability that this event does not occur, is 1 p. Ifwehavetwo events, A and B, then the following formula holds: P(A or B) = P(A)+ P(B) P(A and B) P(A and B) = P(A)P(B A). (A.1) Here P(B A) represents our probability for B when it is known that A has occurred. If A and B are independent, then P(B A) = P(B); in other words, the fact that we know that A has occurred does not affect our probability that B will occur. Suppose that we want to express the probability that two persons will both develop the illness S, if they both have the symptoms V. In other words, we would like to determine P(A 1 and A 2 K) where A 1 represents patient 1 becoming ill and A 2 represents patient 2 becoming ill. We base our analysis on the assignments P(A 1 K) = P(A 2 K) = Is then P(A 1 and A 2 K) = P(A 1 K) P(A 2 K) = = 0.25%? The answer is yes if A 1 and A 2 are independent. But are they independent? If it was known to you that patient 1 had become ill, would it not alter your probability that patient 2 would become ill? Not necessarily it depends on what your background knowledge is: what is known to us initially, whether there is a coupling between these patients in some way or another. For example, if they are both in a weakened physical condition or are related, then it is clear that we know more about patient 2 if we find out that patient 1 has become ill. If our background knowledge is very limited, knowledge that patient 1 has become ill will provide information to us about patient 2. In practice, however, we have so much knowledge about this illness that we can ignore the information that is associated with A 1. We therefore obtain independence since P(A 2 K,A 1 ) = P(A 2 K). If there is coupling between the patients, as illustrated above, then P(A 2 K,A 1 ) will be different from P(A 2 K). Thus we have a dependence between the events A 1 and A 2. A conditional probability, P(A B), is defined by the formula P(A B) = P(A and B)/P (B). We see that this formula is simply a rewriting of formula (A.1). By substituting P(A and B) with P(A)P(B A) (again we use formula (A.1)), the well-known Bayes formula is established: P(A B) = P(A)P(B A)/P (B). We will show the application of this formula in Section A.4.

4 170 PROBABILITY CALCULUS AND STATISTICS A.3 Probability distributions: expected value Let X denote the number of persons who become ill in the course of 1 year for a group of four persons. Assume that you have established the following probabilities that X will take the value i, i = 0, 1, 2, 3, 4: i P(X = i) The expectation, EX, is defined by: EX = = 1.7 The expected value is the centre of gravity of the distribution of X. Ifalever is set up over the point 1.7, then the masses 0.05, 0.40,...,0.05 over the points 0, 1,...,4 will be perfectly balanced. If X can assume one of the values x 1,x 2,..., one can find EX by multiplying x 1 with the corresponding probability P 1, and likewise multiply value x 2, with probability P 2, etc., and sum up all values x j,i.e. EX = x 1 P 1 + x 2 P If X denotes the number of events of a certain type, and this number is either 0 or 1, then the associated probability equals the expected value. This is evident from the formula for expected value, as in this case EX is equal to 1 P (the event will occur). In many situations, we are concerned about rare events in which we, for all practical purposes, can disregard the possibility of two or more such events occurring during the time interval under consideration. The expected number of events will then be approximately equal to the probability that the event will occur once. In applications, we often use the term frequency for the expected value with respect to the number of events. We speak about the frequency of gas leakages, for example, when we actually mean the expected value. We can also regard the frequency as an observation, or prediction, of the number of events during the course of a certain period of time. If we say, for example, that the frequency is 2 per year, we have observed, or we predict, two events per year on the average. The expectation constitutes the centre of gravity of the distribution, as mentioned above, and we see from the example distribution that the actual outcome can be far from the expected value. To describe the uncertainties, a prediction interval is often used. A 90% prediction interval for X is an interval [a, b], where a and b are constants, which is such that P(a X b) = In cases where the probabilities cannot be determined such that the interval has probability 0.90, the interval boundaries are specified such that the probability is larger than, and as close as possible to, In our example, we see that [1, 3] is a 90% prediction interval. We are 90% certain that X will assume one of the values 1, 2 or 3.

5 PROBABILITY CALCULUS AND STATISTICS 171 The variance and standard deviation are used to express the spread around the expected value. The variance of X, VarX, is defined as the expectation of (X EX) 2, while the standard deviation is defined as the square root of the variance. A.3.1 Binomial distribution Let us assume that we have a large population, I, of people (for example, patients) and that we are studying the proportion q of them that become ill over the course of the next year. Let us assume further that we have another similar population II that is composed of n persons. Let X represent the number that develops the illness in this population. What is then our probability that all of those in population II will develop the illness, i.e. P(X = n)? Alternatively, we may think of the populations as comprising technical units, for example machines. In the Bayesian literature, it is common to refer to q as a chance (Singpurwalla 2006). To answer this question, first assume that q is known. You know that the proportion within the larger population I is 0.10, say. Then the problem boils down to determining P(X = n q). If we do not have any other information, it would be natural to say that P(X = n q) = q n. We have n independent trials and our probability for success (illness) is q in each of these trials. We see that when q is known, then X has a so-called binomial probability distribution, i.e. n! P(X = i q) = i!(n i)! qi (1 q) n i, i = 0, 1, 2,...,n, (A.2) where i! = i. The reader is referred to a textbook on probability calculus if understanding this is difficult. When q is small and n is large, we can approximate the binomial probability distribution by using the Poisson distribution: P(X = i r) = ri e r, i = 0, 1, 2,..., i! where r = nq. We know, for example, that (1 q) n is approximately equal to e r. This can be checked using a pocket calculator. We refer to q and r as parameters of the probability distributions. By varying the parameters, we obtain a class of distributions. What do we do if q is unknown? Let us imagine that q can be 0.1, 0.2, 0.3, 0.4 or 0.5. We then use the total probability rule, and obtain: P(X = i) = P(X = i q = 0.1)P (q = 0.1) + P(X = i q = 0.2)P (q = 0.2) P(X = i q = 0.5)P (q = 0.5). By assigning values for P(q = 0.1), P (q = 0.2), etc., we obtain the probability distribution for X, i.e.p(x = i) for various values of i.

6 172 PROBABILITY CALCULUS AND STATISTICS A.4 Statistics (Bayesian statistics) In statistics, focus is often on properties within large populations, for example q in the above example, i.e. the proportion of the large population I that will develop the illness in question. The problem is how to express our knowledge of q based on the available data X, i.e. to establish a probability distribution for q when we observe X. We call this distribution the posterior probability distribution of q. We begin with the so-called prior distribution before we perform the measurements X. Here let us suppose that we only allow q to assume one of the following five values: 0.1, 0.2, 0.3, 0.4 or 0.5. We understand these values such that, for the example 0.5, this means that q lies in the interval [0.45, 0.55). Based on the available knowledge, we assign a prior probability distribution for the proportion q: q P(q = q ) This means that we have the greatest confidence that the proportion q is 0.3 (50%), then 0.2 and 0.4 (20% each) and least likely, 0.1 and 0.5 (5% each). Suppose now that we observe 10 persons and that among these persons there is only 1 that has the illness. How will we then express our uncertainty regarding q? We use Bayes formula and establish the posterior distribution of q. Bayes formula states that P(A B) = P(B A)/P (B) for the events A and B. If we apply this formula, we see that the probability that the proportion will be equal to q when we have observed that 1 out of 10 has the illness is given by P(q = q X = 1) = cf (1 q )P (q = q ), (A.3) where c is a constant such that the sum over the q s is equal to 1, and f is given by f(i q ) = P(X = i q = q ), refer formula (A.2); the quantity X is binomially distributed with parameters 10 and q when q = q is given. Using the formula (A.3), we find the following posterior distribution for q: q P(q = q ) We see that the probability mass has shifted to the left towards smaller values. This was as expected since we observed that only 1 out of 10 became ill, while we, at the start, expected the proportion q to be closer to 30%. If we had a larger observation set, then this data set would have dominated the distribution to an even larger degree.