Probability, its meaning, real and theoretical populations

Transcription

1 3 Probability, its meaning, real and theoretical populations 3.1 Probability for finite populations By a 'finite' population we mean one with a finite (= countable) number of individuals. Thus, the population of wage-rates of Table 2.1 is finite, and so too are populations of men's heights, since the number of men in the world, though large, is countable. (It is possible to envisage infinite populations, as will be seen.) Now, the reader may have some idea of what probability is, but, for the matters in hand, a precise and clear definition must be made. Since the aim of statistics is to gather as much information from as few observations as possible, consider the following question: If, from the wage-rate data of Table 2.1, an 'individual' is selected at random, what is the probability that this is 75 pence per hour (note that an 'individual' is an observation). The answer is 23/211, because 23 men out of 211 have this wage-rate and, if we repeated the operation of choosing a man at random a very large number of times, a 75 pence per hour man would be chosen a proportion 23/211 of times. The perceptive reader will ask 'Exactly what does "at random" mean 7' The answer is that there is no pattern or regularity whatsoever in the selection. (How to achieve random selection in practice is discussed in Chapter 15.) We now express this idea in general terms. First we define probability precisely thus: The probability that a single observation has a given variate value, X say, is the proportion of times the variate-value X turns up when a very large number (theoretically, infinity) of random selections are made. We further ask the reader, as a reasonable person, to agree that, when selection is random, the proportion of times that a value X turns up exactly equals the proportion of individuals in the population who have a variate-value X, in the long run. (There is no proof of this; it is an axiom, that is, it is regarded as a self-evident truth.) Hence, given a population consisting of II individuals with variate-value Xl' of h with variate-value Xi' and so on up to fn with variate-value X N, then the probability that a randomly selected individual has a variate-value Xi is Pi = fi/(h + h fn) (3.1) 15 C. Mack, Essentials of Statistics for Scientists and Technologists C. Mack 1966

2 16 ESSENTIALS OF STATISTICS From this formula we can deduce the very important relation + + +p _ II + h Pl h.... N - II + h IN II IN + IN = II + h IN - 1 (32)... II IN II IN -. This can be put in words thus: The sum of the probabilities that an individual has one or other of the possible variate-values is unity. Examples 3.1(i). Calculate the Pi (i.e. Pl up to Ps) for the data of Table I (ii). Calculate the Pi for the data of Examples 2.1(i) (a), (b). What is the probability that the sky is half (0 5) or more clouded in 2.1(i) (b)? Another relation is the following: Since the proportion of individuals with variate-value Xl is the same as Pl and so on, then the population mean p, can be found from the formula (see 2.1 and 3.1) p, = PlX1 + P2X PNXN (3.3) We now elaborate this notion of probability, occasionally using formulae which the reader may remember if he wishes. He should, however, make sure he understands the underlying ideas clearly. 3.2 Probability for infinite populations with continuons variate The variate-values in many real populations (e.g. wage-rates) have only a finite number of possible values. But populations with a continuous range of possible variate-values can be envisaged, e.g. measurements of the velocity of light, for owing to variation from experiment to experiment even the same observer will never reproduce exactly the same value and, indeed, we are not sure that the velocity is an absolute constant. The idea of a population with a continuous range has been so fruitful we shall consider it in some detail. It originated in astronomy when the observations made by observers at different places and/or by the same observer at different times had to be combined to give the best 'fix' for a given star. The chief cause of variation in the observations is atmospheric refraction which is varying all the time and the great mathematical physicist Gauss suggested that each observation should be regarded as the combination of the true position plus an 'error' term. This error term Gauss regarded as coming from a population of errors (with a continuous variate). By examining the variations of the observations among themselves, we can get quite a good idea of this error population, without a detailed examination 01 the causes 01 the errors. On plausible assumptions

3 PROBABILITY, REAL AND THEORETICAL POPULATIONS 17 Gauss worked out the properties of this error population and from these it is possible to deduce the overall best position for a given star, together with some idea of the final accuracy. Later, the idea of continuous variate populations was extended, and the properties of real populations studied by considering continuous variate populations which closely resemble them. Thus, the 'heights of men' histogram of Fig. 2.1 is closely represented by the frequency curve (drawn through the midpoints of the tops of the t dx...;:. II,, 11 X+dlC lc_ FIGURE 3.1. A frequency curve boxes) of Fig. 2.3(b). In some respects this frequency curve is more realistic because it gives a better idea of how heights vary inside a box. But the great merit of such curves is that they may have a simple formula which enables properties to be calculated simply and quickly. We consider now the extension offormula (3.1) for probability in finite populations to continuous variate populations. Since the frequency curve of a continuous population may have, in theory, any formula we shall call its equation y = f(x), see Fig Instead of the frequency fi of a variate-value Xi as in a frequency diagram or the frequency in a box of a histogram, we consider the frequency of variate-values in a narrow strip lying between x and x + dx, (see the shaded strip in Fig. 3.1). This frequency is f(x)dx (3.4) Again the frequency of variate-values which lie anywhere between x = a and x = b is the integral (see the stippled area of Fig. 3.1) (3.5)

4 18 ESSENTIALS OF STATISTICS To specify completely the population we need to know the lowest possible variate-value (I, say) and the greatest value (g, say). Then the total frequency (i.e. the total number of individuals) in the population is (3.6) Analogous to (3.1), the probability that an individual selected at random has a variate-value between x and x + dx is equal to the proportion of the total frequency with a variate-value between x and x + dx. This proportion is f(x)dx f f(x)dx (3.7) The quantity p(x) = lex) i f(x)dx (3.8) is called the 'probability density function' of the population. It is analogous to the Pi of Section 3.1 and its definition should be remembered. Again, analogous to the important relation (3.2), we have the formula i g x=l i~/(x)dx p(x) dx = f = 1 f(x) dx x=l (3.9) or, in words, the integral of the probability density function p(x) over all possible values of the variate (i.e. from x = 1 to x = g) is unity. (The reader must not think that all theoretical populations have continuous variates. In some problems we consider discrete objects such as particles, or customers, arriving at random. The number of customers is often the variate considered and this can only be 0, 1, 2,... and, for example, cannot be ) The term 'distribution' is nowadays very commonly used instead of 'population' especially for theoretical populations. We finish this section with the formula for the probability that an

5 PROBABILITY, REAL AND THEORETICAL POPULATIONS 19 individual, selected at random, has a variate-value lying somewhere between a and b; this can be seen, from (3.5) and (3.8), to be a result which should be remembered. i~ap(x)dx (3.10) Examples 3.2(i). If a frequency curve has formula f(x) = e-, and I = 0, g = 00, show that p(x) = e-"'. If f(x) = e-exx, I = 0, g = 00, show that p(x) = lxe-"'''' (both these are called 'negative exponential' distributions). 3.2(ii). If f(x) = x 2-2x + 5, I = 0, g = 3, find the probability density function p(x). Find the probability that an observation made at random lies between 1 and Sampling populations (distributions) These are theoretical populations which arise thus: Suppose we take a sample of a given size from a population and calculate the sample mean x from formula (2.9). If we did this a very large number of times we could plot the curve of frequency of occurrence of different values of x. In other words we would get a population of sample means. Such a population is called a 'sampling population' or 'sampling distribution'. If we repeated this process for samples of all sizes we can find out various vitally useful properties. Consider, for example the fact that the sample mean is used to estimate the true or population mean. The sample mean though usually close to the population mean does differ from it. The sampling population properties enable us to say what size of sample is required to estimate the population mean with a given accuracy. We can, each time we take a sample, calculate the sample variance S2, (from formula (2.11» and form a population based on the frequency of occurrence of different values of S2. Orwecouldmeasure the sample 'range' (= largest - smallest observation) and obtain a population from the values of this. In fact any population derived from an original population by taking samples repeatedly and calculating some function of the observations in the sample is called a 'sampling population'. If the original population has a reasonably simple mathematical formula, the formulae for the sampling distributions derived from it can often (though not always) be found (though sometimes some rather clever mathematical tricks have to be used). In fact these mathematical formulae form the basis of modern statistics and its very powerful methods.

6 20 ESSENTIALS OF STATISTICS There is no need to follow the mathematics in detail to appreciate these methods, however, and they are, in general, merely outlined here. However, since the useful formulae are usually given in terms of probability rather than frequency we now rephrase a few definitions and define some useful technical terms. 3.4 Moments of populations in terms of probability Section 2.4 gives the formulae for the moments of a finite population. In terms of the probabilities PI (see formula (3.1» it will be seen that '111' = PlX{ + P-l.X{ PNXN1' (3.11) where '111' is the rth moment about the origin. The corresponding formula for continuous variate populations is '111' = i~l(x) x1'dx (3.12) where p(x) is the probability density function (see formula (3.8». Similarly the rth moment about the mean P1' has the formula or P1' = Pl(X l - p)1' + P2(X2 - p)1' PN(XN - pt (3.13) P1' = i~, (x - py p(x)dx (3.14) where P is the population mean, i.e. P = 'Ill' Note the following relations which is the same as formula (3.9), and '110 = Po = i~l(x)dx = 1 (3.15) P1 = f (x - p)p(x)dx = f xp(x)dx - P fp(x)dx = p - p = 0 (3.16) Note that P2 is identical with the variance C12 just as for finite populations (see Section 2.4), for it is the mean value of the square of the deviations of each observation (i.e. it is the mean value of (x _ p)2). Examples 3.4{i). Find 'Ill' 'lis, P1' Ps for the population with p(x) = e-~, 1= 0, g = 00.

7 PROBABILITY, REAL AND THEORETICAL POPULATIONS (ii). For the population with p(x) = t, 1= 1, g = 4 find '111' '112' 'IIr; fll' fl2' flr. (This is called a 'rectangular' distribution, see Section 10.5.) There are relations between the 'IIr and the flr identical with those of (2.7) and (2.8) for finite populations, namely (3.17) Example 3.4(iii). Prove (3.17) and (3.18) (for the mathematically curious). 3.5 The 'expected' value This is a simple but very useful notation. Suppose we select at random a sample from a population and calculate some quantity such as the mean or variance or 'range' (see Section 3.3). Then the mean value of this quantity averaged over a large number (theoretically, infinity) of randomly selected samples (of the same size) is called the 'expected value' of this quantity (and denoted by the symbol E). Thus, if x denotes the variate-value of a Single individual selected at random (Le. a sample of size 1), then E(x) = fl (3.19) because the mean value of x averaged over a large number of selections is equal to the population mean Il (since we agree that, in the long run, each possible variate-value will appear its correct proportion of times). In the case of a finite population with possible variate-values Xl' X2,... X N whose probabilities are PI, P2'... PN then E(x) = PIXI + P2X PNXN = fl (3.20) We can express the moments about the origin ('IIr) and those about the mean (flr) conveniently in this notation, thus E(xr) = PIX{ + P2X{ PNXN = 'IIr; E(x - fly = plxl - fly PN(XN - fly = flr (3.21) Similarly, for continuous variates, E(Xf) = f xrp(x)dx = 'IIr; E(x - fly = f(x -fl)" p(x)dx = flr (3.22)

8 22 ESSENTIALS OF STATISTICS and, in particular, E(x) = f xp(x)dx = '" E(x - ",)2 = f (x - ",)2p(x)dx = "'2 = a2 (3.23) 3.6 The cumulative distribution function (for continuous variates) The integral of the probability density p(x) from the lowest value 1 of x up to a given value x = X, that is L:/(X)dX = P(X), say (3.24) is called the '(cumulative) distribution function'; the word 'cumulative' is often omitted. Note that X may take any value between 1 andg. We shall not use the distribution function explicitly in this book, but it is often very useful in the theory. Note that the derivative of the distribution function equals the probability density function, that is dp(x)/dx = p(x) (3.25)