MAS275 Probability Modellig 6 Poisso processes 6.1 Itroductio Poisso processes are a particularly importat topic i probability theory. The oe-dimesioal Poisso process, which most of this sectio will be about, is a model for the radom times of occurreces of istataeous evets; there are may examples of thigs whose radom occurreces i time ca be modelled by Poisso processes, for example customers arrivig i a queue, icomig calls to a phoe, eruptios of a volcao, ad so o. Higher dimesioal aalogues of the basic Poisso process are also useful as models for radom locatios of objects i space; we will discuss this i sectio 6.9. We will firstly recall some properties of the Poisso distributio before movig oto the defiitio of the process. 6.2 Properties of the Poisso distributio Recall that the Poisso distributio with parameter λ, deoted P o(λ), has probability fuctio p(x) = λx e λ, x = 0, 1, 2, 3,... x! ad a radom variable with P o(λ) distributio has E(X) = Var(X) = λ. Some further properties of the Poisso distributio follow. Propositio 16. The Poisso distributio is additive. More precisely, if X ad Y are idepedet radom variables with distributios P o(λ) ad P o(µ) respectively, the XY has the distributio P o(λµ). 48
This property exteds i a obvious way to more tha two idepedet radom variables. Proof. See Exercise 17(b)(i). Propositio 17. The Poisso distributio is a limit of biomial distributios. Specifically, if we cosider the sequece of biomial distributios Bi(, µ/) for fixed µ, the the umber of trials is icreasig but the probability of success o a trial is µ/ which is decreasig i proportio, so that the mea µ remais fixed. I the limit, the distributio is Poisso with parameter µ. Proof. This was show i MAS113, sectio 9.3. The proof is repeated below, but will ot be covered i lectures. To verify this, we look at the probability fuctio of Bi(, µ/) ( ) (µ ) x ( 1 µ ) x ( 1)... ( x 1) µ x = (1 µ x x! x ( = 1 1 1 ) ( ) (x 1) µ x... 1 x! 1 µx x! e µ 1 as. This is the probability fuctio of P o(µ). ) ( 1 µ ( 1 µ ) x ) ( 1 µ ) x 6.3 The basic Poisso process We ow move to a cotiuous time scale which is usually regarded as startig at time zero, so that it cosists of the positive real umbers, ad deote our time variable typically by t. Whe we refer to a time iterval, we adopt the covetio that it excludes the left had ed poit but icludes the right had ed-poit, say (u, v] = {t : u < t v}. 49
The Poisso process is described i terms of the radom variables N u,v for 0 u v, where N u,v is the umber of occurreces i the time iterval (u, v]. The process has oe parameter, which is a positive umber λ kow as the rate of the process ad which is meat to measure the average or expected umber of occurreces per uit time. The basic Poisso process is the defied by the followig two assumptios: (a). For ay 0 u v, the distributio of N u,v is Poisso with parameter λ(v u). (b). If (u 1, v 1 ], (u 2, v 2 ],..., (u k, v k ] are disjoit time itervals the N u1,v 1, N u2,v 2,..., N uk,v k are idepedet radom variables. Note that by assumptio (a) ad the mea of a Poisso radom variable, N u,v has mea λ(v u), which gives the correct iterpretatio to the rate λ as described above. To show that a process actually exists which satisfies these assumptios still requires a bit of work. At this poit, we ca observe that the special properties of Poisso distributios are importat: if say u < v < w the N u,w = N u,v N v,w ad because all three of these radom variables are to have Poisso distributios, ad the two o the right had side are idepedet, the assumptios ca oly work because of the additivity property of Poisso radom variables. We will see a bit more o justifyig the existece of a process satisfyig the assumptios at the ed of sectio 6.8. We ca also ask about why the assumptio of Poisso distributios might make sese i a modellig cotext. To see this, approximate the cotiuous time scale by dividig (0, t] up ito small itervals, ad assume that there is the same small probability of a occurrece i each, assumig that the probability of more tha oe occurrece i a iterval is egligible. I order to fix the expected umber of occurreces at λt, this probability must be 50
set equal to λt/. The the umber of occurreces i the iterval has the biomial distributio Bi(, λt/). As we let ted to ifiity, the small itervals become smaller ad so the approximatio to the cotiuous time scale becomes closer, ad, by the relatioship betwee the Poisso ad Biomial distributios (see Propositio 17) the distributio of the umber of occurreces approaches the Poisso distributio with parameter λt. This suggests the Poisso distributio as a sesible model i the geuiely cotiuous time settig. You might like to thik about which other distributios you have ecoutered might have assumptios similar to (a) ad (b) which work. Figure 1: A simulatio of a Poisso process with rate 1 up to time 10 Example 33. Volcaic eruptios 51
6.4 Iter-occurrece times Let T 1 deote the legth of time util the first occurrece, T 2 deote the legth of time betwee the first ad secod occurreces, ad so o, so that T represets the time betwee occurreces 1 ad. These radom variables are called iter-occurrece times. We first show that these caot be zero, ad we will the show which distributio they have. Theorem 18. The probability that i (0, t] two occurreces of a Poisso process with rate λ occur at exactly the same time is zero. Proof. Cosider dividig (0, t] ito small itervals, as i the justificatio of the Poisso model above. Each of these small itervals will be of the form ((i 1)t/, it/] for some i, ad has legth t/, so the umber of occurreces i ay small iterval has a Poisso distributio with parameter λt/. Hece the probability there are at least two occurreces i a give small iterval is 1 e λt λt λt e = 1 e λt ( 1 λt Let Y be the umber of the small itervals which have at least two occurreces. By the idepedece assumptios, Y will have a Biomial distributio: ( ( Y Bi, 1 e λt 1 λt )), ad hece P (Y = 0) = ). ( ( e λt 1 λt )) ( = e λt 1 λt ). As ( 1 λt ) e λt as, we have that P (Y = 0) 1 as. But if there were a probability p > 0 that there were two occurreces at exactly the same time, we would have P (Y = 0) < 1 p for all, so this caot be the case. Hece the probability of there beig two occurreces at exactly the same time is 0. 52
Theorem 19. Iter-occurrece times are idepedet of each other, ad are expoetially distributed with parameter λ. Proof. (Sketch) That T 1 has this distributio is straightforward to show. First, P (T 1 t) is the probability that the first occurrece has happeed by time t, so is the probability that there is at least oe occurrece i (0, t]. Hece P (T 1 t) = P (N 0,t 1) = 1 P (N 0,t = 0) = 1 e λt, which is the distributio fuctio of a expoetial distributio with parameter λ. Now cosider the probability that T is at most t, coditioal o the values of T 1, T 2,..., T 1, P (T t T 1 = t 1, T 2 = t 2,..., T 1 = t 1 ). (There are some techical details to deal with the fact that we are coditioig o a evet of probability zero, which is why this proof is a sketch.) Similarly to the above argumet, this is the probability that there is at least oe occurrece betwee times t 1 t 2... t 1 ad t 1 t 2... t 1 t, which agai is 1 e λt. So, regardless of the values take by T 1, T 2,..., T 1, the coditioal distributio of T is expoetial with parameter λ, which implies the result. 6.5 Variable rate Poisso process We ca make the basic Poisso process more flexible ad realistic as a model by allowig the rate of the process to vary with time, λ(t) say. This ca take ito accout, for example, the fact that traffic is heavier at rush hours, the rate of emissio of particles from a radioactive isotope declies with time, ad so o. The oly chage to the defiitio of the Poisso process is that the assumptio (a) is replaced by the followig: 53
For ay 0 u v, the distributio of N u,v v u λ(t)dt. is Poisso with parameter This assumptio geeralises that of the costat rate case, ad gives the correct iterpretatio of rate whe this rate is varyig. If λ(t) is a costat we recover the basic Poisso process. Figure 2: A simulatio of a variable rate Poisso process with rate 30/(t 1) up to time 10 The idepedece assumptio (b) still holds whe the rate is variable; this still works because of the additivity property of Poisso radom variables, ad also because of the additivity of itegrals, amely that if u < v < w the w u λ(t)dt = Example 34. Email arrivals v u λ(t)dt w v λ(t)dt. 54
6.6 Superpositio I some situatios we have more tha oe Poisso process ruig. If these are idepedet, the the process formed by combiig them is also a Poisso process. Theorem 20. Let (N u,v ) ad (M u,v ) be idepedet Poisso processes with (possibly variable) rates λ(t) ad µ(t) respectively. The (N u,v M u,v ) also forms a Poisso process, with rate λ(t) µ(t). Proof. Because (N u,v ) ad (M u,v ) are Poisso processes, ( v ) N u,v P o λ(t) dt ad ( v ) M u,v P o µ(t) dt. u Because they are idepedet, the additivity of the Poisso distributio (Propositio 16) tells us that ( v v ) N u,v M u,v P o λ(t) dt µ(t) dt u u ( v ) = P o (λ(t) µ(t)) dt. u The idepedece of the umber of occurreces i disjoit itervals i the combied process follows from the same property of the two origial processes. u 6.7 Markig ad thiig Sometimes the occurreces i a Poisso process may be categorised as each belogig to oe of a umber of types. This is sometimes referred to as 55
markig: thik of each occurrece as beig give a radom mark. Specifically we will assume that each occurrece i a Poisso process with (possibly variable) rate λ(t) is give, idepedetly of everythig else, oe of k differet marks with probabilities p 1, p 2,..., p k respectively. Write the total umber of occurreces i (u, v] as N u,v (as before) ad write the umber of occurreces of type i i (u, v] as N (i) u,v. The followig result about Poisso radom variables will be useful. Lemma 21. Let X be a Poisso radom variable with parameter µ, ad imagie that, coditioal o X = x, we have x objects each of which is of oe of k types. Assume further that each of these objects is of type i with probability p i, idepedetly of the other objects. Let the umber of objects of type i be Y i. The (ucoditioal) joit distributio of Y 1, Y 2,..., Y k is such that they are idepedet Poisso, with parameters p 1 µ, p 2 µ,... ad p k µ respectively. Proof. For y 1, y 2,... y k = 0, 1, 2,..., ad lettig x = y 1 y 2... y k, P (Y 1 = y 1, Y 2 = y 2,..., Y k = y k ) = P (X = x)p (Y 1 = y 1, Y 2 = y 2,..., Y k = y k X = x) µ µx x! = e x! y 1!y 2!... y k! py 1 1 p y 2 2... p y k k as required. = e p 1µ (p 1µ) y 1 e p 2µ (p 2µ) y 2... e p kµ (p kµ) y k y 1! y 2! y k! Theorem 22. For each i, the process give by (N (i) u,v) (coutig the occurreces which are type i) is a Poisso process with rate λ(t)p i, ad the k processes for the differet types are idepedet of each other. Proof. This essetially follows from Lemma 21 together with the idepedece properties of Poisso processes. 56
It is eve possible to allow the probabilities of the marks to be depedet o time, say p 1 (t), p 2 (t),..., p k (t). The the geeralised result is that the marked processes are idepedet Poisso processes with variable rates p 1 (t)λ(t), p 2 (t)λ(t),..., p k (t)λ(t) respectively. Oe special case of markig is where k = 2 ad the process of markig cosists of either retaiig the occurrece, with probability p, or deletig it, with probability q = 1 p. The the process of retaied poits is Poisso with rate pλ, ad i this cotext the property is ofte kow as the thiig property. Example 35. Uiversity applicatios 6.8 Coditioig o the umber of occurreces i a iterval Sometimes we kow how may occurreces there are i a give iterval, ad are iterested i how they are distributed withi the iterval. Theorem 23. Assume that we have a Poisso process with costat rate λ. Give that there are occurreces i the time iterval (0, t] say, the positios of these occurreces are distributed as a radom sample of size from the uiform distributio o that iterval. Note that this implies that, coditioal o there beig occurreces i (0, t], the umber of occurreces i ay iterval (u, v] (0, t] (so 0 u < v t) has a Bi(, (v u)/t) distributio, as each of the occurreces would have probability (v u)/t of beig i (u, v], idepedetly of the others. We will prove this latter versio of the statemet. Proof. Note that N 0,t P o(λt), N u,v P o(λ(v u)) ad N 0,t N u,v = N 0,u N v,t P o(λ(t (v u)), ad the latter two radom variables are 57
idepedet. Thus, for 0 a, P (N u,v = a P 0,t = ) = P (N u,v = a, N 0,t = ) P (N 0,t = ) = P (N u,v = a, N 0,t N u,v = a) P (N 0,t = ) = P (N u,v = a)p (N 0,t N u,v = a) P (N 0,t = ) = (λ(v u))a e λ(v u) a! (λ(t (v u))) a e λ(t (v u)) ( a)!! (λt) e λ = e λ(v u) e λ(t (v u))! λ a (v u) a λ a (t (v u)) a e λ a!( a)! (λt) ( ) ( ) a ( v u = 1 v u ) a, a t t which is the probability that a Bi(, (v u)/t) radom variable takes the value a, as required. This result geeralises to the variable rate case, but the uiform distributio is replaced by the distributio which has p.d.f. f(s) = λ(s) t 0 λ(x)dx, which is the distributio o (0, t] whose desity is proportioal to the rate of the origial process. So the umber of occurreces i ay iterval (u, v] (0, t] has a ( v u Bi, λ(s)ds ) t 0 λ(s)ds distributio. The proof is essetially the same. Example 36. Coditioig o umber of evets Note that we ca actually reverse this idea to costruct a Poisso process, for example for simulatio purposes, or to covice ourselves that Poisso processes really exist. Assumig that we wat to costruct a variable rate Poisso process with rate λ(t), we ca do the followig: 58
Divide the positive real lie up ito itervals ( 1, ] for each positive iteger. To each of these itervals ( 1, ] assig a Poisso radom variable X with parameter 1 λ(t) dt. These Poisso radom variables should be idepedet of each other. (This assumes this itegral is fiite; if for oe of the itervals it is ot we will eed to be more careful.) If X = 0, the there will be o occurreces i the iterval ( 1, ]; if X = x > 0, the we create a radom sample of x radom variables o λ(s) ( 1, ] with probability desity fuctio f(s) = λ(x)dx, i a similar 1 maer to above. The values of these radom variables will give the times of occurreces i the iterval. It is ot too hard to show, usig the markig ad additivity properties of the Poisso distributio, that a process of occurreces costructed i this way will satisfy the assumptios with which we defied the Poisso process. 6.9 The spatial Poisso process A importat geeralisatio of the basic Poisso process is to replace the time scale with a space, ad the aim is to model a radom scatterig of poits i this space. The space may be oe-dimesioal for example if we wish to cosider defects o a legth of cable ad i that case it looks like the time scale, or it may be i a higher dimesio two dimesios for positios of spots of rai o a pavemet, three dimesios for positios of stars i space, for example. To geeralise the assumptios we made for the basic Poisso process, we eed a aalogue of the legth of a time iterval. The atural way to do this is to cosider legth i oe dimesio, area i two dimesios, volume i three dimesios, ad so o. We will refer to legth, area or volume, as appropriate, as measure, ad deote the measure of the set A by A. [For 59
those who are familiar with the mathematical cocept of a measure, we ca use other measures o our space here i place of legth, area or volume.] The behaviour of the process ca be described by radom variables N(A) for subsets A with fiite measure: N(A) represets the umber of poits of the process which fall iside the set A. The parameter λ is i this cotext called the desity of the process. A spatial Poisso process is ow defied to be a process which satisfies the followig assumptios, which are geeralisatios of those we used i the time settig. (a). For ay set A of fiite measure, N(A) has the Poisso distributio with parameter λ A. (b). If A 1, A 2,..., A k are disjoit sets of fiite measure, the N(A 1 ), N(A 2 ),..., N(A k ) are idepedet radom variables. The assumptios work for essetially the same reasos as before, otably the additivity of idepedet Poisso radom variables. Most of the properties of the basic Poisso process have aalogues i more tha oe dimesio: Superpositio Two idepedet spatial Poisso processes with rates λ ad µ ca be combied to form a spatial Poisso process with rate λ µ. Markig If the poits of a spatial Poisso process with rate λ are give idepedet marks (from 1, 2,..., k) with probabilities p 1, p 2,..., p k the the poits with mark i form a spatial Poisso process with rate λp i, ad the processes correspodig to the differet marks are idepedet. Coditioig If we kow that there are poits of the process i A, the the coditioal distributio of the locatio of the poits is that of a radom sample of size from the uiform distributio o A. I particular, if B A, the the umber of poits i B is Biomial with parameters ad B / A. 60
The proofs of all of these properties are atural geeralisatios of the proofs of the oe-dimesioal versios. It is also possible to simulate from a spatial Poisso process i a similar way to the oe described for the variable rate time Poisso process. Figure 3: A simulatio of a spatial Poisso process with rate 40 o a uit square Oe property of the basic Poisso process which does ot aturally carry over is the joit distributio of iter-occurrece times, as there is o atural orderig of poits i two or more dimesios, ad so the aalogue of iteroccurrece times does ot exist. However, it is possible to use a similar idea to calculate the distributio of the distace to the earest poit i the process from a give poit. Example 37. Trees i a forest It is also possible to defie variable rate spatial Poisso processes; the parameter of the Poisso distributio givig the umber of poits i a set A will be 61
the itegral over A of the rate fuctio, just as for the oe-dimesioal case, but the itegral is ow a multidimesioal oe. A actual scatter of poits may be clustered relative to the true radomess of the Poisso process for example, positios of plats each of which self-propagates withi a local area or regular relative to the Poisso process for example, positios of birds ests whe there is a territorial effect ihibitig ests from beig too close together. regular Figure 4: A simulatio of a spatial process more regular tha a Poisso process 6.10 Compoud Poisso processes* Suppose that evets occur at radom times but also each evet carries with it some umerical value, ad the chief iterest is i the sum of these umerical values over a period of time. Examples might be claims o a isurace compay, which occur at radom times but they differ i size; or fatalities i 62
cluster Figure 5: A simulatio of a spatial process more clustered tha a Poisso process road accidets, where the accidets occur at radom times but each accidet may icur a umber of deaths. The simplest model for such situatios is to take the times of occurreces as a basic Poisso process ad the to assume that the sizes of the occurreces are radom variables each with some kow distributio, which may be discrete or cotiuous, these radom variables beig idepedet of each other ad of the times of the occurreces. This gives what is kow as a compoud Poisso process. We will use the followig otatio. N(t) deotes the umber of occurreces i the time iterval (0, t], previously writte as N 0,t ; the sizes of the occurreces i chroological order are deoted by Y 1, Y 2,.... The the sum of these over 63
the time iterval (0, t] may be writte N(t) X(t) = Y i. If we thik graphically of plottig N(t) ad X(t) agaist t, the the graph of N(t) jumps upwards by 1 at each occurrece ad stays costat i betwee, whereas the graph of X(t) jumps (upwards or dowwards, sice Y i could be egative) by the radom quatities Y 1, Y 2,... at the times of the occurreces, ad stays costat i betwee. The process is completely specified by the rate λ of the Poisso process ad the commo distributio of the radom variables Y 1, Y 2,.... i=1 Figure 6: A simulatio of a compoud Poisso process with rate 1 ad jumps havig χ 2 4 distributio up to time 10 We ca combie the ideas of compoud Poisso processes with variable rate ad spatial Poisso processes as well. 64
For example, we might be modellig the locatios of ests of some species of bird withi some regio. We could treat the locatios as poits i two dimesioal space to be modelled by a spatial Poisso process. If some parts of the regio are more favourable to the species tha others, the we would expect a higher desity of ests i these areas, so the model would have a variable rate which is higher i favourable areas ad lower elsewhere. If we wated to model the total umber of offsprig raised, the a compoud spatial Poisso model might be appropriate. 65