Six Examples of Fits Using the Poisson. Negative Binomial. and Poisson Inverse Gaussian Distributions

Transcription

1 APPENDIX A APPLICA TrONS IN OTHER DISCIPLINES Six Examples of Fits Using the Poisson. Negative Binomial. and Poisson Inverse Gaussian Distributions Poisson and mixed Poisson distributions have been extensively used in many other disciplines than actuarial science. The examples provided in this appendix, originating from sports, the military, and medicine, provide further comparisons of the Poisson, negative binomial (NB), and Poisson-inverse Gaussian (PIG) distributions. They provide a variety of illustrations of both true and apparent contagion and various degrees of clustering. X 2 values are provided using rule A.

2 44 APPLICATIONS IN OTHER DISCIPLINES Table A-I. Example 1 (Williams, 1954) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk Not possible X2=0.33 X2=1.24 X 2 =1.52 x = A 1..= =9.484 ~ = S2 = a=5.785 fl=0.127 y = y=i.280 y=1.445 y=1.530 In table A-I, k is the number of deaths due to horse kicks in the ten corps of the Prussian army between 1875 and This is a classical example of total randomness of accidents. The Poisson distribution provides a much better fit than both the NB and the PIG, with one parameter only. The moments methods cannot be applied, since S2<X.

3 APPENDIX A 45 Table A-2. Example 2 (Richardson, 1945, Ortiz, 1990) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk npk npk l l.05 14l l X 2 =2.74 X 2 =0.34 X 2 =0.33 X 2 =0.32 X 2 =0.32 x = '" 1-.=0.692 ~ = 1 1. ~ 0 = 't=10.66 ~ = S2 = a= =0.090 a= =0.092 y = l.295 y=l.202 y=1.359 y=i.368 y=i.365 y=1.371 In table A-2, k represents the number of major international wars during each of the 432 years between 1500 and The Poisson hypothesis is not rejected, indicating that the outbreak of war is mostly a random process. However, both the NB and the PIG models provide much better fits, probably due to a low degree of both true and apparent contagion. Apparent contagion may result from technological "advances" in warfare and structural changes that affect nations' ability to wage or defend against war. True contagion is a consequence of the long-lasting memories and bitter rivalries among participant nations elicited by wars: wars destabilize the international system.

4 46 APPLICA TIONS IN OTHER DISCIPLINES Table A-3. Example 3. (Reep, Pollard and Benjamin, 1971), Pollard, Benjamin and Reep, 1977) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk npk npk X2==735.2 X 2 ==2.16 X 2 ==11.02 X2== 1.67 X 2 ==6.65 x == '" A==0.482 ~ = = 0. 8~ 6= 2 = ~ = = ~ 1 = = S2 == a== ==1.160 a== == Y == y==1.440 y==3.253 y==3.863 y==3.322 y==4.436 In table A-3, k is the number of runs scored per half-inning in all the baseball World Series played between 1947 and The World Series, played annually between the best teams of the two Major Leagues, present our best case of a very low degree of apparent contagion and a very high degree of true contagion. Indeed World Series usually match teams with similar skills; their winning percentages during the regular season differ by very few percentage points. On the other hand, baseball is a game where runs are often scored in bunches. Run scoring is a rather rare event, since over 70 percent of all half-innings result in no score. But the scoring of a run greatly increases the probability of further runs in the same half-inning: home runs may score up to four runs; runs scored without the benefit of a

5 APPENDIX A 47 home run usually leave men on base, who can score on a single or a double; men on base create holes in the defense and increase strategic possibilities for the manager (sacrifice flies, hit-and-run, base steals, suicide squeezes). In this extreme case of true contagion, the Poisson model is totally unacceptable. Both the NB and the PIG result in satisfactory fits, but the NB clearly outperforms the PIG. The MLE method provides in both cases significantly better fits.

6 48 APPLICA nons IN OTHER DISCIPLINES Table A-4. Example 4. (Reep, Pollard and Benjamin, 1971, Pollard, Benjamin and Reep, 1977) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk npk npk X 2 =17.75 X 2 =3.87 X 2 =3.65 X 2 =3.88 X 2 =3.64 x = A,,= =6.721 ~ = ~ = ~ 8 = S2 = a=io.16 11=0.149 a= =0.148 y = y=0.813 y=0.985 y=0.999 y=0.981 y=0.999 In table A-4, k is the number of goals scored per game per team in the English First Division (soccer) Football League in the season. First Division teams being of more or less comparable standards, most of the contagion exhibited by the data probably comes from game strategy changes that follow a goal; the team that was scored against has to take more chances and attack more, increasing its vulnerability to counterattacks. Both the NB and the PIG provide good fits of the data, with a slight edge to the PIG. Note the rather high value of parameter a. The outcome of every sports competition results from a combination of skill and chance. The high value of a is an indicator that chance plays an important role in football, probably because it is a low-scoring game. Therefore football is likely to produce major upsets.

7 APPENDIX A 49 Table A-5. Example 5. (Reep, Pollard and Benjamin, 1971, Pollard, Benjamin and Reep, 1977) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk npk npk X 2 =9.21 X 2 =3.38 X 2 =3.57 X 2 =3.39 X 2 =3.55 x = " A=2.979 ~ = 5. 3~ 9= '1=5.296 ~ = S2 = a= =0.185 a= =0.186 y = y=0.579 y=0.729 y=0.745 y=0.732 y=0.746 In table A-5, k is the number of goals scored per game per team in the season of the National Hockey League. Ice hockey is a lowcontagion game in North America. Teams do not differ vastly in skills, and the scoring of a goal does not change strategies much (except at the very end of a game, when a team losing by one goal may pull its goalie during the last minute). Hence the Poisson assumption is not rejected. The NB and the PIG provide better fits, with a slight edge to the NB. Whether the improvement in the quality of the fit is worth an extra parameter is a judgment call.

8 50 APPLICA TIONS IN OTHER DISCIPLINES Table A-6. Example 6. (Gurland, 1959) Moments MLE Observed Poisson NB PIG NB PIG k nk npk npk npk npk npk l J II X 2 = X 2 =9.38 X 2 =27.26 X 2 =5.84 x 2 =17.34 A X = A=2.528 ~ = 0. 3~ 1= ~ = ~ 3 = S2 = a=0.787 ~ = a=0.588 ~ = y=i.450 y=0.629 y=2.275 y=3.303 y=2.62 I y=4.428 In table A-6, k is the number of dental cavities observed in 163 twelveyear-old school children. This example involves a very high degree of both apparent ("brush your teeth and eat less candy!") and true ("contiguity risk") contagion. Only the NB provides satisfactory fits here; the PIG model fails completely. Note that in both cases MLE estimators lead to much better fits than moments estimators.

9 APPENDIX A 51 Notes. 1. Exact Maximum Likelihood Estimators For examples 3 to 6, the sample mean and variance cannot be computed exactly with the data as provided. For instance, in example 3, the data indicate that a team scored seven or more runs in a half-inning three times but do not specify exactly how many runs. Consequently, the sample mean and variance cannot be computed exactly and all parameter estimations are approximations. In order to calculate exact maximum likelihood estimators, we have to use, for example 3, (P7+Pg+Pq+...? as the correct contribution to the likelihood function. The estimates in the tables above use (P7t Generally, we have to use the likelihood function where N k + 1 = n k + 1 +n k and k+1 is the class for which exact data are not available. We then have to substitute k 1 - L Pj j=o for the last term, and set the first derivative of the loglikelihood function equal to 0, as usual. For instance, in the Poisson case, we obtain

10 52 APPLICATIONS IN OTHER DISCIPLINES k II V!) i;o din k k ( k ). j) L(A) = _ n. ~ + ~ ~ J + ' N n. ~ l 1n - ~ ~ = 0, d ' L" L J k+1 d' L" JI. j;o II. j;o II. j=o J, As k A ~ I( 1-): n e- )." oj = da i=o j! k Ai k-i Ai) "_ e( - A ~ L" L" j;o J, j=o J,. k. l - e -~ A ~ L " j;o J, k -e - I -A): )." " j;o J,. ' we obtain the following equation to find Ie:

11 APPENDIX A 53 Similar calculations lead to exact maximum likelihood estimators for the negative binomial and Poisson-inverse Gaussian distributions. Tab[e A-7 presents the results for example 3. Table A-7. Example 3: Exact Maximum Like[ihood Estimators Observed Poisson NB PIG k nk npk npk npk [ [ [ X 2 =735.2 X 2 =1.57 X 2 =6.38 A=0.482 t=0.806 ~ = a=0.390 li= The differences between tables A-3 and A-7 are very small.

12 54 APPLICA nons IN OTHER DISCIPLINES 2. The Likelihood Ratio Test In addition to the X 2 distance, the likelihood ratio test can be used as a way of deciding between the NB/PIG and the Poisson distributions. For instance, to test the null hypothesis Ho: The random sample presented in example 2 comes from a Poisson distribution versus HI: The random sample comes from a negative binomial distribution, consider the parameter space Q= {a, 't} defining the NB, and the subset 0)= {a, 't I a--)ooo, 't--)oo,ai't=constant} defining the Poisson as a special or limiting case. Let L=L(a,'t) be the likelihood function, the joint probability of the sample L(a,'t) m II [pia,'t)tk O The generalized likelihood ratio K is defined to be K= max L(a;r:) w max L(a;r:) Q Maximizing L(a,'t) under Q is accomplished by substituting the maximum likelihood estimators for a and 't into L(a,'t). The numerator of K is obtained by using the Poisson probabilities. A low observed value of K indicates that the NB is superior to the Poisson. An observed value of K near I suggests that the hypothesis of an extra parameter is not justified. A generalized likelihood ratio test is a test that rejects Ho whenever 0 < K ::;; K*, where K* is chosen so that

13 APPENDIX A 55 P[O < K:s K 'H o is true] = a, where a is the level of significance of the test. Given the magnitude of the figures, it is usually more practical to use the loglikelihood -In L(a;t) instead ofl(a,'t). For example 2, -In L = for the Poisson, for the NB, and for the PIG. The observed value of the K statistic is for the comparison of the Poisson and the NB and for the comparison of the Poisson and the PIG. It is clearly not necessary to use the less parsimonious NB or PIG model. Table A-8 provides the values of -In L for all examples, and the observed K statistics. It shows that the Poisson distribution is to be used in examples 1 and 2 and the NB in examples 3 and 6. A two-parameter model should be used in example 4. Example 5 is a borderline case. Note that, with the exception of example 4, the NB outperforms the PIG in all cases. Table A-8. Loglikelihoods and K Statistics for All Examples Example -In L -In L -In L Poisson NB PIG K\ K E E E E-56 K 1 : PoissonlNB K2: Poisson/PIG

14 56 APPLICA nons IN OTHER DISCIPLINES Conclusions. The various examples analyzed in this appendix may be more useful to perform a valid comparison of the different models than auto accidents distributions, as they have more nonempty classes. Overall, it seems that * Maximum likelihood estimators clearly lead to better fits than moments estimators. * In a few cases the PIG provides a better fit than the NB - but only marginally better. * In cases showing a high degree of contagion, the PIG is decisively dominated by the NB (although the PIG exhibits a higher skewness than the NB). Overall, the negative binomial seems to provide the better fits, especially in the case of a very high degree of contagion.