NEGATIVE BINOMIAL DISTRIBUTIONS FOR FIXED AND RANDOM PARAMETERS

Transcription

1 UNIVERSITY OF NAIROBI School of Mathematics College of Biological and Physical sciences NEGATIVE BINOMIAL DISTRIBUTIONS FOR FIXED AND RANDOM PARAMETERS BY NELSON ODHIAMBO OKECH I56/743/2 A PROJECT PRESENTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN MATHEMATICAL STATISTICS. DECEMBER 24

2 DECLARATION I declare that this is my original wor and has not been presented for an award of a degree to any other university Signature Date. Mr. Nelson Odhiambo Oech This thesis is submitted with my approval as the University supervisor Signature. Date. Prof. J.A.M. Ottieno i

3 DEDICATIONS This project is dedicated to my mother Mrs. Lorna Oech, my late father Mr. Joseph Oech and my only sister Hada Achieng Oech for having initiated this project, inspired me and sponsored the fulfillment of this dream. ii

4 ACKNOWLEDGMENT I would lie to acnowledge with gratitude the effort and guidance from my supervisor Prof. JAM Ottieno. The fulfillment of the project was as a result of his selfless devotion to help. I m deeply indebted to the entire School of Mathematics (University of Nairobi) for the support accorded to me at whatever level. The entire academic journey in the mentioned institution was made possible by the relentless support from the dedicated lectures and support staff. Your contribution was immense and may God shower you with his blessings It cannot be gainsaid that my friends, relatives and well wishers played a pivotal role in the accomplishment of this project. They provided moral support, encouraged me whenever the journey seemed rocy and gave me the necessary peace of mind to enable me channel all my energy to this remarable venture: Duncan Oello, Beryl Anyango, Mary Oech, Evelyn Oech, Shadrac Oech, Danson Ngiela and George Oluoch Last but not least, it is indeed my great pleasure to appreciate the support from my teammates and fellow students during the entire period. Above all, I than the Almighty God for the good health and continued protection for the whole period. iii

5 ABSTRACT The first objective of this project is to construct Negative Binomial Distributions when the two parameters p and r are fied using various methods based on: Binomial epansion; Poisson Gamma miture; Convolution of iid Geometric random variables; Compound Poisson distribution with the iid random variables being Logarithmic series distributions; Katz recursive relation in probability; Eperiments where the random variable is the number of failures before the rth success and the total number of trials required to achieve the rth success. Properties considered are the mean, variance, factorial moments, Kurtosis, Sewness and Probability Generating Function. The second objective is to consider p as a random variable within the range and. The distributions used are: i. The classical Beta (Beta I) distribution and its special cases (Uniform, Power, Arcsine and Truncated beta distribution). ii. Beyond Beta distributions: Kumaraswamy, Gamma, Minus Log, Ogive and two sided Power distributions. iii. Confluent and Gauss Hypergeometric distributions. The third objective is to consider r as a discrete random variable. The Logarithmic series and Binomial distributions have been considered. As a continuous random variable, an Eponential distribution is considered for r. The Negative Binomial mitures obtained have been epressed in at least one of the following forms. a. Eplicit form b. Recursive form c. Method of moments form. Comparing eplicit forms and the method of moments, some identities have been derived. For further wor, other discrete and continuous miing distributions should be considered. Compound power series distributions with the iid random variables being Geometric or shifted Geometric distributions are Negative Binomial mitures which need to be studied. Properties, estimations and applications of Negative Binomial mitures are areas for further research. iv

6 Table of Contents DECLARATION... i DEDICATIONS... ii ACKNOWLEDGMENT... iii ABSTRACT... iv CHAPTER... GENERAL INTRODUCTION.... Bacground information... Negative Binomial Distribution... Distribution Mitures... 2 Uncountable mitures... 3 Mitures of parametric families... 3 Negative Binomial mitures Problem statement Objectives Methodology Literature review Significance of the study and its applications... 7 Chapter CONSTRUCTIONS AND PROPERTIES OF NEGATIVE BINOMIAL DISTRIBUTION Introduction NBD based on Binomial epansion NBD based on mitures Construction from a fied number of Geometric random variables Construction from logarithmic series Representation as compound Poisson distribution Construction using Katz recursive relation in probability Iteration technique Using the Probability Generating Function (PGF) technique From eperiment Properties Factorial moments of Negative Binomial distribution v

7 CHAPTER BETA - NEGATIVE BINOMIAL MIXTURES Introduction A brief discussion of the various forms of epressing the mied distribution Eplicit form Method of moments Recursion Classical beta Negative Binomial distribution Construction of Classical Beta Distribution Properties of the Classical Beta distribution The miture In eplicit form Using Method of Moments Miing using Recursive relation Properties of Beta Negative Binomial Distribution... 4 Mean Moment Generating Function Special cases of beta negative Binomial distribution Uniform Negative Binomial distribution Uniform distribution The mitures Power function - negative Binomial distribution Power function distribution The miing Arcsine negative Binomial distribution Negative Binomial Truncated Beta distribution Truncated Beta distribution Construction Negative Binomial Confluent Hypergeometric distribution Confluent Hypergeometric Confluent Hypergeometric Negative Binomial miing Gauss Hypergeometric Negative Binomial Distribution vi

8 3.6.. Gauss Hypergeometric distribution Gauss Hypergeometric Negative Binomial miture CHAPTER NEGATIVE BINOMIAL MIXTURES BASED ON DISTRIBUTIONS BEYOND BETA Negative Binomial Kumaraswamy (I) Distribution Kumaraswamy (I) Distribution Negative Binomial Kumaraswamy (I) distribution miing Negative Binomial Truncated Eponential Distribution Truncated Eponential Distribution (TEX(λ, b)) Negative Binomial Truncated Eponential distribution miing Negative Binomial Truncated Gamma Distribution Truncated Gamma Distribution Negative Binomial Truncated Gamma Distribution miing Negative Binomial Minus Log Distribution Minus Log Distribution Negative Binomial Minus Log Distribution miing Negative Binomial Standard Two Sided Power Distribution Standard Two Sided Power Distribution Negative Binomial Standard Two Sided Power Distribution Negative Binomial Ogive Distribution Ogive Distribution Negative Binomial Ogive Distribution miing Negative Binomial Standard Two Sided Ogive Distribution Standard Two Sided Ogive Distribution Negative Binomial Standard Two Sided Ogive Distribution miing Negative Binomial Triangular Distribution Triangular Distribution Negative Binomial Triangular distribution miing CHAPTER GEOMETRIC DISTRIBUTION MIXTURES WITH BETA GENERATED DISTRIBUTIONS IN THE [, ] DOMAIN Introduction Geometric Classical Beta Distribution... 9 vii

9 5.2.. Classical Beta Geometric distribution from eplicit miing Classical Beta Geometric distribution from method of moments miing Classical Beta Geometric distribution from recursive relation Properties of beta Geometric distribution Geometric Uniform distribution Geometric uniform distribution from eplicit miing Geometric uniform distribution from recursive relation Geometric uniform distribution from Method of moments Negative Binomial power function distribution Power function distribution Geometric Power distribution from eplicit miture Geometric Power distribution from moments method Geometric Power distribution in a recursive format Geometric Arcsine distribution Geometric Arcsine from eplicit miing Geometric Arcsine distribution in a recursive format Geometric Arcsine distribution from method of moments Geometric Truncated beta distribution Truncated beta Geometric Truncated beta from eplicit miing Geometric Truncated beta distribution from method of moments Geometric Confluent Hypergeometric Confluent Hypergeometric Geometric Confluent Hypergeometric distribution from eplicit miing Geometric Hypergeometric distribution from Method of Moments Geometric Gauss Hypergeometric distribution Gauss Hypergeometric distribution Gauss Hypergeometric Geometric distribution from Eplicit miing Gauss Hypergeometric Geometric distribution from method of moments miing... 6 CHAPTER GEOMETRIC MIXTURES BASED ON DISTRIBUTIONS BEYOND BETA FROM NEGATIVE BINOMIAL MIXTURES... 7 viii

10 6.. Introduction Geometric Kumaraswamy (I) Distribution Kumaraswamy (I) Distribution Geometric Kumaraswamy (I) distribution from eplicit miing Geometric Kumaraswamy (I) distribution from method of moments miing Geometric Truncated Eponential Distribution (TEX(λ, b)) Truncated Eponential Distribution (TEX(λ, b)) Geometric Truncated Eponential distribution from eplicit maing Geometric Truncated Gamma Distribution Truncated Gamma Distribution Geometric Truncated Gamma distribution from eplicit miing Geometric Truncated Gamma distribution from moments method miing Geometric Minus Log Distribution Minus Log Distribution Geometric Standard Two Sided Power Distribution Standard Two Sided Power Distribution Geometric Standard Two Sided Power distribution from eplicit miing Geometric Ogive Distribution Ogive Distribution Geometric Ogive distribution from eplicit miing Geometric Standard Two Sided Ogive Distribution Standard Two Sided Ogive Distribution Geometric Standard Two Sided Ogive Distribution from eplicit miing Geometric Triangular Distribution Triangular Distribution Geometric Triangular distribution from eplicit miing Geometric Triangular distribution from method of moments miing... 4 CHAPTER NEGATIVE BINOMIAL MIXTURES WITH P AS A CONSTANT IN THE MIXING DISTRIBUTION Introduction Logarithmic distribution/ logarithmic series distribution/ log series distribution Negative Binomial eponential distribution... 8 i

11 CHAPTER SUMMARY AND CONCLUSION Table : Negative Binomial mitures with [,] domain distribution priors based on Classical Beta Table 2: Negative Binomial mitures with [,] domain Beyond Beta distribution priors A framewor for the mitures Recommendations... 3 REFERENCE... 32

12 CHAPTER. Bacground information GENERAL INTRODUCTION Negative Binomial Distribution Pierre de Montmort first mentioned Negative Binomial distribution in 73. He considered a series of Binomial trials and came up with a finding of the probability of the number of failures, before the rth success in the series. In 838 Poisson Simeon (78-84) developed the Poisson regression. He first introduced the new distribution as the limiting case of the Binomial. He later discovered that we can derive Poisson from the Binomial distribution and he demonstrated how the two distributions actually relate. Poisson regression was developed to handle count data, and later became the standard method used to model counts. It is important to note that Poisson assume equality in its mean and variance. This is a very rare characteristic in real data. Data that has greater variance than the mean is termed as Poisson over dispersed, yet more commonly nown as overdispersed. It is recommended that we apply Negative Binomial distribution instead of Poisson distribution when dealing with count data that is overdispersed. Some of the most prominent and well nown discrete distributions are the Binomial, the Poisson and the Negative Binomial distribution. The theoretical connection between these distributions is too close that it is hardly convenient to discuss any one of them without referring to the other. For instance, the Negative Binomial distribution is based on the other two distributions (Poisson and Binomial) in relation to its construction as you will see later in this project. Negative Binomial distribution can be epressed in two different ways depending on the definition of the parameter r as follows.

13 a. st Form of Negative Binomial Distribution Consider a sequence of independent Bernoulli (p) trials. In each trial the probability of success is p. Let the random variable X denote the trial, at which the rth success occurs, where r is a fied integer, then, Pr X = /r, p = r pr p r = r, r +, r + 2, (.) And we say that X has a Negative Binomial distribution with parameters r and p epressed as X~NB(r, p) b. 2 nd Form of Negative Binomial Distribution Negative Binomial distribution can as well be epressed as follows. = te Number of failures before te rt success in an infinite series of indipendent trials wit a constant probability of success p. + r = te number of trials ecluding te rt success + r = te number of ways of obtaining failures and r success Thus the alternative form of the Negative Binomial distribution is epressed as follows Prob(X = /r, p) = r + p r p.2 for =,,2, and r > Distribution Mitures A miture distribution is the probability distribution of random variable whose values can be interpreted as being derived from an underlying set of other random variables: specifically, the final outcome value is selected at random from among the underlying values, with a certain probability of selection being associated with each. Here the underlying random variables may be random vectors each having the same dimension, in which case the miture distribution is a multivariate distribution. 2

14 In cases where each of the underlying random variable is continuous, the outcome variable will also be continuous and its probability density function is sometimes referred to as a miture density. A miture distribution constitutes a number of components which are often restricted to being FINITE, although in some cases the components may be COUNTABLE. More general cases (i.e. an UNCOUNTABLE set of component distributions), as well as the countable case, are referred to as COMPOUND DISTRIBUTIONS Finite or countable mitures n F = w i P i () j = f = w i p i () The sum is finite and the miture is called a finite miture, and in applications, an unqualified reference to a "miture density" usually means a finite miture. The case of a countable set of components is covered formally by allowingn =. Uncountable mitures Consider a probability density function p(;a) for a variable, parameterized by a. That is, for each value of a in some set A, p(;a) is a probability density function with respect to. Given a probability density function w (meaning that w is nonnegative and integrates to ), the function n j = f = A w(a)p(; a)da is again a probability density function for. A similar integral can be written for the cumulative distribution function. Mitures of parametric families Parameters in a miture distribution can be grouped together into a parametric family and they don t follow any arbitrary probability distributions. In such cases, assuming that it eists, the density can be written in the form of a sum as: f ; a,, a n = w i p i (; a i ) 3 n j =

15 for one parameter, or for two parameters, and so forth. f ; a,, a n, b,, b n = w i p i (; a i b i ) n j = Negative Binomial mitures The Negative Binomial distribution is constituted of two parameters r and p, and either of the parameters can be randomized to achieve the Negative Binomial miture. This project entails the scenarios where the parameter p has a continuous miing distribution with the probability g(p) such that f = r + p r p g(p)dp where f() is the Negative Binomial miture.2 Problem statement The project considers the Negative Binomial of the following format Pr X = /p = r + p r p The problem statement is to find the Negative Binomial mitures in the following distribution i. f = r + p r p g p dp p is within the [,] domain ii. f = r + p r p g r dr We need to develop these new distributions to help in solving the problem associated with over dispersed data. 4

16 Evaluating data that has several factors that affect the final outcome of the analysis need to be fitted using a heterogeneous distribution that will capture a majority of the aspects. This will reduce the ris that is associated with data loss or generality.3 Objectives Main objective To construct Negative Binomial mitures when the miing distributions come from the probability of success and the number of success as random variable. Specific objectives a. To epress the Negative Binomial mied distributions in eplicit forms, recursive forms and epectation forms when the miing distributions are: i. Classical beta distribution and its special cases. ii. The beyond beta distributions b. To construct the Negative binomial mitures when the number of successes taes i. Logarithmic distribution ii. Binomial distribution iii. Negative Binomial distribution c. To construct generalized Negative Binomial miture when the number of successes (r) is Eponential distribution d. To construct Geometric Mitures as special cases of Negative Binomial distribution.4 Methodology The methods applied to construct the Negative Binomial mitures include: i. Eplicit or direct method ii. Moment Generating Function method iii. Recursive relation method 5

17 .5 Literature review Here, we will analyze and access some of the wors that has been done in relation to Negative Binomial distribution and its mitures Wang, Z. (2) has done research on a three parameter distribution which is called the Beta Negative binomial (BNB) distribution. He derived the closed form and the factorial moment of the BNB distribution. A recursion on the pdf of the BNB stopped-sum distribution and a stochastic comparison between BNB and NB distributions are derived as well. He observed that BNB provides a better fit with a heavier tail compared to the Poisson and the NB for count data especially in the insurance company claim data Li Xiaohu et al (2) have studied the Kumaraswamy Binomial Distribution. They considered two models of the Kumaraswamy Distribution, derived their pdfs and other basic properties. The stochastic orders and dependence properties are also wored on by the group. Applications based on the incident of international terrorism and drining days in two wees were highlighted as well. Nadarajah, S. et al (22) proposed a new three parameter distribution for modeling lifetime date. This is the Eponential Negative Binomial distribution. It is advocated as most reasonable among the many eponential miture type distributions proposed in the recent years. Kotz S. et al (24) came up with other Continuous families of Distributions that are Beyond Beta with Bounded Support and applications. Properties studied included moments, CDF, Quartiles, maimum lielihood method of estimating parameters amongst others. The distributions highlighted include Triangular distribution, Standard Two sided Power series Sarabia J. et al (28) have done some wor on construction of multivariate distributions. The paper reviews the following set of methods: (a) Construction of multivariate distributions based on order statistics, (b) Methods based on mitures, (c) Conditionally specified distributions, (d) Multivariate sew distributions, (e) Distributions based on the method of the variables in common and (f) Other methods, which include multivariate weighted distributions, vines and multivariate Zipf distributions. Furman E.(27) has done some wor on the generation of the Negative Binomial Distribution from the sum of random variables. He also tals about the reasons why the negative Binomial distribution has been frequently proposed as a reasonable model for the number of insurance claims. 6

18 Ghitany et al (2) has also shown that Hypergeometric generalized negative binomial distribution has moments of all positive orders, is overdispersed, sewed to the right and leptourtic..6 Significance of the study and its applications This is an important project both in substance and timing. The objectives of this project as well as the analysis, as scheduled for discussion are important in identifying new distributions, their properties, identities as well as relevant applications. Statistical distributions are at the core of statistical science and are a leading requisite tool for its applications. Negative Binomial and especially its mitures are used widely in the insurance industry in the measure of total claims. This can be done by calculating the total claims distribution (according to the different methods nown) by spending a reasonable computing time and without incurring in underflow and overflow (this problem could be defined as a compatibility problem of the parameters). A finite miture of Negative Binomial (NB) regression models has been proposed to address the unobserved heterogeneity problem in vehicle crash data. This approach can provide useful information about features of the population under study. For a standard finite miture of regression models, previous studies have used a fied weight parameter that is applied to the entire dataset. However, various studies suggest modeling the weight parameter as a function of the eplanatory variables in the data. 7

19 Chapter 2 CONSTRUCTIONS AND PROPERTIES OF NEGATIVE BINOMIAL DISTRIBUTION 2.. Introduction Negative Binomial distribution can be constructed from a variety of methods. Some of the techniques are based on:. Binomial epansion 2. Mitures 3. Compound Poisson distribution 4. Katz Recursive relation in probability 5. Logarithmic series 6. Sums of a fied number of Geometric random variables 7. From eperiment Below are the brief discussions of these methods NBD based on Binomial epansion Epanding (a + b) r were r >, we get a + b r = r a r + r a r b + r 2 a r 2 b 2 + = r = a r b (2.) Let a = and b = θ Then θ r = r = ( θ) (2.2) Dividing both sides by θ r = = r θ θ r = = r θ θ r 2.3 8

20 p = r θ θ r for r > ; =,,2,.. (2.4) Is a pmf But r = r r r 2 r 3.. r 2 r 2 3. r = r r + r + 2 r r + 2 r +! r r + = ( ) Thus ( ) r = r + Replacing this in equation above p = r θ θ r for =,,2, p = r + θ θ r for =,,2,. ; < θ < (2.5) This is the Negative Binomial distribution with parameters r and θ = p 2.3. NBD based on mitures Negative Binomial distribution can be developed from miing Poisson distribution with Gamma distribution. Gamma distribution is considered as the prior distribution while the Poisson distribution a posterior distribution Poisson distribution This is epressed as the probability of a given number of events occurring in a fied interval of time and/or space if these events occur with a nown average rate and independent of time since the last event. A discrete random variable X is said to have a Poisson distribution with parameter λ >, if for =,, 2,, the probability mass function of X is epressed as: 9

21 Gamma distribution The miture Pr X = /λ = e λ λ ; =,,2, ; λ > (2.6)! g λ; α, β = λα e λ β ; for λ andα, β > (2.7) Γαβα f = Pr X = /λ g λ; α, β dλ Inserting the equations f =! Γαβ α f() =! Γαβ α e λ λ e λ λ/β λ +α λ α e λ/β dλ dλ f() =! Γαβ α e λ +β β λ +α dλ but +β λ e β λ +α dλ = Γ α + β + β α+ f = β Γ α +! Γαβα + β α+ f() = α +!! α! β α β + β α+ f = α + + β + β (2.8) for =,,2,. ; β, > The marginal distribution of X is a Negative Binomial distribution with r = α and p = +β

22 2.4. Construction from a fied number of Geometric random variables Let s r = X + X 2 + X X r denote te sum of iid random variables X i and r is fied The PGF of s N is given by H s = E s s r (2.9) = E s X +X 2 + +X r = E(s X s X 2s X 3. s X r ) = E(s X )E s X 2 E(s X 3). E(s X r ) (2.) (since X i are independent and identical) X r H(S) = E s (because X i s are identical) = G s r where G s is te pgf ofx i Let X i ~Geometric(p) Case The pmf of X is p = pq for =,,2, ; q = p; < p < and the pgf of Xis G s = E s X = p ; s < p qs Therefore H s = G s r H(s) = p qs r (2.) p qs r is the pgf of a Negative Binomial distribution with < p <, q + p = and r >

23 Proof H(s) = p qs r = p r qs r = p r = r (qs) = p r = r + (qs) = = r + = p = = E s s p r q s where p is the pdf of the negative Binomial distribution and hence the above statement is validated Case 2 The pmf of is p = pq for =,2,3, ; q = p; < p < The pgf of X in this case is given by G s = ps qs if s < q Therefore H s = G s r H(s) = ps qs r ps qs r is the pgf of a Negative Binomial distribution with p, q + p = and r > 2

24 Proof H(s) = ps qs r = ps r qs r = ps r = r (qs) = ps r = r + (qs) = ps r r (qs) + r qs r + 2 (qs) r + (qs) + = ps r = ps r =r =r r r qs r qs r = =r r p r q r s = p =r = E s where p is the pdf of the negative Binomial distribution and hence the above statement is validated p = r pr q r for p, p + q =, = r, r +, r + 2, s 2.5. Construction from logarithmic series Construction of Logarithmic distribution. Integrating both sides with respect to pwe get p = + p + p2 + p 3 + p 4 + p 5 + 3

25 dp p = + p + p2 + p 3 + p 4 + p 5 + dp log p = p + p2 2 + p3 3 + p4 4 + p p + (2.2) = = = = p p log p And hence the logarithmic distribution taes the form below p = p log p for =,2,3,. ; < p < (2.4) If we consider the associated power equation 2.2 and find its derivative, we get + p + p 2 + p p + = p (2.5) multiplying both sides of (2.5) by p ( p) + ( p)p + ( p)p 2 + ( p)p ( p)p + = Note that the generating term of this series ( p)p is the pmf of the Geometric distribution where =,,2, represents the number of successes before the first failure in a sequence of independent Bernoulli trials with parameter p Derivative of both sides of equation (2.5) + 2p + 3p 2 + 4p p + + p + = Multiply across by p 2 and consider the associated generating distribution ( p) 2 (2.6) + p p 2 = + p 2 p ; =,,2, = + = p 2 p p 2 p 4

26 which is the pmf of the negative Binomial distribution with parameters 2 and p for =,,2, Again tae the derivative of (2.6) p + 43p p p p 2 + = ( p) 3 Multiplying across by the reciprocal of 2 ( p) 3and taing the associated pmf p 3 p = p 3 p = p 3 p for =,,2, which is the pmf of the negative Binomial distribution with parameter 3 and p Taing the rth derivative of each side the power series, we find a series from which the NBD with parameters r and p can be obtained r! + r! r + p +! 2! p ! p ! r +!! p + = r! (2.7) p r which is associated with NBD with parameters r and p, and hence r +! r!! p r p = r + p r p (2.8) for =,,2, ; r > ; p This is a negative Binomial distribution with parameters r and p 2.6. Representation as compound Poisson distribution Let Y n n =,2,3, denote a sequence of identical and independent distributed random variables each having a logarithmic distribution log p wit a pmf p y = p y y log p for y =,2,3,. ; < p < (2.9) 5

27 Let N be a random variable independent of the sequence and suppose N~Poisson λ = rln p let s N = X + X 2 + X X N be sum of te independent random variables To calculate the pgf H(s) of X wic is te composition of teprobability Generating functions G N (s) and G y (s) and G N s = e λ s (2.2) G y s = ln ps ln p s < p (2.2) We get H s = G N G y s = e λ G y s = ep λ ln ps ln p = ep r ln ps ln p H(s) = p ps r (2.22) which is the probability generating function for the negative Binomial distribution Proof H(s) = q ps r = q r ps r = q r = r (ps) = q r = r + (ps) 6

28 = = r + = p = = E s s q r p s where p is the pdf of the negative Binomial distribution and hence the above statement is validated 2.7. Construction using Katz recursive relation in probability Consider the following recursive relation in probabilities f( + ) f() = P() Q() (2.23) Where P and Q are polynomials of f(. ) is the probability mass function in particular Let P() = α+β Q() + (2.24) which is the Katz relationship This implies f + f = α + β + ; =,,2 Let α and β Then f + = α + β f ; =,,2 (2.25) + We shall use two approaches to obtain the negative Binomial distribution. 7

29 2.7.. Iteration technique When = f = αf When = f 2 = α+β 2 f = α+β 2 αf When = 2 f 3 = α+2β 3 f 2 = α+2β 3 α+β 2 αf When = 3 f 4 = α+3β 4 f 3 = α+3β 4 α+2β 3 α+β 2 αf When = f = α+ β f f = α + β α + 2 β α + 3β 4 α + 2β 3 α + β 2 αf (2.26) f() = β α β + β α β β α β + 2 β α β + β α β! f f() = β α +! β f α!! β Therefore f = β α β + f ; =,,2,3 (2.27) Since equation 2.27 is a pmf, then; = f = f + f + = = f β = α β + f = 8

30 But f + β f = = = + β α β + α β + = But f = β α β + f ; =,,2,3 This implies that f() will be f = β = β α + β α + β (2.28) =,,2, If r = α is a positive integer β α β + = r + = r = β α β + = β = r = β r < β < Conclusion f + = α + β + f ; =,,2 (2.29) 9

31 For. r = α is a positive integer β 2. < β < 3. α > f = β α β + β r = f = r + α β + < β < ; =,,2,3. β β r This is a Negative Binomial distribution with parameters and p = β β β r (2.3) Using the Probability Generating Function (PGF) technique Remember Let α and β Then f + = α + β f ; =,,2, ; α > ; < β < + [ + ] f + s = = = α + β f s (2.3) Define G s = f() s (2.32) = G s = f = s 2

32 G s = ( + )f + = s G s = α f() s + β = = f() s G s = αg s + βs f = s (2.33) G s = αg s + βsg s βs G s = αg βs G s s = αg s G s G s = α βs s (2.34) ln G(s) = α ln βs + ln C β ln G(s) = ln βs α β + ln C ln G(s) = ln C βs α β G s = C βs α β (2.35) Let s = G = C( β) α = C( β) β C = α ( β) β α β α = ( β) β (2.36) G s = βs β α β 2

33 Let α β = r be a positive integer Suppose p = β and p + q =, ten G s = βs β G s = β βs r r 2.37 G s = p qs r 2.38 This is a PGF of a NBD with parameters r > and p Note: proof is carried out in section 2.4 above under case 2.7. From eperiment Let X be the number of failures preceding the rth success in an infinite series of independent trials with a constant probability of success p r + = te toatal number of trials ecluding te rt success r + = Te number of ways of obtaining failures and r success Prob X = = = r + r + p r p. p p r p (2.39) for r > ; p ; =,,2,3. Alternatively, denote the probability of X= by p r, =,,2, We can formulate a difference equation for p r as follows. p r = Te probability of te first trial being a success followed by a prob of a failure wit r successes Or p r = Te probability of te first trial being a failure followed by failures wit r successes p r = pp r + qp r ; =,2,3, and r

34 In terms of pgf, p r s = p p r s + q p r s 2.4 = = = But G s, r = = p r s But p r = p r and p r = p r Put r = => G s, = But G s, r p r = p G s, r p r + qsg s, r 2.42 G s, r p r = pg s, r p r + qsg s, r p qs qs G s, r = pg s, r G s, G s, r = p G s, r qs G s, = p = s = p + p = s p = Te prob of failures before zero success = p = and p = for G s, = p p G s, = qs qs Put r = 2 G s, 2 = p p G s, = qs qs 2 In general G s, r = p qs r (2.43a) This is the pgf of a NBD where the random variable X is the number of failures before the rth success. 23

35 The other case is to consider X to be the total number of trials required to achieve r successes. Let us denote X= with Probability p r. The corresponding difference equation is: p r = pp r + qp r ; =,2,3, and r 2.44 In terms of pgf, p r s = p p r s + q p r s 2.45 = = = But G s, r = = p r s But p r = p r and p r = p r Put r = => G s, = But G s, r p r = p sg s, r + qsg s, r G s, r p r = psg s, r + qsg s, r ps qs qs G s, r = psg s, r G s, G s, r = ps G s, r qs G s, = p = = p + p = s s p = Te prob of failures before zero success = p = and p = for G s, = G s, = ps qs. = ps qs

36 Put r = 2 G s, 2 = ps ps G s, = qs qs 2 In general G s, r = ps qs 2 (2.43b) This is the pgf of a NBD where the random variable X is the number of trials required to achieve r> successes. Properties From the construction of Negative Binomial distribution, we have established that Negative Binomial distribution can be epressed in the following formats. p = r + p r ( p) for r > ; < p < ; =,,2,3. This is a Negative Binomial distribution with parameters r and p, represents the total number of failures before the rth success 2. p = r pr p r = r, r +, r + 2, If p is a sequence of independent Bernoulli trials and random variable is taen to denote the trial, at which the rth success occurs, where r is a fied integer Probability generating function The probability generating function of a Negative Binomial distribution is given by the following equation G s = E s = p s = = = r + p r q s 2.47 = p r = r + (qs) 25

37 = p r = r (qs) = p r qs r G s = p qs r 2.48 This is the probability generating function for the negative Binomial distribution with Consider qs <, p, q + p = and r > p = r + p r ( p) for r > ; < p < ; =,,2,3. The first derivative of the pgf of this distribution is G s = let s = G = rqp r 2.49 qs r+ rqp r q r+ G = rq p 2.5 The second derivative of the pgf of the negative Binomial distribution is G s = r r + q2 p r 2.5 qs r+2 let s = G = r r + q2 p Mean E X = G E X = r p p 2.53 Variance var X = G + G G 2 var X = rq p

38 Factorial moments of Negative Binomial distribution For a natural number r, the rth factorial moment of a probability distribution on the real or comple numbers, or in other words, a random variable X with that probability distribution is E X = μ X = E X X X 2 X Where E refers to the epectation and = 2 + is the falling factorial. Thus for the negative binomial distribution μ X = Γ r + Γr p p for =,2, μ X = E X = r r! q r! p = rq p 2.57 μ 2 X = E(X 2 X) = Γ r + 2 Γr p 2 p μ 2 X = r r + q2 p So E(X 2 X) = r r + q2 p E(X 2 ) = r r + q2 p 2 + rq p E(X 2 ) = rq p 2 r + q + p 2.6 μ 3 X = E(X 3 3X 2 + 2X) 2.62 = r + 2 r + r q3 p

39 So E(X 3 3X 2 + 2X) = r + 2 r + r q3 p 3 E(X 3 ) = r + 2 r + r q3 p 3 + 3E(X2 ) + 2E(X) E(X 3 ) = r + 2 r + r q3 p rq p r + q p + 2 rq p 2.64 = rq p 3 r + 2 r + q2 + 3p 2 r + q p + 2p2 = rq p 3 r + 2 r + q2 + p 2 3 r + q p + E(X 3 ) = rq p 3 r + 2 r + q 2 + p 3 r + q + p 2.65 μ 4 X = E X X X 2 X = E (X 4 6X 3 + X 2 6X) = E(X 4 ) 6E X 3 + E X 2 6E X μ 4 X = r + 3 r + 2 r + r q4 p Hence E(X 4 ) = r + 3 r + 2 r + r q4 p 4 + 6E X3 E X 2 + 6E X 2.68 = r + 3 r + 2 r + r q4 p rq p 3 rq p 2 r + q + p + 6 rq p r + 2 r + q 2 + p 3 r + q + p E(X 4 ) = rq p 4 r + 3 r + 2 r + q3 + 6p r + 2 r + q 2 + p 3 r + q + p p 2 r + q + p + 6p

40 Sewness and urtosis Sewness is a measure of symmetry or the lac of symmetry. A distribution, or data set, is symmetric if it loos the same to the left and right of the center point. Kurtosis is a measure of whether the data are peaed or flat relative to a normal distribution. That is, data sets with high urtosis tend to have a distinct pea near the mean, decline rather rapidly, and have heavy tails. Data sets with low urtosis tend to have a flat top near the mean rather than a sharp pea. A uniform distribution would be the etreme case. According to Pearson s moment coefficient of sewness, the sewness of a random variable X is the third Standard moment denoted by Suppose X i, i =,2,3,. N are univariate data that follows a Negative Binomial distribution then = E μ s = E μ + 3μ 2 + μ 3 s 3 = E( 3 ) 3μE 2 + 3μ 2 E + μ 3 s = E( 3 ) 3μ E 2 + 3μ 2 + μ 3 s 3 = E 3 3μs 2 + μ 3 s 3 = rq p 3 r + 2 r + q 2 + p 3 r + q + p 3 rq p rq p 2 + rq p 3 p 2 rq Negative values for the sewness indicate data that are sewed left and positive values for the sewness indicate data that are sewed right 29

41 urtosis Kurtosis = N i= X i μ 4 N s 4 = E μ s = E X 4 4μE(X 3 ) + 6μ 2 E(X 2 ) 3μ 4 s = rq p 4 r + 3 r + 2 r + q3 + 6p r + 2 r + q 2 + p 3 r + q + p p 2 r + q + p + 6p 3 4μ rq p 3 + 6μ 2 rq p 2 r + q + p 3 rq p 4 p 2 rq r + 2 r + q 2 + p 3 r + q + p 2 3

42 CHAPTER 3 BETA - NEGATIVE BINOMIAL MIXTURES 3.. Introduction From chapter 2, we have identified the following forms of Negative Binomial distribution.. p = r+ p r p for =,,2,. ; p (3.) with parameters r and p. represents the total number of failures before the rth success and; 2. p = r pr p r = r, r +, r + 2, ; p (3.2) If p is a sequence of independent Bernoulli trials and random variable is taen to denote the number of trials required to produce r successes, where r is a fied integer In this chapter we are going to consider the Negative Binomial distribution as given in (3.) when r is fied and p is varying between and. The distribution of p is the classical beta distribution is given by f = α β B α, β < < ; α, β > 3.3 This will act as acts as the miing distribution. We shall also use the special cases of the classical beta distribution. These are. Uniform distribution 2. Power function distribution 3. Truncated beta distribution 4. Arc sine distribution Apart from the beta distribution and its special cases, we shall also consider 5. Confluent Hypergeometric distribution 6. Gauss Hypergeometric distribution 3

43 The mied negative Binomial distributions obtained by various miing (prior) distributions will be epressed. Eplicitly (where integration is possible) 2. Recursively 3. Using method of Moments In section 3.2 we shall have a brief discussion of the forms. For the other sections we shall briefly introduce the miing distributions before miing them with the negative Binomial distribution A brief discussion of the various forms of epressing the mied distribution Eplicit form The mied distribution is epressed as f = r + p r p g p dp 3.4 If the integration is possible then we say that f() is epressed eplicitly. However, in most cases this is not possible so we resort to alternative forms Method of moments f = r + p r p g p dp (3.5) f = r + = p r+ g p dp (3.6) f = r + = E p r+ f = Γ r + Γ r! = E p r+ 3.7 for p ; r > ; =,,2, E p r+ is te moment of order r + about te origin of te miing distribution 32

44 Recursion One way of obtaining recursions is by considering the ratios of two consecutive probabilities i.e. f()/ f(- ) 3.3. Classical beta Negative Binomial distribution Construction of Classical Beta Distribution Classical beta distribution can be constructed in various ways. Method We can consider a beta function which is epressed in the following format. B α, β = α β d (3.8) If we divide both sides byb α, β, we get = α β d (3..9) B α, β The right hand side of equation 3.6 is a pdf since the integral is equal to and hence the pdf is epressed as follows This is the Beta distribution. f X = ; α, β = α β B α, β ; < < ; α, β > (3.) Method 2 An alternative way of constructing a beta distribution is shown below. Let and 2 be two stochastically independent random variables that have Gamma distributions and joint pdf f, 2 = Γα α e Γβ 2 β e 2 (3.) 33

45 f, 2 = ΓαΓβ α 2 β e e 2 < <, < 2 < Let Y = + 2 and p = + 2 Therefore = yp, 2 = y yp = y( p) Then g p, y = ΓαΓβ py α y p β e yp e y p J (3.2) Where J = d dy d 2 dy d dp d 2 dp = p y p y J = yp y p = y = y Therefore equation 3.2 becomes g p, y = ΓαΓβ pα p β y α +β + yp y( p) e g p, y = ΓαΓβ pα p β y α+β + e y (3.3) wit < y < and < p < Integrating g p, y with respect to y, the results to the marginal pdf is given by g 2 p, y = ΓαΓβ pα p β y α+β e y dy (3.4) Introducing Γ(α + β) 34

46 g 2 p, y = pα p β Γ α + β ΓαΓβ y α+β e y dy Γ α + β g 2 p, y = pα p β Γ(α + β) ΓαΓβ. = p α p β B(α, β) g 2 p, y = pα p β B α, β ; p, α, β > (3.5) Equation 3.6 is also called the Classical Beta distribution with parameters α and β Properties of the Classical Beta distribution The jth moment of this pdf (classical beta) about the origin is given by E P i = p α+i p β B α, β dp E P i = B α + j, β B α, β (3.6i) E P i = α + j! α + β! α + β + j! α! (3.6ii) The Mean of the classical beta is therefore E P = α α + β (3.7) The 2 nd moment about the origin is E P 2 = α + α α + β α + β + (3.8) And finally the variance of the miing distribution becomes Var P = E P 2 (E(P)) 2 Var P = αβ α + β 2 α + β + (3.9) 35

47 3.3.3 The miture In eplicit form f = r + p r p pα ( p) β B(α, β) dp = r + B α, β p r+α p +β dp (3.2) = r + B r + α, + β B α, β were =,,2, (3.2a) = r +! B r + α, + β r!! B α, β were =,,2,, (3.2b) = Γ r+ B r+α,+β Γ r! B α,β ; =,,2, ; r, α, β > (3.2c) = Γ r + Γ α + β Γ r + α Γ + β Γ r! ΓαΓβΓ r + α + + β ; =,,2, ; r, α, β > (3.2d) Z. Wang (2). One mied negative binomial distribution with application. Journal of Statistical Planning and Inference Using Method of Moments f = Γ r + Γ r! = E(p r+ ) E p r+ is te moment of order r + about te origin of te miing distribution But E P i = B α + j, β B α, β f = Γ r + Γ r! = B α + r +, β B α, β ; for =,,2, ; r, α, β > (3.22) 36

48 Miing using Recursive relation There are three ways of applying the recursive relation in this miture, namely. Using ratio of the conditional distribution 2. Using ratio of the mied distribution 3. Using a dummy function. Using ratio of the conditional distribution p r = f /p g p dp (3.23) p(/p)is a Negative Binomial distribution in this case f /p = r + p r p (3.24) Substituting wit in equation 3.24 we get f ( )/p = r + 2 p r p (3.25) Dividing equation 3.24 by equation 3.25 we get f(/p) f ( )/p = r + p r ( p) r + 2 p r p f /p = r + r + 2!! r!! r! ( p)f ( )/p r + 2! f /p = r + p f ( )/p (3.26) Substituting equation (3.26) into equation (3.23) p r () = r + p f ( )/p g p dp 37

49 p r = r + f ( )/p g p dp pf ( )/p g p dp (3.27) Consider f ( )/p g p dp (3.28) This can be epressed as r + 2 p r p g p dp = p r (3.29a) Consider pf ( )/p g p dp (3.3) = r + 2 p r+ p g p dp = r + 2 r r p r+ p g p dp = r + 2 r p r+ ( ) pf ( )/p g p dp = r r + p r+ (3.3a) Substituting equation 3.29a and 3.3a into 3.27 the Beta - Negative Binomial recursive miture becomes p r = r + p r r r + p r+ (3.3) r >, =,,2.. ; 38

50 2. Using ratio of the mied distribution From equation 3.2b f = r +! B r + α, + β r!! B α, β were =,,2, Considering the below ratio. f f r +! B r + α, + β = r!! B α, β r!! B α, β r +! B r + α, + β f f = r + β + r + + α + β f = β + r + r + + α + β f 3.32 for r, α, β >, =,,2 ; 3. Using a dummy function Consider the miture equation below f = r + B α, β p r+α p +β dp 3.33 Introducing the dummy function I r, α, β = Integrating the integral by parts. B α, β f r+ = p r+α p +β dp 3.34 p r+α p +β dp udv = uv vdu = udv Let 39

51 u = p r+α and dv = p +β dp Hence du = r + α p r+α 2 dp and v = p +β + β p r+α p +β dp = p +β p r+α + β + r + α + β p +β p r+α 2 dp = r + α + β I + r, α, β 3.35 I r = B α, β f r+ = r + α + β I + r B α, β f r+ = r + α + β B α, β f + r This leads to f = r + + β r + 2 f 3.37 for r, β, α > and =,2,3, Identity We can draw an identity based on the result from eplicit miing and that from method of moments as follows. B r + α, + β = = B α + r +, β ; for =,,2, ; r, α, β >

52 3.3.4 Properties of Beta Negative Binomial Distribution Mean Considering the result got from the recursive relation miture i.e. equation 3.3 p r () = r + p r ( ) r r + p r+( ) E X = = p r () E X = r + = p r r r + p r+ (3.39) = r + = p r r p r+ = E X = r + = r + 2 p r p r = r + p r+ p (3.4) Consider the first part of the equation = r + r + 2 p r p (3.4a) This can be epressed as follows = r +! p r p r!! (3.4b) But r +! r + = r r!! Hence 4

53 = r +! p r p = r r!! = r + p r p (3.4) = r p = r + p r+ p = r p Since = r + p r+ p = Consider the second part of the equation r = r + p r+ p (3.42) This can be epressed as r = r + p r+ p = r Since The mean therefore becomes = r + p r+ p = E X = r p p (3.43) Variance E X 2 = X 2 p r () = E X 2 = r + = r + 2 p r p r = r + p r+ p (3.44) 42

54 Consider the first part of the equation above = r + r + 2 p r p (3.44a) It can be epressed or manipulated as follows p = r +! p r p r! ( )! = p = 2 r +! r!! p r p = p = 2 r + p r p But considering a Negative Binomial Distribution E X 2 /p = = 2 r + p r p E X 2 /p = r p + r p p 2 And hence p = r +! p r p = r! ( )! r + r p p 2 Consider the second part of the equation r = r + p r+ p (3.44b) This can be epressed as follows = rp p = r +! p r ( p) r r!! 43

55 = rp p = 2 r +! r r!! p r ( p) = p p = 2 r + p r ( p) The second moment therefore becomes = p r p + r p X ( p) p 2 r + r p = p E X 2 = E X 2 = r + r p p 2 r + r p p r + r p p p (3.45) Hence var X = r + r p p p r2 ( p) 2 p 2 var X = r(r2 p) p 2 (3.46) Moment Generating Function Consider the eplicit miture of Beta distribution with the Negative Binomial distribution f X = = ᴦ r + ᴦ r! B r + α, + β B α, β ; =,,2, ; r, α, β > MGF g t = E t n ; n =,,2,. g t = H r, β; r + α + β, t pr Z = (3.47) 44

56 Where H a, b; c, t = n= a (n ) b (n ) c (n ) n! t n is te Hypergeometric function (n) = Γ + n Γ a = r; b = β; c = r + α + β; t = t g t = n= r (n) β (n) r + α + β (n) t n n! g t = n= Γ r + n Γr Γ β + n Γβ Γ r + α + β Γ r + α + β + n t n n! (3.48) Differentiate g(t) with respect to t g t = n n= Γ r + n Γr Γ β + n Γβ Γ r + α + β Γ r + α + β + n t n n! g t = n= Γ r + n Γr Γ β + n Γβ Γ r + α + β Γ r + α + β + n t n n! (3.49) Hence g t = n n n= Γ r + n Γr Γ β + n Γβ Γ r + α + β Γ r + α + β + n t n 2 n! (3.5) g t = n= Γ r + n Γr Γ β + n Γβ Γ r + α + β Γ r + α + β + n t n 2 n 2! (3.5) 45

57 3.4. Special cases of beta negative Binomial distribution Uniform Negative Binomial distribution Uniform distribution Construction Consider a beta distribution defined as follows g p = pα p β B α, β < p < ; α, β > If we let α = β = We get the uniform distribution [,] given by g p = < p < elsewere (3.52) Properties Moment of order j about the origin of the uniform distribution g(p) E P j = = p j j + p dp j + E p j = j + (3.53) Mean E P = 2 (3.54) Variance var P = 2 (3.55) 46

58 The mitures Eplicit miing f = r + p r p g(p)dp where g(p)is the uniform distribution. f = r + p r p dp f = r + B r +, + p r p B r +, + dp f = r + B r +, + f = r +! ᴦ r + ᴦ + r!! ᴦ r f = r r + + (r + ) =,,2, ; r > (3.56) Using recursive relation The miture between Negative Binomial and uniform distribution can be epressed in a recursive format in the following two ways. st Form Here we consider the miture from eplicit miing i.e. equation 3.56 f = r r + + r + Formulation and woring the below ratio. f f = r r + + r + r + r + r 47

59 Hence f = r + f (3.57) r nd Form Here we introduce a dummy functioni r that is of a Beta format as follows. f = r + p r p dp f r+ = pr p dp I r = f r+ = pr p dp (3.58) Using integral by parts Let u = p du = p dp v = pr+ r + dv = p r dp u dv = uv v du I r = p pr+ + r + r + p r+ p dp I r = r + p r+ I r = p dp (3.59) r + I r + 48

60 I r = f r+ = r + f r+ f = r + r+ r+ f Hence f = r f 3.6 r + Using Method of Moments f = Γ r + Γ r! = E(p r+ ) E p r+ is te moment of order r + about te origin of te miing distribution E p j = j + f = Γ r + Γ r! = r for r > ; =,,2, Identity We can draw an identity based on the result from eplicit miing and that from method of moments as follows. r r + + r + = Γ r + Γ r! = r + + r!! r + + r +! = = r + + for r > ; =,,2, 49

61 Power function - negative Binomial distribution Power function distribution Construction Consider the beta function g p = p α ( p) β B(α, β) elsewere < p < ; α, β > Let β = then g p = p α ( p) B(α, ) g p = αpα < p < ; α > elsewere (3.62) This is the pdf of a power function distribution with parameter α Properties The moments of order j about the origin is E P j = αp α p j dp = α p α+j dp = α α+j p α + j E P j = α α + j (3.63) 5

62 Mean E P = α α + (3.64) Variance var p = E p 2 E p 2 var p = var p = α α + 2 a 2 α + 2 α α + 2 α + 2 (3.65) The miing Eplicit miing f = r + p r p g(p)dp g(p)is the power function distribution g p = αp α f = r + p r p αp α dp f = r + α p r+α p dp f = f = r + r + α B(r + α, + ) p r+α p B(r + α, + ) dp α B r + α, + (3.66) for =,2. ; a, r > This is the density function of the Negative Binomial Power function epressed eplicitly 5

63 Using Method of Moments f = Γ r + Γ r! = E(p r+ ) E p r+ is te moment of order r + about te origin of te miing distribution E P j = α α + j f = Γ r + Γ r! = α α + r for r, α > ; =,,2, Using the recursive relation epression We can achieve this by the use of two different ways. st Form Consider the result from eplicit miture as shown below. f = r + α B r + α, + Epressing this a ratio f f = r+ B r + α, + B r + α, r+ 2 Hence f = r + f 3.68 r + α + 52

64 2 nd Form Consider the eplicit epression below f = r + α p r+α p dp Manipulate this to a dummy function I r, α as shown below Let f r+ α = pr+α p dp I r, α = f r+ α = pr+α p dp (3.69) Integrating the integral by parts Let u = p du = p dp dv = p r+α dp v = pr+α r + α u dv = vu v du I r, α = p pr+α + r + α r + α p r+α p dp I r, α = r + α p r+α p dp (3.7) I r, α = f r+ α = r + α I r, α f r+ α = r + α f r+ 3 α

65 f = r + α r+ r+ 3 f Hence f = r + r + 2 r + α r f 3.72 Identity We can draw an identity based on the result from eplicit miing and that from method of moments as follows. r Γ r + = r + + (r + ) Γ r! = α α + r + r!! r + + r +! = = α α + r for =,,2, ; r, α > Arcsine negative Binomial distribution Arcsine distribution Construction The standard Arcsine distribution is a special case of the beta distribution with α = β = 2 Consider a beta distribution below g p = pα p β B α, β p ; α, β > If α = β = 2 Then 54