Objective Bayesian Analysis for the Lomax Distribution arxiv: v1 [stat.me] 26 Feb 2016

Transcription

1 Objectve Bayesan Analyss for the Lomax Dstrbuton arxv:62.845v [stat.me] 26 Feb 26 Paulo Ferrera, Jhon Gonzales, Vera Tomazella, Rcardo Ehlers, Francsco Louzada, Evelny Slva February 26 Abstract In ths paper we propose to make Bayesan nferences for the parameters of the Lomax dstrbuton usng non-nformatve prors, namely the Jeffreys pror and the reference pror. We assess Bayesan estmaton through a Monte Carlo study wth 5 smulated data sets. To evaluate the possble mpact of pror specfcaton on estmaton, two crtera were consdered: the bas and square root of the mean square error. The developed procedures are llustrated on a real data set. Keywords: Bayesan nference, Jeffreys pror, Lomax dstrbuton, reference pror. Introducton The Lomax dstrbuton [7], also known as the Pareto Type II dstrbuton or smply Pareto II), s a heavy-tal probablty dstrbuton often used n busness, economcs and actuaral modelng. It s essentally a Pareto dstrbuton that has been shfted so that ts support begns at zero [22]. The Lomax dstrbuton has been appled n a varety of contexts rangng from modelng the survval tmes of patents after a heart transplant [] to the szes of computer fles on servers [4]. Some authors, such as [9], suggest the use of ths dstrbuton as an alternatve to the exponental dstrbuton when data are heavy-taled. The man objectve of ths paper s to make Bayesan nferences for the parameters of the Lomax dstrbuton usng non-nformatve prors, namely the Jeffreys pror [6] and the reference pror [7]. Then, we perform a smulaton study to compare the effcency of the Bayesan approach for estmatng the model parameters under these two prors, and check for the possble mpact of pror specfcaton. We also show how to represent the Lomax dstrbuton n a herarchcal form by augmentng the model wth a latent varable whch makes the Bayesan computatons easer to mplement. Ths would also allow Correspondng author. Emal: paulohenr@ufba.br

2 the user to mplement nferences usng all-purpose Bayesan statstcal packages lke WnBUGS [8] or JAGS [9]. The remander of ths paper s organzed as follows. In Secton 2, we present the Lomax dstrbuton and lst some of ts propertes. In Secton 3, we formulate the Bayesan model usng non-nformatve prors. In Secton 4, a smulaton study s presented. In Secton 5, the methodology s llustrated on a real data set. Some fnal comments are gven n Secton 6. 2 Model defnton Here, we use the defnton that appears, for example, n [5]. Defnton 2.. A contnuous random varable X has a Lomax dstrbuton wth parameters α and f ts probablty densty functon s gven by fx, α) α + x ) α+), x, where α > and > are the shape and scale parameters, respectvely. We refer to ths dstrbuton as Lomax, α). The medan s 2 /α ) and the mode s zero. The hazard functon s gven by hx, α) α + x ), x, whch s a decreasng functon of x, thus makng ths a sutable model for components that age wth tme. The survval functon s gven by Sx, α) + x ) α, x. We note also that the Lomax dstrbuton can be expressed n the followng herarchcal form ) λ X, λ Exponental, λ α Gammaα, ). Ths follows from wrtng the jont densty of X and λ as fx, λ)fλ α) { Γα) λα exp λ + x )}. So, the margnal densty of X s gven by fx) α Γα) Γα + ) Γα) + x ) α+), { λ α exp λ + x )} dλ + x ) α+) 2

3 and we can conclude that X Lomax, α). Usng ths mxture representaton, t s not dffcult to see that the uncondtonal mean and varance of X are gven by EX) E[λ ] α, α >, V arx) 2 { E [ λ ] 2 + V ar [ λ ]} α 2 α ) 2 α 2), α > 2, snce λ IGα, ), where IGa, b) denotes the Inverse Gamma dstrbuton wth parameters a and b, mean b/a ), a >, and varance b 2 /a ) 2 a 2), a > 2. Now, suppose that X X,..., X n ) s a random sample of sze n from the Lomax dstrbuton. We assume that the mxng parameters λ λ,..., λ n ) are a pror ndependent. The complete condtonal dstrbuton of λ usng the herarchcal form s gven by so that fλ x,, α) fx, λ) fλ α) n n λ exp λ x /) λ α exp λ ) n λ α exp { λ + x )}, λ x, λ, α, Gamma α +, + x ). Agan, usng the herarchcal form, we obtan the complete condtonal dstrbutons of α and as n ) α fα x, λ, ) fλ α) π, α) [Γα)] n λ π, α), f x, λ, α) fx, λ) π, α) n exp { } n λ x π, α). We note that, usng ths representaton of the Lomax dstrbuton, each observaton X s assocated wth one mxng parameter λ, whose posteror mean or medan can be used to dentfy a possble outler. 3 Pror specfcaton We now complete the Bayesan model by specfyng a pror dstrbuton for α and. We consder non-nformatve prors on these parameters and verfy the exstence of ther posteror dstrbuton. 3

4 3. Jeffreys pror A commonly used objectve pror n Bayesan analyss s Jeffreys pror [6], whch s defned as π J, α) I, α) /2, where I ) stands for the Fsher nformaton matrx. Ths s gven by from whch we obtan α 2 α + 2) I, α) n α + ) π J, α) α + ), α 2,, α >. α + )α /2 α + 2) /2 Consderng ndependence between the parameters, the Jeffreys jont pror for, α) s gven by π IJ, α) π)πα) /α. Then, substtutng π, α) n the expressons for the complete condtonal denstes, we obtan fα x, λ, ) for the dependent Jeffreys pror and n ) α λ α + )α /2 α + 2) /2 Γ n α) fα x, λ, ) n ) α αγ n λ α) for the ndependence case. So, the complete condtonal dstrbuton of α s not of standard form and a Metropols-Hastngs algorthm [3, ] s used to sample ts values. The complete condtonal densty of s gven by { } f x, λ, α) n+) exp n λ x for both dependent and ndependent Jeffreys prors. It then follows that ) n x, λ, α IG n, λ x. 3.2 Reference pror Reference prors were frst proposed by [7] and further developed by [2, 3, 4,, 6, 5], among others. The dea s to specfy a pror dstrbuton such that, 4

5 even for moderate sample szes, the nformaton provded by the data should domnate the pror nformaton. In partcular, [3] dscuss the constructon of a non-nformatve pror that gves a dfferent treatment for parameters of nterest and nusance parameters. When there are nusance parameters whch s the case n ths paper), one must establsh an order parametrzaton between the nterest and nusance parameters. The followng proposton s borrowed from [8] and adapted to the Lomax case. Proposton 3.. Let f x, α),, α) α) IR IR be a probablty model. Suppose that the jont posteror dstrbuton of, α) s asymptotcally normal wth covarance matrx S ˆ, ˆα) I ˆ, ˆα), where ˆ and ˆα are consstent estmators of and α, respectvely. Then, f s the parameter of nterest and α s the nusance parameter, ) The condtonal reference pror of α gven s π α ) [I 22, α)] /2, α α ). ) If π α ) s not proper, a compact approxmaton {α ),, 2,... } to α ) s requred and the reference pror of α gven s π α ) α ) [I 22, α)] /2 [I 22, α)] /2 dα, α α ). ) The sequence of prors can be obtaned as [ ] π ) exp π α ) log s /2, α) dα, α ) where s /2, α) I, α) I I 2 I 22 I 2. v) The reference posteror dstrbuton of gven data x x,..., x n ) s [ n ] π x) π) fx, α) πα )dα. α) Proof. See a heurstc justfcaton n [8]. Corollary 3.. If the nusance parameter space α ) α s ndependent of, and the functons s /2, α) and I /2 22, α) factorze n the form [s, α)] /2 f ) g α), [I 22, α)] /2 f 2 ) g 2 α), 5

6 then π) f ) and π α ) g 2 α). Thus, the reference pror relatve to the ordered parametrzaton, α) s gven by π R, α) π)πα ) f ) g 2 α). Proof. See proof of Theorem 2 n [8]. Proposton 3.2. The jont reference pror for the Lomax model wth parameters and α,, α) α) IR IR, s gven by where π) f ). π R, α) π) α, Proof. The nverse of the Fsher nformaton matrx s gven by I, α) S, α) n 2 α + 2)α + ) 2 α αα + 2)α + ) αα + 2)α + ) α 2 α + ) 2 from whch we obtan [ [s, α)] /2 2 α + 2)α + ) 2 ] /2 α /2 α α + 2) /2 α + ). Consderng Corollary 3.,, f ) and g α) α /2 α + 2) /2 α + ). From the Fsher nformaton matrx, we also have that [I 22, α)] /2 and fnally, f 2 ) and g 2 α) /α. [ ] /2 α 2 α, So, consderng as the parameter of nterest and α as the nusance parameter, we conclude that the jont reference pror for, α) s gven by π R, α) f )g 2 α),, α >. α Note that ths reference pror concdes wth the Jeffreys pror under the assumpton of ndependence of parameters π IJ, α)) n the Lomax dstrbuton. Consequently, the complete condtonal dstrbutons are the same as for the ndependent Jeffreys pror. 6

7 Applyng the Bayes theorem, the jont posteror densty s gven by π, α x) α ) n n α n n+) n + x ) α+) α + x ) α+). ) Proposton 3.3. The posteror dstrbuton ) s mproper for n and proper for n >. Proof. See Appendx A. 4 Smulaton study In ths secton, we perform a Monte Carlo study to evaluate the methodology descrbed n the prevous secton. We generated m 5 replcatons of samples of szes n 5,, 5, 2, 3 and 5 from the Lomax dstrbuton, consderng parameter values 2 and α.5. The model was then estmated usng Jeffreys and reference prors. We used the Metropols-Hastngs algorthm and the Gbbs sampler mplemented n software R [2] to smulate two chans of values from the posteror dstrbuton. A total of, teratons wth jumps of and a burn-n of, were performed for each chan, thus leadng to a fnal sample of, values for each chan. Also, the Gelman and Rubn s Monte Carlo Markov Chan MCMC) convergence dagnostc [2] provded n the R package CODA [2] was used to montor convergence of the two parallel chans. Let ˆθ j) be the estmate of parameter θ for the j-th replcaton, j,..., m. These are the parameter posteror means calculated from the 2, smulated values for each replcaton. To evaluate the estmaton method, two crtera were consdered: the bas and square root of the mean square error or smply root mean square error, rmse), whch are defned as bas m ˆθ j) θ, m j rmse m ˆθj) ) 2. θ m The results from the smulated experment appear n Tables -4 and n Fgure. Tables and 3 show the computed posteror mean, standard devaton SD), 95% credblty nterval CI), bas and rmse for each parameter and sample sze, consderng Jeffreys and reference prors, respectvely. We note that the performances are barely smlar whatever the pror we adopt. For the parameter j 7

8 of nterest the bas s relatvely small and negatve for both prors, whle for the nusance parameter α the bas becomes postve for sample szes n when a Jeffreys pror s adopted. Wth respect to accuracy, we obtaned good results for and α wth relatvely small rmse for both Jeffreys and reference prors and moderate sample szes. Overall, the bases tend to reduce when movng from a Jeffreys to a reference pror, however ths s manly observed n the rmse for sample szes n 2 see Fgure ). The Gelman and Rubn s dagnostc values presented n Tables 2 and 4 are close to, whch means that the algorthm converged, ndependent of the ntal values adopted. Note also from these tables that the acceptance rates were better hgher) when usng the reference pror. Table : Results of the smulaton study usng Jeffreys pror, consderng true values of parameters: 2 and α.5. sample sze parameter mean SD 95% CI bas rmse [.8639 ; 9.28] α [.8596 ; 4.627] [.549 ; ] α [.966 ; 2.948] [.6 ; 3.767] α [.897 ; 2.456] [.2327 ; ] α [.63 ; 3.344] [.337 ; ] α [.2423 ; ] [.4456 ; ] α [.496 ;.7245] Table 2: Evaluaton of the algorthm usng Jeffreys pror. sample sze acceptance rate Gelman-Rubn Applcaton In order to llustrate the methodology proposed n ths paper, we consder a sample of computer fle szes n bytes) for all 269 fles wth the *.n extenson on a Wndows-based personal computer. These data can be downloaded from the webste dgles/downloads/data. A prevous work by [4] has demonstrated the superorty of the Lomax dstrbuton over several other compettors for modelng such fle szes. Those authors also pro- 8

9 Table 3: Results of the smulaton study usng reference pror, consderng true values of parameters: 2 and α.5. sample sze parameter mean SD 95% CI bas rmse [.8876 ; 8.234] α [.875 ; ] [.864 ; 5.694] α [.96 ; 3.789] [.86 ; ] α [.79 ; ] [.2539 ; ] α [.99 ; ] [.3876 ; ] α [.825 ; 2.78] [.468 ; 2.899] α [.2259 ;.9224] Table 4: Evaluaton of algorthm usng reference pror. sample sze acceptance rate Gelman-Rubn vde techncal nformaton suggestng that the dstrbuton should have nfnte varance.e. α < 2) n ths context. The samples for the Jeffreys and reference posteror dstrbutons of the parameters and α were obtaned by the Gbbs sampler and Metropols-Hastngs algorthm,.e. through MCMC methods mplemented n software R; see Appendx B. The convergence of the chans were tested by usng the Gelman and Rubn method mplemented n the R package CODA. Graphcal traces of those methods and kernel densty estmaton for each parameter showed that there were no convergence problems. We generated two parallel chans of sze 8, for each parameter. The frst 2, teratons were gnored to elmnate the effect of the ntal values burn-n) and, to avod correlaton problems, we consdered a spacng of sze 2, obtanng a sample of sze 3,. The posteror results from usng both the Jeffreys and reference prors are shown n Table 5. It may be notced that the posteror results are all very smlar. In Fgures 2 and 3 we show plots of the generated samples and the emprcal margnal posterors for model parameters and α, based on the generated chans of the margnal Jeffreys and reference posterors, respectvely. 9

10 bas Jeffreys pror reference pror rmse Jeffreys pror reference pror sample sze sample sze bas Jeffreys pror reference pror rmse Jeffreys pror reference pror sample sze sample sze Fgure : Bas and rmse for upper panels) and α lower panels) parameters usng Jeffreys and reference prors. 6 Concludng remarks In ths paper we evaluated the Bayesan method to estmate the parameters n a Lomax dstrbuton under two non-nformatve pror specfcatons. We showed that the jont posteror dstrbuton s proper no matter whch non-nformatve pror s used. We also obtaned a scale mxture representaton of the Lomax dstrbuton n whch the complete condtonal dstrbuton of the scale parameter s of known closed form and easy to sample. As a by product, ths representaton allows for the mxng parameters to be used to dentfy possble outlers. Overall, the results obtaned ndcate that the Bayesan method estmates the Table 5: Posteror summares for the Lomax parameters usng Jeffreys and reference prors. pror parameter mean SD 95% CI π J [88.99 ; 84.66] α [.427 ;.592] π R [9. ; 82.7] α [.4226 ;.5865]

11 .7 8 α Densty Smulatons α Densty Smulatons Fgure 2: Trace and densty of α upper panels) and lower panels) usng Jeffreys pror. parameters well under both pror specfcatons f the sample sze s not too small. Of course, as n any Monte Carlo study, our results are lmted to our partcular selecton of sample szes and pror dstrbutons. We hope that our fndngs are useful to the practtoners. Acknowledgements The research of Francsco Louzada s funded by the Brazlan organzaton CNPq.

12 .65 8 α Densty Smulatons α Densty Smulatons 5 2 Fgure 3: Trace and densty of α upper panels) and lower panels) usng reference pror. 2

13 A Appendx A A. Verfyng that the posteror s proper under Jeffreys pror Under Jeffreys pror, the jont posteror densty of and α s gven by π, α x) αn /2 n+) n + x ) α+). α + )α + 2) /2 We next show that the ntegral of ths expresson s fnte for any sample sze n. a) Verfyng for n : x α /2 2 α + )α + 2) /2 α /2 α + )α + 2) /2 α /2 α + )α + 2) /2 α /2 α + )α + 2) /2 α /2 α + )α + 2) /2 α /2 α + )α + 2) /2 xα dα + x ) α+) ddα 2 + x ) α+) d ) dα ) α+) + x d) 2 dα ) 2 α+ + x) α+) d dα ) α + x) α+) d dα α /2 α + )α + 2) /2 dα π 2x <. b) Verfyng for n > : Frst, we solve n n+) + x ) α+) d. 2) 2 Consder y mnx 2,..., x n ). Then, t follows that + x ) α+ + y ) α+, α >, 2,..., n n 2 n 2 + x ) α+ + x ) α+) < + y ) n )α+) + y ) n )α+). 3

14 Therefore, n n+) 2 + x ) α+) d < n+) + y ) n )α+) d ) n )α+) + y n+) d n+) nα+n α + y) n )α+) d nα α 2 + y) n )α+) d n )!Γnα α ) y n Γnα α + n ). Then, n )! y n n )! y n α n /2 n+) α + )α + 2) /2 α n /2 n 2 + x ) α+) ddα n )!Γnα α ) α + )α + 2) /2 y n Γnα α + n ) dα α n /2 α + )α + 2) /2 Γnα α ) Γnα α + n ) dα α n /2 α + )α + 2) /2 n 2 3) nα α + j)dα. j But note that nα α + j) nα α), α >, j. Therefore, nα α + j) < nα α) n 2 j nα α + j) < n 2 j nα α) n 2 j and replacng n 3), we have that < n )! y n n )! y n nα α + j) < nα α) n+2 α n /2 α + )α + 2) /2 n 2 nα α + j)dα j α n /2 α + )α + 2) /2 nα α) n+2 dα. We can conclude that the posteror dstrbuton usng Jeffreys pror s proper for n. 4

15 A.2 Verfyng that the posteror s proper under reference pror Usng a reference pror, the jont posteror densty of and α s gven by π, α x) α n n+) n + x ) α+). We next verfy whether the ntegral of ths expresson s fnte. a) Verfyng for n : 2 + x ) α+) ddα ) α+) + x 2 ddα 2 α+ + x) α+) ddα α + x) α+) ddα dα. xα Therefore, the posteror dstrbuton usng a reference pror s mproper for n. b) Verfyng for n > : Note that n n+) 2 + x ) α+) d s the same ntegral 2) prevously resolved. Then, < n )! y n n )! y n α n n+) n 2 + x ) α+) ddα α n nα α + j)dα n 2 j α n nα α) n+2 dα. Thus, the posteror dstrbuton usng reference pror s proper for n >. 5

16 B R code of the Metropols-Hastng wthn Gbbs ## ### c l e a n memory rm l s tl s a l lt) ) g r a p h c s. o f f ) ## datread. t a b l e d a t a s et. txt, headert, sep,, quote \, dec., f l l T, na. s t r n g s NA ) n datnrow dat ) ## ### J e f f r e y s ndependence ) and r e f e r e n c e p r o r s ## $p \ alpha, \ beta ) \ propto \ f r a c {}{\ alpha \ beta }$ # use o f the l o g o f the c o n d t o n a l d s t r b u t o n o f alpha alpha condf u n c t o n x, lambda, tunng ){ r e p e a t { # xn : proposal value xnx+rnorm,, tunng ) f xn>) break } Al e n g t h lambda ) lgamma x) lgamma xn ))+xn x ) sum l o g lambda))+ l o g x) l o g xn ) A2pnorm x, l o g. pt) pnorm xn, l o g. pt) AA+A2 f l o g r u n f ))<A) xxn e l s e xx r eturn x ) } ## # some v a r a b l e d e c l a r a t o n burn 2 jump2 n. amostra3 t e rburn+jump n. amostra # r e f f u l ll s t alphanumerc ), alpha2numerc ), betanumerc ), beta2numerc ) ) # ## # MCMC method : Gbbs sampler wth Metropols Hastngs # n t a l v a l u e s a l p h a t e rrgamma,, ) b e t a t e rrgamma,, ) # tunng 3/3 6

17 # f o r j n : t e r ){ # a u x l a r y v a r a b l e lambda auxu n l s t l a p p l y : n dat, f u n c t o n ) rgamma, shape a l p h a t e r +), r a t e dat [, ] / b e t a t e r )+)))) # c o n d t o n a l d s t r b u t o n o f beta b e t a t e r /rgamma, shapen dat, s c a l e /sum lambda aux dat ) ) # g e n e r a t o n o f v a l u e s o f alpha from t s c o n d t o n a l # d s t r b u t o n alphanalpha cond a l p h a t e r, lambda aux, tunng ) # a l p h a t e ralphan # r e f f u l l $ a l p h a [ j ] a l p h a t e r r e f f u l l $ b e t a [ j ] b e t a t e r } ## 7

18 References [] Ban, L., Engelhardt, M.: Introducton to Probablty and Mathematcal Statstcs. Duxbury Press 992) [2] Berger, J.O., Bernardo, J.M.: Estmatng a Product of Means: Bayesan Analyss wth Reference Prors. Journal of the Amercan Statstcal Assocaton 8445), ) [3] Berger, J.O., Bernardo, J.M.: On the development of reference prors, pp Oxford Unversty Press 992) [4] Berger, J.O., Bernardo, J.M.: Ordered group reference prors wth applcatons to a multnomal problem. Bometrka 79, ) [5] Berger, J.O., Bernardo, J.M., Sun, D.: The formal defnton of reference prors. The Annals of Statstcs 372), ) [6] Berger, J.O., Bernardo, J.M., Sun, D.: Natural nducton: An objectve bayesan approach. RACSAM - Revsta de la Real Academa de Cencas Exactas, Fscas y Naturales, Sere A, Matematcas 3), ) [7] Bernardo, J.: Reference posteror dstrbutons for Bayesan nference wth dscusson). Journal of the Royal Statstcal Socety, Seres B 4, ) [8] Bernardo, J.M.: Reference Analyss, vol. 25, pp Elsever 25) [9] Bryson, M.: Heavy-taled dstrbutons: Propertes and tests. Technometrcs 6, ) [] Chb, S., Greenberg, E.: Understandng the Metropols-Hastng Algorthm. Amercan Statstcan 494), ) [] Clarke, B., Sun, D.: Reference prors under the ch-squared dstance. Sankhya: The Indan Journal of Statstcs, Seres A 96-22) 592), ) [2] Gelman, A., Rubn, D.B.: Inference from teratve smulaton usng multple sequences. Statstcal Scence 74), ) [3] Hastngs, W.: Monte Carlo Samplng Methods Usng Markov Chans and Ther Applcatons. Bometrka 57, ) [4] Holland, O., Golaup, A., Aghvam: Traffc characterstcs of aggregated module downloads for moble termnal reconfguraton. Bometrka 35, ) [5] Howlader, H., Hossan, A.M.: Bayesan survval estmaton of Pareto dstrbuton of the second knd based on falure-censored data. Computatonal Statstcs & Data Analyss 38, ) 8

19 [6] Jeffreys, H.: Theory of Probablty. Oxford Unv. Press, Oxford 96) [7] Lomax, K.: Busness Falures; Another example of the analyss of falure data. Journal of the Amercan Statstcal Assocaton - JSTOR 49, ) [8] Lunn, D.J., Thomas, A., Best, N., Spegelhalter, D.: WnBUGS - a Bayesan modellng framework: Concepts, structure, and extensblty. Statstcs and Computng 4), ) [9] Plummer, M.: JAGS: A program for analyss of Bayesan graphcal models usng Gbbs samplng. In: Proceedngs of the 3rd Internatonal Workshop on Dstrbuted Statstcal Computng 23) [2] Plummer, M., Best, N., Cowles, K., Vnes, K.: CODA: Convergence Dagnoss and Output Analyss for MCMC. R News 6), 7 26). [2] R Core Team: R: A Language and Envronment for Statstcal Computng. R Foundaton for Statstcal Computng, Venna, Austra 24). [22] Van, H.M., Vose, D.: A Compendum of Dstrbutons - ebook 29) 9