THE CONSTRUCTION OF COMBINED BAYESIAN-FREQUENTIST CONFIDENCE INTERVALS FOR A POSITIVE PARAMETER

Transcription

1 STATISTICA, ao LXV,. 4, 5 THE CONSTRUCTION OF COMBINED BAYESIAN-FREQUENTIST CONFIDENCE INTERVALS FOR A POSITIVE PARAMETER 1. INTRODUCTION I the case whe a parameter is assumed to be oegative or positive-valued, we cosider a iterval estimatio problem o a ukow parameter based o the observatios icludig errors. If the magitude of errors is almost same size as that of the value of the parameter, the ordiary cofidece iterval i which the parameter is ot cosidered to be oegative ivolves a set of egative values, ad the iterval restricted to oegative oes ca be degeerate. O the above problem, various methods to costruct such cofidece itervals are proposed by may physicists ad others (see, e.g. Feldma ad Cousis (1998), Madelker (), Madelker ad Schultz (a, b) ad Roe ad Woodroofe (1999, 1)). But, from the viewpoit of statistics, the problem is very simple ad clear. Ideed, there are basically two ways to costruct Bayesia ad frequetist cofidece itervals. I the Bayesia case, it is eough to cofie a prior distributio to the set of existece of a parameter, ad i such a case, cosideratio whether the prior oe is appropriate or ot is ecessary, but it is essetially same as the ordiary Bayesia case. From the viewpoit of frequetist, we cosider a set of testig hypothesis ad take a correspodig acceptace regio. Ideed, let >, ad we cosider the problem of testig the hypothesis H: = agaist the alterative K:, > with level. Let A( ) be a acceptace regio of the test, ad for a radom sample X S( X ) : { X A( )}. Sice, for >, P { S( X )} P { X A( )} 1, it follows that a cofidece regio S(X) for of cofidece coefficiet 1 is obtaied (see, e.g. Lehma (1986), Bickel ad Doksum (1)). I particular, if S(X) becomes a iterval, the it is called a cofidece iterval.

2 35 Now, the problem is a better choice of the test procedure i such a testig problem. Notig that a method of ubiased tests ca ot be applied to the above testig problem, we propose the followig procedure i this paper. Assume that a real radom variable X has a probability desity fuctio (p.d.f.) f (x,) (w.r.t. the Lebesgue measure), where >, The we cosider the problem of testig the hypothesis H: = (> ) agaist the alterative K:, >. Let be a prior distributios defied o the iterval (,), ad take the alterative K : f ( x ) : f ( x, ) d ( ). Here, f might be improper. Ad we cosider the problem of testig the hypothesis H : f (x, ) agaist the alterative K : f (x). The the acceptace regio of the most powerful test is of the form A( ) : { x f ( x ) f ( x, ) }, where is determied from the coditio P { X A( )} 1 for give (<<1). For various prior distributios, we ca get may cofidece itervals. They are all admissible.. ORDINARY CONFIDENCE INTERVALS Suppose that X 1,...,X are idepedet ad idetically distributed (i.i.d.) radom variables with the ormal distributio N(µ, ), where µ > ad is kow. Sice the pivotal quatity T( X, ) : ( X ) is distributed accordig to N(,1), it follows that the iterval I( X ) : = max, X u /, max, X + u / is the cofidece iterval (c.i.) for µ of cofidece coefficiet (c.c.) 1, where u / is the upper 1(/) percetile (see figure 1). Ideed, while I( X ) ca become sometimes degeerate, it is always true that P { I( X )} 1, sice the evet I( X ) is equivalet to the evet X u /, which always has the probability 1. But, for X 1, the c.i. I (X ) is degeerate, hece there is still room for improvemet.

3 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 353 Figura % cofidece limits for µ from the pivotal quatity. I order to improve the above, Feldma ad Cousis (1998) cosider the c.i. based o the acceptace regio of the likelihood ratio (LR) test as follows. Suppose that X 1,...,X are i.i.d. radom variables with the ormal distributio N(µ,1), where µ>. Sice the likelihood fuctio L of µ, give X : (1 ) Xi (1 ) x i : x, i 1 i 1 1 L( x ) ( ) exp ( x x ) ( x ) / i i 1 for µ >, it follows that the maximum likelihood estimator (MLE) is give by ˆ : max{ X,}. The ML / 1 ( ) exp ( x i x ) for x, i 1 L( ˆ ML x ) / 1 ( ) exp xi for x, i 1 hece the LR R ( x ) : exp{ ( /)( x ) } for x, L( x ) L( ˆ ML x ) exp x for x.

4 354 If we ca obtai a(µ) ad b(µ) such that R( a( )) R( b( )) ad 1 ( ( b( ) )) ( ( a( ) )), the the iterval [a(µ), b(µ)] is a acceptace iterval, where is the cumulative distributio fuctio (c.d.f.) of N(,1). If S( X ) : { X [ a( ), b( )]} is a iterval, the it is c.i. for µ of c.c. 1 (see figure ). Figure 68.7% cofidece limits for µ derived from the LR test. 3. BAYESIAN CONFIDENCE INTERVALS Suppose that X 1,..., X are i.i.d. radom variables with the ormal distributio N(µ,1), where µ >. Let (µ) be a improper prior distributio, i.e. 1 for, ( ) for. Sice X is ormally distributed as N(µ, 1/), the posterior desity of µ give X x is f ( x ) : exp ( x ), ( x ) where is the c.d.f. of N(, 1). If there exist ( x ) ad ( x ) such that

5 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 355 [ ( x ), ( x )] { f ( x ) c}, P { ( X ) ( X )} 1, the the iterval [ max{ X d,}, X d ] is the c.i. for µ of c.c. 1, where d 1 1 (1 ( X )) for X x, (1 ) ( X ) for X x with x : 1 (see Madelker () ad figure 3). Figure % cofidece limits for µ based o the Bayesia approach. 4. COMBINED BAYESIAN-FREQUENTIST CONFIDENCE INTERVALS FOR THE NORMAL MEAN IN CASE OF KNOWN VARIANCE Uder the same setup as sectio 3, we costruct cofidece itervals for µ by the combied Bayesia-frequetist approach. First we cosider the problem of testig the hypothesis H: µ = µ (> ) agaist K: µ ~ (µ), i.e. µ is distributed as the

6 356 same improper prior as sectio 3. The the acceptace regio of the most powerful (MP) test is of the form X d X exp ( ) ( ) T( X, ) :, exp ( X ( ( )) ) X where, for give (<<1), is determied by y y y ( u) du. Here, T( X, ) is the itegrated LR or the Bayes factor (see, e.g. Robert (1) ad O Haga ad Forster (1994)). Next, we shall show that there exist at most two solutios y( ) ad y( ) of the equatio ( ( y )), (1) ( y) if they does. Ideed, we cosider the equatio (t + c)/(t) =. Puttig G( t ) : ( t c ) ( t ), () we have ct ( c /) G ( t ) : ( t c ) t( t ) ( t )( e t ). Here, ote that lim t G(t) =, lim t G(t) = 1 ad >. If c <, the there is the oly oe solutio of the equatio G'(t) =, hece that of () is uique. If c >, the there are at most two solutios of the equatio G'(t) =, hece the solutios of () are also so. Therefore there are at most two solutios of (1), if they exist. Sice y ad y are solutio of (1), if follows that ( ( y ( ( )) y )). ( y) ( y) Puttig z y, z y ad m :, we have ( z m) ( z m). ( z ) ( z )

7 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 357 Let Y be a radom variable with the ormal distributio N(, 1/). The 1 P{ z Y z} ( z ) ( z ). Sice z m z m 1 P{ z Y z} P X, we have the acceptace regio 1 1 ( z m), ( z m), hece we ca costruct a cofidece iterval (see figure 4). Figure % cofidece limits for µ based o the combied Bayesia-frequetist approach. Next, as a proper prior distributio, we cosider a expoetial distributio 1 µ e for µ, ( ) for µ, where >. Here, ote that the proper prior (µ) coverges to the improper uiform prior of type (µ) as i the sese that (µ) 1 as. I a similar way to the above, it is show that the acceptace regio of the MP test is of the form

8 358 T( X, ) : 1 exp ( X ) e µ d exp ( X ) 1 1 X 1 X exp. ( ( X )) Here, for give (<<1), is determied by exp ( x ) d x. x T ( x, ) Now, there are at most two solutios t 1 (µ ) ad t (µ ) (t 1 (µ ) < t (µ )) of the equatio T( x, µ ) if they exist. The we have 1 P { T( X, ) } P { t ( ) X t ( )}, 1 ad if the above equality is reduced to P { a( X ) b( X )} 1, the iterval [ a( X ), b( X )] is c.i. for µ of c.c. 1 (see figure 5). Figure 5 Bayesia-frequetist 68.7% cofidece limits for µ whe the prior distributio is expoetial oe Exp( ). Next we umerically compare ordiary, LR, Bayesia, combied Bayesiafrequetist cofidece limits (see figure 6 ad table 1). As is see i figure 6 ad table 1, if the value of X is less tha 1, the LR c.i. is comparatively better tha

9 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 359 others, ad i a eighborhood of X the combied Bayesia-frequetist c.i. is better tha others i the sese that the legth of c.i. is shorter. However, the combied oe seems to be good i the sese that its width is mootoe icreasig with some appropriate extet for X. I purely o-bayesia sese, our formulatio ca be iterpreted as follows. We have the problem of testig H: µ = µ agaist K: µ µ whe µ >. Now deote by (µ) the power of a test uder µ. We wat to maximize (µ). Sice there exists o uiformly most powerful test, we ca ot maximize (µ) simultaeously for all µ >. We may istead maximize the average power of (µ) over µ > with some weight fuctio (µ), that is, ( ) ( ) d( ), where is a probability measure over the iterval (,). It is easily show that maximizig ( ) is give by the most powerful test of the hypothesis agaist the simple alterative hypothesis : µ~(µ). Figure % cofidece limits for µ. TABLE 1 Ordiary, Bayesia, ad LR 68.7% cofidece itervals of µ for =1 x Ordiary LR Bayes -4 [, ] [,.8] [,.64] -3 [, ] [,.38] [,.334] - [, ] [,.69] [,.446] -1 [, ] [,.7] [,.64] [, 1.] [, 1.] [, 1.] 1 [,.] [.41,.] [.3, 1.797] [1., 3.] [1., 3.] [1.3,.968] 3 [., 4.] [., 4.] [.1, 3.998] 4 [3., 5.] [3., 5.] [3., 5.]

10 36 Bayesia-frequetist 68.7% cofidece itervals of µ for =1 x Uiform Exp(1) Exp() Exp(3) -4 [,.54] [,.361] [,.435] [,.466] -3 [,.66] [,.46] [,.495] [,.534] - [,.75] [,.467] [,.581] [,.63] -1 [,.961] [,.574] [,.78] [,.794] [, 1.37] [,.841] [, 1.54] [, 1.146] 1 [.55,.89] [.488, 1.54] [.53, 1.77] [.54, 1.86] [1.99, 3.1] [.83,.5] [1.53,.64] [1.135,.74] 3 [.8, 4.] [1.48, 3.494] [1.651, 3.68] [1.798, 3.74] 4 [3.11, 5.] [1.83, 4.493] [.443, 4.67] [.648, 4.73] 5. COMBINED BAYESIAN-FREQUENTIST APPROACH IN CASE OF UNKNOWN VARIANCE I the previous sectios, we assume that the variace is kow. I this sectio we costruct a Bayesia-frequetist type cofidece itervals of µ whe the variace is ukow. Suppose that X 1,...,X are i.i.d. radom variables accordig to the ormal distributio N(µ, ) with a p.d.f. f(x ; µ, ), where. Now we cosider the problem of testig H : µ = µ (> ), ~ () = 1/ for > agaist K : (µ, ) ~ (µ, ) = 1/ for µ >, >. The we have i 1 d f ( xi ;, ) 1 1 exp ( x ) 1 i ( ) i ( ) exp ( x ) i d ( ) i 1 ( ) 1 ( ) i 1 i 1 d ( ) 1 y ( xi ) y e dy after a traformatio y ( x ) C ( xi ) i 1 C {( 1) s ( x ) } C s Cs ( x ) 1 ( 1) s ( ) ( x ) 1 ( 1) ( ( 1) ) ( 1) s 1 i Cs f ( ( x ) s ) (say), (3)

11 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 361 where ( ) 1 ( ) C :, C : C ( 1), ( ) ( 1) ( ( 1) ) 1 C : C, s : ( xi x ). ( ) 1 Note that f 1 is a p.d.f. of the t-distributio with degrees 1 of freedom. From (3) we have uder the hypothesis H f xi Cs f1 i 1 s i 1 d ( x ) ( ;, ), ad uder the alterative i 1 1 f ( x i ;, ) dd ( x ) Cs f1 d s 1 Cs x s ( ) t 1 ( 1) ( ( 1) ) 1 dt (after a trasformatio t ( x ) s ) 1 Cs F ( x s ), 1 where F 1 is the c.d.f. of f 1. I the above testig problem, the acceptace regio of the MP test is of the form T X S ( S ) F ( X S ) 1 (,, ) :. f1( ( X ) S ) (4) By the approximatio of the c.d.f. F 1 to the ormal distributio, we have 1 1 F 1( t ) ( t ) t( t 1) ( t ) O 4( 1) 1 : 1( t ) O (say), (5)

12 36 where ad are the c.d.f. ad p.d.f. of the stadard ormal distributio N(,1), respectively. The 1 1 f1( t ) F 1( t ) ( t ) t ( t 1) ( t ) O 4( 1) 1 : 1( t ) O (say). (6) So, istead of T i (4) we use ( S ) 1( X S ) T ( X, S, ) : ( ( X ) S ) 1 as the approximatio of T by (5) ad (6). Puttig t ( x ) s, we have at most two solutios t ad t of t of the equatio ( s ) 1 t s 1 (7) ( t ), if they exist. But t ad t ca ot be represeted as fuctios of µ /s sice s ad t are ot idepedet. So, istead of s i (7) we use as a kow stadard deviatio for the preset, ad the solutios t ad t become fuctios of µ / such that P t Sice ( X ) t 1. P t X t 1, it follows that the acceptace regio is t, t. (8) Sice is geerally ukow, we propose the acceptace regio

13 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 363 S t S, t, S S (9) where S is substituted for i (8). From (9) we umerically costruct a c.i. for µ (see figure 7 ad table ). Figure 7 Bayesia-frequetist 95% cofidece limits for µ i the case of ukow variace. TABLE The coverage probability (%) of the Bayesia-frequetist type 95% cofidece iterval for µ i the case whe is ukow (i) = 5 µ (ii) = 1 µ As is see i table, the value of coverage probability is very close to that of cofidece coefficiet. 6. CONCLUDING REMARKS I this paper, we propose the combied Bayesia-frequetist c.i. for µ i both of cases whe is kow ad ukow. We give a umerical compariso of the

14 364 combied oe with ordiary, LR ad Bayesia c.i. s. As a result, the combied oe is comparatively better tha others i the sese that its width is mootoe icreasig with some appropriate extet for X. Note that usig the itegrated LR is closely coected with the Bayes factor of H agaist K (see, e.g. Robert (1) ad O Haga ad Forster (1994)). I the case whe is ukow as is see i table, the Bayesia-frequetist type c.i. ca be recommeded. For ukow case, oe may choose as the prior dµd/ istead of dµd/. Now, i the formula (3) we have + 1 istead of, ad the results will ot make much differece. However, it seems to be more atural to the degree 1 of freedom istead of. Hece our choice is dµd/. O the choice of the prior, see, e.g. Berger ad Pericchi (1). Istitute of Mathematics Uiversity of Tsukuba, Japa Istitute of Mathematics Uiversity of Tsukuba, Japa Faculty of Iteratioal Studies Meiji Gakui Uiversity, Japa MASAFUMI AKAHIRA ATSUSHI SHIMIZU KEI TAKEUCHI ACKNOWLEDGEMENTS The authors wish to thak the referee for the commets from the Bayesia viewpoit. REFERENCES J. BERGER, L.PERICCHI (1), Objective Bayesia methods for model selectio: Itroductio ad compariso (with discussio), i P. LAHIRI (eds.), Model Selectio, Istitute of Mathematical Statistics, Beachwood, Ohio, pp P. J. BICKEL, K. A. DOKSUM (1), Mathematical Statistics, vol. 1, (d ed.), Pretice Hall, New Jersey. G. J. FELDMAN, R. D. COUSINS (1998), Uified approach to the classical statistical aalysis of small sigals. Physical Review, D57, pp E. L. LEHMANN (1986), Testig Statistical Hypotheses, (d ed.), Wiley, New York. M. MANDELKERN (), Settig cofidece itervals for bouded parameters, Statistical Sciece, 17, pp M. MANDELKERN, J. SCHULTZ (a), The statistical aalysis of Gaussia ad Poisso sigals ear physical boudaries, Joural of Mathematical Physics, 41, pp M. MANDELKERN, J. SCHULTZ (b), Coverage of cofidece itervals based o coditioal probability, Joural of High Eergy Physics, 11, 36, pp A. O HAGAN, J. FORSTER (1994), Kadall s Advaced Theory of Statistics, vol. B: Bayesia Iferece, Chapma ad Hall, Lodo. C. P. ROBERT (1), The Bayesia Choice, d ed., Spriger, New York. B. P. ROE, M. B. WOODROOFE (1999), Improved probability method for estimatig sigal i the presece of backgroud, Physical Review, D6, 539, pp B. P. ROE, M. B. WOODROOFE (1), Settig cofidece belts, Physical Review, D63, 139, pp. 1-9.

15 The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter 365 RIASSUNTO La costruzie di itervalli di cofideza combiati bayesiai-frequetisti per u parametro positivo Negli esperimeti della fisica modera, si verificao umerosi casi i cui il parametro può assumere solo valori o egativi o positivi. Per tali casi el cotributo si adotta u approccio combiato bayesiao-frequetista per costruire gli itervalli di cofideza. I seguito gli itervalli vegoo cofrotati co quelli otteuti dai due approcci bayesiao e classico per i casi ormali. SUMMARY The costructio of combied Bayesia-frequetist cofidece itervals for a positive parameter I curret physics experimets, there are may cases whe the value of a parameter is theoretically assumed to be oegative or positive. I such cases, a combied Bayesiafrequetist approach to cofidece itervals for a positive parameter is adopted i this paper, ad the cofidece itervals are costructed. Comparisos of the cofidece itervals with ordiary ad Bayesia oes are doe i the ormal cases.