Ordinary and Bayesian Shrinkage Estimation. Mohammad Qabaha

Transcription

1 An - Najah Univ. J. Res. (N. Sc.) Vol., 007 Ordinary and ayesian Shrinkage Estimation التقدير باستخدام طريقتي التقلص العادية والتقلص لبييز Mohammad Qabaha محمد قبها Department of Mathematics, Faculty of Science, An-Najah National University, Nablus, Palestine mohqabha@mail.najah.edu Received: (//006), Accepted: (9/5/007) Abstract In this paper a variety of shrinkage methods for estimating unknown population parameters has been considered. Aprior distribution for the parameters around their natural origins has been postulated and the ordinary ayes estimators are used in place of natural origins in the ordinary shrinkage estimators to obtain ayesian shrinkage estimators. The results are applied to the problem of estimating the location and scale parameters and the reliability function of the two-parameter exponential distribution. Simulation experiments are used to study the performances of these estimators. Key words: Estimation, parameter, reliability, shrinkage, ayes, guess value, exponential distribution, simulation. ملخص يهدف هذا البحث الى ايجاد تقديرات للمعلمتين والفاعلية للتوزيع الاسي ذي المعلمتين اعتمادا على طريقتي التقلص العادية والتقلص لبييز.. Introduction In the estimation of an unknown parameter, some form of a prior knowledge about the parameter which one would like to utilize in order

2 Estimation Ordinary and ayesian Shrinkage 0 to get a better estimate often exists. Thompson (968) described a shrinkage technique for estimating the mean of a population. Mehta and Srinivasan (97) proposed another class of shrunken estimator for the mean of a population and showed that this class had a better performance than that of Thompson in terms of mean squared error. Pandey and Singh (977) and Pandey (979) described shrinkage techniques for estimating the variance of a normal population. Lemmer (98) gave the concept of using an ordinary ayes estimator instead of natural origin in the ordinary shrinkage estimator and thus derived the concept of ayesian shrinkage estimation. He considered the estimation of binomial, Poisson and normal parameters through ayesian shrinkage techniques. Pandey and Upadhyay(985) considered the ayesian shrinkage estimation of reliability of one-parameter exponential failure model.yousef (986) proved that the mean squared error of Thompson type estimator is smaller than the remaining shrinkage estimators for estimating the parameters of the two-parameter exponential distribution.yousef (99) derived confidence bounds for reliability of the two-parameter exponential distribution. In this paper we consider the problem of estimating the parameters θ, µ and the reliability function R(t) of the two-parameter exponential distribution when the prior information regarding θ, µ and R(t) is available in the form of guess values. More specifically, it is assumed that the guessed values θ 0, µ 0 and R 0 (t) are close or approximately equal to the true values of θ, µ and R(t), respectively. A variety of shrinkage methods proposed by Thompson (968), Mehta and Srinivasan (97), Pandey (979) and Lemmer (98) are used for this purpose. We propose the corresponding ayesian shrinkage estimators of θ, µ and R(t) after deriving the expressions for their ordinary ayes estimators from type ІІ censored sample of life testing data from the two-parameter exponential distribution. Simulation experiments are used to study the performances of these estimators. Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

3 0 Mohammad Qabaha. Ordinary Shrinkage Estimators Let X be the life length of a certain system which has the probability density function f (X; θ, µ) = θ exp[- (X-µ) / θ ], 0 µ X, θ >0. Then the reliability function of this system at time t is defined by R(t) = exp [- (t - µ) / θ]. Let us consider a random sample of n items of such a system subjected to test and the test terminated as soon as the first r ( n) items fail. Let X = {X (), X (),, X (r) } be the first r ordered failure times. It is reasonable to take the minimum variance unbiased estimators θ, μ and R(t) of θ, µ and R(t) respectively, and modified these estimators by moving them closer to θ, µ and R(t) so that the resulting estimators, perhaps biased, have smaller mean squared error than that of θ, μ and R(t). It is well known from Epstein and Sobel (954) and asu (964) that r θ = [ X(i) + (n-r) X (r) - nx () ] / (r-), r >, i μ = X () - θ /n, and (r - ) R(t) = n n - t - X() -, r >, (r -)θ (.) Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

4 Estimation Ordinary and ayesian Shrinkage 04 are the minimum variance unbiased estimators of the parameters θ, µ and R(t), respectively. The variances of these estimators (see Lee (978), p 6), are given by var( θ ) = θ r, r>, r θ var ( μ ) =, r>, n (r) and var ( R(t) ) = (n-) n (r-) r - 4 i 0 r 4 i! Γ(r -i ) i i n The first estimator considered is n(μ - t) i ( ) m θ m 0 m! - (t) R, r >. μ T = µ 0 + c ( μ - µ 0 ), 0 c, (.) where µ 0 is the guessed value of µ and μ T is the actual Thompson type estimator. Thompson (968) suggested that c determined from MSE (μ T) c error of μ T. = 0 with MSE( μ T It follows that c = (µ - µ 0 ) / [ ( µ - µ 0 ) + var ( μ ) ]. ) = E ( μ Tμ), the mean squared Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

5 05 Mohammad Qabaha In practice c is estimated by replacing the unknown parameters by their sample estimates. Substituting the estimated value of c in (.) we have μ T = µ 0 + ( μ - µ0 ) / [( μ - µ0 ) + r θ /(n (r-))]. For any value of c, 0 c, and when µ 0 tends to µ, it is easily seen from (.) that MSE ( μ T ) = c var ( μ ) + (-c) (µ - µ 0 ) var( μ ). Secondly, we consider the Mehta and Srinivasan (97)-type estimator. This is given by μ = μ - a( μ - µ0 )exp[-b( μ -µ0 ) /var( μ )], (.) M where a and b are positive constants to be suitably chosen such that 0<a< and b>0. No general guidance has been given on how a and b should be chosen. Substituting var ( μ ) and unknown parameters by their sample estimates in (.) we obtain μ = μ - a ( μ -µ0 ) exp [-bn (r-) ( μ - µ0 )/r θ ]. M It can be verified from (.) that the minimum and maximum values of μ M is attainable when b tends to 0 and respectively by a suitable choice of a, 0<a<. So we take lim MSE ( μ M ) = (-a) var( μ ) + a(µ - µ0 ), b 0 and Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

6 Estimation Ordinary and ayesian Shrinkage 06 by lim MSE ( μ M ) = var ( μ ). b Hence for 0<a<, b>0 and µ 0 tends to µ we have MSE ( μ M ) MSE ( μ ). Thirdly, we consider the Pandey (979) - type estimator of µ is given μ = a[k μ + (-k)µ0 ], 0 k, (.4) p with k is a constant specified by the experimenter according to his belief MSE( μp) in µ 0 and a is determined from = 0. It follows that a a = d µ / [k var ( μ ) + d µ ] where d = k + (-k)μ 0/ μ. Usually a is estimated by replacing the unknown parameters by their sample estimates. Substituting the estimated value of a in (.4) we obtain μ P = d μ / [ d μ + k r θ / (n (r -)) ], with d = [k+(- k) µ 0 / μ ]. It can be shown from (.4) that MSE ( μ P ) = ak var ( μ ) + [(-ak) µ - a (-k) µ0 ]. It follows that MSE ( μ P ) MSE ( μ ) only when a= and µ0 tends to µ, it is not clear otherwise. Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

7 07 Mohammad Qabaha Finally, we consider the Lemmer (98)-type estimator for µ. This is given by μ L = k μ + (-k) µ0. (.5) It can be seen from (.4) and (.5) that μ p = μ L if a=. All the above approaches can be used to define a variety of shrunken estimators for the parameter θ and the reliability function R (t). The estimators considered for the parameters µ, θ and R(t) are presented in Table. Table (): Shrunken estimators for µ, θ and R (t) Parameter Type of Estimator Thompson Estimator μ T (μ - μ0) = µ 0 + (μ - μ0) rθ /n (r -) Location Parameter µ Mehta- Srinivasan Pandey Lemmer μ M μ P μ L = μ - a ( μ -µ0 ) exp[-bn (r -)( μ -µ0 )/ r θ = d d μ μ k rθ /n (r -) = k μ + (- k)µ0 ] Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

8 Estimation Ordinary and ayesian Shrinkage 08 Parameter Scale Parameter θ Reliability Function R(t) Type of Estimator Thompson Mehta- Srinivasan Pandey Lemmer Thompson Mehta- Srinivasan Pandey Lemmer Estimator θ T θ M θ P θ L RT R M var RP = θ 0 + (θ - θ (θ - θ ) θ 0 0 ) /(r - ) Continue table () = θ - a ( θ - θ 0 ) exp [-b(r-) ( θ - θ 0 ) / θ = d θ d k θ θ /(r -) = k θ + (-k) θ 0 (R(t) R 0(t)) (t)= R 0 (t) + (R(t) R 0(t)) var(r(t)) (t) = R (t) (R(t) )] (t) = d - a ( R(t) d R(t) R(t) k var (R(t)) R L (t) = k R (t) + (-k) R0 (t) - R 0 (t)) exp [- b ( R(t) -R 0 (t))/ ] Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

9 09 Mohammad Qabaha where d = k + (-k) µ 0 / μ, d = k + (-k) θ 0 / θ, d = k + (-k)r 0 (t) / R(t), k is a known constant between zero and one, a and b are positive constants to be suitably chosen such that 0<a< and b>0, µ 0, θ 0 and R 0 (t) are the guessed values for µ, θ and R(t) respectively and var ( R(t) ) is the estimated variance of the estimator R (t) which is given by (n-) var ( R(t) )= n (r-) r- i4 0 n(μ-t) ( ) m r 4 i i! Γ(r-i ) i θ i n m0 m! - R (t), r>.. ayesian Shrinkage Estimators Using the set up of section, the likelihood function of X is given by n! L(X / θ, µ) = exp[-(r θ +n μ - n µ) /θ]. (n r)!θ r Assume that our prior knowledge about θ and µ can be expressed as β α g(θ,µ) = ( ) α δ Γα θ exp[-β/θ], α, β, δ, θ >0, 0 µ δ, where α, β, δ are known constants. Combining the above prior with the likelihood function we obtain the posterior probability density function of θ and µ as h(θ, µ /X ) = S - ( rα θ ) exp[-(r θ +n μ -nµ+β)/θ], α, β, δ, θ>0, o µ M, where M = min (δ, X ()), Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

10 Estimation Ordinary and ayesian Shrinkage 0 and Γ(rα) S = n ], [ r α - r α R R r with R = X(i) + (n-r)x (r) - nm + β and R = R + nm. i ayes estimators of θ, µ and R(t) with respect to the squared error loss function are given by θ 0 = θ h(θ, μ/x) u θ = μ 0 = μ h( θ, μ/x) θ u = ], r α- R Γ(rα) n s Γ(rα) n s ], [ r - - r - R R (r α )nm R [ rα- and R 0 (t) = e -((t-μ) / θ ) h(θ, μ/x) u θ Γ(rα-) = [ - ], (n)s rα- rα- (R t - M) (R t) respectively. If we substitute the ayes estimators θ0, μ 0 and R 0 (t) in place of natural origins θ 0, µ 0 and R 0 (t) in a shrunken estimators presented in Table, we obtain the ayesian shrunken estimators for the parameters θ, µ and R(t). For example μ T = μ 0 + ( μ - μ 0 ) / [( μ - μ 0 ) + r θ R /n (r-)], - (.) Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

11 Mohammad Qabaha μ M = μ - a ( μ - μ 0 ) exp [-bn (r-) ( μ - μ 0 ) / r θ ], μ P = d μ L = k μ / [ d μ + (-k) μ 0. μ + k r θ /n (r-) ] with d = k +(-k) μ 0 / μ, are Thompson, Metha-Srinivasan, Pandey and Lemmer ayesian shrinkage estimators respectively for the parameter µ. In the same manner we can find the other types of ayesian shrinkage estimators for the parameters θ and reliability function R(t). 4. Simulation The researcher uses simulation experiments to study the performances of the estimators obtained in Sections and.a random sample of size n from the two-parameter exponential distribution with µ =80 and θ=7 is generated. The vector X = {X (), X (),, X (r) } of the first r-ordered observation is recorded. Then the minimum variance unbiased estimators μ, θ and R(t) of µ, θ and R(t) respectively are computed using the formulas in (.). For a known constant k between zero and one and for specific values µ 0, θ 0 and R 0 (t) the quantities d = k+(-k) µ 0 / μ, d = k+(-k) θ 0 / θ, d = k+(-k) R 0 (t)/ R(t) are obtained. Then the ordinary shrinkage estimators μ T, μ M, μp and μl of µ, θ T, θ M, θp and θl of θ, and RT(t), RM(t), RP(t) and RL(t) of R(t) are computed using the corresponding formulas shown in Table. For given values of α, β, and δ, the ayesian estimators μ 0, θ 0 and Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

12 Estimation Ordinary and ayesian Shrinkage R0 (t) of µ, θ and R(t) respectively are obtained by using the formulas given in (.). Then the ayesian estimates μ 0, θ 0 and R0 (t) are obtained and substituted in place of natural origins µ 0, θ 0 and R 0 (t) in shrunken estimators obtained in Table. Thus the ayesian shrunken estimators μ T, μ M, μ P, and μ L of µ, θ T, θ M, θp and θl of θ and RT (t), RM (t), RP (t), and RL (t) of R(t) are computed. Monote Carlo experiments are repeated 500 times. The average of the 500 sample values of each squared error, e.g. ( μ - µ), is taken as an estimate of the corresponding mean squared error which is denoted by MSE.The estimates of the mean squared errors of the various estimators of µ, θ and R(t) and the relative efficiencies, e.g. R( μ T / μ ) = MSE( μ T )/ MSE( μ ), R( μ T / μ T ) = MSE( μ T )/ MSE( μ T ), are calculated for n=0, r=0,0,0, k=0.05,0.5, a=0.,0.5, b =40, 500, α =β =, δ = 8, µ = µ 0 = 80 and θ = θ 0 = 7. Results of the simulation experiments are given in Tables Conclusions Although the results derived above apply strictly to limited cases, they are suggestive of some general conclusions regarding the relative efficiencies of the various methods. It can be seen from Tables -4 that MSE of Thompson, Mehta and Srinivasan, and Lemmer estimators are Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

13 Mohammad Qabaha smaller than that of μ, marked when r is small. θ and R(t). The advantages ofμt andμl are most Further comparison statistics in Tables 5-7 show that when the natural origins are close to the true values of µ, θ and R(t), the MSE of Thompson, Mehta and Srinivasan and Lemmer ordinary shrinkage estimators are smaller than the MSE of their corresponding ayesian shrinkage estimators, while the MSE of Pandey ayesian shrinkage estimators is smaller than that of the corresponding ordinary shrinkage estimator. If the natural origins are far away from the true values, then the MSE of the various ordinary shrinkage estimators is higher than that of μ, θ and R(t) smaller MSE than that of, while the ayesian shrinkage estimators still have μ, θ and R(t). Table : Relative efficiencies of various ordinary shrunken estimators of µ with respect to μ. Sample size n=0, µ = µ 0 = 80, θ 0 = 7. No. of failures R( μ T / μ ) a=., b=40 a=.5, b=500 k=.05 k=.50 k=.05 k=.50 R( μ M / μ ) R( μ P / μ ) R( μ L / μ ) 0 4.4x x x x x x x0-59 Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

14 Estimation Ordinary and ayesian Shrinkage 4 Table : Relative efficiencies of various ordinary shrunken estimators of θ with respect to θ. Sample size n=0, µ = 80, θ = θ 0 = 7 No. of failures R( R T / θ ) a=., b=40 a=.5, b=500 k=.05 k=.50 k=.05 k=.50 R( θ M / θ ) R( θ P / θ ) R( θ L / θ ) 0 9.4x x x x x x x Table (4): Relative efficiencies of various ordinary shrunken estimators of R(t) with respect to R(t) Sample size n=0, µ = µ 0 = 80, θ = θ 0 = 7, t=85, R(t)=R 0 (t) =.490 No. of failures R( RT(t) R(t) ) / a=., b=40 R( RM(t) a=.5, b=500 / R(t) k=.05 k=.50 k=.05 k=.50 ) R( RP(t) / R(t) ) R( RL(t) / R(t) ) 0 4.9x x x x x x x Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

15 5 Mohammad Qabaha No. of failures Table 5: Relative efficiencies of various ordinary shrunken estimators of µ with respect to their corresponding ayesian shrunken estimators. Sample size n=0, µ = 80, θ = 7, α = β =, δ = 8 a=., a=.5, k=.05 k=.50 k=.05 k=.50 b=40 b=500 R( μ T / μ T ) R( μ M / μ M ) R( μ P / μ P ) R( μ L / μ L ) x x x x x 0-9.x0 -.66x x x 0 -.0x0 -.56x x Table (6): Relative efficiencies of various ordinary shrunken estimators of θ with respect to their corresponding ayesian shrunken estimators Sample size n=0, µ = 80, θ = 7, α = β =, δ = 8 a=., a=.5, R( θ T / b=40 b=500 k=.05 k=.50 k=.05 k=.50 θ T ) R( θ M / θ M ) R( θ P / θ P ) R( θ L / θ L ) x x No. of failures Table 7: Relative efficiencies of various ordinary shrunken estimators of R(t) with respect to their corresponding ayesian shrunken estimators Sample size n=0, µ = 80, θ = 7, t=85, R(t) =.490, α β, δ8 No. of failures R( RT(t) / R T(t) ) a=., b=40 R( RM(t) a=.5, b=500 / R M(t) k=.05 k=.50 k=.05 k=.50 ) R( RP(t) / R P(t) ) R( RL(t) / R L(t) ) 0 9.x0 -.7 x0-6.89x x x0 -.0 x0-7.45x x x0 -.45x x0 -.8 Vol., 007 An - Najah Univ. J. Res. (N. Sc.)

16 Estimation Ordinary and ayesian Shrinkage 6 References - asu, A. P. (964). Estimates of Reliability for Some Distributions Useful in Life Testing. Technometrics. (6) Epstein,. & Sobel, M. (954). Some Theorems Relevant to Life Testing From an Exponential Distribution. Ann. Math. Statist. (5) Lee, J.. (978). "Statistical Analysis of Reliability and Life-Testing Models". Marcel Dekker, Inc. Newyork Lemmer, H. H. (98). From Ordinary to ayesian Shrinkage Estimators. South African Statist. J. (5) Mehta, J. S. & Srinivasan, R. (97). Estimation of the Mean by Shrinkage to a Point. J. Amer. Statist. Asso. (66) Pandey,.N. (979). On Shrinkage Estimation of Normal Population Variance. Commun. Statist. (8) Pandey,.N. & Singh, J. (977). Estimation of Varience of Normal Population Prior Information. J. Indian Assoc. (5) Pandey, M. & Upadhyay, S.K. (985). ayesian Shrinkage Estimation of Reliability From Censord Sample With Exponential Failure Model. South African Statist. J. (9) Thompson, J.R. (968). Some Shrinkage Techniques for Estimating the Mean. J. Amer. Statist. Assoc. (6) Yousef, M.A.Q. Abusalih, M.S. & Ali, M.A. (986). On Some Shrinkage Techniques for Estimating the Parameter of Exponential Distribution. Qatar Univ. Sci. ull. (6) Yousef, M.A.Q. (99). Confidence ounds of Reliability of a Series System. Journal of Information and Optimization Sciences. () Vol., 007 An - Najah Univ. J. Res. (N. Sc.)