LOGISTIC regression model is often used as the way

Transcription

1 Vol:6, No:9, 22 Secon Orer Amissibilities in Multi-parameter Logistic Regression Moel Chie Obayashi, Hiekazu Tanaka, Yoshiji Takagi International Science Inex, Mathematical an Computational Sciences Vol:6, No:9, 22 waset.org/publication/642 Abstract In multi-parameter family of istributions, conitions for a moifie maximum likelihoo estimator to be secon orer amissible are given. Applying these results to the multi-parameter logistic regression moel, it is shown that the maximum likelihoo estimator is always secon orer inamissible. Also, conitions for the Berkson estimator to be secon orer amissible are given. Keywors Berkson estimator, moifie maximum likelihoo estimator, Multi-parameter logistic regression moel, secon orer amissibility. I. INTRODUCTION LOGISTIC regression moel is often use as the way of statistical analysis of binary ata. In the moel, [] asserte that the minimum logit chi-square estimator MLχ 2 E for short is better than the maximum likelihoo estimator MLE by comparing the exact mean square errors MSEs of the two estimators. The problem which the MLE or the MLχ 2 E is better, calle the Berkson s bioassay problem, is iscusse by many researchers. In this moel, there exists a completely sufficient statistic. So the MLχ 2 E can be improve by the Rao-Blackwell theorem, which is calle Berkson estimator. [2] evaluate the Taylor expansions of the MSEs of these estimators up to the secon orer an showe that the MSE of the Berkson estimator is asymptotically smaller than that of the MLE. [3] trie to solve the problem in terms of asymptotic amissibility. First, they erive a necessary an sufficient conition for a moifie MLE to be secon orer amissible SOA uner the quaratic loss function in general setup of one-parameter case. Especially, in the logistic regression moel, they showe that the MLE is always secon orer inamissible an the Berkson estimator is SOA if an only if the number of the oses is greater than or equal to 4. However, in multi-parameter logistic regression moel, whether the two estimators are SOA or not is open. Recently, [4] erive conitions for a moifie MLE to be SOA in general setup of two-parameter case. Also, they showe that the MLE is always an the Berkson estimator is SOA if an only if the number of the oses is greater than or equal to 6. The purpose in this article is to exten the results in [4] to the p-parameter case where p 3. This article is organize as follows: In Section 2, we present notations, efinition an some theorems for secon orer amissibility in general setup. In Section 3, we ientify C. Obayashi is with the Grauate School of Engineering, Osaka Prefecture University, Osaka, , Japan obayashi@ms.osakafu-u.ac.jp. H. Tanaka is with the Faculty of Liberal Arts an Sciences, Osaka Prefecture University, Osaka, , Japan tanaka@ms.osakafu-u.ac.jp. Y. Takagi is with the Faculty of Eucation, Nara University of Eucation, Nara, , Japan takagi@nara-eu.ac.jp. whether the MLE an the Berkson estimator are SOA or not in the multi-parameter logistic regression moel. Some concluing remarks are given in Section 4. Finally, we give proofs of lemmas. II. PRELIMINARIES In this section, we present necessary conition an sufficient conition for secon orer amissibility in general setup. Suppose that X,...,X n are inepenent an ientically istribute i.i.. ranom vectors accoring to a probability istribution P θ, where θ θ,θ 2,...,θ p R p is unknown an p 3. Suppose that P θ has a probability ensity function fx, θ with respect to some σ-finite measure. Also, we assume the regularity conitions i to v given in [5]. Then, we consier the estimation problem of θ uner the norme quaratic loss function lθ, δ :δ θ Iθδ θ in estimating θ by δ, where Iθ is the Fisher information matrix per one observation. Let λ min θ an λ max θ be the smallest an the largest eigenvalues of Iθ. Henceforth, tra} an A enote the trace an the eterminant of matrix A, an θ : θ θ. Also, /θfθ : /θ fθ, /θ 2 fθ,...,/θ p fθ, /θ fθ : /θfθ for vector value function fθ. [3] propose a concept of secon orer amissibility. Definition : Let D be a set of first orer efficient estimators. An estimator δ D of θ is D-secon orer inamissible as n if there exists an estimator δ D such that lim n n 2 Rθ, δ Rθ, δ} for all θ R p, an lim n n 2 Rθ,δ Rθ,δ} < for some θ R p.an estimator δ D is D-secon orer amissible as n if δ is not D-secon orer inamissible as n. Let ˆθ c be the moifie MLE of θ by function c, that is, ˆθ c : ˆθ ML +cˆθ ML /n, where ˆθ ML is the MLE of θ. Accoring to [5], first orer efficient estimators can be represente as a moifie MLE up to the orer o p /n. Therefore, in this paper, we restrict estimators to the class D : ˆθ c : c C R p }, where C R p is the set of all continuously ifferentiable functions over R p. Henceforth, secon orer amissibility means D-secon orer amissibility as n uner the norme quaratic loss function. Let b c θ be the asymptotic bias of ˆθ c, that is, b c θ : lim n ne θ [ˆθ c θ] b ML θ+cθ. International Scholarly an Scientific Research & Innovation scholar.waset.org/ /642

2 Vol:6, No:9, 22 International Science Inex, Mathematical an Computational Sciences Vol:6, No:9, 22 waset.org/publication/642 Then, the risk ifference between ˆθ c an ˆθ D is given by Rθ, ˆθ Rθ, ˆθ c n 2 tr gθg θiθ+2b c θg θiθ+2 +o } θ gθ n 2 as n, where gθ :θ cθ. Therefore, a necessary an sufficient conition for the moifie MLE ˆθ c to be SOA is that if g C R p satisfies tr gθg θiθ+2b c θg θiθ+2 } θ gθ 2 for all θ R p, then gθ for all θ R p. Here, we present an assumption, which correspons to a potential function. Assumption : There exists a continuously ifferentiable function γ c : R p R + such that Iθb c θ θ log γ c θ for all θ R p. For the moifie MLE ˆθ c satisfying Assumption, 2 is equivalent to } γ c θ h θiθhθ 2tr θ hθ, 3 where hθ :gθγ c θ. Theorem : Suppose that the moifie MLE ˆθ c satisfies Assumption an there exists ξ such that [ ] λmax x Hθ : r < γ c x xθ+rω ξ for all θ R p, where ω ξ : ω ξ,,ω ξ,2,...,ω ξ,p, ξ : ξ,ξ 2,...,ξ p, cos ξ i, i ω ξ,i : cos ξ i j sin ξ j i 2,...,p, p j sin ξ j i p, : ξ R p : ξ i [,πi,...,p 2, ξ p [, 2π}. Furthermore, we assume that the ifferential of Hθ can be obtaine by the ifferential in the integral sign, that is, [ ] λmax x Hθ r 4 θ i θ i γ c x xθ+rω ξ for i,...,p, then ˆθ c is. The proof is omitte since it can be shown that hθ : ω ξ /Hθ satisfies 3 by the similar way to [4]. Theorem may be complicate since it may be ifficult to show the exchangeability of the ifferential sign an integral one. The next result is easy to hanle it. Corollary : Suppose that the moifie MLE ˆθ c satisfies Assumption. If λ max θ γ c θ θ < for all θ 2,...,θ p, then ˆθ c is. The proof is omitte since it can be obtaine from Theorem. The next lemma is use in Theorem 2. The proof is given in Appenix. Lemma : Suppose that h C R p. Then, } tr θ hθ θ u p ω ξhuω ξ Jξξ hols for u>, where is efine in Theorem, : θ R p : θ u}, Jξ : p 2 sin p i ξ i. Theorem 2: Suppose that the moifie MLE ˆθ c satisfies Assumption. Put [ ] γc θ η c r : Jξξ, λ min θ θrω ξ where ω ξ is given in Theorem. If ε r r p η c r for some ε>, then ˆθ c is SOA. Taking account of Lemma, the proof of Theorem 2 can be obtaine by the similar way to [4]. III. ADMISSIBILITIES IN LOGISTIC REGRESSION MODEL In this section, we consier the secon orer amissibilities of MLE an the Berkson estimator in p-parameter logistic regression moel where p 3 is a given number. Suppose that R,...,R k are inepenently istribute ranom variables accoring to the binomial istribution Bn, P i θ for i,...,k with P i θ : +exp x i θ, where θ : θ,θ 2,...,θ p R p is unknown an the oses x i, i,2,..., i,p is known for i,...,k. For convenience, we put i,. Here the conition x i,x i2,...,x ip for some i > >i p shoul be assume. Let Y i,...,y in be i.i.. ranom variables accoring to the binomial istribution B,P i θ for i,...,k. Put Y j : Y j,...,y kj for j,...,n. Then, Y,...,Y n are i.i.. ranom vectors an the Fisher information matrix is given by Iθ P i θ P i θx i x i. Let ˆθ ML an ˆθ B be the MLE an the Berkson estimator of θ, that is, ˆθ B ˆθ ML + n b logitˆθ ML b ML ˆθ ML, International Scholarly an Scientific Research & Innovation scholar.waset.org/ /642

3 Vol:6, No:9, 22 International Science Inex, Mathematical an Computational Sciences Vol:6, No:9, 22 waset.org/publication/642 where b logit θ is the asymptotic bias of the MLχ 2 E ˆθ logit see 74 in [2]. Then, by the similar arguments to [4], we see that both ˆθ ML an ˆθ B satisfy Assumption with γ ML θ, 5 Iθ γ B θ Iθ k P i θexp } 2 x iθ. 6 To apply Theorems an 2 to the logistic regression moel, we prepare some lemmas. The proofs are given in Appenix. Lemma 2: The eterminant of the Fisher information matrix is represente as Iθ P i θ P i θ P ip θ P ip θ i > >i p x i,x i2,...,x ip 2. Lemma 3: For all θ >, θ 2,...,θ p R, the followings hol: i triθ} C θ } e θ, ii Iθ C 2 θ } e pθ, k iii P i θexp } 2 x iθ C 3 θ } exp k 2 θ for some C θ }, C 2 θ } an C 3 θ }, where θ } : θ 2,...,θ p. Lemma 4: For l i l j, put l,...,l p : ξ : x l x l 2 } x l p x i i l,...,l p. For all ξ l,l 2,...,l p an r>, the followings hol: i trirω ξ }C 4 exp r x l, ii Irω ξ C 5 exp r, iii m k P i rω ξ exp r } exp r x l 2 m, where C 4 an C 5 are constants. Theorem 3: The MLE ˆθ ML of θ is always. Proof: From 5, Lemma 3 an the fact λ max θ triθ}, wehave λ max θ γ ML θ C θ } C 2 θ } exp } 2 p +2θ for all θ >, θ 2,,θ p R. Therefore, we see that λ max θ γ ML θ θ C θ } C 2 θ } exp } 2 p +2θ θ <, which implies the secon orer inamissibility of ˆθ ML by Corollary. Theorem 4: The Berkson estimator ˆθ B is if p k 2p +. Proof: By the similar argument to the proof of Theorem 3, we have λ max θ γ B θ C θ } C 2 θ } C 3 θ } exp } 2 2p +2 kθ for all θ >, θ 2,...,θ p R. Therefore, we see that ˆθ B is if p k 2p +. Theorem 5: The Berkson estimator ˆθ B is SOA if 2p x l 4 + m > 7 m ξ l,l 2,...,l p hols for all l,l 2,...,l p l i l j, where l,l 2,...,l p is efine in Lemma 4. In particular, if k 4p 3, the Berkson estimator ˆθ B is SOA Proof: Note that η B r is written by η B r l i l j It is easy to show that l,l 2,...,lp γ B rω ξ λ min rω ξ Jξξ. λ min θ trp Iθ}. Iθ So, from 6 an Lemma 4, we have γ B rω ξ λ min rω ξ trp Irω ξ } k Irω ξ 2 C 6 exp r 2 P i rω ξ exp r } 2p x l 4 m + } m for all ξ l,...,l p, r>, where C 6 is a constant. If 7 hols, we have lim r r p η B r. So, we get the first part from Theorem 2. Next, we show the secon part. From the assumption p 3, we can show that 2p x l 4 + m 2p 5 x l 3 2p 5 x l > m + m2 if k 4p 3. This completes the proof. mp+ International Scholarly an Scientific Research & Innovation scholar.waset.org/ /642

4 Vol:6, No:9, 22 International Science Inex, Mathematical an Computational Sciences Vol:6, No:9, 22 waset.org/publication/642 IV. CONCLUDING REMARKS In one-parameter logistic regression moel, [3] showe that the MLE is always an the Berkson estimator is SOA if only if the number of the oses is greater than or equal to 4 uner the quaratic loss function. The loss function in [3] is ifferent from the one in [4] an this article, however the amissibility result uner the quaratic loss function coincies with the one uner the norme quaratic loss function. Table summarizes the amissibility or inamissibility of the two estimators for p,...,4. The symbol means that the case can not be consiere since the number of the oses k is larger than the imension of the parameter space p. The part of? means that the case is still open. There are some reasons why it is open. One is that the lower boun of λ min θ is not sharp enough to show the complete inamissibility of ˆθ B. Another is that we applie Corollary not but Theorem to show the inamissibility. It seems that showing the valiity of 4 is not so easy. The authors conjecture that there exist both cases. TABLE I ADMISSIBILITIES OF MLE AND BERKSON ESTIMATOR FOR p,...,4 p p 2 p 3 p 4 [3] [4] [Th.3,4,5] [Th.3,4,5] ˆθ ML k k 2 k 3 k 4 ˆθ B k 5 k 6, 7 k 8 SOA? k 9 SOA k,, 2 SOA? k 3 SOA APPENDIX A In this section, we give the proofs of lemmas presente in the previous Sections. Proof of Lemma : Changing the variables as θ : r cos ξ, i θ i : r cos ξ i sin ξ j i 2,...,p, j p θ p : r sin ξ j, j where r> an ξ, we see that r θ, ξ i Tan p ji+ θ2 j θ i i,...,p, an the Jacobian is r p Jξ. Put Hr, ξ :hrω ξ. Then, by the chain rule, we have h l θ θ l r H r, ξcosξ l r H lr, ξcosξ l sin ξ i l + cos ξ l r H l r, ξ r ξ l p r H pr, ξ sin ξ i + r p H r, ξ sin ξ ξ r ξ i H l r, ξcosξ i sin ξ l l j sin ξ j ξ i H p r, ξcosξ i l ji+ sin ξ j i j sin ξ j p ji+ sin ξ j i j sin ξ j l, l 2,...,p, l p. Using this relation an integration by parts, we have h θθ θ u r H r, ξcosξ H r, ξ sin ξ } ξ r r p Jξrξ u cos ξ Jξ u u p p 2 r p 2 } i2 π r p } r H r, ξr ξ sin p i ξ i sin p ξ H r, ξξ ξ } r ξ Jξcosξ H u, ξξ, where ξ } : ξ 2 ξ p an } : ξ 2,...ξ p : ξ }. Similarly, we have h l θθ u p θ l h p θθ u p θ p l JξH l u, ξcosξ l sin ξ j ξ j l 2,...,p, p JξH p u, ξ sin ξ j ξ. j Therefore, it follows that } tr θ hθ θ u p ω ξhuω ξ Jξξ. This completes the proof. Lemma may be prove by the ivergence theorem, which is easier than the proof of Lemma. Proof of Lemma 2: Let S p be the symmetric group of egree p an let sgnσ be the sign of σ S p. Since the International Scholarly an Scientific Research & Innovation scholar.waset.org/ /642

5 Vol:6, No:9, 22 International Science Inex, Mathematical an Computational Sciences Vol:6, No:9, 22 waset.org/publication/642 α, β-th element of Iθ is we have Iθ I αβ θ sgnσ i P i θ P i θ i,α i,β, P i θ P i θ i, i,σ i P ip θ P ip θ ip,p ip,σp i p P i θ P i θ P ip θ P ip θ i p i, ip,p sgnσ i,σ ip,σp P i θ P i θ P ip θ P ip θ i i p i, ip,p x i,...,x ip P i θ P i θ i σ > >i σp P ip θ P ip θ i, ip,p x i,...,x ip. Changing the variables as i σ i,...,i σp i p,weget Iθ i >i 2> >i p P i θ P i θ P ip θ P ip θ iσ, iσ p,p x iσ,...,x iσ p P i θ P i θ P ip θ P ip θ i >i 2> >i p iσ, iσp,p x iσ,...,x iσp sgnσ P i θ P i θ P ip θ P ip θ i >i 2> >i p x i,x i2,...,x ip 2. Proof of Lemma 3: It is easily obtaine that P i θ P i θ exp x iθ exp θ + i,m θ m m2 for all θ >, θ 2,...,θ p R. Accoring to triθ} j 2 i,jp i θ P i θ an Lemma 2, we have i an ii. Furthermore, it can be shown that P i θexp Hence, we get iii. 2 x iθ expx i θ/2 + exp x i θ/2 2 exp 2 x iθ } 2 exp θ + i,m θ m. 2 Proof of Lemma 4: Since m2 4 exp r x i P i rω ξ P i rω ξ exp r x i for all ξ l,l 2,...,l p an r>, weget trirω ξ } j 2 i,j exp r x i exp r x l By using the result of Lemma 2, we get Irω ξ 4 p i > >i p x i,x i2,...,x ip 2 exp 4 p x l,x l2,...,x lp 2 exp Furthermore, the inequality P i rω ξ exp r implies iii. r j r 2 i,j. x i m m. m exprx i ω ξ/2 + exp rx i ω ξ/2 exp r 2 x i exp r 2 x l i ACKNOWLEDGMENT This research was partially supporte by Grant-in-Ai for Young Scientists B, REFERENCES [] J. Berkson, Maximum likelihoo estimates of the logistic function, J. Amer. Statist. Assoc., vol. 5, pp. 3 62, 955. [2] T. Amemiya, The n 2 -orer mean square errors of the maximum likelihoo an the minimum chi-square estimator, Ann. Statist., vol. 9, pp , 98. [3] J. K. Ghosh an B. K. Sinha, A necessary an sufficient conition for secon orer amissibility with applications to Berkson s bioassay problem, Ann. Statist., vol. 9, pp , 98. [4] H. Tanaka, C. Obayashi an Y. Takagi, On secon orer amissibilities in two-parameter logistic regression moel, submitte for publication. [5] K. Takeuchi an M. Akahira, Asymptotic optimality of the generalize Bayes estimator in multiparameter cases, Ann. Inst. Statist. Math., vol. 3, pp , 979. International Scholarly an Scientific Research & Innovation scholar.waset.org/ /642