Iteratioal Joural of Cotemporary Mathematical Scieces Vol. 1, 215, o. 6, 275-286 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/1.12988/ijcms.215.563 Asymptotic Properties of MLE i Stochastic Differetial Equatios with Radom Effects i the Diffusio Coefficiet 1 School Alsukaii Mohammed Sari 1,2, Alkreemawi Walaa Khazaal 1,2 ad Wag Xiag-ju 1* of Mathematics ad Statistics, Huazhog Uiversity of Sciece ad Techology, Wuha, Hubei, 4374 P. R. Chia 2 Departmet of Mathematics, College of Sciece, Basra Uiversity, Basra, Iraq Copyright 215 Alsukaii Mohammed Sari et al. This article is distributed uder the Creative Commos Attributio Licese, which permits urestricted use, distributio, ad reproductio i ay medium, provided the origial work is properly cited. Abstract We cosider idepedet stochastic processes( X i (t), t [, T i ], i = 1,, ), defied by a stochastic differetial equatio with diffusio coefficiets depedig oliearly o a radom variable i.the distributio of the radom effect i depedig o ukow parameters which are to be estimated from the cotiuous observatios of the processes X i (t). Whe the distributio of the radom effects is expoetial, the asymptotic properties of the maximum likelihood estimator (MLE) are derived whe ted to ifiity. We also discussed the asymptotic properties of the estimator whe the radom effect is Gaussia. Keywords: Stochastic differetial equatios, maximum likelihood estimator, oliear radom effect, cosistecy, asymptotic ormality 1 Itroductio Stochastic differetial equatios (SDEs) are a atural choice to model the time evolutio of dyamic systems which are subject to ifluece. I the recet years, the stochastic differetial equatios with radom effects have bee the subject of diverse applicatios such as pharmacokietic/pharmacodyamics, euroal
276 Alsukaii Mohammed Sari et al. modelig (Delattre ad Lavelle, 213[2], Doet ad Samso, 213[6], Picchii et al. 21[1]). I biomedical research, studies i which repeated measuremets are take o series of idividuals or experimetal aimals play a importat role, models icludig radom effects to model this kid of data ejoy a icreasig popularity. Maximum likelihood estimator (MLE) of the parameters of the radom effect, is geerally ot possible, because of the likelihood fuctio is ot available i most cases, except i the specific case of SDE (Ditlevse ad De Gaetao (25) [5]) ad [1]. May refereces proposed approximatios for the ukow likelihood fuctio, for geeral mixed SDEs a approximatios of the likelihood have bee proposed (Picchii ad Ditlevse, 211[9]), liearizatio (Beal ad Sheier (1982)[1]), or Laplace's approximatio (Wolfiger, (1993)[12]). Delattre et al.(212)[4], studied the maximum likelihood estimator for radom effects i more geerally for fixed T ad tedig to ifiity ad foud a explicit expressio for likelihood fuctio ad exact likelihood estimator by ivestigate the liear radom effect i the drift together with a specific distributio for the radom effect. Almost researcher studied the radom effect i the drift ot i diffusio except Delattre et al.(214) [3] who use the radom effect i the diffusio coefficiet with a specific distributio ad focus o discretely observed SDEs. I the preset paper we focus o stochastic differetial equatio with radom effect i diffusio term ad without radom effect i drift term. We cosider real valued stochastic processes( X i (t), t [, T i ], i = 1,, ), with dyamics ruled by the followig SDEs: dx i (t) = b(x i (t))dt + σ(x i (t), i ))dw i (t), X i () = x i, i = 1,, (1) Where W 1,, W are idepedet wieer processes, 1,, are i. i. d. R + -valued radom variables, 1,, ad W 1,, W are idepedet ad x i, i = 1,, are kow real values. The fuctios b(x) (drift term) ad σ(x) (diffusio term) are kow real valued fuctios. Each process X i (t) represets a idividual, the variable i represets the radom effect of idividual i, the radom variables 1,, have a commo distributio g(φ, θ)dυ(φ) o R + where θ is a ukow parameter belogig to a set Θ R d where υ is a domiatig measure. Our aim is to estimate θ from the cotiuous observatios ( X i (t), t [, T i ], i = 1,, ), we focus o a special case of oliear radom effect i the diffusio coefficiet i the model (1), i.e. σ(x, i ) = 1 i σ(x), where σ is a kow real fuctio ad i is a expoetial ad Gaussia, we fid a explicit
Asymptotic properties of MLE i stochastic differetial equatios 277 likelihood formula ad the maximum likelihood estimator of θ. We will use also the sufficiet statistics U i ad V i as i [4]: T i U i = b(x i(s)) σ 2 (X i (s)) T i dx i (s), V i = b2 (X i (s)) σ 2 (X i (s)) ds, i = 1,, We prove the cosistecy ad the asymptotic ormality of the maximum likelihood estimator of θ (MLE of θ). The orgaizatio of our paper is as follows. Sectio 2 cotais the otatio ad assumptios that we will eed throughout the paper. The explicit likelihood fuctio ad a specific distributio for the radom effect are itroduced i sectio 3. I sectio 4 we study the cosistecy ad asymptotic ormality of the maximum likelihood estimator whe the radom effect is expoetial distributio. I sectio 5 we itroduce Gaussia oe dimesioal oliear radom effect. 2 Notatios ad Assumptios Cosider real valued stochastic processes (X i (t), t ), i = 1,, with dyamics ruled by (1). The processes W 1,, W ad the radom variables 1,, are defied o a commo probability space (Ω, F, P).Cosider the filtratio (F t, t ) defied byf t = σ( i, W i (s), s t, i = 1,,. As F t = σ( i, W i (s), s t) F t i with F t i = σ( i, j, W j (s), s t, j i) idepedet of W i, each process W i is a (F t, t ) Browia motio. Moreover, the radom variables i are F measurable. We assume that: H1 the fuctio σ belogs to c 2 (R R + ) ad for all x R, σ 2 2 (x, φ) σ 1. < σ 2 Uder H1 the process (X i (t)) is well defie ad ( i, X i (t)) adapted to filtratio (F t, t ). The processes( i, X i (t)), i = 1,, are idepedet. For all φ, ad all x i R, the stochastic differetialequatio: dx i φ (t) = b(x i φ )dt + σ(x i φ, φ)dw i (t), X i φ () = x i (2) Admits a uique strog solutio process (X i φ (t), t ) adapted to filtratio (F t, t ). We deduce that the coditioal distributio of X i give i = φ idetical to the distributio of X i φ.
278 Alsukaii Mohammed Sari et al. 3 Likelihood, specific distributio for the radom effects 3.1 Likelihood We itroduce the distributio Q φ x i,t i of (X i φ (t), t [, T i ]).Let P θ i = x g(φ, θ)dv(φ) Q i,t i i φ deote the joit distributio of ( i, X i (t)) ad let Q θ deote the margial distributio of (X i (t), t [, T i ]). Let us cosider the followig assumptio: H2 1. For i = 1,, ad for all φ, φ, Q φ x i,t i ( b 2 φ T i (X i (t)) σ 2 φ (X i (t),φ ) dt < + ) = Propositio 3.1 uder H1-H2 ad let φ R +, we have, the distributio Q φ x i,t i are absolutely cotiuous w.r.t. Q i = Q φ x i,t i with desity: dq φ x i,t i dq i (X i ) = L Ti (X i, φ) = exp ( b(x i(s)) σ 2 (X i (s)) T i T i dx i (s) 1 2 b2 (X i (s)) σ 2 (X i (s)) ds) (3) (see Liptser ad Shiryaev[7]), which is admits a cotiuous versio Q i a.s. Proof: (see the proof of propositio 2) i [4]. By idepedet of idividuals, P θ = i P θ is the distributio of ( i, X i (. )), i = 1,, ad Q θ = i Q θ is the distributio of the sample(x i (t), t [, T i ], i = 1,, ). We ca compute the desity of Q θ w.r.t. Q = Q i as follow: dq θ dq i (X i) = L Ti (X i, φ)g(φ, θ)dv(φ) = λ i (X i, θ) R + The distributio Q θ admits a desity give by: dq θ dq (X 1,, X ) = λ i (X i, θ) Ad the exact likelihood of whole sample ( X i (t), t [, T i ], i = 1,, ) is A (θ) = λ i (X i, θ)
Asymptotic properties of MLE i stochastic differetial equatios 279 3.2 Specific distributio for the radom effects I this sectio we cosider model (1) with oliear radom effect i the diffusio coefficiet σ(x, φ) = 1 φ σ(x) where φ R+ ad b(. ), σ(. ) are kow fuctios. We assume that: T i b2 (X i (s)) x ds <, Q i,t i σ 2 φ (X i (s)) a. s. for all φ. Ad for i = 1,, ; T i = T, x i = x, so that ( X i (t), t [, T], i = 1,, ) are i. i. d. We will use the well defie statistics as follow: T U i = b(x i(s)) σ 2 dx (X i (s)) i (s), V i = b2 (X i (s)) σ 2 (X i (s)) So that the desity λ i (X i, θ) is give by: T ds (4) λ i (X i, θ) = g(φ, θ) exp (φ 2 U i 1 2 φ2 V i ) dv(φ) (5) R + For a geeral distributio g(φ, θ)dv(φ) of the radom effect i, there is o explicit expressio for λ i (X i, θ) above. We propose a specific distributios (expoetial (θ) ad Gaussia (, ω 2 ), where ω 2 is ukow) for the radom effects, which will give a explicit likelihood. I this sectio we shall use expoetial distributio ad i sectio 5 we shall use Gaussia distributio. I the ext propositio a explicit expressio for λ i (X i, θ) is obtaied whe the distributio of the radom effects is expoetial with ukow parameter θ Θ = R +, the true value is deoted by θ ο. Propositio 3.2 suppose that g(φ, θ)dv(φ) = exp (θ), θ > ad its desity θe φθ, the Proof: sice λ i (X i, θ) = θ 2π θ 2 λ i (X i, θ) = exp ( ). V i 2U i 2(V i 2U i ) R + g(φ, θ) exp (φ 2 (U i 1 V 2 i)) dv(φ), so we must compute first g(φ, θ) exp (φ 2 (U i 1 2 V i)) which is equal to exp (φ 2 (U i 1 2 V i)). θexp ( φθ) From the expoet we have:
28 Alsukaii Mohammed Sari et al. We set: φ 2 (U i 1 2 V i) θφ = 1 2 (V i 2U i )φ 2 θφ σ i 2 = (V i 2U i ) 1 = = 1 2 (V i 2U i ) [φ 2 + 2θ V i 2U i φ] = 1 (V 2 i 2U i ) [(φ θ ) 2 θ2 V i 2U i (V i 2U i ) 2] = 1 2 (V i 2U i )(φ θ V i 2U i ) 2 + 1 V i 2U i, m i = θ 2 2(V i 2U i ) θ V i 2U i The λ i (X i, θ) = θ exp ( 1 (V R + 2 i 2U i )(φ m i ) 2. exp ( ) dφ 2(V i 2U i ) θ 2π θ 2 λ i (X i, θ) = exp ( ). V i 2U i 2(V i 2U i ) Ad the coditioal distributio of i give X i is N(m i, σ i 2 ). I order to fid MLE θ, the log-likelihood fuctio is θ 2 Ad hece, L (θ) = loga (θ) θ 2 = log θ exp ( ). 2(V i 2U i ) = log 2π V i 2U i + logθ + 2π V i 2U i θ 2 (6) 2(V i 2U i ) L (θ) θ = ( 1 + θ ) ad the MLE of θ is θ = θ V i 2U i 1 U i 2V i (7) 4 properties of maximum likelihood estimator of θ I this sectio we shall study the properties of θ (cosistecy ad asymptotic ormality), the followig assumptios ad results are importat for our purpose:
Asymptotic properties of MLE i stochastic differetial equatios 281 Let we assume β i (θ) = ( 1 θ + be a radom variable ad the score fuctio is β i θ V i 2U i ) (8) = β θ i(θ). (9) Propositio 4.1 Let θ R +, E θ (β i (θ)) = ad var(β i (θ)) = E(β i 2 (θ)),where E θ meas the expectatio w.r.t. P θ. Proof: We set β 1 (θ) = β 1, Sice λ 1 (X 1, θ) = dq θ 1, Q dq 1 θ 1 = λ 1 (X 1, θ)dq 1 = 1, where C C T is the space of T real cotiuous fuctios (x(t), t [, T i ]) defied o [, T i ]. By iterchage derivatio w.r.t. θ ad itegratio w.r.t. Q 1 we have : λ 1 θ dq1 =. C T Sice λ 1 = exp ( θ 2 ). θ 2(V i 2U i ) ad hece: θ 2π θ. ( + 1 ),the λ 1 = λ V i 2U i V i 2U i θ θ 1. β 1 (θ) λ 1 C θ dq1 = λ 1 β T C 1 (θ)dq 1 T = E(β 1 (θ)) =. Sice var(β 1 ) = E(β 1 2 ) (E(β 1 )) 2, var(β 1 ) = E(β 1 2 ). Remark 1 by usig the law of large umbers ad the cetral limit theorem, it is easy to prove that the radom variable 1 L θ (θ) coverges i distributio to ormal (N(, I(θ)) as ted to ifiity for all θ uder Q θ,where I(θ) is fisher iformatio, ad the radom variable 1 L θ (θ) coverges i probability to 2 I(θ)= E(β 2 1 (θ)) ( the covariace of β 1 (θ)). Further the assumptios H4-H6 i [4] we eed the followig assumptio before verificatio of the asymptotic properties of the maximum likelihood estimator of the parameter θ. H3 there is a ope set M θ s.t. θ M θ Θ. 2
282 Alsukaii Mohammed Sari et al. 4.1 cosistecy of MLE We cosider the theorem 7.49 ad 7.54 of Schervish (1995) [11] ad verify the coditios i this theorem for our purpose. Theorem 1 [11]: Let {x } =1 be coditioally i. i. d give θ with desity f 1 (x θ) with respect to a measure v o a space(χ 1, B 1 ). Fix θ ο Ω, ad defie, for each M Ω ad x χ 1, Z(M, x) = if ψ M log f 1(x θ ο ) f 1 (x ψ) Assume that for each θ θ ο, there is a ope set N θ such that θ N θ ad that E θο Z(N θ, X i ) >. Also assume that f 1 (x.) is cotiuous at θ for every θ, a.s. [P θο ]. The if θ is the MLE of θ correspodig to observatios. It holds that lim θ = θ ο a.s. [P θο ]. Trisha Maitra el. (214) [8] used this theorem for the same purpose. We ote that for ay x, f 1 (x θ) = λ 1 (x, θ) = λ(x, θ). Which is clearly cotiuous i θ. I our case we compute for every θ θ ο : f 1 (x θ ο ) f 1 (x θ) = θ ο 2π exp ( V i 2U i 2(V i 2U i ) ) θ ο 2 θ 2π θ exp ( 2 V i 2U i 2(V i 2U i ) ) = θ ο θ exp ( θ ο 2 θ 2 2(V i 2U i ) ) Ad hece log f 1(x θ ο ) f 1 (x θ) = log θ ο θ + θ ο 2 θ 2 2(V i 2U i ) We ote that E θο log θ ο θ is fiite ad by usig lemma (1) i [4], E( θ 2 ο θ 2 ) is 2(V i 2U i ) also fiite, ad by usig H3, follows that E θο Z(N θ, X i ) >, hece lim θ = θ ο a.s. [P θο ].
Asymptotic properties of MLE i stochastic differetial equatios 283 4.2 Asymptotic ormality of MLE Propositio 4.2 Assume H1-H2 ad H4-H6 i [4], the maximum likelihood estimator of θ satisfies as ted to ifiity, (θ θ ο ) d 1 N(, ). I(θ ο ) Proof: Sice MLE θ is maximizer of L (θ) = 1 log λ(x i, θ), L (θ ) = (logλ(x i, θ)) = By usig the Tayler formula; = L (θ ) = L (θ ο ) + L (θ 1)(θ θ ο ), where θ 1 [θ, θ ο ], (θ θ ο ) = L (θ ο ) L (θ 1) ad (θ θ ο ) = L (θ ο ) L (θ 1) (1) Sice θ ο is the maximizer of L (θ), we have: L (θ ο ) = E θο l (X, θ ο ) =, (l (X, θ ο ) = (logλ(x i, θ)) ) Ad so, E θο l (X, θ ο )=E θο 2 The umerator i (1) L (θ ο ) = ( 1 (x i, θ ο ) ) θ 2 logλ(x i, θ) = I(θ ο ) = ( 1 (x i, θ ο ) E θο l (X 1, θ ο )) d N(, var(l (X 1, θ ο ))) (11) Coverge i distributio by CLT. From the deomiator i (1) we have for all θ, L (θ) = 1 l (X i, θ) Coverge to E θο l (X 1, θ) by the law of large umbers. Sice θ 1 [θ, θ ο ] ad θ θ ο, we have θ 1 θ ο. So we have L (θ 1) E θο l (X 1, θ ο ) = I(θ ο ) hece L (θ ο ) L (θ 1) d N(, var(l (X 1,θ ο ) (I(θ ο )) 2 )
284 Alsukaii Mohammed Sari et al. ad sice var θο (l (X 1, θ ο )) = (E θο (l (X, θ ο )) 2 ) (E θο l (x, θ ο )) 2 = I(θ ο ) By substitutig i (1), we have: (θ θ ο ) d 1 N (, I(θ ο ) ). 5 Gaussia oe dimesioal oliear radom effect I this sectio we discuss the case whe the radom effect is Gaussia, whe μ = ad ω 2 is ukow (g(φ, θ)dv(φ) = N(, ω 2 )), that is mea 1 the desity fuctio is exp ( 1 2πω 2ω 2 φ2 ), So by a aalogous method of propositio 3.2 we get the joit desity fuctio of ( i, X i ) w.r.t. dφ dq i as follow: 1 g i (X i, θ) = ω 2 (V i 2U i ) + 1 Ad the coditioal distributio of i give X i is Gaussia as follow: Ad the likelihood fuctio is: ω 2 N (, ω 2 (V i 2U i ) + 1 ) Ad so, L (θ) = 1 2 log(ω2 (V i 2U i ) + 1), Let we assume that: L (θ) ω 2 = 1 2 V i 2U i ω 2 (V i 2U i ) + 1 Ad the score fuctio is α i (ω 2 ) = L (θ) ω 2 V i 2U i ω 2 (V i 2U i ) + 1 = 1 2 α i(ω 2 )
Asymptotic properties of MLE i stochastic differetial equatios 285 Remark we ca ivestigate the properties of the radom variable α i (ω 2 ) as i sectio 4 ad by usig theorem 1, propositio (4), assumptios H1-H3 ad the assumptios H4-H6 i [4], the cosistecy ad asymptotic ormality of MLE ω 2 will be hold. Refereces [1] S. Beal ad L. Shier, Estimatig populatio kietics, Critical Reviews i Biomedical Egieerig, 8 (1982), 195-222. [2] M. Delattre ad M. Lavielle, Couplig the SAEM algorithm ad the exteded Kalma filter for maximum likelihood estimatio i mixed-effects diffusio models, Statistics ad Its Iterface, 6 (213), 519-532. http://dx.doi.org/1.431/sii.213.v6.4.a1 [3] M. Delattre, V. Geo-Catalot, ad A. Samso, Estimatio of populatio parameters i stochastic differetial equatios with radom effects i the diffusio coefficiet, Preprit MAP, 5 (214), 214-7. [4] M. Delattre, V. Geo-Catalot, ad A. Samso, Maximum likelihood estimatio for stochastic differetial equatios with radom effects, Scadiavia Joural of Statistics, 4 (212), 322-343. http://dx.doi.org/1.1111/j.1467-9469.212.813.x [5] S. Ditlevse ad A. De Gaetao, Mixed effects i stochastic differetial equatio models, REVSTAT Statistical Joural, 3 (25), 137-153. [6] S. Doet ad A. Samso, A review o estimatio of stochastic differetial equatios for pharmacokietic-pharmacodyamics models, Advaced Drug Delivery Reviews, 65 (213), 929-939. http://dx.doi.org/1.116/j.addr.213.3.5 [7] R. S. Liptser ad A. N. Shiryaev, Statistics of Radom Processes, I. Geeral Theory, 2d editio. Spriger-Verlag, Berli, Heidelberg, (21). http://dx.doi.org/1.17/978-3-662-1343-8 [8] T. Maitra ad S. Bhattacharya, O asymptotic related to classical iferece i stochastic differetial equatios with radom effects, ArXiv: 147.3968v1, (214). 1-12. [9] U. Picchii ad S. Ditlevse, Practical estimatio of high dimesioal stochastic differetial mixed-effects models, Computatioal Statistics & Data Aalysis, 55 (211), 1426-1444. http://dx.doi.org/1.116/j.csda.21.1.3
286 Alsukaii Mohammed Sari et al. [1] U. Picchii, A. De Gaetao, ad S. Ditlevse, Stochastic differetial mixedeffects models, Scad. J. Statist., 37 (21), 67-9. [11] M. J. Schervish, Theory of Statistics, Spriger-Verlag, New York. (1995). http://dx.doi.org/1.17/978-1-4612-425-5 [12] R. Wolfiger, Laplace s approximatio for oliear mixed models, Biometrika, 8 (1993), 791-795. ISSN 6-3444. http://dx.doi.org/1.193/biomet/8.4.791 Received: July 7, 215; Published: August 3, 215