ESTIMATION OF THE STATIONARY DISTRIBUTION OF A SEMI-MARKOV CHAIN

Transcription

1 Journal of Relablty and Statstcal Studes; ISSN (Prnt): , (Onlne): Vol. 5, Issue Specal (01): 15-6 ESTIATION OF THE STATIONARY DISTRIBUTION OF A SEI-ARKOV CHAIN Vlad Stefan Barbu *, Jan Bulla and Antonello aruott * Unversté de Rouen, Laboratore de athématques Raphaël Salem, UR 6085, Avenue de l' Unversté, BP.1, F76801 Sant-Étenne-du-Rouvray, France Unversté de Caen, Laboratore de athématques Ncolas Oresme, CNRS UR 6139, 1403 Caen Cedex, France Unverstà d Roma Tre, Dpartmento d Isttuzon Pubblche, Economa e Socetà, Rome, Italy Emal: barbu@unv-rouen.fr; bulla@uncaen.fr; antonello.maruott@unroma3.t Abstract Ths artcle s concerned wth the estmaton of the statonary dstrbuton of a dscretetme sem-arov process. After brefly presentng the dscrete-tme sem-arov settng, we propose an estmator of the assocated statonary dstrbuton. The man results concern the asymptotc propertes of ths estmator, as the sample sze becomes large. A numercal example llustrates the asymptotc propertes of the estmators. Key words sem-arov chans, statonary dstrbuton, nonparametrc estmaton, asymptotc propertes. 1. Introducton Sem-arov processes and arov renewal processes represent a class of stochastc processes that generalze the arov and the renewal processes. As t s well nown, for a dscrete-tme (respectvely contnuous-tme) arov process, the sojourn tme n each state s geometrcally (respectvely exponentally) dstrbuted. In the sem- arov case, the sojourn tme dstrbuton can be any dstrbuton on * (resp. ). For ths reason, the sem-arov approach s much more sutable for applcatons, than the arov one (see, e.g., [1], [], [5], [6], [8]). A quantty related to a sem-arov process s the so-called lmt dstrbuton (assumed to exst), whch descrbes the lmt behavor of the process, when tme becomes large (cf. Defnton 4). In a certan sense, ths s also the statonary dstrbuton of the chan (see the dscusson after Defnton 4). Es tmatng the statonary dstrbuton of a sem-arov chan s an mportant queston, at least for two reasons. Frstly, from a theoretcal or appled pont of vew, one s always nterested n the equlbrum behavor of a process (when ths equlbrum exsts). Secondly, when a certan phenomenon has started suffcently far n the past, one can always consder that t has reached t's equlbrum behavor when we actually begn the observaton. When ths s the case, t s justfed to consder that the statonary dstrbuton s the ntal

2 16 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) dstrbuton of the process. As ths ntal dstrbuton appears n the computaton of many quanttes we are nterested n (for example, when computng the relablty/survval functon, the avalablty, the falure rate, the mean tme to falure/repar), t s mportant to be able to estmate ths statonary dstrbuton and to fnd estmators that have nce asymptotc propertes. To conclude, the purpose of ths paper s to estmate the statonary dstrbuton of a dscrete-tme sem-arov process and to nvestgate the asymptotc propertes of ths estmator, as the sample sze becomes large. Smlar results have already been obtaned n [7] for a contnuous-tme sem-arov process. The present artcle s structured as follows: n the next secton we brefly ntroduce the sem-arov framewor and gve the necessary notaton and defntons. In Secton 3 we defne the statonary dstrbuton of a sem-arov chan, propose emprcal estmators of the mean sojourn tmes of the sem-arov chan and of the statonary dstrbuton of the so called embedded arov chan. Consequently, we obtan the correspondng estmator of the statonary dstrbuton of the sem-arov chan. Secton 4 s devoted to the asymptotc propertes of the estmator of the statonary dstrbuton of the sem-arov chan, namely to the strong consstency and asymptotc normalty. We llustrate the theoretcal results by a numercal example.. Dscrete-tme Sem-arov Framewor In ths secton we ntroduce the basc notaton concernng a dscrete-tme sem-arov model. We consder a random system wth fnte state space E {1,, s}, whose evoluton n tme s governed by a stochastc process Z () Z. Let us denote by S () S n n the successve tme ponts when state changes n () Z occur and by J () Jn n the successvely vsted states at these tme ponts. Set also X X * for the successve sojourn tmes n the vsted () n n states; thus, X n Sn Sn 1, * n. Fgure 1 gves a representaton of the evoluton of the system. The relaton between process Z and process J of the successvely vsted states s gven by Z J (), or, equvalently, J Z, n,, where N n S n N() : max{ n S } s the dscrete-tme countng process of the number of jumps n [1, ]. n Fgure 1: Sample path of a sem-arov chan

3 Estmaton of the Statonary Dstrbuton of a Sem-arov Chan 17 We suppose that Z () Z s a sem-arov chan (SC), or, equvalently, that the couple ( J,)( S,) Jn Sn n s a arov renewal chan (RC) and we denote by q ((); q, j,) E the assocated dscrete-tme sem-arov ernel defned j q () :( J,). j X J by j n1 n1 n We also ntroduce the cumulatve sem-arov ernel Q (();) Q defned by Q () :( J,)(), j, X, J. q l j E j n1 n1 n j l0 Note that, for ( J,) S a arov renewal chan, we can easly see that () Jn n, s a arov chan, called the embedded arov chan (EC ) assocated to the RC ( J,) S. We denote by p () p j, j E E the transton matrx of () J, n n p (), J, j, J. j E n j n1 n Let the row vector α ( 1,,) s denote the ntal dstrbuton of the sem-arov chan Z () Z, where :()(), Z0. J0 E We also assume that p 0,() q 0,,. E We defne now the sojourn tme dstrbutons n a gven state and the condtonal dstrbutons dependng on the next state to be vsted. Defnton 1 (Condtonal dstrbutons of the sojourn tmes) For all, j E, let us defne: 1. f j ( ), the condtonal dstrbuton of X 1, n : n f ()( X J,), J. j j n1 n n1. F j ( ), the condtonal cumulatve dstrbuton of X 1, n : j n1 n n1 j l0 n F ()( X J,)(), J. j f l Defnton ( Sojourn tmes dstrbutons n a gven state) For all E, let us denote by: 1. h ( ), the sojourn tme dstrbuton n state : h ()()(), X. J q n1 n j je. H ( ), the sojourn tme cumulatve dstrbuton functon n state : *

4 18 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) H ()()(), X. J h l n1 n l1 Let us also denote by m the mean sojourn tme n a state E, m :()()(1()). S J j h H j 1 0 j j 1 1 Note that, for all, j E and such that p 0, the sem-arov ernel verfes the relaton q ()(). p f j j j j * If we suppose that the sojourn tmes n a state depend only on the present vsted state, a partcular type of sem-arov chan s obtaned, whose sem-arov ernel s q ()(), p h, j E,. For ths partcular type of sem-arov chan we wll j j prove the asymptotc normalty of the statonary dstrbuton estmator (Proposton ). For G the cumulatve dstrbuton functon of a r.v. X, we denote ts survval functon by G() n: 1()(), G n. X n n Thus, for all states, j E we put F j and H for the correspondng survval functons. Defnton 3 The transton functon P (();) P of the sem-arov chan Z s defned by P () :(),, Z, jz. j E j 0 3. Estmaton of the Statonary Dstrbuton Defnton 4 (lmt dstrbuton of a SC) For a sem-arov chan () Z the lmt dstrbuton π ( 1,,) s s defned, when t exsts, by j lm(), Pj for every, j E. Let us denote by U n the bacward recurrence tme Un : n SN () n of the sem-arov chan. It s worth notng that the lmt dstrbuton π s also the statonary dstrbuton of the SC () Z n the sense that t s the margnal dstrbuton of the statonary dstrbuton of the arov chan ( Zn,) Un n, that s j ({ j },), j E (see [1] & [3]). For these reasons, the lmt dstrbuton π wll be also called the statonary dstrbuton of the SC. All along ths paper, we consder that the SC Z s rreducble, aperodc, wth fnte mean sojourn tmes.

5 Estmaton of the Statonary Dstrbuton of a Sem-arov Chan 19 Let () j Sn n be the successve passage tmes n a fxed state j E. For any arbtrary states, j E, j, we denote by j the mean frst passage tme from state j to j for the SC, j :() S0, and by jj the mean recurrence tme of state j for j the SC, jj :(), j S1 where s the condtonal expectaton gven { J0 }. Proposton 1 The lmt dstrbuton of an SC s gven by 1 ()() j m j j m j j m j, j E, () m m jj E where the row vector ν ((1),,()) s s the statonary dstrbuton of the EC J and we denoted by m :() m the mean sojourn tme of the SC. () n n E A proof of ths result can be found n [1]. A dfferent proof, based on generatng functons, can be found n [4]. Let us assume now that we have an observaton of ths SC, censored at fxed arbtrary tme *, ( Z0,,) Z, or, equvalently, an observaton of the assocated arov renewal chan ( J,) S, ( J0, X1(), 1()(), J N, X N, J N,), u where N () n n n u : S s the censored sojourn tme n the last vsted state (). For all states, j E, let us ntroduce: N () : N () 1 { Jn } { Jn, Sn1 } n0 n0 J, up to tme ; EC () n n N () : N () J N 1 1 the number of vsts to state of the j { Jn1, Jn j} { Jn1, Jn j, Sn } n1 n1 EC () J n n 1 1 the number of transtons of the from to j, up to tme. Let us consder the emprcal estmator of the statonary dstrbuton of the EC () J n n defned by: N () ˆ(,),. E (1) N() For any state E, wrtng the mean sojourn tme n state as m (1 H ())() H and usng the emprcal estmator of the survval 0 0 functon n state, H (), we get an estmator for m,

6 0 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) N () 1 mˆ () X. N () () 1 Consequently, an estmator of the mean sojourn tme of the SC, m, s N j () N () j N (3) je 1 1 ˆ 1 1 m () X X, N()() and we get the followng estmator of the statonary dstrbuton of the SC 4. Asymptotc Results N () 1 ˆ () X,. ˆ E m ()() (4) N 1 Frst of all, note that we have the followng asymptotc results: a. s. N () /()(), N (5) a. s. N () /()() N, p (6) j a. s. j N () / 1/. (7) The frst two results are mmedately obtaned from classcal arov chan asymptotc propertes, whereas the thrd one s a drect applcaton of the Strong Law of Large Numbers (SLLN) to the smple renewal chan () Sn. S0 n Lemma 1 For any state E of the SC, the estmators ˆ(,), mˆ (), m ˆ (), and ˆ () proposed n Equatons (1-4) for the statonary dstrbuton of the EC, mean sojourn tme n state mean sojourn tme of the SC, and statonary dstrbuton of the SC, respectvely, are strongly consstent, as tends to nfnty. Proof. The consstency of ˆ(,) has been already stated n (5). From the SLLN and the fact that N () a. s. j, we obtan the strong consstency of m ˆ () These results, together wth contnuous mappng theorem, yeld the strong consstency of m ˆ () and ˆ (), as tends to nfnty. The asymptotc normalty of the statonary dstrbuton estmator of a SC wll be proved for a partcular sem-arov model, defned by the sem-arov ernel q ()(), p h, j E,. Remar 1 gven after the proof of the result gves j j detals on ths choce. j

7 Estmaton of the Statonary Dstrbuton of a Sem-arov Chan 1 Proposton For any fxed arbtrary state E, we have wth asymptotc varance [() ˆ ](0,), (8) () m 1 m, 1 1 m m where s the varance of the recurrence tme of state and sojourn tme n state. (9) s the varance of the Proof. The proof s essentally based on the lmtng dstrbuton of the total sojourn tme n a state of sem-arov process (cf. Theorem 3.1 of [9]). Wthout loss of generalty, for any fxed arbtrary state, we can consder the ntal vsted state J 0 to be. Frst, let us denote by S () the total tme spent by the SC n state up to tme, wthout tang nto account the last censored tme u S,.e., N () N () S ()()() X mˆ N 1 and by S * () the total tme spent by the SC n state up to tme, tang nto account the last censored tme,.e., * ()(). 1{ J } S S u N () Second, let us express the varable of nterest as follows: S ()() / [() ˆ S ] u 1 u / S ()() S u (1(1)) o p S S u u * ()() (1(1)) op 1{ J. } N () As u a. s. / 0 and Slutsy's theorem that S ()() N m, we get from a. s. ˆ m () [() ˆ ] has the same lmt n dstrbuton as

8 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) S * (). Consequently, applyng the result from [9] on the lmtng dstrbuton of the total sojourn tme n a state of sem-arov process, we get the desred result. Remar 1 The proof of Theorem 3.1 of [9] s based on Taács's paper [10], that * consders an alternatng renewal chan () V n * n, wth Vn : X n Yn, n, under the assumptons that () X and n * () Y * n n n are sequences of..d. random varables, and () X and n * () Y * n n n are ndependent between them. In order to apply ths result n our framewor, we need to consder sojourn tmes n a state dependng only on the present vsted state,.e., a sem-arov ernel of the form q ()(). p h The followng result llustrates how the varance of the recurrence tmes can be recursvely computed. A proof of ths result could be found n [4]. Lemma For any state j, the varance jj of the recurrence tme of state j s gven by 1 jj ()() m () m, j p jj () j j where the mean frst passage tmes j,, j E, can be computed usng the followng recurrence formulas (see [4] or[1] for a proof) m p,, j E. j j j 5. Numercal Example In the followng we demonstrate the fndngs of the prevous secton by means of a short smulaton study. ore precsely, we chose a 3-state SC wth shfted Posson sojourn tme dstrbutons, that s, 1 h () e. ( 1)! The true parameter values of the model equal p and Addtonally, a unform dstrbuton s assumed for the ntal dstrbuton. From ths parameterzaton drectly follows m 5 6 4, , , and j j,.

9 Estmaton of the Statonary Dstrbuton of a Sem-arov Chan 3 Thus, the true values of and are avalable for checng the consstency of the estmators presented n Equaton ( 7) and ( 4). Therefore, we smulate 00 sequences wth N() = 500 each, whch s equvalent to values of moderately superor to 000. Fgure : Estmated values of N ()/ (gray lnes) from smulated seres together wth the true value of 1/ (blac lnes) for states =1,, and 3 Fgures & 3 provde a vsual mpresson of convergence towards the true parameter values by means of 0 randomly selected sample paths. Whle the blac horzontal lnes represent the true values of 1/ and, respectvely, the gray lnes result from the correspondng estmators. To confrm the optcal mpresson, we calculate the emprcal 5%- and 95%-quantle of the two estmators for N() = 50, 00 and 500, respectvely, from the 00 smulated trajectores. Table 1 dsplays the results, showng that the estmated quanttes converge toward the true values for ncreasng N() (or ).

10 4 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) Fnally, we nvestgate the fndngs of Proposton. Note that Equaton (9) requres the calculaton of, the varance of the recurrence tme of all states. Usng a parametrc bootstrap approach, we obtan the estmate ˆ Usng ths estmate and the true values of the remanng quanttes descrbed above, we obtan the ''true'' In order to chec for normalty of ˆ, we carry out the Shapro-Wl test. Furthermore, a potental dependence on ( N()) s nvestgated, we consder each 00 estmates ˆ obtaned for fxed N() 1,,,500 and test for normalty by state. The results show that normalty s rejected n a majorty of cases for N() 10, sometmes for 10() N50, and rarely for N() 50. oreover, for ncreasng the varance of the quantty on the left hand sde of equaton (8) converges towards the ''true'' values of. Fgure 4 dsplays the evoluton of varance of ths quantty for ncreasng values of N() (or ). Recall that 00 observatons serve for the varance estmaton for each value of N(). Fgure 3: Estmated values of ˆ () (gray lnes) from smulated seres together wth the true value of (blac lnes) for states =1,, and 3

11 Estmaton of the Statonary Dstrbuton of a Sem-arov Chan 5 Table 1: Smulated results: 5%-and 95% quantle of & π for N()=50, 100, 500 N()=50 N()=00 N()=500 true q 05 q 95 q 05 q 95 q 05 q 95 1/ / / Fgure 4: Estmated value of (gray lnes) from smulated seres together wth the true values (blac lnes) for states =1,, and 3 Smlar statements to those artculated above n the context of tests for normalty hold true: For small values of N(), the sample varance s not too close to the target value, but ths qucly changes for ncreasng N(). Acnowledgment The authors would le to than the Fédératon Normande-athématques (FR CNRS 3335) for ther contnuous support.

12 6 Journal of Relablty and Statstcal Studes, Aprl 01, Vol. 5 (S) References 1. Barbu, V. and Lmnos, N. (008). Sem-arov Chans and Hdden Sem- arov odels toward Applcatons - Ther use n Relablty and DNA Analyss. Lecture Notes n Statstcs, vol. 191, Sprnger, New Yor.. Cnlar, E. (1975). Introducton to Stochastc Processes. Prentce Hall, New Yor. 3. Chryssaphnou, O., Karalopoulou,., and Lmnos, N. (008). On dscrete tme sem-arov chans and applcatons n words occurrences. Comm. Statst. Theory ethods, 37, p Howard, R. (1971). Dynamc Probablstc Systems, v., Wley, New Yor. 5. Janssen, J. and anca, R. (006). Appled Sem-arov Processes. Sprnger, New Yor. 6. Lmnos, N. and Oprşan, G. (001). Sem-arov Processes and Relablty. Brhäuser, Boston. 7. Lmnos, N., Ouhb, B. and Sade, A. (005). Emprcal Estmator of Statonary Dstrbuton for Sem-arov Processes. Comm. Statst. Theory ethods, 34(4), p Ouhb, B. and Lmnos, N. (1999). Nonparametrc estmaton for sem-arov processes based on ts hazard rate functons. Stat. Inference Stoch. Process., (), p Taga, Y. (1963). On the lmtng dstrbutons n arov renewal processes wth fntely many states. Ann. Inst. Statst. ath., 15(1), p Taács, L. (1959). On a sojourn tme problem n the theory of stochastc processes. Trans. Am. ath. Soc., 93, p