RENEWAL PROCESSES. Chapter Introduction

Transcription

1 Chaper 5 RENEWAL PROCESSES 5.1 Inroducion Recall ha a renewal process is an arrival process in which he inerarrival inervals are posiive, 1 independen and idenically disribued (IID) random variables (rv s). Renewal processes (since hey are arrival processes) can be specified in hree sandard ways, firs, by he join disribuions of he arrival epochs S 1, S 2,..., second, by he join disribuions of he inerarrival imes X 1, X 2,..., and hird, by he join disribuions of he couning rv s, N() for >. Recall ha N() represens he number of arrivals o he sysem in he inerval (, ]. Figure 5.1 reviews he connecion beween he sample values of hese rv s. r X 3(!) - r 6 X 2(!)- N(,!) = 2 r N(,!) = n for S n(!) apple < S n+1(!) X 1(!)- S 1(!) S 2(!) S 3(!) Figure 5.1: A sample funcion of an arrival process for a given sample poin! wih is arrival epochs {S 1 (!), S 2 (!),... }, is inerarrival imes {X 1 (!), X 2 (!),... }, and is couning process {N(,!); > }. The sample funcion of he couning process is he sep funcion illusraed wih a uni sep a each arrival epoch. The simples characerizaion is hrough he inerarrival imes X i, since hey are IID. Each arrival epoch S n is simply he sum X 1 + X X n of n IID rv s. The characerizaion 1 Renewal processes are ofen defined in a slighly more general way, allowing he inerarrival inervals X i o include he possibiliy 1 > Pr{X i = } >. All of he heorems in his chaper are valid under his more general assumpion, as can be verified by complicaing he proofs somewha. Allowing Pr{X i = } > allows muliple arrivals a he same insan, which makes i necessary o allow N() o ake on posiive values, and appears o inhibi inuiion abou renewals. Exercise 5.3 shows how o view hese more general renewal processes while using he definiion here, hus showing ha he added generaliy is no worh much. 223

2 224 CHAPTER 5. RENEWAL PROCESSES of greaes ineres in his chaper is he renewal couning process, {N(); > }. Recall from (2.2) and (2.3) ha he arrival epochs and he couning rv s are relaed in each of he following equivalen ways. {S n apple } = {N() n}; {S n > } = {N() < n}. (5.1) The reason for calling hese processes renewal processes is ha he process probabilisically sars over a each arrival epoch, S n. Tha is, if he nh arrival occurs a S n =, hen, couning from S n =, he j h subsequen arrival epoch is a S n+j S n = X n X n+j. Thus, given S n =, {N( + ) N( ); } is a renewal couning process wih IID inerarrival inervals of he same disribuion as he original renewal process. This inerpreaion of arrivals as renewals will be discussed in more deail laer. The major reason for sudying renewal processes is ha many complicaed processes have randomly occurring insans a which he sysem reurns o a sae probabilisically equivalen o he saring sae. These embedded renewal epochs allow us o separae he long erm behavior of he process (which can be sudied hrough renewal heory) from he behavior wihin each renewal period. Example (Visis o a given sae for a Markov chain). Assume ha a recurren finie-sae Markov chain wih ransiion marix [P ] sars in sae i a ime. Then on he firs reurn o sae i, say a ime n, he Markov chain, from ime n on, is a probabilisic replica of he chain saring a ime. Tha is, he sae a ime 1 is j wih probabiliy P ij, and, given a reurn o i a ime n, he probabiliy of sae j a ime n + 1 is P ij. In he same way, for any m >, Pr{X 1 = j,..., X m = k X = i} = Pr{X n+1 = j,..., X n+m = k X n = i}. (5.2) Each subsequen reurn o sae i a a given ime n sars a new probabilisic replica of he Markov chain saring in sae i a ime. Thus he sequence of enry imes o sae i can be viewed as he arrival epochs of a renewal process. This example is imporan, and will form he key o he analysis of Markov chains wih a counably infinie se of saes in Chaper 6. A he same ime, (5.2) does no quie jusify viewing successive reurns o sae i as a renewal process. The problem is ha he ime of he firs enry o sae i afer ime is a random variable raher han he given ime n indicaed in (5.2). This will no be a major problem o sor ou, bu he resoluion will be more insighful afer developing some basic properies of renewal processes. Example (The G/G/m queue:). The cusomer arrivals o a G/G/m queue form a renewal couning process, {N(); > }. Each arriving cusomer eners one of he m servers if he server is no busy and oherwise wais in a common queue wih firs-come firs-serve service unil a server becomes free. The service ime required by each cusomer is a rv, IID over cusomers, and independen of arrival imes and servers. The sysem is assumed o be empy for <, and an arrival, viewed as cusomer number, is assumed a ime. The subsequen inerarrival inervals X 1, X 2,..., are IID. Noe ha N() for each > is he number of arrivals in (, ], so arrival number a = is no couned in N(). 2 2 There is always a cerain amoun of awkwardness in saring a renewal process, and he assumpion of

3 5.1. INTRODUCTION 225 We define a new couning process, {N r (); > }, for which he renewal epochs are hose arrival epochs in he original arrival process {N(); > } a which an arriving cusomer sees an empy sysem (i.e., no cusomer in queue and none in service). 3 We will show in Secion ha {N r (); > } is acually a renewal process, bu give an inuiive explanaion here. Noe ha cusomer arrives a ime o an empy sysem, and given a firs subsequen arrival o an empy sysem, a say epoch S1 r >, he subsequen cusomer inerarrival inervals are independen of he arrivals in (, S1 r ) and are idenically disribued o hose earlier arrivals. The service imes afer S1 r are also IID from hose earlier. Finally, he number of cusomers in he queue is he same funcion of iner-arrival inervals and service compleions for he sysem saring a ime as for any subsequen arrival o an empy sysem. In mos siuaions, we use he words arrivals and renewals inerchangeably, bu for his ype of example, he word arrival is used for he couning process {N(); > } of he acual arrivals o he queueing sysem and he word renewal is used for {N r (); > }. The reason for being ineresed in {N r (); > } is ha i allows us o analyze very complicaed queues such as his in wo sages. Firs, {N(); > } les us analyze he disribuion of he inerrenewal inervals X r n of {N r (); > }. Second, he general renewal resuls developed in his chaper can be applied o he disribuion on X r n o undersand he overall behavior of he queueing sysem. Throughou our sudy of renewal processes, we use X and E [X] inerchangeably o denoe he mean iner-renewal inerval, and use 2 X or simply 2 o denoe he variance of he iner-renewal inerval. We will usually assume ha X is finie, bu, excep where explicily saed, we need no assume ha 2 is finie. This means, firs, ha 2 need no be calculaed (which is ofen di cul if renewals are embedded ino a more complex process), and second, since modeling errors on he far ails of he iner-renewal disribuion ypically a ec 2 more han X, he resuls are relaively robus o hese kinds of modeling errors. Much of his chaper will be devoed o undersanding he behavior of N() and N()/ as becomes large. As migh appear o be inuiively obvious, and as is proven in Exercise 5.1, N() is a rv (i.e., no defecive) for each >. Also, as proven in Exercise 5.2, E [N()] is finie for all >. I is hen also clear ha N()/, which is inerpreed as he ime-average renewal rae over (,], is also a rv wih finie expecaion. One of he major resuls abou renewal heory, which we esablish shorly, concerns he behavior of he rv s N()/ as! 1. For each sample poin! 2 of he over-all process, N(,!)/ is a nonnegaive number for each, so ha, for given!, we can view {N(,!)/; > } as a funcion of ime. Thus lim!1 N(,!)/, if i exiss, is he imeaverage renewal rae over (, 1) for he sample poin!. The srong law for renewal processes saes ha his limiing ime-average renewal rae exiss for a se of! ha has probabiliy 1, and, over ha se, his limiing value is 1/X. an arrival a ime which is no couned in N() seems srange, bu simplifies he noaion. The process is defined in erms of he IID iner-renewal inervals X 1, X 2,.... The firs renewal epoch is a S 1 = X 1, and his is he poin a which N() changes from o 1. 3 Readers who accep wihou quesion ha {N r () > } is a renewal process should be proud of heir probabilisic inuiion, bu should also quesion exacly how such a conclusion can be proven.

4 226 CHAPTER 5. RENEWAL PROCESSES We shall ofen refer o his resul by he less precise saemen ha he ime-average renewal rae is 1/ X. This resul is a direc consequence of he srong law of large numbers (SLLN) for IID rv s. In he nex secion, we firs sae and prove he SLLN for IID rv s and hen esablish he srong law for renewal processes. There is a cerain inuiive appeal o he idea of aking a limi over a sample pah as opposed o showing, for example, ha he random variable N()/ has a variance ha approaches as! 1, bu he reader will need some paience o undersand boh how o deal wih limis over individual sample pahs and he reason for waning o deal wih such quaniies. Anoher imporan heoreical resul in his chaper is he elemenary renewal heorem, which saes ha E [N()/] also approaches 1/X as! 1. Surprisingly, his is more han a rival consequence of he srong law for renewal processes, and we shall develop several widely useful resuls such as Wald s equaliy, in esablishing his heorem. The final major heoreical resul of he chaper is Blackwell s heorem, which shows ha, for appropriae values of, he expeced number of renewals in an inerval (, + ] approaches /X as! 1. We shall hus inerpre 1/X as an ensemble-average renewal rae. This rae is he same as he above ime-average renewal rae. We shall see he benefis of being able o work wih boh ime averages and ensemble averages. There are a wide range of oher resuls, ranging from sandard queueing resuls o resuls ha are needed in all subsequen chapers. 5.2 The srong law of large numbers and convergence WP1 The concep of a sequence of rv s converging wih probabiliy 1 (WP1) was inroduced briefly in Secion We discuss his ype of convergence more fully here and esablish some condiions under which i holds. Nex he srong law of large numbers (SLLN) is saed for IID rv s (his is essenially he resul ha he parial sample averages of IID rv s converge o he mean WP1). A proof is given under he added condiion ha he rv s have a finie fourh momen. Finally, in he following secion, we sae he srong law for renewal processes and use he SLLN for IID rv s o prove i Convergence wih probabiliy 1 (WP1) Recall ha a sequence {Z n ; n 1} of rv s on a sample space is defined o converge WP1 o a rv Z on if n o Pr! 2 : lim Z n(!) = Z(!) = 1, n!1 i.e., if he se of sample sequences {Z n (!); n 1} ha converge o Z(!) is an even and has probabiliy 1. This becomes slighly easier o undersand if we define Y n = Z n Z for each n. The sequence {Y n ; n 1} hen converges o WP1 if and only if he sequence {Z n ; n 1} converges o Z WP1. Dealing only wih convergence o raher han o an arbirary rv doesn cu any seps from he following proofs, bu i simplifies he noaion and he conceps a no cos o generaliy.

5 5.2. THE STRONG LAW OF LARGE NUMBERS AND CONVERGENCE WP1 227 We sar wih a simple lemma ha provides a useful condiion under which convergence o WP1 occurs. We shall see laer how o use his lemma in an indirec way o prove he SLLN. Lemma Le {Y n ; n 1} be a sequence of rv s, each wih finie expecaion. If P 1 n=1 E [ Y n ] < 1, hen Pr{! : lim n!1 Y n (!) = } = 1. Discussion: Noe ha he Y n are no IID and usually no independen. If one wans a simple example of such rv s, hink of Y n = (X X n )/n where he X n are IID. Noe also ha he condiion P 1 n=1 E [ Y n ] < 1 implies ha lim n!1 E [ Y n ] =, bu also implies ha he convergence of E [ Y n ] is fas enough o make he sum converge. Proof: For any > and any ineger m 1, he Markov inequaliy says ha ( m ) X Pr Y n > apple E [P m n=1 Y P m n ] n=1 = E [ Y n ]. (5.3) n=1 Since Y n is nonnegaive, P m n=1 Y n > implies ha P m+1 n=1 Y n >. Thus he lef side of (5.3) is non-decreasing in m and is upper bounded by he limi of he righ side. Thus boh sides have a limi, and ( m ) lim Pr X P 1 n=1 Y n > apple E [ Y n ]. (5.4) m!1 n=1 We nex show ha he limi on he lef side of (5.4) can be brough inside he probabiliy. Le A m = {! : P m n=1 Y n(!) > }. As seen above, he sequence {A m ; m 1} is nesed, A 1 A 2, so from propery (1.9) of he axioms of probabiliy, ( m ) lim Pr X n[ 1 o Y n > = Pr A m m!1 m=1 n=1 = Pr n! : X 1 o Y n(!) >, (5.5) n=1 where he second equaliy uses he fac ha for any given!, P 1 n=1 Y n(!) > if and only if P m n=1 Y n(!) > for some m 1. Combining (5.4) wih (5.5), ( ) 1X P 1 n=1 Pr! : Y n (!) > apple E [ Y n ]. n=1 Looking a he complemenary se and assuming > P 1 n=1 E [ Y n ], ( ) 1X P 1 n=1 Pr! : Y n (!) apple 1 E [ Y n ]. (5.6) n=1 For any! such ha P 1 n=1 Y n(!) apple, we see ha { Y n (!) ; n 1} is simply a sequence of nonnegaive numbers wih a finie sum. Thus he individual numbers in ha sequence mus approach, i.e., lim n!1 Y n (!) = for each such!. I follows hen ha n o Pr! : lim Y n(!) = n!1 Pr (! : ) 1X Y n (!) apple. n=1

6 228 CHAPTER 5. RENEWAL PROCESSES Combining his wih (5.6), n o Pr! : lim Y n(!) = n!1 1 P 1 n=1 E [ Y n ]. This is rue for all, so Pr{! : lim n!1 Y n (!) = } = 1. The same limi holds for Y n (!) in place of Y n (!), compleing he proof. Noe ha he inerchange of limis in (5.5) also shows ha {! : P 1 n=1 Y n(!) > } is an even, which is by no means obvious for a counably infinie sum of rv s. I is insrucive o recall Example 1.7.1, illusraed in Figure 5.2, where {Y n ; n 1} converges in probabiliy bu does no converge wih probabiliy one. Noe ha E [Y n ] = 1/(5 j+1 5 j ) for n 2 [5 j, 5 j+1 ). Thus lim n!1 E [Y n ] =, bu P 1 n=1 E [Y n] = 1. Thus his sequence does no saisfy he condiions of he lemma. This helps explain how he condiions in he lemma exclude such sequences. 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Figure 5.2: Illusraion of a sample pah of a sequence of rv s {Y n ; n } where, for each j, Y n = 1 for an equiprobable choice of n 2 [5 j, 5 j+1 ) and Y n = oherwise. Before proceeding o he SLLN, we wan o show ha convergence WP1 implies convergence in probabiliy. We give an incomplee argumen here wih precise versions boh in Exercise 5.5 and Exercise 5.7. Exercise 5.7 has he added meri of expressing he se {! : lim n Y n (!) = } explicily in erms of counable unions and inersecions of simple evens involving finie ses of he Y n. This represenaion is valid wheher or no he condiions of he lemma are saisfied and shows ha his se is indeed an even. Assume ha {Y n ; n 1} is a sequence of rv s such ha lim n!1 (Y n ) = WP1. Then for any >, each sample sequence {Y n (!); n 1} ha converges o saisfies Y n apple for all su cienly large n. This means (see Exercise 5.5) ha lim n!1 Pr{ Y n apple } = 1. Since his is rue for all >, {Y n ; n } converges in probabiliy o Srong law of large numbers (SLLN) We nex develop he srong law of large numbers. We do no have he mahemaical ools o prove he heorem in is full generaliy, bu will give a fairly insighful proof under he addiional assumpion ha he rv under discussion has a finie 4h momen. The heorem has a remarkably simple and elemenary form, considering ha i is cerainly one of he mos imporan heorems in probabiliy heory. Mos of he hard work in undersanding he heorem comes from undersanding wha convergence WP1 means, and ha has already been discussed. Given his undersanding, he heorem is relaively easy o undersand and surprisingly easy o prove (assuming a 4h momen).

7 5.2. THE STRONG LAW OF LARGE NUMBERS AND CONVERGENCE WP1 229 Theorem (Srong Law of Large Numbers (SLLN )). For each ineger n 1, le S n = X X n, where X 1, X 2,... are IID rv s saisfying E [ X ] < 1. Then S Pr! : n (!) lim = X = 1. (5.7) n!1 n Proof (for he case where X = and E X 4 < 1): Assume ha X = and E X 4 < 1. Denoe E X 4 by. Le x be a real number. If x apple 1, hen x 2 apple 1, and if x > 1, hen x 2 < x 4. Thus x 2 apple 1 + x 4 for all x. I follows ha 2 = E X 2 apple 1 + E X 4. Thus 2 is finie if E X 4 is. Now le S n = X X n where X 1,..., X n are IID wih he disribuion of X. E Sn 4 = E [(X X n )(X X n )(X X n )(X X n )] 2! 1! nx nx nx nx X`!3 = E 4 X A X k 5 i=1 j=1 X j k=1 `=1 = nx nx nx nx E [X i X j X k X`], i=1 j=1 k=1 `=1 where we have muliplied ou he produc of sums o ge a sum of n 4 erms. For each i, 1 apple i apple n, here is a erm in his sum wih i = j = k = `. For each such erm, E [X i X j X k X`] = E X 4 =. There are n such erms (one for each choice of i, 1 apple i apple n) and hey collecively conribue n o he sum E Sn 4. Also, for each i, k 6= i, here is a erm wih j = i and ` = k. For each of hese n(n 1) erms, E [X i X i X k X k ] = 4. There are anoher n(n 1) erms wih j 6= i and k = i, ` = j. Each such erm conribues 4 o he sum. Finally, for each i 6= j, here is a erm wih ` = i and k = j. Collecively all of hese erms conribue 3n(n 1) 4 o he sum. Each of he remaining erms is since a leas one of i, j, k, ` is di eren from all he ohers, Thus we have E S 4 n = n + 3n(n 1) 4. Now consider he sequence of rv s {S 4 n/n 4 ; n 1}. 1X n=1 apple S 4 E n n 4 = 1X n=1 n + 3n(n 1) 4 n 4 < 1, where we have used he facs ha he series P n 1 1/n2 and he series P n 1 1/n3 converge. Using Lemma applied o {Sn/n 4 4 ; n 1}, we see ha lim n!1 Sn/n 4 4 = WP1. For each! such ha lim n!1 Sn(!)/n 4 4 =, he nonnegaive fourh roo of ha sequence of nonnegaive numbers also approaches. Thus lim n!1 S n /n = WP1. The above proof assumed ha E [X] =. I can be exended rivially o he case of an arbirary finie X by replacing X in he proof wih X X. A proof using he weaker condiion ha 2 X < 1 will be given in Secion

8 23 CHAPTER 5. RENEWAL PROCESSES The echnique ha was used a he end of his proof provides a clue abou why he concep of convergence WP1 is so powerful. The echnique showed ha if one sequence of rv s ({S 4 n/n 4 ; n 1}) converges o WP1, hen anoher sequence ( S n /n ; n 1}) also converges WP1. We will formalize and generalize his echnique in Lemma as a major sep oward esablishing he srong law for renewal processes. The SLLN appears o be fairly inuiive. We sar wih a sequence of IID rv s {X i ; i 1}, nex form a sequence of sample averages, {Y n ; n 1} where Y n = (X 1 + X n )/n, and finally look a a sample sequence {y n ; n 1} of he sample averages. Each sample sequence {y n ; n 1} is a sequence of real numbers and we would expec ha sequence o converge o X. The SLLN says ha his convergence akes place WP1. The inuiive naure of he SLLN fades slighly if we look a i in he following way: consider a Bernoulli process {X i ; i 1} wih p = p X (1). Then {! : lim n Y n (!) = p} is an even in he sample space wih probabiliy 1. Consider a di eren Bernoulli process {Xn; n 1}, wih probabiliy of success p 6= p. The se of sample poins and he se of evens for his process is he same as before. The se {! : lim n Yn(!) = p} is sill he same even as before, bu i has probabiliy under his new probabiliy measure. The parameer p of a Bernoulli process can ake on an uncounably infinie number of values in he inerval (, 1). Thus here are uncounably many evens {! : Y n (!) = p}, one for each value of p. Each such even has probabiliy 1 for he Bernoulli process wih ha value of p and probabiliy for each oher value of p. There is no mahemaical conradicion here, bu we should realize ha inuiion abou hese evens requires more subley han migh be imagined iniially. 5.3 Srong law for renewal processes To ge an inuiive idea why N()/ should approach 1/X for large, consider Figure 5.3. For any given sample funcion of {N(); > }, noe ha, for any given, N()/ is he slope of a sraigh line from he origin o he poin (, N()). As increases, his slope decreases in he inerval beween each adjacen pair of arrival epochs and hen jumps up a he nex arrival epoch. In order o express his as an equaion for arbirary >, le he rv S N() denoe he arrival epoch of he N()h arrival and le S N()+1 denoe he arrival epoch of he firs arrival following (see Figure 5.3). If N() =, we ake S N() = by convenion. Thus we have S N() apple < S N()+1. For all >, hen N() S N() N() > N() S N()+1. (5.8) We wan o show inuiively why he slope N()/ in he figure approaches 1/X as! 1. As increases, we would guess ha N() increases wihou bound, i.e., ha for each arrival, anoher arrival occurs evenually. Assuming his, he lef side of (5.8) increases wih increasing as 1/S 1, 2/S 2,..., n/s n,..., where n = N(). Since S n /n converges o X WP1 from he srong law of large numbers, we migh be brave enough or insighful enough o guess ha n/s n converges o 1/X.

9 5.3. STRONG LAW FOR RENEWAL PROCESSES 231 Slope = N() N() S N() N() Slope = N() S N()+1 A AAU S 1 S N() S N()+1 Figure 5.3: Comparison of a sample funcion of N()/ wih N() S N() and N() S N()+1 for he same sample poin. Noe ha for he given sample poin, N() is he number of arrivals up o and including, and hus S N() is he epoch of he las arrival before or a ime. Similarly, S N()+1 is he epoch of he firs arrival sricly afer ime. We are now ready o sae he srong law for renewal processes as a heorem. Before proving he heorem, we formulae he above wo guesses as lemmas and prove heir validiy. Theorem (Srong Law for Renewal Processes). For a renewal process wih mean iner-renewal inerval X < 1, lim!1 N()/ = 1/X WP1. Lemma Le {N(); > } be a renewal couning process wih iner-renewal rv s {X n ; n 1}. Then (wheher or no X < 1), lim!1 N() = 1 WP1 and lim!1 E [N()] = 1. Proof of Lemma 5.3.1: Noe ha for each sample poin!, N(,!) is a non-decreasing real-valued funcion of and hus eiher has a finie limi or an infinie limi. The firs par of he lemma, i.e., lim!1 N() = 1 WP1, will be proven by showing ha he se of! for which his limi is finie has probabiliy. Pr n o! : lim N(,!) < 1!1 = Pr n! : [ n 1 lim!1 N(,!) < n o apple X n Pr! : lim N(,!) < n!1 n 1 o. (5.9) I is su cien o show ha each erm in his sum is. Wriing Pr{! : lim!1 N(,!) < n} as Pr{lim!1 N() < n}, we now show ha Pr{lim!1 N() < n} = for each n 1. Using (5.1), lim Pr{N() < n} = lim Pr{S n > } = 1 lim Pr{S n apple }.!1!1!1 Since he X i are rv s, he sums S n are also rv s (i.e., nondefecive) for each n (see Secion 1.5.1), and hus lim!1 Pr{S n apple } = 1 for each n. Thus lim!1 Pr{N() < n} = for each n, and from (5.9), lim!1 N() = 1 WP1. Nex, E [N()] is non-decreasing in, and hus has eiher a finie or infinie limi as! 1. For each n, Pr{N() n} 1/2 for large enough, and herefore E [N()] n/2 for such. Thus E [N()] can have no finie limi as! 1, and lim!1 E [N()] = 1.

10 232 CHAPTER 5. RENEWAL PROCESSES The following lemma is quie a bi more general han he second guess above, bu i will be useful elsewhere. This is he formalizaion of he echnique used a he end of he proof of he SLLN. Lemma Le {Z n ; n 1} be a sequence of rv s such ha lim n!1 Z n = WP1. Le f be a real valued funcion of a real variable ha is coninuous a. Then lim f(z n) = f( ) WP1. (5.1) n!1 Proof of Lemma 5.3.2: Firs le z 1, z 2,..., be a sequence of real numbers such ha lim n!1 z n =. Coninuiy of f a means ha for every >, here is a > such ha f(z) f( ) < for all z such ha z <. Also, since lim n!1 z n =, we know ha for every >, here is an m such ha z n apple for all n m. Puing hese wo saemens ogeher, we know ha for every >, here is an m such ha f(z n ) f( ) < for all n m. Thus lim n!1 f(z n ) = f( ). If! is any sample poin such ha lim n!1 Z n (!) =, hen lim n!1 f(z n (!)) = f( ). Since his se of sample poins has probabiliy 1, (5.1) follows. Proof of Theorem 5.3.1, Srong law for renewal processes: Since Pr{X > } = 1 for a renewal process, we see ha X >. Choosing f(x) = 1/x, we see ha f(x) is coninuous a x = X. I follows from Lemma ha lim n!1 n S n = 1 X WP1. From Lemma 5.3.1, we know ha lim!1 N() = 1 wih probabiliy 1, so, wih probabiliy 1, N() increases hrough all he nonnegaive inegers as increases from o 1. Thus N() n lim = lim = 1!1 S N() n!1 S n X WP1. Recall ha N()/ is sandwiched beween N()/S N() and N()/S N()+1, so we can complee he proof by showing ha lim!1 N()/S N()+1 = 1/X. To show his, lim!1 N() S N()+1 = lim n!1 n n+1 = lim S n+1 n!1 S n+1 n n+1 = 1 X WP1. We have gone hrough he proof of his heorem in grea deail, since a number of he echniques are probably unfamiliar o many readers. If one reads he proof again, afer becoming familiar wih he deails, he simpliciy of he resul will be quie sriking. The heorem is also rue if he mean iner-renewal inerval is infinie; his can be seen by a runcaion argumen (see Exercise 5.9). As explained in Secion 5.2.1, Theorem also implies he corresponding weak law of large numbers for N(), i.e., for any >, lim!1 Pr N()/ 1/X = ). This weak law could also be derived from he weak law of large numbers for S n (Theorem 1.7.4).

11 5.3. STRONG LAW FOR RENEWAL PROCESSES 233 We do no pursue ha here, since he derivaion is edious and uninsrucive. As we will see, i is he srong law ha is mos useful for renewal processes. Figure 5.4 helps give some appreciaion of wha he srong law for N() says and doesn say. The srong law deals wih ime averages, lim!1 N(,!)/, for individual sample poins!; hese are indicaed in he figure as horizonal averages, one for each!. I is also of ineres o look a ime and ensemble averages, E [N()/], shown in he figure as verical averages. Noe ha N(,!)/ is he ime-average number of renewals from o for a given!, whereas E [N()/] averages also over he ensemble. Finally, o focus on arrivals in he viciniy of a paricular ime, i is of ineres o look a he ensemble average E [N( + ) N()] /. Given he srong law for N(), one would hypohesize ha E [N()/] approaches 1/X as! 1. One migh also hypohesize ha lim!1 E [N( + ) N()] / = 1/X, subjec o some minor resricions on. These hypoheses are correc and are discussed in deail in wha follows. This equaliy of ime averages and limiing ensemble averages for renewal processes carries over o a large number of sochasic processes, and forms he basis of ergodic heory. These resuls are imporan for boh heoreical and pracical purposes. I is someimes easy o find ime averages (jus like i was easy o find he ime-average N(,!)/ from he srong law of large numbers), and i is someimes easy o find limiing ensemble averages. Being able o equae he wo hen allows us o alernae a will beween ime and ensemble averages. N(,! 1 ) Time and ensemble Average over (, ) Ensemble Average a (1/ )E [N( + ) N()] - Time Ave. a! 1 N(,! 2 ) - Time Ave. a! 2 N(,! 3 ) - Time Ave. a! 3 Figure 5.4: The ime average a a sample poin!, he ime and ensemble average from o a given, and he ensemble average in an inerval (, + ]. Noe ha in order o equae ime averages and limiing ensemble averages, quie a few condiions are required. Firs, he ime average mus exis in he limi! 1 wih probabiliy one; ha ime average mus also be he same for all sample poins in a se of probabiliy one. Second, he ensemble average mus approach a limi as! 1. Third, he ime average and ensemble average mus be he same. The following example, for a sochasic process very di eren from a renewal process, shows ha equaliy beween ime and ensemble averages is no always saisfied for arbirary processes. Example Le {X i ; i 1} be a sequence of binary IID random variables, each aking he value wih probabiliy 1/2 and 2 wih probabiliy 1/2. Le {M n ; n 1} be he produc

12 234 CHAPTER 5. RENEWAL PROCESSES process in which M n = X 1 X 2 X n. Since M n = 2 n if X 1 o X n each ake he value 2 (an even of probabiliy 2 n ) and M n = oherwise, we see ha lim n!1 M n = wih probabiliy 1. Also E [M n ] = 1 for all n 1. Thus he ime average exiss and equals wih probabiliy 1 and he ensemble average exiss and equals 1 for all n, bu he wo are di eren. The problem is ha as n increases, he aypical even in which M n = 2 n has a probabiliy approaching, bu sill has a significan e ec on he ensemble average. Furher discussion of ensemble averages is posponed o Secion 5.6. Before ha, we briefly sae and discuss he cenral limi heorem for couning renewal processes and hen inroduce he noion of rewards associaed wih renewal processes. Theorem (Cenral Limi Theorem (CLT) for N()). Assume ha he inerrenewal inervals for a renewal couning process {N(); > } have finie sandard deviaion >. Then ( ) lim Pr N() /X!1 X 3/2 p < = ( ). (5.11) where (y) = R y 1 1 p 2 exp( x 2 /2)dx. This says ha he CDF of N() ends o he Gaussian disribuion wih mean /X and sandard deviaion X 3/2 p. n -6? p n p n X Slope = 1 X nx E [S n ] Figure 5.5: Illusraion of he cenral limi heorem (CLT) for renewal processes. A given ineger n is shown on he verical axis, and he corresponding mean, E [S n ] = nx is shown on he horizonal axis. The horizonal line wih arrows a heigh n indicaes sandard deviaions from E [S n ], and he verical line wih arrows indicaes he disance below (/X). The heorem can be proved by applying Theorem (he CLT for a sum of IID rv s) o S n and hen using he ideniy {S n apple } = {N() n}. The general idea is illusraed in Figure 5.5, bu he deails are somewha edious, and can be found, for example, in [22]. We simply ouline he argumen here. For any real, he CLT saes ha where ( ) = R 1 limi n! 1. Leing 1 p 2 exp( Pr S n apple nx + p n ( ), x 2 /2) dx and where he approximaion becomes exac in he = nx + p n,

13 5.4. RENEWAL-REWARD PROCESSES; TIME AVERAGES 235 and using {S n apple } = {N() n}, Pr{N() n} ( ). (5.12) Since is monoonic in n for fixed, we can express n in erms of, geing n = X p n X X 1/2 (X) 3/2. Subsiuing his ino (5.12) esablishes he heorem for, which esablishes he heorem since is arbirary. The omied deails involve handling he approximaions carefully. 5.4 Renewal-reward processes; ime averages There are many siuaions in which, along wih a renewal couning process {N(); > }, here is anoher randomly-varying funcion of ime, called a reward funcion {R(); > }. R() models a rae a which he process is accumulaing a reward. We shall illusrae many examples of such processes and see ha a reward could also be a cos or any randomly varying quaniy of ineres. The imporan resricion on hese reward funcions is ha R() a a given depends only on he locaion of wihin he iner-renewal inerval conaining and perhaps oher random variables local o ha inerval. Before defining his precisely, we sar wih several examples. These examples also illusrae ha reward funcions can be defined o sudy fundamenal quesions concerning he renewal process iself. The reamen here considers ime averages for hese quesions, whereas Secion 5.7 views he corresponding ensemble averages.. Example (Time-average residual life) For a renewal couning process {N(), > }, le Y () be he residual life a ime. The residual life is defined as he inerval from unil he nex renewal epoch, i.e., as S N()+1. For example, if we arrive a a bus sop a ime and buses arrive according o a renewal process, Y () is he ime we have o wai for a bus o arrive (see Figure 5.6). We inerpre {Y (); } as a reward funcion. The ime average of Y (), over he inerval (, ], is given by 4 (1/) R Y ( )d. We are ineresed in he limi of his average as! 1 (assuming ha i exiss in some sense). Figure 5.6 illusraes a sample funcion of a renewal couning process {N(); > } and shows he residual life Y () for ha sample funcion. Noe ha, for a given sample funcion {Y () = y()}, he inegral R y( ) d is simply a sum of isosceles righ riangles, wih par of a final riangle a he end. Thus i can be expressed as Z y( )d = 1 2 n() X i=1 Z x 2 i + y( )d, =s n() where {x i ; < i < 1} is he se of sample values for he iner-renewal inervals. R 4 Y ( )d is a rv jus like any oher funcion of a se of rv s. I has a sample value for each sample funcion of {N(); > }, and is CDF could be calculaed in a sraighforward bu edious way. For arbirary sochasic processes, inegraion and di ereniaion can require grea mahemaical sophisicaion, bu none of hose subleies occur here.

14 236 CHAPTER 5. RENEWAL PROCESSES X 1 N() X 2 - S 1 S 2 S 3 S 4 S 5 S 6 X 2 Y X @@ S 1 S 2 S 3 S 4 S 5 S 6 X 5 Figure 5.6: Residual life a ime. For any given sample funcion of he renewal process, he sample funcion of residual life decreases linearly wih a slope of 1 from he beginning o he end of each iner-renewal inerval. Since his relaionship holds for every sample poin, we see ha he random variable R Y ( )d can be expressed in erms of he iner-renewal random variables X n as Z = Y ( )d = 1 2 N() X n=1 Z Xn 2 + Y ( )d. =S N() Alhough he final erm above can be easily evaluaed for a given S N() (), i is more convenien o use he following bound: N() 1 X Xn 2 apple 1 2 n=1 Z = Y ( )d apple 1 2 N()+1 X n=1 X 2 n. (5.13) The erm on he lef can now be evaluaed in he limi! 1 (for all sample funcions excep a se of probabiliy zero) as follows: lim!1 P N() n=1 X2 n 2 = lim!1 P N() n=1 X2 n N() N() 2. (5.14) Consider each erm on he righ side of (5.14) separaely. For he firs erm, recall ha lim! N() = 1 wih probabiliy 1. Thus as! 1, P N() n=1 X2 n/n() goes hrough he same se of values as P k n=1 X2 n/k as k! 1. Thus, assuming ha E X 2 < 1, we can apply he SLLN o {Xn; 2 n 1}, lim!1 P N() n=1 X2 n N() P k n=1 = lim X2 n k!1 k = E X 2 WP1. The second erm on he righ side of (5.14) is simply N()/2. Since X < 1, he srong law for renewal processes says ha lim!1 N()/2 = 1/(2E [X]) WP1. Thus boh limis

15 5.4. RENEWAL-REWARD PROCESSES; TIME AVERAGES 237 exis WP1 and lim!1 P N() n=1 X2 n 2 = E X 2 2E [X] WP1. (5.15) The righ hand erm of (5.13) is handled almos he same way: lim!1 P N()+1 n=1 X 2 n 2 = lim!1 P N()+1 n=1 X 2 n N() + 1 N() + 1 N() N() 2 = E X 2 2E [X]. (5.16) Combining hese wo resuls, we see ha, wih probabiliy 1, he ime-average residual life is given by lim!1 R = Y ( ) d = E X 2 2E [X]. (5.17) Noe ha his ime average depends on he second momen of X; his is X X 2, so he ime-average residual life is a leas half he expeced iner-renewal inerval (which is no surprising). On he oher hand, he second momen of X can be arbirarily large (even infinie) for any given value of E [X], so ha he ime-average residual life can be arbirarily large relaive o E [X]. This can be explained inuiively by observing ha large inerrenewal inervals are weighed more heavily in his ime average han small iner-renewal inervals. Example As an example of he e ec of improbable bu large iner-renewal inervals, le X ake on he value wih probabiliy 1 and value 1/ wih probabiliy. Then, for small, E [X] 1, E X 2 1/, and he ime average residual life is approximaely 1/(2 ) (see Figure 1/ - Figure 5.7: Average Residual life is dominaed by large inerarrival inervals. Each large inerval has duraion 1/, and he expeced aggregae duraion beween successive large inervals is 1 Example (ime-average Age) Le Z() be he age of a renewal process a ime where age is defined as he inerval from he mos recen arrival before (or a) unil, i.e., Z() = S N(). By convenion, if no arrivals have occurred by ime, we ake he age o be (i.e., in his case, N() = and we ake S o be ).

16 238 CHAPTER 5. RENEWAL PROCESSES As seen in Figure 5.18, he age process, for a given sample funcion of he renewal process, is almos he same as he residual life process he isosceles righ riangles are simply urned around. Thus he same analysis as before can be used o show ha he ime average of Z() is he same as he ime average of he residual life, lim!1 R = Z( ) d = E X 2 2E [X] WP1. (5.18) Z() S 1 S 2 S 3 S 4 S 5 S 6 Figure 5.8: Age a ime : For any given sample funcion of he renewal process, he sample funcion of age increases linearly wih a slope of 1 from he beginning o he end of each iner-renewal inerval. Example (ime-average Duraion) Le X() e be he duraion of he iner-renewal inerval conaining ime, i.e., X() e = XN()+1 = S N()+1 S N() (see Figure 5.9). I is clear ha X() e = Z() + Y (), and hus he ime average of he duraion is given by lim!1 R = e X( ) d = E X 2 E [X] WP1. (5.19) Again, long inervals are heavily weighed in his average, so ha he ime-average duraion is a leas as large as he mean iner-renewal inerval and ofen much larger. ex() X 5 - S 1 S 2 S 3 S 4 S 5 S 6 Figure 5.9: Duraion e X() = X N() of he iner-renewal inerval conaining General renewal-reward processes In each of hese examples, and in many oher siuaions, we have a random funcion of ime (i.e., Y (), Z(), or e X()) whose value a ime depends only on where is in he curren iner-renewal inerval (i.e., on he age Z() and he duraion e X() of he curren

17 5.4. RENEWAL-REWARD PROCESSES; TIME AVERAGES 239 iner-renewal inerval). We now invesigae he general class of reward funcions for which he reward a ime depends a mos on he age and he duraion a, i.e., he reward R() a ime is given explicily as a funcion R(Z(), X()) e of he age and duraion a. For he hree examples above, he funcion R is rivial. Tha is, he residual life, Y (), is given by R(Z(), X()) e = X() e Z(). Similarly, he age is given direcly as R(Z(), X()) e = Z(). This illusraes ha he general funcion R(Z(), X()) e need no involve boh Z() and ex(); i simply can involve anyhing else. 1 We now find he ime-average value of R(), namely, lim!1 R( ) d. As in examples o above, we firs wan o look a he accumulaed reward over each iner-renewal period separaely. Define R n as he accumulaed reward in he nh renewal inerval, R n = Z Sn S n 1 R( ) d( ) = Z Sn R S n 1 R[Z( ), e X( )] d. (5.2) For residual life (see Example 5.4.1), R n is he area of he nh isosceles righ riangle in Figure 5.6. In general, for in (S n 1, S n ], we have Z( ) = S n 1 and X( ) e = S n S n 1 = X n. I follows ha R n is given by R n = Z Sn S n 1 R( S n 1, X n ) d = Z Xn z= R(z, X n ) dz. (5.21) Noe ha z is inegraed ou in his expression, so R n is a funcion only of X n alhough, of course, he form of ha funcion is deermined by he funcion R(z, x). Assuming ha his inegral exiss for all sample values of X n and ha he resuling R n is a rv, i is clear ha {R n ; n 1} is a sequence of IID random variables. For residual life, R(z, X n ) = X n z, so he inegral in (5.21) is Xn/2, 2 as calculaed by inspecion before. In general, from (5.21), he expeced value of R n is given by Z 1 Z x E [R n ] = R(z, x) dz df X (x). (5.22) x= z= Breaking R R( ) d ino he reward over successive renewal periods, we ge Z Z S1 Z S2 Z SN() Z R( ) d = R( ) d + R( ) d + + R( ) d + R( )d S 1 S N() 1 S N() = N() X n=1 Z R n + R( ) d. (5.23) S N() The following heorem now generalizes he resuls of Examples 5.4.1, 5.4.3, and o general renewal-reward funcions. Theorem Le {R(); > } be a nonnegaive renewal-reward funcion for a renewal process wih expeced iner-renewal ime E [X] = X < 1. If each R n is a rv wih E [R n ] < 1, hen wih probabiliy 1, lim!1 1 Z = R( ) d = E [R n] X. (5.24)

18 24 CHAPTER 5. RENEWAL PROCESSES Proof: Using (5.23), he accumulaed reward up o ime can be bounded beween he accumulaed reward up o he renewal before and ha o he nex renewal afer, P N() n=1 R n apple R = R( ) d The lef hand side of (5.25) can be separaed ino apple P N()+1 n=1 R n. (5.25) P N() n=1 R n = P N() n=1 R n N() N(). (5.26) Each R n is a given funcion of X n, so he R n are IID. As! 1, N()! 1, and, hus, as we have seen before, he srong law of large numbers can be used on he firs erm on he righ side of (5.26), geing E [R n ] wih probabiliy 1. Also he second erm approaches 1/X by he srong law for renewal processes. Since < X < 1 and E [R n ] is finie, he produc of he wo erms approaches he limi E [R n ] /X. The righ-hand inequaliy of (5.25) is handled in almos he same way, P N()+1 n=1 R n = P N()+1 n=1 R n N() + 1 N() + 1 N() N(). (5.27) I is seen ha he erms on he righ side of (5.27) approach limis as before and hus he erm on he lef approaches E [R n ] /X wih probabiliy 1. Since he upper and lower bound in (5.25) approach he same limi, (1/) R R( ) d approaches he same limi and he heorem is proved. The resricion o nonnegaive renewal-reward funcions in Theorem is slighly arificial. The same resul holds for non-posiive reward funcions simply by changing he direcions of he inequaliies in (5.25). Assuming ha E [R n ] exiss (i.e., ha boh is posiive and negaive pars are finie), he same resul applies in general by spliing an arbirary reward funcion ino a posiive and negaive par. This gives us he corollary: Corollary Le {R(); > } be a renewal-reward funcion for a renewal process wih expeced iner-renewal ime E [X] = X < 1. If each R n is a rv wih E [ R n ] < 1, hen lim!1 1 Z = R( ) d = E [R n] X WP1. (5.28) Example (Disribuion of Residual Life) Example reaed he ime-average value of he residual life Y (). Suppose, however, ha we would like o find he ime-average CDF of Y (), i.e., he fracion of ime ha Y () apple y for any given y. The approach, which applies o a wide variey of applicaions, is o use an indicaor funcion (for a given value of y) as a reward funcion. Tha is, define R() o have he value 1 for all such ha Y () apple y and o have he value oherwise. Figure 5.1 illusraes his funcion for a given sample pah. Expressing his reward funcion in erms of Z() and X(), e we have R() = R(Z(), e X()) = ( 1 ; e X() Z() apple y ; oherwise.

19 5.4. RENEWAL-REWARD PROCESSES; TIME AVERAGES 241 y - y - y - - X 3 S 1 S 2 S 3 S 4 Figure 5.1: Reward funcion o find he ime-average fracion of ime ha {Y () apple y}. For he sample funcion in he figure, X 1 > y, X 2 > y, and X 4 > y, bu X 3 < y Noe ha if an iner-renewal inerval is smaller han y (such as he hird inerval in Figure 5.1), hen R() has he value one over he enire inerval, whereas if he inerval is greaer han y, hen R() has he value one only over he final y unis of he inerval. Thus R n = min[y, X n ]. Noe ha he random variable min[y, X n ] is equal o X n for X n apple y, and hus has he same CDF as X n in he range o y. Figure 5.11 illusraes his in erms of he complemenary CDF. From he figure, we see ha E [R n ] = E [min(x, y)] = Z 1 x= Pr{min(X, y) > x} dx = Z y x= Pr{X > x} dx. (5.29) Pr{min(X, y) > x} x - y Pr{X > x} Figure 5.11: R n for disribuion of residual life. Le F Y (y) = lim!1 (1/) R R( ) d denoe he ime-average fracion of ime ha he residual life is less han or equal o y. From Theorem and Eq.(5.29), we hen have F Y (y) = E [R n] X = 1 X Z y x= Pr{X > x} dx WP1. (5.3) As a check, noe ha his inegral is increasing in y and approaches 1 as y! 1. Noe also ha he expeced value of Y, calculaed from (5.3), is given by E X 2 /2X, in agreemen wih (5.17). The same argumen can be applied o he ime-average disribuion of age (see Exercise 5.14). The ime-average fracion of ime, F Z (z), ha he age is a mos z is given by F Z (z) = 1 X Z z x= Pr{X > x} dx WP1. (5.31) In he developmen so far, he reward funcion R() has been a funcion solely of he age and duraion inervals, and he aggregae reward over he nh iner-renewal inerval is a funcion only of X n. In more general siuaions, where he renewal process is embedded in some more complex process, i is ofen desirable o define R() o depend on oher aspecs of he process as well. The imporan hing here is for {R n ; n 1} o be an IID sequence.

20 242 CHAPTER 5. RENEWAL PROCESSES How o achieve his, and how i is relaed o queueing sysems, is described in Secion Theorem clearly remains valid if {R n ; n 1} is IID. This more general ype of renewal-reward funcion will be required and furher discussed in Secions o where we discuss Lile s heorem and he M/G/1 expeced queueing delay, boh of which use his more general srucure. Limiing ime averages are someimes visualized by he following ype of experimen. For some given large ime, le T be a uniformly disribued random variable over (, ]; T is independen of he renewal-reward process under consideraion. Then (1/) R R( ) d is he expeced value (over T ) of R(T ) for a given sample pah of {R( ); >}. Theorem saes ha in he limi! 1, all sample pahs (excep a se of probabiliy ) yield he same expeced value over T. This approach of viewing a ime average as a random choice of ime is referred o as random incidence. Random incidence is awkward mahemaically, since he random variable T changes wih he overall ime and has no reasonable limi. We will no use i in wha follows, since, while somewha inuiive, i is ofen confusing. 5.5 Sopping imes for repeaed experimens Visualize performing an experimen repeaedly, observing successive sample oupus from a sequence of rv s. Depending on he sample values, x 1, x 2,... he observaions are sopped afer some rial n, where n is chosen based on he sample values x 1,..., x n already observed. This ype of siuaion occurs frequenly in applicaions. For example, we migh be required o choose beween several hypoheses, and migh repea an experimen unil he hypoheses are su cienly discriminaed. We will find ha if he number of rials is allowed o depend on he oucome, hen he mean number of rials required o achieve a given error probabiliy is ypically a small fracion of he number of rials required when he number is chosen in advance. Anoher example occurs in ree searches where a pah is explored unil furher exensions of he pah appear o be unprofiable. The firs careful sudy of experimenal siuaions where he number of rials depends on he daa was made by he saisician Abraham Wald and led o he field of sequenial analysis (see [29]). We sudy hese siuaions now since one of he major resuls, Wald s equaliy, will be useful in sudying E [N()] in he nex secion. Sopping imes are frequenly useful in he sudy of random processes, and in paricular will be used in Secion 5.7 for he analysis of queues, and again in Chaper 9 as cenral opics in he sudy of random walks and maringales. An imporan par of experimens ha sop afer a random number of rials is he rule for sopping. Such a rule mus specify, for each sample pah, he rial a which he experimen sops, i.e., he final rial afer which no more rials are performed. Thus he rule for sopping should specify a posiive ineger-valued, random variable J, called he sopping ime, or sopping rial, mapping sample pahs o his final rial a which he experimen sops. We view he sample space as including all possible oucomes for he never-ending sequence of random variables X 1, X 2,.... Tha is, even if he experimen is sopped a he end of he

21 5.5. STOPPING TIMES FOR REPEATED EXPERIMENTS 243 second rial, we sill visualize he 3rd, 4h,... random variables as having sample values as par of he sample pah. In oher words, we visualize ha he experimen coninues forever, bu he observer sops waching a he end of he sopping poin. From an applicaion sandpoin, he experimen migh or migh no coninue afer he observer sops waching. From a mahemaical sandpoin, however, i is far preferable o view he experimen as coninuing. This avoids confusion and ambiguiy abou he meaning of rv s when he very exisence of laer rv s depends on he sample values of earlier rv s. The inuiive noion of sopping a sequenial experimen should involve sopping based on he daa (i.e., he sample values) gahered up o and including he sopping poin. For example, if X 1, X 2,... represen he successive changes in our forune when gambling, we migh wan o sop when our cumulaive gain exceeds some fixed value. The sopping rial n hen depends on he sample values of X 1, X 2,..., X n. The whole noion of sopping on he basis of pas sample values would be violaed by rules ha allow he experimener o peek a subsequen values before making he decision o sop or no. For example, poker players do no ake kindly o a player who aemps o wihdraw his ane if he doesn like his cards. Similarly, a saisician gahering daa on produc failures should no respond o a failure by choosing an earlier rial as a sopping ime, hus no recording he failure. I is no immediaely obvious how o specify a rv J ha is defined as a funcion of a sequence X 1, X 2,..., bu has he propery ha he even J = n depends only on he sample values of X 1,..., X n. A sensible approach, embodied in he following definiion, defines J in erms of he successive indicaor funcions I {J=n} of J. Definiion A sopping rial (or sopping ime 5 ) J for a sequence of rv s X 1, X 2,..., is a posiive ineger-valued rv such ha for each n 1, he indicaor rv I {J=n} is a funcion of {X 1, X 2,..., X n }. The las clause of he definiion means ha any given sample values x 1,..., x n for X 1,..., X n uniquely deermine wheher he corresponding sample value of J is n or no. Noe ha since he sopping rial J is defined o be a posiive ineger-valued rv, he evens {J = n} and {J = m} for m < n are disjoin evens, so sopping a rial m makes i impossible o also sop a n for a given sample pah. Also he union of he evens {J = n} over n 1 has probabiliy 1. Aside from his final resricion, he definiion does no depend on he probabiliy measure and depends solely on he se of evens {J = n} for each n. In many siuaions, i is useful o relax he definiion furher o allow J o be a possibly-defecive rv. In his case he quesion of wheher sopping occurs wih probabiliy 1 can be posponed unil afer specifying he disjoin evens {J = n} over n 1. Example Consider a Bernoulli process {X n ; n 1}. A very simple sopping rial for his process is o sop a he firs occurrence of he sring (1, ). Figure 5.12 illusraes his sopping rial by viewing i as a runcaion of he ree of possible binary sequences. 5 Sopping rials are more ofen called sopping imes or opional sopping imes in he lieraure. In our firs major applicaion of a sopping rial, however, he sopping rial is he firs rial n a which a renewal epoch S n exceeds a given ime. Viewing his rial as a ime generaes considerable confusion.

22 244 CHAPTER 5. RENEWAL PROCESSES r 1 r r 1 1 r s r r r s r s r r rx s rx s rx s rx r Figure 5.12: A ree represening he se of binary sequences, wih a sopping rule viewed as a pruning of he ree. The paricular sopping rule here is o sop on he firs occurrence of he sring (1, ). The leaves of he ree (i.e., he nodes a which sopping occurs) are marked wih large dos and he inermediae nodes (he oher nodes) wih small dos. Noe ha each leaf in he pruned ree has a one-o-one correspondence wih an iniial segmen of he ree, so he sopping nodes can be unambiguously viewed eiher as leaves of he ree or iniial segmens of he sample sequences. The even {J = 2}, i.e., he even ha sopping occurs a rial 2, is he even {X 1 =1, X 2 =}. Similarly, he even {J = 3} is {X 1 =1, X 2 =1, X 3 =} S {X 1 =, X 2 =1, X 3 =}. The disjoinness of {J = n} and {J = m} for n 6= m is represened in he figure by erminaing he ree a each sopping node. I can be seen ha he ree never dies ou compleely, and in fac, for each rial n, he number of sopping nodes is n 1. However, he probabiliy ha sopping has no occurred by rial n goes o zero geomerically wih n, which ensures ha J is a random variable. This example is generalized in Exercise 5.35 o sopping rules ha sop on he firs occurrence of some arbirary sring of binary digis. This also uses renewal heory o provide a simple way o find he expeced number of rials unil sopping. Represening a sopping rule by a pruned ree can be used for any discree random sequence, alhough he ree becomes quie unwieldy in all bu rivial cases. Visualizing a sopping rule in erms of a pruned ree is useful concepually, bu sopping rules are usually saed in oher erms. For example, we shorly consider a sopping rial for he inerarrival inervals of a renewal process as he firs n for which he arrival epoch S n saisfies S n > for some given > Wald s equaliy An imporan quesion ha arises wih sopping rials is o evaluae he sum S J of he random variables up o he sopping rial, i.e., S J = P J n=1 X n. Many gambling sraegies and invesing sraegies involve some sor of rule for when o sop, and i is imporan o undersand he rv S J (which can model he overall gain or loss up o and including ha rial). Wald s equaliy relaes he expeced value of S J o E [J], and hus makes i easy o