IEOR 316: Fall 213, Profeor Whitt Topic for Dicuion: Tueday, November 19 Alternating Renewal Procee and The Renewal Equation 1 Alternating Renewal Procee An alternating renewal proce alternate between two tate, which we call up and down, thinking of the next example in which a computer i either working (up) or not working (down). The ytem alternate between ucceive up interval and down interval. The ytem i firt up for a time U 1 and then down for a period D 1. Next it i up for a period U 2 and then down for a period D 2, and o on. Often, the equence of up time {U n : n 1} and down time {D n : n 1} are independent equence of i.i.d. nonnegative random variable. However, a in the traffic cop example and the computer example below, all we really require i that the equence of pair {(U n, D n ) : n 1} be i.i.d. random vector (pair of nonnegative random variable). We allow U n and D n to be dependent for each n, but different pair are i.i.d. Let U be a generic up time; let D be a generic down time. The ditribution of U need not be the ame a the ditribution of D. Let Z(t) be the tate at time t; i.e., Let Z(t) 1 if the alternating renewal proce i up at time t, tarting at time at the beginning of an up period; otherwie let Z(t). We can apply the renewal reward theorem to deduce that the long-run proportion of time that the alternating renewal proce i up i long-run proportion up 1 lim t t Z() d E[U] E[U] + E[D]. (1) 2 An Example: ZeLun Gu Computer Zelun computer ha three critical part, each of which i needed for the computer to work. The computer run continuouly a long a the three required part are working. The three part have mutually independent exponential lifetime before they fail. The expected lifetime of part 1, 2 and 3 are 1 week, 2 week and 3 week, repectively. When a part fail, the computer i hut down and an order i made for a new part of that type. When the computer i hut down (to order a replacement part), the remaining two working part are not ubject to failure. The time to replace part 1 i exponentially ditributed with mean 1 week; the time to replace part 2 i uniformly ditributed between 1 week and 3 week; and the time to replace part 3 ha a gamma ditribution with mean 3 week and tandard deviation 1 week. (a) Auming that all part are initially working, what i the expected time until the firt part fail? Let T be the time until the firt failure. Then T i exponential with rate equal to the um of the rate; ee Chapter 5; i.e., ET 1 λ 1 + λ 2 + λ 3 1 (1/1) + (1/2) + (1/3) 1 (11/6) 6 5.45 week. 11 (b) What i the probability that part 1 i the firt part to fail?
Let N be the index of the firt part to fail. Since the failure time are mutually independent exponential random variable (ee Chapter 5), P (N 1) λ 1 λ 1 + λ 2 + λ 3 (1/1) (1/1) + (1/2) + (1/3) 6 11.545. (c) What i the long-run proportion of time that the computer i working? Now for the firt time we need to conider the random time it take to get the replacement part. Actually thee ditribution beyond their mean do not affect the anwer to the quetion aked here. Only the mean matter here. Ue elementary renewal theory. The ucceive time that the computer i working and hut down form an alternating renewal proce. Or, equivalently, apply renewal reward procee, a in Section 7.4: Look at the expected reward per cycle divided by the expected length of a cycle. Let a reward be earned at rate 1 whnenver the computer i working. Let T be a time until a failure (during which the computer i working) and let D be a down time. A cycle i T + D. Then the long-run proportion of time that the computer i working i 1 lim t t N(t) 1 R() d lim t t i1 T i ET ET + ED, where N(t) i the renewal counting proce counting the number of complete cycle up to time t. (Thi ignore the lat cycle in proce at time t, but the contribution of it i aymptotically negligible.) By part (a) above, ET 6/11. It uffice to find ED. To find ED, we conider the three poibilitie for the part that fail: ED P (N 1)E[D N 1] + P (N 2)E[D N 2] + P (N 3)E[D N 3] (6/11)E[D N 1] + (3/11)E[D N 2] + (2/11)E[D N 3] (6/11)1 + (3/11)2 + (2/11)3 18/11. Hence, ET/(ET + ED) (6/11) (6/11) + (18/11) 6 78 3 39.769 (d) Suppoe that new part of type 1 each cot $5; new part of type 2 each cot $1; and new part of type 3 each cot $4. What i the long-run average cot of replacement part per week? Thi i yet another application of the renewal reward theorem. The cycle i the ame a before, but now we have a new reward R. Now ER EC ER ET + ED 2
P (N 1)5 + P (N 2)1 + P (N 3)4 78/11 (6/11)5 + (3/11)1 + (2/11)4 78/11 (3/11) + (3/11) + (8/11) 78/11 14/11 78/11 14 17.95 dollar per week 78 3 Harder Reult for Alternating Renewal Procee We now want to etablih a harder (more refined) reult (needing extra condition). Now we want to how that E[U] lim P (Z(t) 1) t E[U] + E[D]. (2) The anwer in (2) i the ame a in (1), but the form of the limit i tronger. For example, uppoe that all up time are of length 1 and all down time are of length 2. Then P (Z(3k +.5) 1) 1 while P (Z(3k + 1.5) 1) for all k 1 integer. One ubequence ha one limit, while another ubequence ha a different limit. We will need to impoe extra condition that rule out thi cae. We will do o by auming that the ditribution of a cycle length i nonlattice. That mean that the cdf of X, ay F, doe not concentrate all it ma on the integer or a ubet {cn : n 1}; the integer cae i the mot common lattice ditribution; then c 1. Any nonnegative random variable with a probability denity function ha a nonlattice ditribution. Any integer-valued dicrete ditribution ha a lattice ditribution. Let P (t) P (Z(t) 1) be the probability that an up-down alternating renewal proce, tarting at time at the beginning of an up period, i in the up tate at time t. We obtain an explicit expreion for P (t) by etting up a renewal equation. We then contruct the unique olution to the renewal equation and apply the key renewal theorem to obtain the deired limit in (2). 3.1 The Renewal Equation for an Alternating Renewal Proce Let H(t) P (U t) and F (t) P (X t), where X U + D i a generic inter-renewal time. (We do not need the cdf of D or the two-dimenional cdf of (U, D).) Let H c (t) 1 H(t) be the complementary cdf (ccdf). We expre P (t) by conditioning on the value of X 1 and unconditioning: P (t) P (Z(t) 1) P (Z(t) 1, X 1 > t) + P (Z(t) 1, X 1 t) P (U 1 > t) + P (U 1 > t) + H c (t) + P (Z(t) 1 X 1 )f X1 () d, P (Z(t ) 1)f X1 () d, P (t )f() d. (3) 3
Line one above i the definition. In line two above we ue the law of total probability: P (A) P (A B)+P (A B c ). For line three, we oberve that the event {Z(t) 1, X 1 > t} i equivalent to the event {U 1 > t} for the firt term. Hence, they necearily have the ame probability. For the econd term on line three, we ue the formula P (AB) P (A B) P (A B)P (B). In line four of (3), we ued the renewal property, P (Z(t) 1 X 1 ) P (Z(t ) 1) P (t ) for all, t. The final diplay in (3) i the a renewal (integral) equation, i.e. P (t) H c (t) + P (t )f() d, (4) Thi renewal equation in (4) i valid for all t. Equation (4) i an integral functional equation, i.e., an equation for the function P (t) regarded a a function of t. Note that the function {P (t) : t } appear on both ide of the equation. To obtain an explicit expreion for the function P (t), we mut olve the equation. The firt term in (4) decribe the probability aociated with the firt renewal interval. The econd term decribe what happen after time and before time t if i the time that the firt cycle end. 3.2 Solving the Renewal Equation and Etablihing the Limit We now how how to olve the renewal equation. We recall the definition of a renewal proce and we introduce the notion of the renewal function. 3.2.1 A Renewal Proce Let {X n : n 1} be a equence of IID nonnegative real-valued random variable with cdf F having finite mean EX. Avoid trivialitie by auming that P (X > ) >. Let with S and S n X 1 + + X n, n 1, (5) N(t) max {n : S n t}, t. (6) Then N {N(t) : t } i a renewal (counting) proce, for which X n i the length of the interval between the (n 1) t point and the n th point, for n 1. (We do not put a point at unle X 1.) In addition, for the aymptotic reult we will want to aume that the ditribution of X 1 i non-lattice. A ditribution i a lattice ditribution if it concentrate all ma (ha upport) on the equence {cn : n } for ome contant c >. The mot common lattice ditribution have ma concentrated on the integer (the cae c 1). For lattice ditribution, there are correponding limit, along the lattice of upport. 3.2.2 The Renewal Function Then the renewal function i the mean of the renewal proce N(t) a a function of t, i.e., m(t) E[N(t)], t. (7) We have the following expreion for the renewal function: m(t) E[N(t)] P (N(t) n) P (S n t) where S n X 1 + + X n and F n (t) P (S n t). 4 F n (t), (8)
3.2.3 The Solution to the Renewal Equation in (4) It i ignificant the we can olve the functional equation (4) for the function {P (t) : t }. We now decribe the reult, but not yet the derivation. Equation (4) ha olution (a can be een by taking tranform; ee the 5.2 below) P (t) H c (t) + H c (t ) dm(), (9) where m(t) E[N(t)] i the renewal function aociated with the renewal counting proce N(t), counting the number of completed cycle up to time t. If m(t) ha a derivative with repect to t, denoted by m (t) ν(t), then dm() ν() d in the integral in (9). Conider that cae if you are not familiar with the Riemann-Stieltje integral with dm(). Then we write the expreion in (9) a a uual integral P (t) H c (t) + H c (t )m () d H c (t) + H c (t )ν() d. (1) Notice that in (9) the function {P (t) : t } now only appear on the left. Equation (9) give an explicit expreion for P (t). We go from (4) to (9) by replacing the two term inide the integral: Inide the integral, we replace P by H c and we replace the denity f()d by dm() ν()d. 3.2.4 Determining the Deired Limit The tandard application of the renewal equation (4) and it olution (9) or (1) i to drive the limit of the function on the left a t. That i, we apply (1) to derive the limit lim t P (t). We need regularity condition. Firt, we need the renewal interval X n to have a nonlattice ditribution. That mean that the cdf of X, which i F, doe not concentrate all it ma on the integer or a ubet {cn : n 1}; the integer cae i the mot common lattice ditribution; then c 1. And nonnegative random variable with a probability denity function ha a nonlattice ditribution. Any integer-valued dicrete ditribution ha a lattice ditribution. We alo need other regularity condition. The main theorem i the key renewal theorem. It conclude that, if regularity condition are atified, including the cdf F being nonlattice, then the olution of the renewal equation converge to a limit. Theorem 3.1 (key renewal theorem for alternating renewal proce) If the interval between renewal X ha a finite mean and a nonlattice ditribution, then the function P (t) on the left of (4) and (9) converge to a proper limit a t, in particular, lim P (t) t H c () d. E[X] Let u conider the pecific cae of an alternating renewal proce. Here i what we expect. By the elementary renewal theorem, we have m(t)/t λ 1/E[X] a t, where E[X] E[U] + E[D] Hence, we expect that ν(t) m (t) λ a t. Clearly, H c (t) a t. By the tail integral formula for the mean, the integral of H c i the mean E[U], where U ha cdf H. Thu, H c (t ) d H c () d 5 H c () d E[U] a t. (11)
Thu we ee that the key renewal theorem tate what we expect: H c (t )ν() d H c ()ν(t ) d λ A a conequence, in the pecific cae of an alternating renewal proce, we get P (t) H c (t) + + λ H c (t ) dm() H c (t) + H c () d H c () d. (12) H c () dm(t ) E[U] E[U] + E[D]. (13) In the lat line, 1/λ i the mean cycle length E[U] + E[D], o that λ 1/(E[U] + E[D]), while E[U] H c () d, by the tail integral formula for the mean. Many application can be treated by thi argument. The challenging careful mathematical treatment find precie condition for all of the above to be fully rigorou. The book, Applied Probability and Queue by S. Amuen, econd edition, Springer, 23, doe a nice job on that, but it i at a level beyond thi coure. 4 Eay and Hard Renewal Theory for the Age Proce We now give more general background, including a treatment of the age, dicued in a previou cla. Very broadly, the word renewal connote tarting over. Renewal theory involve only a few key idea: Firt, renewal theory i about renewal procee. A key quantity aociated with a renewal proce i the renewal function. The renewal function i important becaue it i a key component of the olution of the renewal equation. We ue the renewal equation to olve harder problem in renewal theory. The eay renewal theory concern application of the renewal reward theorem, yielding long-run average or long-run proportion. 4.1 Eay Renewal Theory For example, when we conider the age proce, the eay renewal theory tell u that the long-run proportion of time that the age A(t) i le than or equal to a contant c i F e (c) 1 E[X] c F c (t) dt, (14) where X i an interval between renewal, having cdf F (x) P (X x) and F c i the complementary cdf, i.e., F c (x) 1 F (x). The tail integral formula for the mean implie that F e i a bonafide cdf. The cdf F e i called the tationary (or equilibrium) age ditribution. From the eay renewal theory (note lat time), we conclude that 1 {A() c}() d E[ X 1 {A(t) c} dt] F e (c) a t. (15) t E[X] 4.2 Harder Renewal Theory In contrat, the harder renewal theory how, under general condition, that the following tronger limit than (15) i valid P (A(t) c) F e (c) a t. (16) 6
In general, the limit (16) implie the limit (15), wherea the limit (15) doe not imply the limit (16). Of coure, the actual value of the limit i the ame in both cae. Thu, to a large extent, the eay renewal theory already provide mot of the information. The harder renewal theory involve the aymptotic form of the renewal function and the olution of the renewal equation. Several theorem decribe thi aymptotic behavior: (i) the elementary renewal theorem, (ii) Blackwell renewal theorem, and (iii) the key renewal theorem. Thee note explain. 5 Uing Laplace Tranform (OPTIONAL FROM HERE ON) A relatively eay way to olve the renewal equation i to apply Laplace tranform, but ome care i needed, becaue the olution of the renewal equation i expreed in term of dm(t) a oppoed to m(t). We explain below. a For a real-valued function g of a nonnegative real variable, it Laplace tranform i defined ĝ() e t g(t) dt, where in general i a complex variable with poitive real part, but it uffice to think of a a poitive real number. 5.1 Laplace Tranform of a Random Variable For any nonnegative random variable Z with probability denity function (pdf) g, it Laplace tranform i defined a ĝ() E[e Z ] e t g(t) dt. That i, the Laplace tranform of the random variable i undertood to be the ordinary Laplace tranform of it denity (pdf). 5.2 The Laplace Tranform of the Renewal Function We now develop an expreion for the Laplace tranform of m(t), i.e., for ˆm() e t m(t) dt. From above, the Laplace tranform of the cdf F n (t) (of S n ) appearing in (8) i ˆF n () e t F n (t) dt. (17) We now develop further expreion. Firt, we write the Laplace tranform of the denity directly a ˆf n () e t f n (t) dt. (18) Since the cdf F n (t) i directly the integral of the pdf f n (t), baic Laplace tranform theory tell u that formula (17) and (18) imply that ˆF n () ˆf n (). (19) 7
(The relation (19) follow by applying integration by part.) Next, becaue S n X 1 + + X n, where the random variable X i are i.i.d., the pdf f n i the n-fold convolution of the pdf f 1 with itelf. (See below.) Thu, we have the following important implifying power relation We thu can combine formula (17)-(2) to obtain ˆf n () ˆf() n. (2) ˆF n () e t F n (t) dt. ˆf n () ˆf() n, (21) where ˆf n () i the Laplace tranform of f n, the pdf aociated with the cdf F n, and ˆf() ˆf 1 () i the Laplace tranform of the pdf f 1 f. Hence, we can combine (8) and (21) to compute the Laplace tranform of the renewal function m: Hence, ˆm() e t m(t) dt ˆm() ˆF n () ( ˆf() n /) ( ˆf n ()/) ( ˆf() n /). (22) ˆf() (1 ˆf()) ˆF () 2 ˆF (). (23) A a conequence, we ee that there i a one-to-one correpondence between cdf F and renewal function m. In particular, we can expre ˆf in term of ˆm, a well a vice vera: In particular, given (23), we have the following expreion going the other way: ˆF () ˆf() ˆm() 1 + ˆm(). (24) A another conequence of (23), we can compute any renewal function m(t) by numerically inverting it Laplace tranform, provided that you can evaluate the Laplace tranform of F, which i ˆF () ˆf()/. That i o becaue we can compute the tranform ˆm() if we can compute the tranform ˆf(). We can alo obtain the expreion for the Laplace tranform ˆm from the renewal equation for m(t), m(t) F (t) + m(t y) df (y) F (t) + m(t y)f(y) dy. (More on the renewal equation below.) By taking Laplace tranform in thi integral equation, we get ˆm() ˆF () + ˆm() ˆf() ˆf() + ˆm() ˆf() ˆF () + ˆF () ˆm(), from which we derive the ame formula above. 5.3 Convolution and Tranform In thi ection we digre to provide ome general technical background. (a) definition of convolution 8
Conider two equence of real number, a {a k : k } and b {b k : k }. We can form a new equence c {c k : k } that i the convolution of the firt two equence, denoted by c a b, by letting kn c n a k b n k, n. k Similarly, conider two function of a nonnegative real variable, f {f(t) : t } and g {g(t) : t }. We can form a new function h {h(t) : t } that i the convolution of the firt two function, denoted by h f g, by letting h(t) (b) aociated tranform f(y)g(t y) dy, f(t y)g(y) dy, t. For the equence a, b and c, (under appropriate regularity condition), we can form aociated generating function, by letting â(z) z k a k k and imilarly for ˆb(z) and ĉ(z). It i eay to ee that the convolution property for the equence a, b and c i equivalent to the tranform (generating function) equation for all (allowed) z. ĉ(z) â(z)ˆb(z) Similarly, for the function f, g and h, (under appropriate regularity condition), we can form aociated Laplace tranform, by letting ˆf() e t f(t) dt, for all complex variable with poitive real part, and imilarly for ĝ() and ĥ(). It i eay to ee that the convolution property for the function f, g and h i equivalent to the tranform (Laplace tranform) equation ĥ() ˆf()ĝ() for all (allowed). (c) probability application The above convolution and tranform relation are general propertie of equence and function, without probability playing a role. There are important application to probability. However, there can be confuion if you are not careful in your treatment of pmf, pdf, cdf and random variable. Almot all difficulty come from not being careful about the definition of thee baic probability model element. Let X and Y be nonnegative random variable with cdf F and G, repectively. Aume that X and Y are independent. Suppoe that F and G have pdf f and g. Let ˆf X () be the Laplace tranform of X, which mean ˆf X () E[e X ] e x df (x) 9 e x f(x) dx.
Let H be the cdf and h be the pdf of X + Y. Then the pdf h of X + Y i the convolution of the pdf of f and g h(t) If you take Laplace tranform, you get f(t x)g(x) dx ĥ() ˆf()ĝ(), f(x)g(t x) dx. a above. A imilar equation hold for cdf, but you have to be careful: H(t) F (t x)dg(x) F (t x)g(x) dx F (x)g(t x) dx. When we conider Laplace tranform, you have to remember that the Laplace tranform of the cdf i not the ame a the Laplace tranform of the pdf. (Thi i an important iue in renewal theory.) In particular, applying integration by part (p. 15 of Feller II), we ee that Then ˆF () Ĥ() ĥ() e t F (t) dt ˆf() ˆF ()ĝ() ˆf() ĝ().. 6 The Solution of the Renewal Equation in General Equation (4) and (9) are pecial cae. In general we have the renewal equation g(t) h(t) + g(t ) df () h(t) + g(t )f() d, (25) where F i the cdf of the inter-renewal time X and h i an arbitrary function (atifying regularity condition). In application, the challenge i to et up thi equation properly, which mean identifying the appropriate function h(t) to ue in (25). (Here the function h i an arbitrary function, choen appropriately for the particular application. It i not intended to be the pdf of the cdf H in the previou ection. The function H c there i a pecial cae of the function h here.) Once we have equation (25), it uffice to olve it for g. (In equation (25) the deired function g appear on both ide; we need to find an explicit expreion for g. Equation (25) ha olution g(t) h(t) + h(t ) dm() h(t) + h(t )ν() d. (26) It i undertood that the final expreion in each cae depend on extra regularity condition. A with (4) and (9), we go from (25) to (26) by replacing the two term inide the integral: We replace g by h and we replace df by dm. To jutify going from (25) to (26), we can take Laplace tranform. Starting from (25), we get ĝ() ĥ() + ĝ() ˆf() ĥ() 1 ˆf() (27) 1
On the other hand, tarting from (26), and applying (7), we have agreeing with (27). ĝ() ĥ() + ĥ()ˆν() ĥ() + ĥ() ˆm() 7 The Key Renewal Theorem ĥ() + ĥ() ˆf() ĥ(), (28) 1 ˆf() 1 ˆf() The key renewal theorem provide a limit a t, given the olution of the renewal equation in (26). We need the inter-renewal cdf F to be non-lattice and we need the function h to be directly Riemann integrable (DRI), an involved technical term. A ufficient condition for h to be DRI i for h to be nonnegative, non-increaing and integrable (over [, )), which alway hold for the alternating renewal proce, o we did not need the extra condition there. Given (26), h(x) dx g(t) a t. E[X] We have applied the argument in (13) above. 11