Runs of Identical Outcomes in a Sequence of Bernoulli Trials

Transcription

1 Western Kentucky University TopSCHOLAR Masters Theses & Specialist Projects Graduate School Spring 2018 Runs of Identical Outcomes in a Sequence of Bernoulli Trials Matthew Riggle Western Kentucky University, matthew.riggle262@topper.wku.edu Follow this and additional works at: Part of the Applied Mathematics Commons, and the Discrete Mathematics and Combinatorics Commons Recommended Citation Riggle, Matthew, "Runs of Identical Outcomes in a Sequence of Bernoulli Trials" (2018). Masters Theses & Specialist Projects. Paper This Thesis is brought to you for free and open access by TopSCHOLAR. It has been accepted for inclusion in Masters Theses & Specialist Projects by an authorized administrator of TopSCHOLAR. For more information, please contact topscholar@wku.edu.

2 RUNS OF IDENTICAL OUTCOMES IN A SEQUENCE OF BERNOULLI TRIALS A Thesis Presented to The Faculty of the Department of Mathematics Western Kentucky University Bowling Green, Kentucky In Partial Fulfillment Of the Requirements for the Degree Master of Science By Matthew Riggle May 2018

3

4 Dedicated to Mom and Dad and Michael

5 ACKNOWLEDGMENTS There are many individuals who contributed to the success of this project. First and foremost, I would like to thank my advisor, Dr. David Neal. He has provided a wealth of guidance and insight throughout the course of the project, and it is an honor to have written the last master s thesis that he will advise. (Happy retirement!) Next, I would like to thank the other readers of my committee, Dr. Melanie Autin and Dr. Ngoc Nguyen. Both of them have been supportive during the project, and were great professors for the many statistics classes I was able to take during my time at WKU. In addition to those named above, I would also like to thank the many other mathematics professors who have taught and advised me during my undergraduate and graduate career. I have enjoyed working with all of you, and I am grateful for all of the challenges and opportunities that helped me develop as a mathematician. Lastly, I would like to thank my friends and family who have supported me during my academic journey. In particular, Kathleen Bell deserves special recognition for doing countless homework assignments together, sharing in the experience of being calculus TAs, being a great office mate, and finally for joining me for several Thesis Thursday sessions. Also, I would like to thank Emily Keith, who came all the way from Illinois to attend my defense. iv

6 CONTENTS List of Tables vii Abstract viii Chapter 1. Introduction Chapter 2. Expectation for Obtaining a Run of Length The Direct Counting Approach for E[T 2,2 ] The Direct Counting Approach for E[T 2 ] Conditional Average for E[T 2,2 ] Numerical Examples Chapter 3. Markov Chains and Recursive Sequences Markov Chain Solutions A Larger Markov Chain Example Pseudo-Fibonacci Sequences Chapter 4. Expectation for Obtaining a Run of Length n Deriving E[T n ] Recursively Deriving E[T n,m ] Recursively Relationships Between E[T n ] and E[T n,m ] Conditional Expectations Applications of Conditional Runs Chapter 5. Simulations with Mathematica Chapter 6. Generalizations and Further Study Obtaining a Run with a Non-Constant Probability Obtaining a Run in a Finite Set v

7 6.3. Obtaining a Run with a Categorical Distribution Obtaining a Run from Multiple Sequences Appendix A Code for Theorem Appendix B Code for Theorem Appendix C Code for Theorems 4.4.1/ Appendix D Code for Conjecture References vi

8 LIST OF TABLES Table 3.1 Selected Pseudo-Fibonacci Sequences Table 4.1 Probabilities for a Fair Run Game Table 5.1 Simulation Results vii

9 RUNS OF IDENTICAL OUTCOMES IN A SEQUENCE OF BERNOULLI TRIALS Matthew Riggle May Pages Directed by: David Neal, Melanie Autin, and Ngoc Nguyen Department of Mathematics Western Kentucky University The Bernoulli distribution is a basic, well-studied distribution in probability. In this thesis, we will consider repeated Bernoulli trials in order to study runs of identical outcomes. More formally, for t N, we let X t Bernoulli(p), where p is the probability of success, q 1 p is the probability of failure, and all X t are independent. Then X t gives the outcome of the t th trial, which is 1 for success or 0 for failure. For n, m N, we define T n to be the number of trials needed to first observe n consecutive successes (where the n th success occurs on trial X Tn ). Likewise, we define T n,m to be the number of trials needed to first observe either n consecutive successes or m consecutive failures. We shall primarily focus our attention on calculating E[T n ] and E[T n,m ]. Starting with the simple cases of E[T 2 ] and E[T 2,2 ], we will use a variety of techniques, such as counting arguments and Markov chains, in order to derive the expectations. When possible, we shall also provide closed-form expressions for the probability mass function, cumulative distribution function, variance, and other values of interest. Eventually we will work our way to general formulas for E[T n ] and E[T n,m ]. We will also derive formulas for conditional averages, and discuss how famous results from probability such as Wald s Identity apply to our problem. Numerical examples will also be given in order to supplement the discussion and clarify the results. viii

10 CHAPTER 1 INTRODUCTION Throughout, we let X t Bernoulli(p), where p is the probability of success, q 1 p is the probability of failure, and all X t are independent. X t gives the outcome of the t th trial, which is 1 for success and 0 for failure. For integers n, m 2, we define T n to be the number of trials needed to first observe n consecutive successes (where the n th success occurs on trial X Tn ). Likewise, we define the stopping time T n,m to be the number of trials needed to first observe either n consecutive successes or m consecutive failures. What are E[T n ] and E[T n,m ]? Motivation for the main problem can be found in a variety of applications. In particular, this problem is closely related to the concept of the hot-hand fallacy, which was discussed in a famous psychology paper by Gilovich, Vallone, and Tversky [2]. The idea is that basketball fans have a mental picture of a player who makes several consecutive shots as being on fire, when in reality most such streaks can be ascribed to random chance. The hot-hand fallacy is closely related to the gambler s fallacy [8], which falsely assumes that even for independent events, long streaks of identical outcomes are less likely than probability would dictate. While we are largely interested in the average number of trials needed to observe a run of a given length, other work has been done on related problems. Several papers by Schilling discuss the average length of the longest run in a given finite number of Bernoulli trials [9], [10], [11]. These papers provide examples of some of the techniques we will use for our problem, including counting arguments and recursive methods. Another application of studying runs is related to the concept of optimal stopping, which seeks to identify when to take a certain action so as to maximize 1

11 (or minimize) some value. An example of a gambling application will be presented in this project, in which the player will quit after some number of consecutive wins or losses. We will investigate the probability of coming out ahead and the associated conditional winnings. Other research related to optimal stopping and the Bernoulli distribution can be found in [6] and [12]. Finally, while not the main focus of this project, it is possible to closely examine the probability mass function for the case T 2 in order to develop recursive Fibonacci-like sequences. It turns out that using p 1/2 returns precisely the Fibonacci sequence, and varying the probability parameter will provide a different recursive sequence. Using a modified version of Binet s Formula for the Fibonacci numbers allows us to define such sequences as a side result. Similar recursive sequences are examined in [1]. The two trivial cases, where p 0 or p 1, can be handled with little effort. If p 1, we will obtain a success on every trial, so it takes n trials to obtain a run of n successes. Thus E[T n ] n and E[T n,m ] n. If instead p 0, we will obtain a failure on every trial, so it takes m trials to obtain a run of m failures and E[T n,m ] m. However, we will never obtain a run of n successes, so E[T n ]. We will thus focus on the case where 0 < p < 1. We begin in Chapter 2 by studying the case where we wish to obtain a run of length 2, in order to illustrate the techniques used to derive formulas for the regular and conditional expectations. In Chapter 3, we use Markov chains as an alternate approach to the problem, which allows use to highlight some nice results related to the Fibonacci sequence. In Chapter 4, we generalize the problem for runs of length n or m where n, m 2 and compare the results with those obtained in Chapter 2. Chapter 5 describes simulation programs used to verify the results obtained in previous chapters. Finally, we present further generalizations and avenues for future study in Chapter 6. 2

12 CHAPTER 2 EXPECTATION FOR OBTAINING A RUN OF LENGTH 2 In this chapter, we utilize a variety of approaches to calculate E[T 2 ] and E[T 2,2 ], beginning with a direct counting approach. Surprisingly, this approach turns out to be more straightforward for the calculation of E[T 2,2 ], so we begin there The Direct Counting Approach for E[T 2,2 ] We begin by deriving the probability mass function f 2,2 (t) P (T 2,2 t), defined for t 2. We note that we must end either by obtaining a run of 2 successes (which occurs with probability p 2 ) or by obtaining a run of 2 failures (which occurs with probability q 2 ). Also, since we cannot obtain a run of length 2 before the observed run, all previous trials must alternate between success and failure. This means that for any given value of t, there are two valid options, depending on the parity of t. If t is even, then the only valid options are or , where 1 indicates a success and 0 indicates a failure. If t is odd, then the two valid options are or Thus, the outcome of the starting trial will depend on whether t is even or odd. We may then find the probability for any particular value of t, depending on whether it is even or odd. If t is even, then there will be either t successes and t 2 1 failures, or vice versa. Also, each valid sequence of trials has only one possible ordering. The corresponding probabilities are p t 2 +1 q t 2 1 and p t 2 1 q t 2 +1, so the total probability is p t 2 +1 q t p t 2 1 q t 2 +1, which can be rewritten as (pq) t 2 1 (p 2 + q 2 ). 3

13 If instead t is odd, then there will be either t+1 2 successes and t 1 2 failures, or vice versa. The corresponding probabilities are p t+1 2 q t 1 2 and p t 1 2 q t+1 2, so the total probability is p t+1 2 q t p t 1 2 q t+1 2, which can be simplified to (pq) t 1 2. Thus the probability mass function f 2,2 (t) for T 2,2 is as follows: p t 2 1 q t 2 1 (p 2 + q 2 ), if t is even f 2,2 (t) (pq) t 1 2, if t is odd. (2.1) Summing f 2,2 (t) over all possible values of t allows us to prove the following result. Theorem For 0 p 1, we will obtain a run of length 2 almost surely. Proof. For p 0 and p 1, it is guaranteed that the run will occur in the first two trials. For 0 < p < 1, we let even t 2k and odd t 2k + 1, and consider the piecewise sum of f 2,2 (t): P (T 2,2 < ) (pq) k 1 (p 2 + q 2 ) + k1 (pq) k (p 2 + q 2 ) + p2 + q 2 1 pq + pq 1 pq p2 + q 2 + pq 1 pq p2 + q(q + p) 1 p(1 p) p2 + q 1 p + p 2 p2 + q q + p 2 1. (pq) k k1 (pq) k+1 Since the sum of f 2,2 (t) over all values of t is equal to 1, we will obtain a run of 4

14 two successes or two failures almost surely. Theorem holds even though there are two sequences which never give a run of 2; i.e., those which forever alternate between success and failure. It is not difficult to show that these two sequences occur with probability 0. However, just because T 2,2 is finite-valued, it is not obvious that E[T 2,2 ] is finite. As a corollary to our work thus far, it is now convenient to utilize the probability mass function to develop a formula for the cumulative distribution function F 2,2 (t) P (T 2,2 t). We note that for even t 2k, the even sum will contain an extra term. For odd t 2k + 1, we have F 2,2 (t) k (pq) i 1 (p 2 + q 2 ) + i1 k (pq) i i1 (p2 + q 2 )(1 (pq) k ) pq (pq)k pq 1 pq (p2 + q 2 )(1 (pq) k ) pq (pq)k pq 1 pq (p2 + q 2 )(1 (pq) k ) + pq (pq) k+1 1 pq (p2 + q 2 )(1 (pq) k ) + pq(1 (pq) k ) 1 pq (p2 + q 2 + pq)(1 (pq) k ) 1 (1 q)q (p(p + q) + q2 )(1 (pq) k ) 1 q + q 2 (p + q2 )(1 (pq) k ) p + q 2 1 (pq) k, and for even t 2k, we have F 2,2 (t) k k 1 (pq) i 1 (p 2 + q 2 ) + (pq) i i1 i1 5

15 (p2 + q 2 )(1 (pq) k ) pq (pq)k + 1 pq 1 pq (p2 + q 2 )(1 (pq) k ) + pq (pq) k 1 pq (p2 + q 2 )(1 (pq) k ) + pq 1 + (1 (pq) k ) 1 pq (p2 + q 2 + 1)(1 (pq) k ) + pq 1 1 pq (p2 + q 2 + 1)(1 (pq) k ) 1 1 pq (p2 + (1 p) 2 + 1)(1 (pq) k ) 1 1 p(1 p) (p p + p 2 + 1)(1 (pq) k ) 1 1 p + p 2 (2p2 2p + 2)(1 (pq) k ) 1 p + p 2 1 2(1 (pq) k ) 1 1 2(pq) k. Then we reach the following corollary. Corollary The cumulative mass function F 2,2 (t), which gives the probability of obtaining a run of two identical trials within t trials, is given by 1 2(pq) t 2, if t is even F 2,2 (t) 1 (pq) t 1 2, if t is odd. Returning to the probability mass function allows us to calculate E[T 2,2 ]. Theorem Let T 2,2 be the number of trials required to obtain either a run of two successes or a run of two failures. Then E[T 2,2 ] 2 + pq 1 pq. Proof. We use two sums, one for even t and one for odd t. For even t 2k, we 6

16 have (2k)(pq) k 1 (p 2 + q 2 ), k1 and for odd t 2k + 1, we have (2k + 1)(pq) k. k1 Combining the even and odd sums, we obtain: E[T 2,2 ] (2k)(pq) k 1 (p 2 + q 2 ) + (2k + 1)(pq) k. k1 k1 Applying the formulas rk 1 1 r and that p + q 1 allows us to simplify the sum: E[T 2,2 ] (2k)(pq) k 1 (p 2 + q 2 ) + k1 2(p 2 + q 2 ) k1 krk 1 (2k + 1)(pq) k k1 k(pq) k 1 + 2pq k1 k(pq) k 1 + pq k1 2(p2 + q 2 ) (1 pq) 2 + 2pq (1 pq) 2 + pq (1 pq) 2p2 + 2q 2 + 2pq + pq(1 pq) (1 pq) 2 2p2 + 2q 2 + 3pq p 2 q 2 (1 pq) 2 (2p2 + 2pq) + (2q 2 + 2pq) pq p 2 q 2 (1 pq) 2 (2p2 + 2pq) + (2q 2 + 2pq) pq p 2 q 2 (1 pq) 2 2p(p + q) + 2q(p + q) pq p2 q 2 (1 pq) 2 1 and the fact (1 r) 2 (pq) k 7

17 2 pq p2 q 2 (1 pq) 2 (2 + pq)(1 pq) (1 pq) pq 1 pq. We note that the formula is also valid for p 0 or p 1; in either case, we obtain E[T 2,2 ] The Direct Counting Approach for E[T 2 ] We now consider the probability mass function f 2 (t) P (T t), defined for t 2, for first obtaining a run of two successes ending on the t th trial. The first few values of f 2 (t) are straightforward to calculate: t 2 corresponds to a run of two successes, so f 2 (2) p 2. Likewise, for t 3, we must obtain successes on trials 2 and 3, and a failure on trial 1 (or else we would have achieved the run at t 2). Thus f 2 (3) p 2 q. It is clear that for t > 3, the last three trials must be failure, success, and success, in that order, but there are more possibilities for the first t 3 trials (hereafter referred to as the pre-run ). The only restriction is that we cannot obtain two consecutive successes. Note that the number of successes in the pre-run cannot exceed t 2 2 run of two successes earlier than the t th trial., since otherwise we would have a We now consider the choice of successes within the pre-run. If the prerun contains no successes, this can only occur in one way (we will write this as ) for reasons that will soon be apparent). If the pre-run contains exactly ( t success, there are ( t 3 1 ) ways this may occur. If the pre-run contains more than 1 success, then each success we place prior to the last will necessarily be preceded by a failure, which removes 1 position from consideration. Thus a prerun that contains exactly 2 successes can occur in ( ) t 4 2 ways, and a pre-run that 8

18 contains exactly k successes can occur in ( ) t 2 k k ways. Finally, note that a prerun of length t 3 with k successes occurs with probability p k q t 3 k, so the entire observed sequence occurs with probability p k+2 q t 2 k. Thus, the probability mass function can be written as f 2 (t) t 2 2 ( t 2 k which leads to the following expected value k ) p k+2 q t 2 k, defined for integers t 2, (2.2) E[T 2 ] t2 t 2 2 ( t 2 k t k ) p k+2 q t 2 k. (2.3) As can be seen, the formula in (2.3) is quite unwieldy, so we now pursue a different method of enumeration. Instead of developing a formula based on the number of trials t, we will consider the number of failures. We know that the sequence must end with two successes, but before that, there may be any number of failures k {0, 1, 2,...}. Between any two failures or before the first failure, there can be at most one success. Thus there are i {0, 1, 2,..., k} successes, in addition to the two successes in the run. Thus the probability of our sequence containing exactly k failures is k i0 ( ) k p 2+i q k p 2 q k i Using this probability allows us to prove the following result. k i0 ( ) k p i p 2 q k (p + 1) k. (2.4) i Theorem For 0 < p 1, we will obtain a run of two successes almost surely. Proof. For p 1, it is guaranteed that the run will occur in the first two trials. 9

19 For 0 < p < 1, we have P (T 2 < ) p 2 q k (p + 1) k p 2 [q(p + 1)] k p 2 1 q(p + 1) p 2 1 pq q p2 p pq p 2 p(1 q) 1. As the total probability is equal to 1, we will obtain the run almost surely. In this case, there are infinitely many sequences which never give a run of 2 successes; however, the union of all such sequences has probability 0. We can also use the probability function in (2.4) to obtain a formula for E[T 2 ]. Theorem Let T 2 be the number of trials required to obtain a run of 2 successes. Then E[T 2 ] 1 + p p 2. Proof. We sum on the number of failures k, for k 0. Since we might have a success before each failure, there are i successes before the run, for some 0 i k. Together with the run of 2 successes, there is a total of 2 + k + i trials. Then E[T 2 ] k i0 ( ) k p 2+i q k (2 + k + i) i 10

20 k ( ) k p 2 q k p i (2 + k + i) i i0 k ( ) k p 2 q ((2 k + k) p i + i i0 k i0 ( ) ) k p i i i p 2 q ( k (2 + k)(p + 1) k + kp(p + 1) k 1) ( p 2 q k (2 + k)(p + 1) k + p 2 (2 q k (p + 1) k + ) q k kp(p + 1) k 1 q k (p + 1) k k + pq ) kq k 1 (p + 1) k 1 ( ) p q(p + 1) + (p + 1)q (1 q(p + 1)) + pq 2 (1 q(p + 1)) 2 p2 (2 2q(p + 1) + (p + 1)q + pq) (1 q(p + 1)) 2 p2 (2 q(p + 1) + pq) (1 qp q) 2 p2 (2 qp q + pq) (p pq) 2 p2 (2 q) (p(1 q)) 2 2 q p p p 2. The formula also works for p 1, as E[T 2 ] Conditional Average for E[T 2,2 ] We may also be interested in obtaining a run of n successes before we obtain a run of m failures. In order to derive the conditional expectation of the number of trials needed in this case, we will need the probability of first obtaining a run of successes. As before, we first seek a derivation for the case where n 2 and m 2. 11

21 Theorem Let P (S2) be the probability of obtaining a run of 2 successes before obtaining a run of 2 failures. Then P (S2) p(1 q2 ) 1 pq. Proof. Suppose we obtain the run of 2 successes first and that it takes t trials. Then the sequence of trials must end with two successes and be preceded by alternating successes and failures. If t is even, we will have two successes (occurring with probability p 2 ), plus an equal number of successes and failures in the preceding t 2 trials. If instead t is odd, we will have two successes (occurring with probability p 2 ), and there will be one additional success in the preceding t 2 trials. Again, by summing on the number of failures k, we obtain P (S2) p 2 (pq) k + qp 2 p2 1 pq + qp2 1 pq p2 + qp 2 1 pq p2 (1 + q) 1 pq p(1 q)(1 + q) 1 pq p(1 q2 ) 1 pq. (pq) k By symmetry, we also obtain the analogous result. Corollary Let P (F 2) be the probability of obtaining a run of 2 failures before obtaining a run of 2 successes. Then P (F 2) q(1 p2 ) 1 pq. Now, we shall derive a formula for E[T 2,2 S2]. 12

22 Theorem Let S2 be the event of obtaining a run of 2 successes before 2 + 3q pq2 obtaining a run of 2 failures. Then E[T 2,2 S2] (1 + q)(1 pq). Proof. We begin with two sums, corresponding to an even number of trials and an odd number of trials, respectively. We multiply by the number of trials in order to calculate the conditional expectation, and we divide by the probability of the given event P (S2): ( E[T 2,2 S2] 1 pq p 2 p(1 q 2 ) ( 1 pq 2p 2 p 2 (1 + q) ( 2pq (pq) k (2k + 2) + qp 2 (pq) k k + 2p 2 (pq) k (2k + 3) (pq) k + 2qp 2 1 pq 1 + q (1 pq) pq + 2q2 p (1 pq) + 3q 2 1 pq 1 pq ( ) 2pq + 2(1 pq) + 2q 2 p + 3q(1 pq) 1 + q (1 pq) 2 1 pq ( ) 2 + 3q q 2 p 1 + q (1 pq) q pq2 (1 + q)(1 pq). ) ) (pq) k k + 3qp 2 (pq) k ) Again, we obtain the analogous result by symmetry. Corollary Let F 2 be the event of obtaining a run of 2 failures before 2 + 3p qp2 obtaining a run of 2 successes. Then E[T 2,2 F 2] (1 + p)(1 pq) Numerical Examples We now present an example utilizing the formulas derived in the previous sections. Example Let p 0.4 be the probability of success on each trial. 13

23 By Theorem 2.2.2, the average number of trials needed to obtain a run of two successes is E[T 2 ] 1 + p p By Theorem 2.1.3, the average number of trials needed to obtain a run of two successes or a run of two failures is E[T 2,2 ] 2 + pq 1 pq 2 + (0.4)(0.6) 1 (.4)(.6) By Theorem 2.3.1, the probability of obtaining a run of two successes before a run of two failures is P (S2) p(1 q2 ) 1 pq (0.4)( ) 1 (.4)(.6) , and, similarly, by Corollary 2.3.2, the probability of obtaining a run of two failures before a run of two successes is P (F 2) q(1 p2 ) 1 pq (0.6)( ) 1 (.4)(.6) Finally, by Theorem 2.3.3, the average number of trials needed to obtain a run of two successes, given that we obtain such a run before a run of two failures, is E[T 2,2 S2] 2 + 3q pq2 (1 + q)(1 pq) 2 + 3(.6) (.4)(.6)2 (1 +.6)(1 (.4)(.6))

24 Likewise, by Corollary 2.3.4, the average number of trials needed to obtain a run of two failures, given that we obtain such a run before a run of two successes, is E[T 2,2 F 2] 2 + 3p qp2 (1 + p)(1 pq) 2 + 3(.4) (.6)(.4)2 (1 +.4)(1 (.4)(.6)) The previous two results demonstrate that T 2,2 depends on which run occurs first. The next example will treat the repeated Bernoulli trials as corresponding to individual steps in a one-dimensional random walk. A success corresponds to an upward step, and a failure corresponds to a downward step. More formally, if upward steps are a units each, and downward steps are b units each, then we can define the height H t as H t t (a1 {Xi 1} b1 {Xi 0}) i1 where 1 {Xt1} is an indicator variable that returns 1 for a success and 0 for a failure, and 1 {Xt0} is an indicator variable that returns 0 for a success and 1 for a failure. Then H t represents the height at time t (i.e., after t trials). A very useful result for studying random walks is Wald s Identity. Next, we will present a proof of Wald s Identity from Harper and Ross [3]. Theorem Wald s Identity: Suppose that H t is the height of a random walk defined by H t t i1 (a1 {X i 1} b1 {Xi 0}). If X t are i.i.d., T is a stopping time, and E[X 1 ] and E[T ] are both finite, then E[H T ] E[X 1 ] E[T ]. 15

25 Proof. As X t and the indicator 1 {T n} are independent, we have E[H T ] E[X n 1 {T n} ] n1 E[X n ] E[1 {T n} ] n1 E[X 1 ] P (T n) n1 E[X 1 ] n P (T n) n1 E[X 1 ] E[T ]. We will use Wald s Identity in the next example. Example Let p 0.4 be the probability of success on each trial, and let each upward step be a units and each downward step be b units. We would like to compute E[H T2 ] and E[H T2,2 ]. First, we observe that E[X 1 ] ap bq. Then, by applying Wald s Identity, we find E[H T2 ] E[X 1 ] E[T 2 ] ( ) 1 + p (ap bq) p 2 (0.4a 0.6b)(8.75) 3.5a 5.25b, 16

26 and E[H T2,2 ] E[X 1 ] E[T 2,2 ] ( ) 2 + pq (ap bq) 1 pq ( ) 56 (0.4a 0.6b) a b

27 CHAPTER 3 MARKOV CHAINS AND RECURSIVE SEQUENCES Markov Chain Solutions The formula for the probability mass function f 2 (t) derived with the direct counting approach, (2.2), is difficult to use directly. For this reason, we attempt to develop a simpler formula by using a Markov chain. In this situation, we have three different states, corresponding to having a current run of either 0, 1, or 2 successes. Thus we have a 1 3 initial state matrix B and a 3 3 transition matrix A: ( ) B p p 0 A 1 p 0 p The entries in B correspond to the initial probabilities; since we always begin with a run of 0 successes, there is a 1 in the first entry and 0 elsewhere. Each entry in A corresponds to the probability of moving from its row state to its column state. Note that the third row represents the absorbing state, since once we obtain a run of 2 successes, we will remain in that state indefinitely. Then the 1 3 matrix B A t represents the probabilities of being in each state after t trials. Because computing A t is quite difficult, we seek a diagonalization of A using standard techniques (see [7]). The eigenvalues of A can be found by solving its characteristic equation: (1 p λ)( λ)(1 λ) (1 p)(p)(1 λ) 0 18

28 (1 λ)[λ 2 (1 p)λ p(1 p)] 0 Solving for λ, we obtain: λ 1 or λ (1 p) ± (1 p)(1 + 3p). 2 Then the three eigenvalues of A are λ 1 1, λ 2 (1 p) + (1 p)(1 + 3p) 2, and λ 3 (1 p) (1 p)(1 + 3p). 2 We can also find the corresponding matrix of eigenvectors P, which is 1 (1 p)+ (1 p)(1+3p) (1 p) (1 p)(1+3p) 2(1 p) 2(1 p) P , where the i th column of P represents the eigenvector corresponding to λ i. Note that P is invertible, and P 1 1 p (1 p)(1 + 3p) (1 p)+ (1 p)(1+3p) 2(1 p) (1 p)+ (1 p)(1+3p) 2(1 p) (1 p)(1+3p) 1 p (1 p) (1 p)(1+3p) 2(1 p) (1 p) (1 p)(1+3p) 2(1 p). Then we have the diagonalization A P DP 1, where D is simply the diagonal matrix containing the eigenvalues on the main diagonal: D 0 (1 p)+ (1 p)(1+3p) (1 p) (1 p)(1+3p) 2.

29 This allows us to easily calculate A t P D t P 1. Note that the matrix ( ) B A t P (0) P (1) P (2) represents the final probabilities of being in each state after t trials. In particular, P (2) is the probability of obtaining a run of 2 successes in t or fewer trials, which is the cumulative mass function F 2 (t). Thus, when we perform the matrix multiplication A t P D t P 1, we can focus on this entry of the final state matrix, which is as follows: [ 1 p (1 p)(1 + 3p) P (2) (1 p)(1 + 3p) 1 p ( (1 p) + ) 2 ( (1 p)(1 + 3p) (1 p) + ) t (1 p)(1 + 3p) 2(1 p) 2 + ( (1 p) (1 p)(1 + 3p) 2(1 p) ) 2 ( (1 p) (1 p)(1 + 3p) 2 ) t. The above result simplifies to yield F 2 (t) 1 [ (1 p) + ] t+2 [ (1 p)(1 + 3p) (1 p) ] t+2 (1 p)(1 + 3p) 2 t+2 (1 p), (1 p)(1 + 3p) (3.1) from which we can derive f 2 (t) F 2 (t) F 2 (t 1). A remarkable simplification is possible in the case where p 1/2; in this case, we obtain [(1 12 ) + (1 12 )( ) ] t+2 [(1 12 ) (1 12 )( ) ] t+2 F 2 (t) 1 2 t+2 (1 1 2 ) (1 1 2 )( ) 20

30 ( ) t+2 ( ( ) t ( ) t+2 ( We note that t ) t+2 ) t ϕ, where ϕ is the golden ratio, and ϕ. This allows us to apply Binet s Formula, which states F n ϕn (1 ϕ) n ϕ (1 ϕ), where F n is the n th Fibonacci number. Thus we obtain the following theorem. Theorem The cumulative distribution function F 2 (t) for the probability of obtaining a run of 2 successes within t trials is given by F 2 (t) 1 F t+2 2 t, where F t is the t th Fibonacci number. Using Theorem 3.1.1, we can also obtain the pdf f 2 (t): f 2 (t) F 2 (t) F 2 (t 1) 1 F t+2 2 t 2F t+1 F t+2 2 t ( 1 F t+1 2 t 1 2F t+1 (F t+1 + F t ) 2 t F t+1 F t 2 t. ) We thus obtain the following result. Theorem The probability mass function f 2 (t) for the probability of obtaining a run of 2 successes ending on the t th trial is given by f 2 (t) F t 1 2 t, where F t is the t th Fibonacci number. We can use the probability distribution function in Theorem to provide an alternate proof that a run of 2 successes must have finite expectation in 21

31 the case where p 1/2. Theorem Suppose p 1/2. Then T 2 is finite almost surely, and E[T 2 ] is finite. Proof. To prove the first part of the theorem, we will show that the following sum is equal to 1: f 2 (t) t2 F t 1 2 t t2 ϕ t 1 (1 ϕ) t 1 2 t [ϕ (1 ϕ)] t2 1 ( ϕ t 1 2[ϕ (1 ϕ)] t2 (1 ϕ)t 1 2t 1 2 t 1 [ 1 (ϕ ) ( ) ] t t (1 ϕ) 2[ϕ (1 ϕ)] 2 2 t1 [ 1 ( ϕ ) t ( ) ] t (1 ϕ) 4ϕ t1 t1 ( ) ϕ 2 1 ϕ ϕ ( ) ϕ 1 1+ϕ 4ϕ ( 1 2 4ϕ 2 2 ϕ 2 ) 1 + ϕ ( ) ϕ 4 + 2ϕ 4ϕ 2 (2 ϕ)(1 + ϕ) ϕ ϕ ( 1+ 5) )

32 For the second part of the theorem, we will use the ratio test on the series E[T 2 ] t2 F t 1 2 t t, obtaining the following limit: ( ) ( ) ( ) Ft 2 t t + 1 lim t F t 1 2 t+1 t ( Ft lim t F t 1 ) ( ) 1 ϕ 2 2 < 1. Hence E[T 2 ] is finite A Larger Markov Chain Example We may apply the same Markov chain technique for longer runs where n > 2. In these cases, however, it will not be as straightforward to solve the characteristic equation and obtain a diagonalization of the transition matrix. We can, however, obtain numerical approximations for given values of p and t. Example Let p What is the probability of obtaining a run of 4 successes within 20 trials? Within 50 trials? What is the probability of first obtaining a run of 4 successes on the 10 th trial? To set up the Markov chain, we note that our initial state matrix will be a 1 5 matrix (allowing for runs of length 0, 1, 2, 3, and 4), and the transition matrix will be a 5 5 matrix. As before, the initial state matrix will contain a single 1 followed by all zeros. All but the last row of the transition matrix will contain 1 p in the first entry, p on row i, column i + 1, and zeros elsewhere. The last row, representing the absorbing state, will contain all zeros followed by a single 1 in the last entry. Thus, ( ) B

33 and A Then, to find the probability of obtaining a run of 4 successes within 20 trials, we simply need to compute ( ) B A 20 P (0) P (1) P (2) P (3) P (4), which yields the following (the values do not sum to 1 due to rounding): ( ) B A Therefore, from the last entry in the matrix, there is about a 79.53% chance of obtaining a run of 4 successes within 20 trials. Similarly, we can compute ( ) B A to find that there is about a 98.60% chance of obtaining a run of 4 successes within 50 trials. Essentially, we are computing specific values of the cumulative distribution function. Thus, in order to answer the last question, we need to compute two of these cdf values and subtract them. f 4 (10) F 4 (10) F 4 (9) (B A 10 ) 5 (B A 9 ) 5 24

34 Thus, there is about a 4.64% chance of first completing a run of 4 successes on the tenth trial Pseudo-Fibonacci sequences We can derive Binet s Formula for the n th Fibonacci number by using a generating function. We let G(x) F 0 + F 1 x + F 2 x 2 + F 3 x be the generating function, with coefficient F n representing the n th Fibonacci number. We also have the initial conditions F 0 0, F 1 1, and F n F n 1 + F n 2. Then we have G(x) F 0 + F 1 x + F 2 x 2 + F 3 x xg(x) F 0 x + F 1 x 2 + F 2 x 3 + F 3 x x 2 G(x) F 0 x 2 + F 1 x 3 + F 2 x 4 + F 3 x Subtracting the last two lines from the first line gives (1 x x 2 )G(x) F 0 + (F 1 F 0 )x, or G(x) x 1 x x 2. Thus we have solved for the generating function G(x). Note that the denominator can be factored (using ϕ and ϕ); hence, 2 2 G(x) x (1 ϕx)(1 (1 ϕ)x). 25

35 The right side of the equation can be rewritten using the partial fraction expansion G(x) A 1 ϕx + B 1 (1 ϕ)x. Through some straightforward algebra, we obtain A 1 5 and B 1 5. Note that 5 ϕ (1 ϕ). Thus we have G(x) ( ) 1 1 ϕ (1 ϕ) 1 ϕx (1 ϕ)x Note that we can rewrite the above using power series as follows: G(x) ( 1 (ϕx) n ϕ (1 ϕ) n0 ) [(1 ϕ)x] n. Then the coefficient F n for the x n term of G(x) is ϕn (1 ϕ) n, which is exactly ϕ (1 ϕ) Binet s Formula. F 2 (t) 1 We now consider the more general formula for F 2 (t) from (3.1): [ (1 p) + ] t+2 [ (1 p)(1 + 3p) (1 p) ] t+2 (1 p)(1 + 3p) 2 t+2 (1 p). (1 p)(1 + 3p) n0 We let α (1 p) + (1 p)(1 + 3p) and β (1 p) (1 p)(1 + 3p), and note that α β 2 (1 p)(1 + 3p), which is present in the denominator of [ (1 p) + ] n [ (1 p)(1 + 3p) (1 p) ] n (1 p)(1 + 3p) 2. (1 p)(1 + 3p) Then for n 0 we obtain 0, and for n 1 we obtain 1, matching the initial values of the Fibonacci sequence. However, we will need a different recursive relation than the one for the Fibonacci sequence. 26

36 For the Fibonacci generating function, we used the polynomial 1 x x 2 in order to eliminate all but a finite number of terms to give (1 x x 2 )G(x) F 0 + (F 1 F 0 )x. Now, the Fibonacci recursive relation may be written as F n F n 1 F n 2 0, which reveals the key insight. The coefficients of the polynomial and the recursion match (constant with F n, x with F n 1, and x 2 with F n 2 ). Thus, we seek a polynomial that factors into (1 αx)(1 βx). This turns out to be 4p(1 p) 2(1 p)x x 2. Then we can define a new recursive sequence, P n, of pseudo-fibonacci numbers with conditions P 0 0, P 1 1 P n 2(1 p)p n 1 + 4p(1 p)p n 2. Then by following the same argument as in the derivation of Binet s Formula, we obtain P n [ (1 p) + ] n [ (1 p)(1 + 3p) (1 p) ] n (1 p)(1 + 3p) 2 (1 p)(1 + 3p). (3.2) For p 1/2, (3.2) gives Binet s Formula for F n. Thus we have defined recursive sequences {P n } corresponding to probability values, which we can use to rewrite F 2 (t) P (T 2 t), as follows: F 2 (t) 1 P t+2 2 t+1 (1 p) 1 2(1 p)p t+1 + 4p(1 p)p t 2 t+1 (1 p) 27

37 1 P t+1 + 2pP t 2 t. It may be of interest to examine the sequences {P n }. Table 3.1 presents the beginning of the sequences for selected values of p. Table 3.1: Selected Pseudo-Fibonacci Sequences p First 7 terms of {P n } 0 0, 1, 2, 4, 8, 16, 32, , 1, 1.6, 3.2, 6.144, , , , 1, 1.4, 2.8, 5.096, , , , 1, 1, 2, 3, 5, 8, , 1, 0.5, 1, 0.875, , 1.25,... ϕ/2 0, 1, , , , , , , 1, 0.2, 0.4, 0.152, , , , 1, 0, 0, 0, 0, 0,... Note that some sequences are strictly increasing, while others oscillate. Some properties of the family {P n } may be observed. For instance, the only values which yield sequences of only integers are p 0, p 1/2, and p 1. Using p 0 gives powers of 2, and of course p 1/2 returns the familiar Fibonacci sequence. Examining the end behavior of the sequences also provides some interesting properties. For 0 p 1/2, {P n } is a monotone increasing sequence. For 1/2 < p < ϕ/2, {P n } oscillates a finite number of times before eventually increasing to infinity. For p ϕ/2, {P n } oscillates infinitely often, but the oscillations become smaller and smaller, and the sequence converges to the limit ϕ 1. For p > ϕ/2, {P n } eventually becomes decreasing and converges to 0. 28

38 Variations on the Fibonacci sequence are not a new concept. For instance, the generalized Fibonacci sequence modifies the starting values while preserving the coefficients in the recursive relation. In contrast, the family of pseudo- Fibonacci sequences defined here modified the coefficients in the recursive relation while preserving the starting values of 0 and 1. More results and applications related to generalized Fibonacci numbers can be found in Koshy [5]. 29

39 CHAPTER 4 EXPECTATION FOR OBTAINING A RUN OF LENGTH N Again, we define the stopping time T n to be the number of trials needed to first observe n consecutive successes, and we define the stopping time T n,m to be the number of trials needed to first observe either n consecutive successes or m consecutive failures. Because the counting arguments presented in Chapter 2 would seemingly be quite complicated for n, m > 2, we seek a more general method to calculate E[T n ] and E[T n,m ] Deriving E[T n ] Recursively Before we begin our procedure to derive a formula for E[T n ], we must first show that it is finite for any non-zero choice of p and any choice of n. Theorem For 0 < p 1 and fixed n, E[T n ] is finite. Proof. We have already proven the result for the trivial case of p 1, so suppose 0 < p < 1. Consider the trials in blocks of n; i.e., trials X 1 through X n, trials X n+1 through X 2n, and so on. Each block contains the run we desire with probability p n. The blocks correspond to a geometric distribution with parameter p n, so the expected number of blocks required to observe a run is 1/p n, which corresponds to n/p n trials. This expectation, which is finite, in fact provides an upper bound for E[T n ], since the first run might occur across two different blocks. Thus we can safely conclude that E[T n ] must be finite for any choice of n. As an aside, note that the case n 1 is equivalent to the geometric distribution with parameter p, which has expected value 1/p. We have already derived a formula for n 2 in Theorem 2.2.2, but we would like a general formula. We now develop such a formula by recursively calculating the weighted average. 30

40 Theorem Let T n be the number of trials required to obtain a run of n consecutive successes. Then for 0 < p < 1, E[T n ] 1 pn p n (1 p). Proof. Suppose we obtain a run of n successes immediately. This situation occurs with probability p n and requires n trials. If instead we do not obtain a run of n successes, then we must have first obtained a run of k successes followed by a failure, for some k {0, 1,..., n 1}. This situation occurs with probability p k (1 p). We have already observed k + 1 trials, and since the failure resets the run, we will require E[T n ] additional trials, for a total of k +1+E[T n ] trials. Thus we obtain the following: n 1 E[T n ] p n n + p k (1 p)(k E[T n ]) n 1 n 1 p n n + p k (1 p)e[t n ] + p k (1 p)(k + 1) n 1 n 1 p n n + (1 p n )E[T n ] + p k (k + 1) p k+1 (k + 1). Because E[T n ] is finite, we can subtract it from the right-hand side. Solving for E[T n ], we obtain n 1 p n E[T n ] p n n + p k (k + 1) n p k k k1 n 1 n 1 p n n p k (k + 1) p k k p n n n n 1 k1 p k 1 pn 1 p, p k k1 31 k1

41 which gives the result. Note that if we use n 1 in Theorem 4.1.2, we obtain the expectation E[T 1 ] 1, which is consistent with the geometric distribution. Furthermore, for p n 2, we obtain E[T 1 ] 1 p2 p 2 (1 p) 1 + p, which agrees with Theorem p 2 Thus we have solved for E[T n ] in all cases:, p 0 1 p E[T n ] n p n (1 p), 0 < p < 1 n, p 1. (4.1) We now turn our attention to E[T n,m ] Deriving E[T n,m ] Recursively For 0 < p < 1, it is not difficult to see that E[T n,m ] must be finite by using Theorem 4.1.1, as it is clear that E[T n,m, ] E[T n ]. Corollary For 0 p 1 and fixed n and m, E[T n,m ] is finite. Since Corollary guarantees E[T n,m ] must be finite for any parameter p, we now seek its derivation. The resulting formula is given as the following theorem. Theorem Let T n,m be the number of trials required to obtain either a run of n successes or a run of m failures. Then E[T n,m ] is given by m, p 0 (1 p E[T n,m ] n )(1 q m ) p n q + pq m p n q, 0 < p < 1 m n, p 1. 32

42 Proof. In order to facilitate the creation of formulas in this proof, we define two additional variables. We let A 1 be the expected number of additional trials needed when starting with a single success, and we let A 2 be the expected number of additional trials needed when starting with a single failure. Both A 1 and A 2 must be finite, as both values are less than or equal to T n,m. Now, consider the first trial, which results in success with probability p and failure with probability q. We may then write E[T n,m ] 1 + pa 1 + qa 2. (4.2) Suppose we have a current run of one success. If we obtain successes in the next n 1 trials, we are finished. This situation occurs with probability p n 1. If instead we do not obtain a run of n 1 successes, then we must have first obtained a run of k successes followed by a failure, for some k {0, 1,..., n 2}. This situation occurs with probability p k q. We have then observed k + 1 trials, and since we now have a current run of one failure, we will require A 2 additional trials, for a total of k A 2 trials. Thus we obtain the following: n 2 A 1 p n 1 (n 1) + p k q(k A 2 ) n 2 n 2 p n 1 (n 1) + p k (1 p)a 2 + p k (1 p)(k + 1) n 2 n 2 p n 1 (n 1) + (1 p n 1 )A 2 + p k (k + 1) p k+1 (k + 1). We proceed by moving the A 2 term to the left side: n 2 n 1 A 1 A 2 (1 p n 1 ) p n 1 (n 1) + p k (k + 1) p k k k1 n 2 n 2 p n 1 (n 1) p k (k + 1) p k k p n 1 (n 1) 33 k1 k1

43 The result is as follows: n k1 n 2 p k. p k A 1 A 2 (1 p n 1 ) 1 pn 1 1 p. (4.3) If we apply the same steps with success and failure reversed, we also obtain A 2 A 1 (1 q m 1 ) 1 qm 1. (4.4) 1 q Thus, (4.3) and (4.4) give a system of two equations in the two variables A 1 and A 2. Solving the system gives the following: A 1 (1 pn 1 )(1 q m ) p n q + pq m p n q m, A 2 (1 qm 1 )(1 p n ) p n q + pq m p n q m. (4.5) For (4.2), we need pa 1 and pa 2. First, we have pa 1 p(1 pn 1 )(1 q m ) p n q + pq m p n q m p pn pq m + p n q m p n q + pq m p n q m p pn + (p n q p n q) pq m + p n q m p n q + pq m p n q m p pn + p n q p n q + pq m p n q 1 m p p n+1 p n q + pq m p n q 1. m 34

44 Similarly, we have pa 2 q q m+1 p n q + pq m p n q m 1. Substituting these expressions into (4.2) and simplifying, we obtain E[T n,m ] 1 + pa 1 + qa 2 p p n+1 p n q + pq m p n q m + q q m+1 p n q + pq m p n q m 1 p pn+1 + q q m+1 p n q pq m + p n q m p n q + pq m p n q m 1 pn (p + q) q m (q + p) + p n q m p n q + pq m p n q m 1 pn q m + p n q m p n q + pq m p n q m (1 pn )(1 q m ) p n q + pq m p n q m, which is valid for 0 < p < 1. It should be noted that the formula in Theorem confirms the formula for n, m 2 derived in Chapter 2. Making the substitutions n 2 and m 2 and simplifying yields E[T 2,2 ] (1 p2 )(1 q 2 ) p 2 q + pq 2 p 2 q 2 1 p2 q 2 + p 2 q 2 pq(p + q) p 2 q 2 p + q p2 q 2 + p 2 q 2 pq p 2 q 2 p(1 p) + q(1 q) + p2 q 2 pq(1 pq) 2pq + p2 q 2 pq(1 pq) 35

45 2 + pq 1 pq, which agrees with Theorem Relationships Between E[T n ] and E[T n,m ] For 0 < p < 1, we have obtained the following expectations E[T n ] 1 pn p n (1 p), E[T n,m] (1 pn )(1 q m ) p n q + pq m p n q m. (4.6) Also, suppose that we wish to obtain a run of m consecutive failures, where the probability of failure is q. We denote the number of trials required by T m. Then, by symmetry, E[T m] is given by E[T m] 1 qm q m (1 q). We then have the following identity. Theorem For 0 < p < 1, 1 E[T n ] + 1 E[Tm] 1 E[T n,m ]. Proof. Simplifying the left side of the identity gives 1 E[T n ] + 1 E[Tm] pn (1 p) + qm (1 q) 1 p n 1 q m pn q(1 q m ) + q m p(1 p n ) (1 p n )(1 q m ) pn q p n q m+1 + q m p q m p n+1 (1 p n )(1 q m ) pn q + pq m p n q m (1 p n )(1 q m ) 1 E[T n,m ]. 36

46 In a sense, we can think of E[T n ] and E[Tm] as two components of E[T n,m ]. Theorem then says that the reciprocal of E[T n,m ] is equal to the sum of the reciprocals of its two components. In the special case where p q 1/2, we can relate E[T n ] and E[T n,n ]. Theorem For p q 1/2, E[T n ] 2E[T n,n ]. Proof. For p 1/2, we have the following: ) n E[T n ] 1 ( 1 2 ) n (1 1 ( 1 2 ) n 2 ) 1 ( 1 2 ( 1 n+1 ( 2) ( 1 1 ) n ) 2 2 ) n ( 1 2 2(2 n 1); [ ( 1 1 ) n ] [ ( n ] E[T n,n ] ( 2) 1 ) n+1 ( ) n+1 ( 2 1 2n 2) [ 1 ( 1 2) n ] 2 ( 1 ) n ( 2 1 2n 2) [ ( 1 1 n ] 2 ( 2) 1 ) n [ ( n ] 2) 1 ( 1 2 ( 1 n 2) ) n 2 n 1 E[T n]. 2 37

47 4.4 - Conditional Expectations Suppose we are now interested in the event that we obtain a run of n successes before we obtain a run of m failures, which we denote R nbm. We shall derive P (R nbm ) by considering cases of how smaller runs may occur before a run of n successes. Theorem Let R nbm be the event that we obtain a run of n successes before p n 1 (1 q m ) a run of m failures. Then P (R nbm ) p n 1 + q m 1 p n 1 q. m 1 Analogously, if Rnbm C is the event that we obtain a run of m failures before a run of n successes, then P (Rnbm C ) q m 1 (1 p n ) p n 1 + q m 1 p n 1 q m 1 Proof. For R nbm, we have two cases. In Case I, we start with a success, which occurs with probability p. We might immediately obtain all remaining n 1 successes, or our potential run of successes might be interrupted. An interruption will have two parts: some number of successes followed by a failure (part 1), then some number of failures followed by a success (part 2). Note that in part 1, we must obtain a failure within n 1 trials, and in part 2, we must obtain a success within m 1 trials (or else we will obtain a run of m failures, which is not allowed). After the interruption, we have one success, which returns to the original Case I scenario. To summarize, we have one success, followed by some number of interruptions (possibly 0), followed by the remaining n 1 successes. Also note that we can use the geometric distribution for each interruption; i.e., the probability of obtaining a failure within n 1 trials is 1 p n 1, and the probability of obtaining a success within m 1 trials is 1 q m 1. Thus, by summing on the number of 38

48 possible interruptions k, the probability of Case I is P 1 p n [(1 p n 1 )(1 q m 1 )] k. In Case II, we start with a failure. We must then obtain some number of failures followed by a success, which returns us to the beginning of case I. We then obtain some number of interruptions followed by the final n 1 successes. In this case, we will have the probability of the failure, the probability of the n 1 successes, and a single geometric probability outside the summation for the interruptions. The probability in this case is P 2 qp n 1 (1 q m 1 ) [(1 p n 1 )(1 q m 1 )] k. The total probability, P (R nbm ), is simply the sum of P 1 and P 2 : P (R nbm ) P 1 + P 2 p n [(1 p n 1 )(1 q m 1 )] k + qp n 1 (1 q m 1 ) [(1 p n 1 )(1 q m 1 )] k [p n + qp n 1 (1 q m 1 )] [(1 p n 1 )(1 q m 1 )] k pn + qp n 1 (1 q m 1 ) 1 (1 p n 1 )(1 q m 1 ) pn 1 (p + q(1 q m 1 )) p n 1 + q m 1 p n 1 q m 1 p n 1 (1 q m ) p n 1 + q m 1 p n 1 q. m 1 We can define Rnbm C analogously, and we can find P (RC nbm ) similarly. Thus, 39

49 we have P (R nbm ) p n 1 (1 q m ) p n 1 + q m 1 p n 1 q m 1, and P (R C nbm) q m 1 (1 p n ) p n 1 + q m 1 p n 1 q m 1. The derivation of the conditional probabilities leads to the following result. Theorem Let T n,m R nbm be the number of trials required to obtain a run of n successes given that we obtain such a run before a run of m failures. Then E[T n,m R nbm ] n + (1 pn )(q q m (m+q mq))+(1 q m )(1 q m 1 )(p p n (n+p np)) (1 q m )(p n q+pq m p n q m ). Also, if T n,m R C nbm is the number of trials required to obtain a run of m failures given that we obtain such a run before a run of n successes, then E[T n,m R C nbm ] m + (1 qm )(p p n (n+p np))+(1 p n )(1 p n 1 )(q q m (m+q mq)) (1 p n )(p n q+pq m p n q m ). Proof. As before, we consider the average in each of the two cases. To facilitate this, let us define conditional averages for the geometrically distributed interruptions. For X Geometric(p), we define e 1 E[X X m 1], and likewise, for Y Geometric(q), we define e 2 E[Y Y n 1]. Then for Case I we have A 1 p n [n + k(e 1 + e 2 )][(1 p n 1 )(1 q m 1 )] k, (4.7) 40

50 and for Case II we have A 2 qp n 1 (1 q m 1 ) We then obtain [n + e 1 + k(e 1 + e 2 )][(1 p n 1 )(1 q m 1 )] k. (4.8) We will now derive e 1 and e 2. Lemma Suppose X Geometric(p). Then E[T n,m R nbm ] A 1 + A 2 P (R nbm ). (4.9) E[X X m 1] 1 qm 1 (m+q mq). p(1 q m 1 ) m 1 k1 kp (k) Proof. We have E[X X m 1], and since X Geometric(p), P (X m 1) P (k) pq k 1 and P (X m 1) 1 q m 1. Thus we obtain E[X X m 1] m 1 k1 kpqk 1 1 q m 1 p[1 qm 1 (m + q mq)] p 2 (1 q m 1 ) 1 qm 1 (m + q mq) p(1 q m 1 ) Then, by Lemma 4.4.3, we have e 1 1 qm 1 (m + q mq) p(1 q m 1 ) 1 p n 1 (n + p np). Now consider A q(1 p n 1 1 from (4.7): ) and e 2 A 1 p n [n + k(e 1 + e 2 )][(1 p n 1 )(1 q m 1 )] k [ p n n[(1 p n 1 )(1 q m 1 )] k + ( ) 1 q m 1 (m + q mq) k [(1 p n 1 )(1 q m 1 )] k p(1 q m 1 ) 41

51 ( ) ] 1 p n 1 (n + p np) + k [(1 p n 1 )(1 q m 1 )] k q(1 p n 1 ) np n [(1 p n 1 )(1 q m 1 )] k + [ p n 1 (1 p n 1 )(1 q m 1 (m + q mq)) ] + pn q (1 qm 1 )(1 p n 1 (n + p np)) k[(1 p n 1 )(1 q m 1 )] k 1 np n 1 (1 p n 1 )(1 q m 1 ) + pn 1 q(1 p n 1 )(1 q m 1 (m + q mq)) q[1 (1 p n 1 )(1 q m 1 )] 2 + pn (1 q m 1 (1 p n 1 (n + p np)) q[1 (1 p n 1 )(1 q m 1 )] ( ) 2 1 (np n q(p n 1 + q m 1 p n 1 q m 1 ) q[p n 1 + q m 1 p n 1 q m 1 ] 2 +p n 1 q(1 p n 1 )(1 q m 1 (m + q mq)) +p n (1 q m 1 )(1 p n 1 (n + p np)) ). Consider A 2 from (4.8). A 2 qp n 1 (1 q m 1 ) [n + e 1 + k(e 1 + e 2 )][(1 p n 1 )(1 q m 1 )] k [ qp n 1 (1 q m 1 ) n[(1 p n 1 )(1 q m 1 )] k ( 1 q m 1 (m + q mq) + (k + 1) p(1 q m 1 ) ( 1 p n 1 (n + p np) + k q(1 p n 1 ) nqp n 1 (1 q m 1 ) 1 (1 p n 1 )(1 q m 1 ) + qp n 2 (1 q m 1 (m + q mq)) ) [(1 p n 1 )(1 q m 1 )] k ) [(1 p n 1 )(1 q m 1 )] k ] (k + 1)[(1 p n 1 )(1 q m 1 )] k + pn 1 (1 q m 1 )(1 p n 1 (n + p np)) 1 p n 1 42 k[(1 p n 1 )(1 q m 1 )] k