arxiv: v2 [math.pr] 9 Sep 2017

Transcription

1 Urn models with two types of strategies Manuel González-Navarrete and Rodrigo Lambert arxiv: v2 [math.pr] 9 Sep 2017 Abstract We introduce an urn process containing red and blue balls U n = (R n, B n ), n N 0. We assume that there are two players employing different strategies to add the new balls. Player A uses a generalized Pólya-type strategy, instead player B employs an i.i.d strategy choosing the color with the same probability all the time, without looking the current urn composition. At step n, player A is chosen randomly with probability θ n. We study the behaviour of this family of urn models with two strategies, as n, proving a law of large numbers and a central limit theorem for the proportion of red balls into the urn. Keywords: Pólya-type processes, memory lapses, law of large numbers, central limit theorem. 1 Introduction The urn models are a very well-exploited theme in probability theory. Its literature is extensive and an essential survey of this issue can be found in the book of Mahmoud [21]. The applications of urn processes are related to analysis of algorithm and data structures, dynamical models of social networks and evolutionary game theory, among others [19, 21, 22]. One of the earliest work was the paper by Eggenberger and Pólya [4]. This so-called Pólya urn is stated as follows. An urn starts with an initial quantity of R 0 red and B 0 blue balls and draws are made sequentially. After each draw, the ball is replaced and other a balls of the same color are added to the urn. The interest is in the current composition of the urn, for each time n N. We consider the two-dimensional vector U n = (R n, B n ), where R n and B n represent the number of red and blue balls at time n, respectively. Posteriorly, Friedman [8] generalized such urn model and then several extensions have been studied (for details, see [21]). We highlight here the Bagchi-Pal mechanism, which will be used in this work (for more details on this model, see [2]). The model is represented, as usual, by the so-called reinforcement or replacement matrix R B ( a b c d ). (1.1) In that case, R-row denotes the balls added if the chosen color is red. This means that we put a red and b blue balls. If a blue-colored ball is obtained we proceed to the B-row. It indicates that we add c red and d blue balls into the urn. As in [2], we suppose that the urn is balanced, which means that a + b = c + d. Therefore, at each step we add a fixed number of balls. In general, the interest is the proportion of red balls, R n /(R n +B n ), as n. There are several approaches to address this problem. For instance, the use of generating functions and convergences

2 of moments [7, 18, 21], embedding the urn into continuous-time branching processes [1, 15, 22], ideas based on convergence results for martingales [10, 11, 20] or combinatorial analytic and algebraic approaches [6, 23]. In this work we consider the possibility of having different mechanisms of adding the balls into the urn at each step. In other words, we imagine there are two players designed to reinforce the urn with new balls. In particular, suppose that one of them is a good worker employing a generalized Bagchi-Pal scheme. On the other hand, second worker is a carefree one. Then with probability p and independently of the current composition of the urn he (she) chooses color blue (B-row in (1.1)), otherwise he uses R-row. Moreover, the player who will play at time n is chosen according to a Bernoulli sequence with probability of success θ n. That is, at each step the player A is chosen with probability θ n and player B with probability (1 θ n ). We study the influence of the strategies in the asymptotic behaviours of the model. Moreover we complement the discussions in González-Navarrete and Lambert [9] analysing the memory lapses property in applications of urn models (Definition 3.1). Namely, a memory lapse is a sequence of consecutive replacements done by Player B. That is, by an i.i.d strategy, without taking the composition of the urn into account. The problem is addressed by considering two approaches. The first one is a martingale theory approach, based on [10, 12]. The second follows the ideas introduced by Janson [15, 16], which allow us to characterize convergence of moments of R n, by interpreting the urn in Section 2 as an urn with random reinforcement matrix. The main results include convergence theorems for the proportion of red balls in the urn, namely R n /(R n + B n ). The first one is a strong law of large numbers, and the second is a central limit theorem (CLT). As a corollary, we obtain a CLT for a process that we call knight random walk. The rest of this paper is organized as follows. Section 2 states the model and presents the main results. In section 3 we introduce two auxiliary processes, used to analyse the law of random variable R n. The proofs for the results are provided in Section 4. 2 The urn process and main results Imagine a discrete-time urn process with initial R 0 red and B 0 blue balls. The composition of the urn at time n N is given by (R n, B n ), where R n means the number of red and B n the number of blue balls. The quantity of balls into the urn is T n = Kn + T 0, where K = a + b from matrix (1.1) and T 0 = R 0 + B 0. Assume there exist two players which follow different strategies. One of them looks to the urn composition, and the other does not. Nonetheless, both players employ the replacement matrix (1.1), each of them following a distinct rule. Now, we describe the dynamics of this process. Player A follows a (generalized) Pólya-type regime. That is, she (he) draws a ball uniformly at random, observe its color and put it back to the urn. Then she chooses with probability p the same color and with probability 1 p the opposite color. Finally, she uses the replacement matrix (1.1). Player B behaves simpler. He (she) chooses a color, say blue or red, with probability p and 1 p, respectively. Then, he puts the balls into the urn following (1.1). We remark that player B behaves independent of the current urn composition. The player choice is made as follows. At each step, a coin is flipped, and the result says who will add the next balls into the urn. Particularly, player A is chosen with probability θ n. In this sense, this process can be interpreted as an urn process with interferences. In applications, this phenomena for instance, could be related to an environmental factor that remove the dependence on the current composition of the system. In Section 3.1 we will complement these ideas. 2

3 We remark some particular cases in the literature. These can be obtained by taking specific values of the parameters. Letting θ n = 1, for all n 1, the player A is always chosen and if p = 1 we have the original Bagchi-Pal urn. The case p = 0 is like a color blindness situation where we choose the opposite color, and the replacement matrix (1.1) changes its rows to obtain again the Bagchi-Pal mechanism. Moreover, in the case θ n = 0, n 1, player B is always chosen and we have a well-studied class of i.i.d models with p (0, 1). We refer Figure 1 for an illustration of these cases. Note that for p = 1/2 we have a class of symmetric random walks with steps of different length. This means that does not matter which strategy is employed. It happens because always with probability 1/2 we choose (a, b) balls to reinforce the urn. Figure 1: Characterization of the models included in the urn process above. See the behaviours of the limit in (2.1) and the matrix (3.7) One example of reinforcement matrix (1.1) can be interpreted as an analogy with a random walk done by chess knight piece. The unusual moves, compared with other chess pieces, can be analysed in the positive quadrant of the two-dimensional Cartesian system. We realize the evolution by plotting (R n, B n ), at each step we move in the direction of one of the vectors r = (2, 1) or b = (1, 2). For instance, let R 0 = B 0 = 1 and see Figure 2 for details. We call this process the knight random walk (KRW). As we will proof in Theorem 2.1, for instance if p = 1/2, the random variable R n /T n converges almost surely to 1/2, as n goes to. That is, the KRW fluctuates around the green dotted line in Figure Main results As usual, we restrict the attention to a class of well-behaved urn processes. In this line, we assume the urn is tenable, i.e., that it is impossible to get stuck. Formally, assume that (a) T 0 > 0. (b) T n = Kn + T 0, for some K 1. (c) The urn R n is not deterministic, that is a c. (d) All entries in (1.1) are positive. We recall that assumption (b) says that the urn is balanced, and assumption (d) is stated in order to simplify the analysis. However the case of (possible) non-positive entries can be studied by using similar tools. 3

4 Figure 2: Space-state of the knight random walk starting at (R 0, B 0 ) = (1, 1). The gray dashed lines represent the deterministic cases, that is, θ n = 0, for all n 1 and p {0, 1}. Now we are ready to present the results about asymptotic proportion of red balls in the urn. In what follows we present the main results about convergence of the random variable R n. The first one is a strong law of large numbers. Theorem 2.1. Let a tenable urn process {U n } n 1 as defined above. If lim n θ n exists, say θ [0, 1]. Then, for θ(2p 1)(a c) < K R n lim = n T n pc + (1 p)a K + θ(2p 1)(c a) a.s (2.1) It is also possible to obtain versions of the central limit theorem. A general form can be stated as follows. Theorem 2.2. If 2θ(2p 1)(a c) < K, then R n E(R n ) n d N(0, σ 2 ), as n, (2.2) where σ is given by expression (4.10) and d means convergence in distribution. The next result is an immediate consequence of previous theorem. It provides a CLT for the KRW defined above. Corollary 2.3. Let a = d = 2 and b = c = 1. Then, for all θ(2p 1) < 3 4 and p 1 2, where γ = R n 3(2 p) 3 θ(2p 1) n n d N ( 0, γ 2 ), as n, (2.3) θ(2p 1) 3(2 p)(1 (θ(2p 1) p)). 3 θ(2p 1) 3 2θ(2p 1) The proofs of these results will be given in Section 4. In order to obtain the proofs, in next section we introduce two forms to analyse the family of urn models defined above. 4

5 3 Auxiliary constructions In this section we introduce related models. Namely, a class of dependent Bernoulli sequences with memory lapses, which allows us to analyse the urn model above by a martingale approach. The other case is defined by a family of urn processes with random reinforcement matrix, giving tools to obtain the convergence of the mean and variance of R n. 3.1 Memory lapses in Bernoulli sequences and urn models We define the so-called Bernoulli sequences with random dependence (BSRD). That models were proposed by González-Navarrete and Lambert [9]. In such work the property of memory lapses and several applications were discussed. Consider a sequence of Bernoulli trials {X i, i 1} and an auxiliary collection of independent Bernoulli random variables {Y i, i 0}, with a given probability and also independent of S i = X X i, for all i 1. The dependence structure of {X i } (for short notation) will be associated to the sequence {Y i }, which we call memory switch sequence. Let denote P i (s, y) = P(X i+1 = 1 S i = s, Y i = y), (3.1) for all i 0, 0 s i and y {0, 1}. We remark that, for y = 0 the probability P i (s, 0) will not depend on the previous successes S i. However, if y = 1, the dependence exists. We think random variable Y i as a latent factor that determines the choice of dependence (or independence) on the past S i for the trial X i+1. In this case, this relation is explicitly given in (3.3). The urn process in previous section is related to BSRD in the following sense. At each step n 1, we denote X n = 1 if the color used is the red (or the R-row from (1.1)), otherwise X n = 0. In that analogy, we denote Y n 1 = 1 if the chosen player at n-step was A, where P(Y i = 1) = θ i, for i 0. Then, we have for all n 0, where S 0 = 0. In addition, by notation (3.1), we know that R n = as n + c(n S n ) + R 0, (3.2) ( cn + R0 P n (s, y) = 1 p + (2p 1)y + T n ) (a c)s, (3.3) T n for all n 0, y = 0, 1 and 0 s n. Parameters a, c, p and R 0 are from the urn model in Section 2. At this point, we stress that the family of dependent Bernoulli sequences defined in [9] can be classified, for instance, by the pattern behaviours of the memory lapses. A memory lapse is a period in which the decisions are done without taking into account the history of the process. Its definition is given as follows. Definition 3.1. A memory lapse in the BSRD {X i } is an interval I N such that Y i = 0 for all i I and there is no interval J I such that Y i = 0 for all i J. The length of the lapse is given by l = I. In some sense, we think this period as a lapse because after these, the model always will recover the dependence on the whole past. This includes the consequences of the decisions taken in the memory lapse period. In view of the problem of this paper, it means that the law of player A is influenced by player s B choices (provided that player B assume the game for a period). In this direction, it is natural to ask about the behaviour of the memory lapses on the process. 5

6 Therefore, we define the random variables M l (n), l {1, 2,..., n}, representing the number of memory lapses of length l in the first n trials of a BSRD sequence {X i }. Formally, we address that question by the analysis of the switch sequence Y i. Then, define M l (n) = n l 1 k=1 Y k (1 Y k+1 ) (1 Y k+l )Y k+l+1 + (1 Y 1 ) (1 Y l )Y l+1 + Y n l (1 Y n l+1 ) (1 Y n ), (3.4) for n > l, and where second and third terms represent the possibility of having a memory lapse at the beginning and at the end of the sequence, respectively. Note that, as defined by Feller [5] in the case θ i = θ for i 1, the random variable M l (n) is the number of runs of 0 s of length l in the first n trials of an independent Bernoulli sequence {Y i }. Then, we have E(M l (n)) = (1 θ) l [2θ + θ 2 (n l 1)], n > l, (3.5) and E(M n (n)) = (1 θ) n. In the literature there exist general results about functionals of random variables M l (n), for example, see Holst and Konstantopoulos [14]. We remark that a memory lapse of length l should be understood as a period of l consecutive replacement in the urn done by player B. We refer to the reader Figure 1 as an illustration of this construction. The relation between Bernoulli sequences and urn processes has been provided by (3.2). This leads us to the the next step. It consists in analysing the behaviour of S n by a martingale approach given by [12]. A similar idea was used to provide limiting results for BSRD in [9]. Then, we are able to provide convergence for the random proportion R n /T n in the urn model. We note that the martingale approach was also used in the class of urn processes analysed in [3], which was related to non-markovian random walks (for details, see [9]). Although this approach is a powerful mathematical tool, in practical terms the hard calculations are related to the values of E(S n ), usually done by iterations, see a detailed example in [13]. Regarding the asymptotic of the mean and variance of R n we need to introduce the approach proposed by Janson in [15] and complemented in the recent works [16, 17]. In particular, we will use such algebraic approach as a tool to obtain E(R n ). However, the theorems in [15] are much more stronger than stated here. 3.2 Urn models with random reinforcement matrix In the paper [15], Janson established a powerful approach to obtain functional limit theorems for branching processes, which can be applied in the study of Pólya-type urns. In this sense, recent works [16, 17] were dedicated to the analysis of the moments of random variables (R n, B n ) in the context of a two color urn model. The particularity of such urns is the replacement matrix. In that case, the rows are given by random vectors. In other words, at each step the number of balls added into the urn are given by the realization of random variables. We call it a random replacement matrix. In this sense, let the random vectors ξ i = (ξ i1, ξ i2 ), for i = 1, 2, which represent the rows of the random replacement matrix. That is, if we pick a red ball, then we add to the urn a random number of balls, sampled from the vector (ξ 11, ξ 12 ), otherwise picking a blue ball we add a sample of the random vector (ξ 21, ξ 22 ). Therefore, we aim to obtain the distribution of random variables ξ ij, i, j = 1, 2 in such a way that the associated urn model, with random reinforcement matrix, has the same law than the urn process defined in Section 2. Then, assume that ξ ij {0, K}, for i, j = 1, 2, and let 6

7 c + (a c) P(ξ 11 = K) = (θ(2p 1) + (1 p)), K c + (a c) P(ξ 21 = K) = (1 p), K P(ξ 12 = K) = P(ξ 11 = 0) and (3.6) P(ξ 22 = K) = P(ξ 21 = 0). As a consequence, it is proved that the moments of R n for the urn model in Section 2 are equal to those obtained by the urn process with random reinforcement matrix defined by random variables {ξ ij } with laws (3.6). After that, we are able to use the results of Janson [16], that is, to obtain E(R n ) and E(Rn), 2 we define the matrix ( ) E(ξ11 ) E(ξ A = 12 ), (3.7) E(ξ 21 ) E(ξ 22 ) and then we analyse its eigendecomposition. That is, we obtain eigenvalues (namely λ 1, λ 2 ) and its correspondent eigenvectors (namely v 1, v 2 ). The main results (detailed in page 7 of Janson [16]) can be summarized as follows Theorem 3.2. Let a Pólya urn being tenable and balanced, (i) If Reλ 2 < λ 1, then E(R n, B n ) = nλ 1 v 1 + o(n). (3.8) (ii) If the urn is strictly small, i.e. Reλ 2 < 1 2 λ 1, then V ar(r n, B n ) n where Reλ means the real part of λ and Σ I is given in (2.15) of [16]. Σ := λ 1 Σ I, (3.9) This theorem is an adaptation of main results of [16]. We recall that in that paper the moments convergence was stated for urn processes with q colours (types), where 2 q <. In this sense, with respect to Σ I, to proof Theorem 2.2 we need just one element of this matrix. Therefore the calculations to obtain σ are simpler, as expressed in the proofs. A particular case of the referred result (Theorem 2.2) can be found in [3]. In that work the authors studied the asymptotics for a non-markovian random walk. This process is obtained by setting θ = 1, a = d = 1 and b = c = 0. The results follow then from Janson [15]. In next section we will explain how these two auxiliary processes allow us to proof the main results. Namely, using the martingale approach based on the BSRD or applying the algebraic tools developed by Janson [15, 16] for urn models with random reinforcement matrix. 4 Proofs One possibility is to exploite the relation in Section 3.1 and use the tools by martingale theory, see Hall and Heyde [12]. In this sense, let denote a 1 = 1 and a n = n 1 k=1 ( 1 + θ ) k(2p 1)(a c) T k 7 for n 2, (4.1)

8 now, we define a discrete-time martingale given by M n = S n E(S n ) a n. (4.2) In other words, if we denote D 1 = M 1 and D n = M n M n 1, n 2, it is possible to prove that {D n, F n, n 1} are bounded martingale differences. In particular, note from (4.2) that E[M n+1 F n ] = 1 a n+1 ( Sn E(S n ) + E[X n+1 F n ] E(X n+1 ) ). (4.3) As done in Section 5 of [9], we obtain E[X n+1 F n ] = 1 p + θ n (2p 1)R n and E(X n+1 ) = 1 p + θ n (2p 1)E(R n ). Then, M n is a martingale. In addition, observe that for all n 2 D n = X n E(X n ) S n 1 E(S n 1 ) a n T n 1 β n 1 a n, (4.4) and D n 2 a n, n 1. Then, by Theorem 2.17 and Corollary 3.1 included in Hall and Heyde [12], we have that (i) If n=1 E[D2 n F n 1 ] < a.s., then n i=1 D i converges almost surely. (ii) Assume that there exists a sequence of positive constants {W n } such that W n as n and 1 n Wn 2 j=1 E[D2 j F j 1] p n σ 2 j=1. Then D j d W n N(0, σ 2 ). Therefore, by obtaining the moments of S n and using the relation in (3.2) we are able to complete the proofs of (2.1) and (2.2). However, we use the approach given by Janson [15] to characterize the convergence of the moments of R n directly and then complete the proof. Let denote λ 1 and λ 2 the eigenvalues of matrix A in (3.7), a general form is given by and λ 1 = E(ξ 11) + E(ξ 22 ) + E(ξ 11 ) 2 + 4E(ξ 12 )E(ξ 21 ) 2E(ξ 11 )E(ξ 22 ) + E(ξ 22 ) 2 2 λ 2 = E(ξ 11) + E(ξ 22 ) E(ξ 11 ) 2 + 4E(ξ 12 )E(ξ 21 ) 2E(ξ 11 )E(ξ 22 ) + E(ξ 22 ) 2, (4.6) 2 where, from (3.6) we get (4.5) E(ξ 11 ) = c + (a c)(θ(2p 1) + (1 p)), E(ξ 12 ) = K E(ξ 11 ), E(ξ 21 ) = c + (a c)(1 p) and E(ξ 22 ) = K E(ξ 21 ). Finally, by a straightforward calculation we obtain (4.7) λ 1 = K and λ 2 = θ(a c)(2p 1). (4.8) Note that, λ 2 is actually the difference E(ξ 11 ) E(ξ 21 ), and λ 1 is given by E(ξ i1 ) + E(ξ i2 ), i = 1, 2. Proof. (Theorem 2.1) Let Λ = λ 2 λ 1, using Theorem 3.2 we have E(R n ) = n E(ξ 21) 1 Λ = nk pc + (1 p)a K + θ(2p 1)(c a). (4.9) Since T n = nk + T 0 and by the result above obtained with the martingale approach, then (2.1) holds. 8

9 Proof. (Theorem 2.2) The central limit theorem for the process in Section 3.2 was stated in [15] (see Theorems and Lemma 5.4), showing that if the urn is small, then R n is asymptotically normal, with the asymptotic covariance matrix equal to the limit in Theorem 3.2. In our case, we obtain using (4.7) the theorem holds. σ = Λ E(ξ21 )E(ξ 12 ) 1 Λ 1 2Λ, (4.10) Proof. (Corollary 2.3) By letting a = d = 2 and b = c = 1, we obtain Then, E(ξ 11 ) = 2 + (θ(2p 1) p) and E(ξ 21 ) = 2 p. (4.11) E(R n ) = 3(2 p) n. (4.12) 3 θ(2p 1) Moreover, from (4.10) we obtain θ(2p 1) 3(2 p)(1 (θ(2p 1) p)) γ =. (4.13) 3 θ(2p 1) 3 θ(2p 1) Acknowledgements MGN is supported by Fundação de Amparo à Pesquisa do Estado de São Paulo, FAPESP (grant 2015/ ). RL is partially supported by the project Statistics of extreme events and dynamics of recurrence (FAPESP Process 2014/ ). He kindly thanks Edmeia da Silva, for her teachings in linear algebra. References [1] Athreya, K. and Karlin, S. (1968) Embedding of urn schemes into continuous time Markov branching processes and related limit theorems. Ann. Math. Statist. 39, [2] Bagchi, A. and Pal, A.K. (1985) Asymptotic normality in the generalized Pólya-Eggenberger urn model, with an application to computer data structures. SIAM J. Algebraic Discrete Methods 6, [3] Baur, E. and Bertoin, J. (2016) Elephant random walks and their connection to Pólya-type urns. Phys. Rev. E 94, [4] Eggenberger, F. and Pólya, G. (1923) Mech. 3, Über die Statistik verketteter Vorgänge. Z. angew. Math [5] Feller, W. (1968) An Introduction to Probability Theory and Its Applications, Vol. 1. John Wiley, New York. [6] Flajolet, P., Gabarró, J. and Pekari, H. (2005) Analytic urns. Ann. Probab. 33,

10 [7] Freedman, D.A. (1965) Bernard Friedman s urn. Ann. Math. Statist. 36(3), [8] Friedman, B. (1949) A simple urn model. Comm. Pure Appl. Math. 2, [9] González-Navarrete, M. and Lambert, R. (2016) On dependent Bernoulli sequences with memory lapses. Submitted to Statis. Probab. Lett. arxiv: [10] Gouet, R. (1989) A martingale approach to strong convergence in a generalized Pólya- Eggenberger urn model. Statis. Probab. Lett. 8, [11] Gouet, R. (1993) Martingale functional central limit theorems for a generalized Pólya urn. Ann. Probab. 21, [12] Hall, P. and Heyde, C.C. (1980) Martingale limit theory and its application. Academic Press, New York. [13] Heyde, C.C. (2004) Asymptotics and criticality for a correlated Bernoulli process. Aust. N. Z. J. Statist. 46, [14] Holst, L. and Konstantopoulos, T. (2015) Runs in coin tossing: a general approach for deriving distributions for functionals. J. Appl. Probab. 52, [15] Janson, S. (2004) Functional limit theorems for multitype branching processes and generalized Pólya urns. Stoch. Proc. Appl. 110(2), [16] Janson, S. (2016) Mean and variance of balanced Pólya urns. arxiv: [17] Janson, S. and Pouyanne, N (2016) Moment convergence of balanced Pólya processes. arxiv: [18] Johnson, N.L. and Kotz, S. (1976) Two variants of Pólya s urn models (towards a rapprochement between combinatorics and probability theory). Am. Stat. 13, [19] Johnson, N.L. and Kotz, S. (1977) Urn Models and Their Application. Wiley, New York. [20] Kuba, M. and Sulzbach, H. (2017) On martingale tail sums in affine two-color urn models with multiple drawings. J. Appl. Probab. 54(1), [21] Mahmoud, H. (2008) Pólya urn models. CRC Press. [22] Pemantle, R. (2007) A survey of random processes with reinforcement. Probab. Surveys 4, [23] Pouyanne, N. (2008) An algebraic approach to Pólya processes. Ann. Inst. Henri Poincaré Probab. Stat. 44(2), Instituto de Matemática e Estatística, Universidade de São Paulo, SP, Brasil. address: manuelg@ime.usp.br Faculdade de Matemática, Universidade Federal de Uberlândia, MG, Brasil. address: rodrigolambert@ufu.br 10