MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015

Size: px

Start display at page:

Download "MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015"

Merilyn Perry
5 years ago
Views:

1 ID NAME SCORE MATH 56/STAT 555 Applied Stochastic Processes Homework 2, September 8, 205 Due September 30, 205 The generating function of a sequence a n n 0 is defined as As : a ns n for all s 0 for which the series converges In particular, if 0 a n, then As is defined for all s [0, If X is a non-negative, integer-valued random variable and p n PX n, n 0, then P s : p ns n is called the probability generating function of X Note that P s is defined for all s [0, ] and P s E [s X ] a point Let q n PX > n, n 0, and let Qs q ns n be the generating function of q n n 0 Prove that Qs P s s, 0 s < b point Conclude that P Q E X possibly + where P denotes the left derivative of P at a Since q n n 0 is decreasing, hence bounded, the series q ns n converges for all s [0,, implying that Q is well defined on [0, Further, q n in+ PX i in+ p i, and therefore Qs p i s n i in+ s i s p i i0 s p i b Let s in the formula for Q Then lim Qs s i0 q n i i s n p i s i s p i p i s i i0 P s s PX > n E X, where both sides can be + On the other hand, P s lim Qs lim s s s lim s P s P s P

2 2 First passage decomposition Let X n n 0 be a Markov chain with transition matrix P p ij For j S let T j min{n > 0 : X n j} and define f n ij P i T j n Let P ij s p n ij sn, F ij s f n ij s n, be generating functions of p n ij n 0 and f n ij n 0 a point Justify the identity p n ij k f k ij pn k jj, n b point Deduce that P ij s δ ij + F ij sp jj s c point Prove from above that P i T i < if and only if pn ii gives an alternative prove of a part of dichotomy result from class a If X n j, then T j k for some k n Therefore p n ij P i X n j P i X n j, T j k k P i T j k, X Tj +n k j k This By the strong Markov property, conditional on X Tj j, X Tj +l : l 0 is δ j, P - Markov chain independent of F Tj that is X 0, X,, X Tj Since X Tj j on T j k n, it follows that X Tj +l : l 0 is δ j, P -Markov chain independent of X 0, X,, X Tj Therefore, P i T j k, X Tj +n k i P i T j kp j X n k i f k ij pn k jj, b by definition f 0 ij 0 P ij s δ ij n k0 p n ij sn f k ij k0 k0 nk n k pn k jj s n p n k jj s n k f k ij pn k jj s n f k ij sk P jj sf k ij sk P jj sf ij s, where in the second line we used f 0 ij the order of summation 0, and the third line follows by interchanging 2

3 c Note that lim s F ii s f n ii b we have By letting s, it follows that P i T i < and lim s P ii s P ii s p n ii F ii s P i T i <, which proves that pn ii if and only of P i T i < pn ii From 3 Let Y i i be a sequence of iid random variables with PY p, PY q p Define the simple random walk S n n 0 as S 0 0, S n Y + + Y n, n Let T min{n > 0 : S n } be the hitting time to, f n PT n, n 0, and F s f ns n a point Justify the equation n 2 f n qf k f n k, n 2 b 2 points Deduce that F s ps qsf 2 s and conclude that k c point Derive from the above that F s pqs 2 2qs PT < d point Finally, prove that p q 2q {, p q, p/q, p < q { +, p q, E T, p > q p q a First note that f 0 0 and f p the first step must be to the right Let T 0 min{n > 0 : S n 0} and note that since S n n 0 is spatially homogeneous, we have that P T 0 k P 0 T k f k P 0 P Let n 2 If T n the first step must 3

4 be to the left Therefore, f n PT n, X qpt n X n 2 qp T n q P T n, T 0 k k n 2 q P T 0 kp T n T 0 k k n 2 q P T 0 kp 0 T n k k n 2 q f k f n k k Here the second and the penultimate lines follow by the strong Markov property of the random walk b Use f 0 0, f s, and part a for f n, n 2, to get F s ps + ps + ps + ps + ps + f n s n n2 n 2 qf k f n k s n n2 k n 2 qf k f n k s n n2 k0 k0 nk+2 f n k s n k f k s k qs f m s m f k s k qs k0 m F sf k s k qs ps + qsf s 2 k0 By solving the quadratic equation we get c + pqs Since lim 2 s 0 qs F s ± pqs 2 qs, but F 0, the solution with the + sign is impossible PT < F pq 2q { p q, p q, 2q p/q, p < q

5 d If p < q, then PT > 0, and therefore E T For p q we use Problem to calculate E T F We first find F s and then compute F to get F 2p p q p q 2q { +, p q,, p > q p q 2 points Let X n n 0 be a Markov chain on 0,, with transition probabilities given by α i + p 0, p i,i+ + p i,i, p i,i+ p i,i, i, i where α 0, For each α compute the value of Plim n X n First note that X is irreducible Exactly as in Problem 0 of Homework, we compute that P 0 X n for all n j α Since j converges only for α >, we see that the above probability is strictly positive j α for α > and equal to 0 for α 0, ] Hence, we deduce that P 0 T 0 < < for α >, and P 0 T 0 < for α In the first case the process is transient, hence P i lim n X n for all i 0, and consequently the required probability is also In the second case, α, the process is recurrent, hence the required probability is equal to points The rooted binary tree is an infinite graph T with one distinguished vertex R from which comes a single edge; at every other vertex there are three edges and there are no closed loops The random walk on T jumps from a vertex along each available edge with equal probability Show that the random walk is transient Let X n n 0 denote the random walk on T Denote by d the path distance on T : dv, w the length of the shortest path connecting v and w Let Y n dx n, R - distance of X n to the root R Then Y n n 0 is a random process with state space Z + {0,, 2, } It follows from the structure of the tree T that Y n n 0 is Markov chain Moreover, since at any vertex v R, the random walk will move away from the root with probability 2/3 and move towards the root with probability /3 there is only one neighbor of v leading to R, the transition probabilities are p i,i+ 2/3, p i,i /3, i, and obviously p 0 It was shown in class, cf Example 33 from the textbook, that Y n n 0 is transient In fact, it is shown in the example that, for the chain Y, P i T 0 < /3/2/3 i 2 i Therefore, P 0 T 0 < P T 0 < /2 6 2 points Find all invariant distributions of the transition matrix P j 5

6 The linear system π πp reads π 2 π + 2 π 5 π 2 2 π 2 + π π 3 π 3 + π π 2 π 2 + π π 5 2 π 2 + π + 2 π 5 The third equation gives π 0, the second π 2 π 0 and the first π π 5 Hence π α/2, 0, α, 0, α/2 with 0 α The solution can be guessed by looking at communicating classes: the class {2, } is transient, hence all mass eventually escapes Two other classes, {, 5} and {3} are recurrent, so any stationary distributions is a convex combination of stationary distributions for these two classes: π α/2, 0, 0, 0, /2 + α0, 0,, 0, 0 7 A particle moves on the eight vertices of a cube in the following way: at each step the particle is equally likely to move to each of the three adjacent vertices, independently of its past motion Let i be the initial vertex occupied by the particle, o the vertex opposite i Calculate each of the following quantities: a point the expected number of steps until the particle returns to i; b point the expected number of visits to o until the first return to i; c point the expected number of steps until the first visit to o a First note that by symmetry the invariant measure π is equal to 8 Hence E i T i π i 8 at every vertex b From class T i E i Xno ν o E i T i π o c Denote by x vertices adjacent to i and by y vertices adjacent to o Let h i E i T o, h x E x T o and h y E y T o By the first step analysis and symmetry h i + h x h x + 3 h i h y h y h x The solution of this system is h i 0, h x 9, h y 7 6

7 8 2 points Find all invariant measures for the asymmetric random walk X X n n 0 on Z with transition probabilities p i,i q, p i,i+ p, p + q p q Does uniqueness up to scalar multiplication hold? Does X have an invariant distribution? The invariant measure λ satisfies λ λp or in components λ i λ i p + λ i+ q The general solution of this recurrence relation is λ i A + B i p, A, B 0 q By choosing A, B 0 and A 0, B we see that uniqueness up to scalar multiplication does not hold Since both series i Z and i Z p/qi diverge, an invariant distribution does not exist 7

Math Homework 5 Solutions

Math Homework 5 Solutions Math 45 - Homework 5 Solutions. Exercise.3., textbook. The stochastic matrix for the gambler problem has the following form, where the states are ordered as (,, 4, 6, 8, ): P = The corresponding diagram