Lecture 6. 2 Recurrence/transience, harmonic functions and martingales
|
|
- Hope Foster
- 6 years ago
- Views:
Transcription
1 Lecture 6 Classification of states We have shown that all states of an irreducible countable state Markov chain must of the same tye. This gives rise to the following classification. Definition. [Classification of irreducible countable state Markov chains] An irreducible Markov chain is called transient, res. null recurrent or ositive recurrent, if all its states are transient, res. null recurrent or ositive recurrent. Recall that τ y := inf{n : X n = y} and ρ xy = P x (τ y < ). The restriction to irreducible Markov chains is artly justified by the following result. Theorem.2 [Decomosition theorem] Let R = {x S : ρ xx = } be the set of recurrent states of a countable state Markov chain. Then R can be decomosed as the disjoint union of R i, each of which is irreducible and closed in the sense that if x R i and ρ xy > 0, then y R i. For each i, the states in R i are either all null recurrent or all ositive recurrent. Proof. If x R and ρ xy > 0, then we have shown that y R and ρ yx =. If ρ xy > 0 and ρ yz > 0, then also ρ xz ρ xy ρ yz > 0. Therefore the relation x y for x and y with ρ xy > 0 defines an equivalence relation on R. The equivalence classes {R i } are what we seek. Restricted to each R i, we have an irreducible Markov chain whose states are all of the same tye. Examle.3 The symmetric simle random walk on Z is recurrent, while the asymmetric simle random walk on Z is transient. 2 Recurrence/transience, harmonic functions and martingales We have shown that the Green s function, G(x, y) = n=0 P x(x n = y), of an irreducible countable state Markov chain (X n ) n 0 is finite if and only if the chain is transient. This rovides one way of determining the transience or recurrence of an irreducible Markov chain. As an examle, we consider the simle symmetric random walk on Z d defined by X 0 = x Z d and X n = X 0 + n i= ξ i, where ξ i are i.i.d. with P(ξ = e) = 2d for each of the 2d unit vectors e Z d. Theorem 2. [Simle symmetric random walk on Z d ] The simle symmetric random walk on Z d is recurrent in dimensions d =, 2, and transient in dimensions d 3. Proof. We will comute the Green s function for the simle symmetric random walk (X n ) n 0 on Z d using Fourier transform. Clearly X is an irreducible Markov chain on Z d. Since all states are of the same tye, we may assume X 0 = 0, the origin. By translation invariance,
2 we can let n (x, y) := n (y x) denote the n-ste transition robability of the random walk. Then for x Z d, n (x) = (x), x,,x n Z d (x ) (x 2 x ) (x x n ) = n where n denotes the n-fold convolution of. For any k [ π, π] d, n has Fourier transform ˆ n (k) = x Z d e i<k,x> n (x) = φ n (k), where φ(k) = d d i= cos k i is the Fourier transform of. By Fourier inversion, n (0) = (2π) d φ n (k)dk, [ π,π] d and hence G(0, 0) = n (0) = n=0 n=0 (2π) d φ n (k)dk = [ π,π] d (2π) d [ π,π] d φ(k) dk. Note that φ(k) = if and only if k = 0, and φ(k) = k 2 2d + o( k 2 ) where k denotes the Euclidean norm. For k > ɛ and k [ π, π] d, φ(k) is uniformly bounded. Therefore to determine whether G(0, 0) is finite, we can relace φ(k) by its Taylor exansion at 0, and consider 2d (2π) d k 2 dk = 2d ɛ (2π) d r d 3 dr, k ɛ which is finite if and only if d 3. Therefore G(0, 0) < and the simle symmetric random walk is transient if and only if d 3. 0 Exercise 2.2 Prove that if the random walk increments ξ i = X i X i, i N, are i.i.d. with E[ξ i ] = 0 and E[ ξ i 2 ] <, then the walk on Z d is also recurrent in dimensions d = and 2, and transient in dimensions d 3. On the other hand, if E[ξ i ] 0, then the walk is always transient regardless of the dimension. In general, it is not ossible to comute exlicitly the Green s function of a Markov chain. An alternative aroach is by finding suitable non-negative (suer) martingales. Let us first illustrate this technique by a concrete examle. Examle 2.3 [Recurrence of a simle symmetric random walk on Z] Let X n = X 0 + n i= ξ i be a simle symmetric random walk on Z where ξ i are i.i.d. with P(ξ i = ±) = 2. To rove recurrence, it suffices to show that P (τ 0 < ) =, where P denotes robability law for X with X 0 =, and τ 0 = inf{n : X n = 0}. Note that X n is a martingale, and X n τ0 τ L is a bounded martingale for any L N. Since τ 0 τ L < almost surely, by the otional stoing theorem, X 0 = = E [X τ0 τ L ] = LP (τ L < τ 0 ), and hence P (τ 0 < τ L ) = L. Let L, we deduce that P (τ 0 < ) =. The crucial fact we use here is that f(x n ), where f(x) = x, is a martingale, and f is non-negative on [0, ) and increases to at. Note that the roof still goes through if f(x n ) is a suer-martingale instead of a martingale. 2
3 Examle 2.4 [Transience of an asymmetric simle random walk on Z] Let X n = X 0 + n i= ξ i where ξ i are i.i.d. with P(ξ i = ) = P(ξ i = ) = for some > 2. To rove transience, it suffices to show that P (τ 0 < ) <. Note that f(x n ) := ( )Xn is a martingale. For X 0 =, τ 0 τ L < almost surely for any L N, therefore, [( ) Xτ0 ] ( τ L ) LP = E = P (τ 0 < τ L ) + (τ L < τ 0 ) P (τ 0 < τ L ). Let L, we deduce that P (τ 0 < ) <, thus roving transience. The crucial fact we use here is that f(x n ), where f(x) = ( )x, is a non-negative suer-martingale, and f(0) > f(). We now generalize the above roof aroach to general irreducible Markov chains. Definition 2.5 [Harmonic functions for a Markov chain] Let X be a time-homogeneous irreducible Markov chain with countable state sace S and one-ste transition robability matrix Π(x, y). A function f : S R is said to be harmonic for X at x S if (Πf)(x) = y S Π(x, y)f(y) = f(x), (2.) and sub-harmonic, res. suer-harmonic, at x if f(x) (Πf)(x), res. f(x) (Πf)(x). If f is (sub/suer)-harmonic at all x S, then f is said to be (sub/suer)-harmonic for X. Lemma 2.6 If f is a harmonic function for an irreducible Markov chain X, then f(x n ) is a martingale. If f is sub/suer-harmonic, then f(x n ) is a sub/suer-martingale. Proof. Let (F n ) n 0 be the canonical filtration generated by (X n ) n 0. By the Markov roerty, E[f(X n+ ) F n ] = E[f(X n+ ) X n ] = y S Π(X n, y)f(y) = f(x n ). Therefore f(x n ) is a martingale. The case when f is sub/suer-harmonic is similar. Theorem 2.7 [Sufficient condition for recurrence] Let X be an irreducible Markov chain with countable state sace S. If there exists a finite set F S and a function φ : S [0, ) such that φ is suer-harmonic for X at all x S\F, and φ(x) as x in the sense that the level set {x S : φ(x) M} is finite for all M > 0, then X is recurrent. Proof. Let the Markov chain start from x 0 S\F. Let τ F = inf{n : X n F }, and let τ M = inf{n : φ(x n ) > M} for any M > φ(x 0 ). By our assumtion on φ, φ(x n τf τ M ) is a suer-martingale. Therefore φ(x 0 ) E[φ(X n τf τ M )] MP(τ M < n τ F ). (2.2) Note that because {x : φ(x) M} is a finite set, by the irreducibility of the Markov chain, τ M < almost surely. Sending n in (2.2) then yields P(τ M < τ F ) φ(x 0 )/M, and hence P(τ F < τ M < ) φ(x 0) M. Sending M then gives P(τ F < ) =. Therefore the set F is recurrent in the sense that X will return to F infinitely often almost surely. In articular, y F G(x 0, y) =. Since F is a finite set, G(x 0, y) = for some y F, which imlies that X is recurrent. 3
4 Theorem 2.8 [Necessary and sufficient condition for transience] Let X be an irreducible Markov chain with countable state sace S. A necessary and sufficient condition for X to be transient is the existence of a non-constant, non-negative suer-harmonic function φ. Proof. Suose that X is recurrent and φ is a non-negative non-constant suer-harmonic function. We will derive a contradiction. Let x, y S with φ(x) < φ(y). Let the Markov chain start at x, whose law we denote by P x. Let τ y = inf{n : X n = y}. Then τ y is a stoing time, and φ(x n τy ) is a suer-martingale. In articular, φ(x) E x [φ(x n τy )] φ(y)p x (τ y n). By recurrence, τ y < almost surely. Therefore sending n leads to φ(x) φ(y), which is a contradiction. For the converse, we claim that if X is transient, then there exists a non-negative nonconstant suer-harmonic function. Fix an x 0 S. We claim that φ(x) = P x ( τ x0 < ) is a suer-harmonic function, where τ x0 = inf{n 0 : X n = x 0 }. Note that φ [0, ], φ(x 0 ) =, and by transience, there exists y S with Π(x 0, y) > 0 and φ(y) <. By examining the Markov chain starting at x after one ste, we note that, if x = x 0, P x ( τ x0 < ) = Π(x, y)p y ( τ x0 < ), if x x 0. y S Therefore φ is harmonic at x x 0 and suer-harmonic at x 0, and hence φ is a non-negative non-constant suer-harmonic function. We now illustrate Theorems 2.7 and 2.8 by an examle. Examle 2.9 [Birth-death chains] Let X be a time-homogeneous Markov chain on {0,, 2, } with one-ste transition robabilities (i, i+) = i, (i, i ) = q i and (i, i) = r i with q 0 = 0. This is called the birth-death chain. We assume that i > 0 for i 0 and q i > 0 for i so that the chain is irreducible. Let us find a non-negative φ which is harmonic for the birth-death chain at all x N. W.l.o.g., we may assume φ(0) = 0 and φ() =. For φ to be harmonic at each x N, we must have which sets u the recursion relation Therefore and φ(x) = x φ(x + ) + r x φ(x) + q x φ(x ), φ(x + ) φ(x) = q x x (φ(x) φ(x )). φ(x) φ(x ) = φ(x) = + x i x i= i=2 j= By construction, φ is harmonic for X at all x N and is non-negative and monotonically increasing. We claim that the chain is recurrent if and only if φ( ) := lim x φ(x) = + 4 q i i, q j j. i i=2 j= q j j =.
5 Indeed, if φ( ) =, then Theorem 2.7 alies with F = {0}. If φ( ) <, then φ(x) = φ( ) φ(x) defines a non-negative non-constant suer-harmonic function for X, so that Theorem 2.8 alies. More directly, we note that φ(x n τ0 τ N ) is a martingale for any N N, which by the otional stoing theorem imlies that if the chain starts at X 0 =, then and hence φ() = P (τ 0 < τ N )φ(0) + P (τ N < τ 0 )φ(n), P (τ N < τ 0 ) = φ(n). Sending N, we conclude that 0 is a recurrent state if and only if φ( ) =. A secial case of the birth-death chain is an asymmetric simle random walk where i = q i = for all i. Then x ( ) i. φ(x) = i= Therefore φ( ) = if and only if 2. If > 2, then P (τ 0 = ) = φ( ) = 2. 3 Dirichlet roblem and Poisson equation We now elaborate more on the connection between Markov chains and otential theory. The first link concerns the so-called the Dirichlet roblem. Assume that X is an irreducible recurrent Markov chain with transition matrix Π, and let A be a given set. We are interested in finding bounded solutions of the Dirichlet boundary value roblem (Π I)f(x) = 0 for x A, f(x) = g(x) for x / A, (3.3) where g is a given bounded function on A c. Observe that such an f is harmonic for X at all x A. Therefore f(x n τa c ) is a bounded martingale, where τ A c = inf{n 0 : X n / A} is finite a.s. by the recurrence of X. By the otional stoing theorem, f(x) = E x [f(x τa c )]. (3.4) Therefore (3.3) has a unique bounded solution, which is given by (3.4). If we let g(x) = {x B} for some B A c, then the solution to (3.3) is f(x) = P x (X τa c B), which is the robability that the Markov chain starting from x enters the set A c by entering the subset B. The collection P x (X τa c = y) for y A c defines a robability measure on A c, called the harmonic measure. Thus comuting the hitting robabilities for a Markov chain is equivalent to solving a Dirichlet roblem with suitable boundary conditions. When X is transient, bounded solutions to (3.3) are not unique in general, as can be seen from the fact that when g = on A c, taken to be a finite set, both f and f(x) = P x (τ A c < ) are bounded solutions to (3.3). However, the second solution can be easily seen to be the minimal non-negative solution, using the fact that any solution induces a martingale. 5
6 Another general way of generating martingales for Markov chains is to take any bounded function f on the state sace S, and then extract a martingale from f(x n ) by subtracting a suitable redictable sequence. More recisely, let Then n = E[f(X n ) f(x n ) X n ] = (Π I)f(X n ). M n = f(x n ) f(x 0 ) n n = f(x n ) f(x 0 ) i= n (Π I)f(X i ) (3.5) is a martingale. If we want to comute the exected hitting time of a set A, i.e., E x [τ A ], then it suffices to solve the equation (Π I)f(x) = for x A, f(x) = 0 for x / A. i= (3.6) This is sometimes called Dynkin s equation. Indeed, if f is a bounded solution of (3.6), then by (3.5), f(x n τa c ) f(x 0 ) + n τ A c is a martingale. If the Markov chain is recurrent, then it is easy to see that E x [τ A c] = f(x). More generally, comuting E x [ τa c i=0 g(x i ) ] reduces to solving (3.6) with (Π I)f = g on A, which is called the Poisson equation. Of course to establish this corresondence using martingales and the otional stoing theorem, one has to be careful with (uniform) integrability issues. Analogues of (3.5) and (3.6) also exist for continuous time Markov rocesses, with (Π I) relaced by the generator of the Markov rocess, the most well known being the Lalacian, which is the generator of a Brownian motion. We can even find robabilistic reresentations for solutions of equations of the following general form: (Π I + V )f(x) = g(x) for x A, (3.7) f(x) = h(x) for x / A, where to avoid integrability issues, let us assume that A is a bounded set, V : A R satisfies max x A V (x) <, h : A c R is a bounded function, and g = 0 on A c. Indeed, the solution of the equation (Π I + V )f(x) = 0 for x A, f(x) = h(x) for x / A (3.8) admits the reresentation f(x) = h(x) if x A c [ and f(x) = E x h(xτa c ) τ A c i=0 V (X i )], using the observation that f(x n ) n i=0 V (X i ) is a martingale if f solves (3.8). This is called the Feynman-Kac formula. On the other hand, the solution of the equation (Π I + V )f(x) = g(x) for x A, f(x) = 0 for x / A [ admits the reresentation f(x) = 0 when x A c τa c, and f(x) = E x i=0 g(x i ) i j=0 when x A, observing that if f solves (3.9) then n f(x n ) i=0 n V (X i ) + i=0 i g(x i ) j=0 V (X j ) (3.9) ] V (X j ) is a martingale. Finally note that by linearity, any solution to (3.7) can be written as the sum of a solution to (3.8) and another solution to (3.9). 6
Lecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.
1 Markov chain: definition Lecture 5 Definition 1.1 Markov chain] A sequence of random variables (X n ) n 0 taking values in a measurable state space (S, S) is called a (discrete time) Markov chain, if
More information1 Gambler s Ruin Problem
Coyright c 2017 by Karl Sigman 1 Gambler s Ruin Problem Let N 2 be an integer and let 1 i N 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins
More informationSolution: (Course X071570: Stochastic Processes)
Solution I (Course X071570: Stochastic Processes) October 24, 2013 Exercise 1.1: Find all functions f from the integers to the real numbers satisfying f(n) = 1 2 f(n + 1) + 1 f(n 1) 1. 2 A secial solution
More informationStochastic integration II: the Itô integral
13 Stochastic integration II: the Itô integral We have seen in Lecture 6 how to integrate functions Φ : (, ) L (H, E) with resect to an H-cylindrical Brownian motion W H. In this lecture we address the
More informationMATH 2710: NOTES FOR ANALYSIS
MATH 270: NOTES FOR ANALYSIS The main ideas we will learn from analysis center around the idea of a limit. Limits occurs in several settings. We will start with finite limits of sequences, then cover infinite
More informationElementary Analysis in Q p
Elementary Analysis in Q Hannah Hutter, May Szedlák, Phili Wirth November 17, 2011 This reort follows very closely the book of Svetlana Katok 1. 1 Sequences and Series In this section we will see some
More informationON THE NORM OF AN IDEMPOTENT SCHUR MULTIPLIER ON THE SCHATTEN CLASS
PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 00, Number 0, Pages 000 000 S 000-9939XX)0000-0 ON THE NORM OF AN IDEMPOTENT SCHUR MULTIPLIER ON THE SCHATTEN CLASS WILLIAM D. BANKS AND ASMA HARCHARRAS
More informationIMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES
IMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES OHAD GILADI AND ASSAF NAOR Abstract. It is shown that if (, ) is a Banach sace with Rademacher tye 1 then for every n N there exists
More informationLecture 7. µ(x)f(x). When µ is a probability measure, we say µ is a stationary distribution.
Lecture 7 1 Stationary measures of a Markov chain We now study the long time behavior of a Markov Chain: in particular, the existence and uniqueness of stationary measures, and the convergence of the distribution
More informationLecture 10. Theorem 1.1 [Ergodicity and extremality] A probability measure µ on (Ω, F) is ergodic for T if and only if it is an extremal point in M.
Lecture 10 1 Ergodic decomposition of invariant measures Let T : (Ω, F) (Ω, F) be measurable, and let M denote the space of T -invariant probability measures on (Ω, F). Then M is a convex set, although
More information1 Probability Spaces and Random Variables
1 Probability Saces and Random Variables 1.1 Probability saces Ω: samle sace consisting of elementary events (or samle oints). F : the set of events P: robability 1.2 Kolmogorov s axioms Definition 1.2.1
More informationEinführung in Stochastische Prozesse und Zeitreihenanalyse Vorlesung, 2013S, 2.0h March 2015 Hubalek/Scherrer
Name: Mat.Nr.: Studium: Bitte keinen Rotstift verwenden! 15.593 Einführung in Stochastische Prozesse und Zeitreihenanalyse Vorlesung, 213S, 2.h March 215 Hubalek/Scherrer (Dauer 9 Minutes, Permissible
More informationAnalysis of some entrance probabilities for killed birth-death processes
Analysis of some entrance robabilities for killed birth-death rocesses Master s Thesis O.J.G. van der Velde Suervisor: Dr. F.M. Sieksma July 5, 207 Mathematical Institute, Leiden University Contents Introduction
More information2. Transience and Recurrence
Virtual Laboratories > 15. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 2. Transience and Recurrence The study of Markov chains, particularly the limiting behavior, depends critically on the random times
More informationδ(xy) = φ(x)δ(y) + y p δ(x). (1)
LECTURE II: δ-rings Fix a rime. In this lecture, we discuss some asects of the theory of δ-rings. This theory rovides a good language to talk about rings with a lift of Frobenius modulo. Some of the material
More informationHENSEL S LEMMA KEITH CONRAD
HENSEL S LEMMA KEITH CONRAD 1. Introduction In the -adic integers, congruences are aroximations: for a and b in Z, a b mod n is the same as a b 1/ n. Turning information modulo one ower of into similar
More informationApplications to stochastic PDE
15 Alications to stochastic PE In this final lecture we resent some alications of the theory develoed in this course to stochastic artial differential equations. We concentrate on two secific examles:
More informationApproximating min-max k-clustering
Aroximating min-max k-clustering Asaf Levin July 24, 2007 Abstract We consider the roblems of set artitioning into k clusters with minimum total cost and minimum of the maximum cost of a cluster. The cost
More information1 Martingales. Martingales. (Ω, B, P ) is a probability space.
Martingales January 8, 206 Debdee Pati Martingales (Ω, B, P ) is a robability sace. Definition. (Filtration) filtration F = {F n } n 0 is a collection of increasing sub-σfields such that for m n, we have
More information6 Stationary Distributions
6 Stationary Distributions 6. Definition and Examles Definition 6.. Let {X n } be a Markov chain on S with transition robability matrix P. A distribution π on S is called stationary (or invariant) if π
More information16.2. Infinite Series. Introduction. Prerequisites. Learning Outcomes
Infinite Series 6.2 Introduction We extend the concet of a finite series, met in Section 6., to the situation in which the number of terms increase without bound. We define what is meant by an infinite
More information16.2. Infinite Series. Introduction. Prerequisites. Learning Outcomes
Infinite Series 6. Introduction We extend the concet of a finite series, met in section, to the situation in which the number of terms increase without bound. We define what is meant by an infinite series
More informationElementary theory of L p spaces
CHAPTER 3 Elementary theory of L saces 3.1 Convexity. Jensen, Hölder, Minkowski inequality. We begin with two definitions. A set A R d is said to be convex if, for any x 0, x 1 2 A x = x 0 + (x 1 x 0 )
More informationLECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]
LECTURE 7 NOTES 1. Convergence of random variables. Before delving into the large samle roerties of the MLE, we review some concets from large samle theory. 1. Convergence in robability: x n x if, for
More informationHomework Solution 4 for APPM4/5560 Markov Processes
Homework Solution 4 for APPM4/556 Markov Processes 9.Reflecting random walk on the line. Consider the oints,,, 4 to be marked on a straight line. Let X n be a Markov chain that moves to the right with
More informationANALYTIC NUMBER THEORY AND DIRICHLET S THEOREM
ANALYTIC NUMBER THEORY AND DIRICHLET S THEOREM JOHN BINDER Abstract. In this aer, we rove Dirichlet s theorem that, given any air h, k with h, k) =, there are infinitely many rime numbers congruent to
More information1 Random Variables and Probability Distributions
1 Random Variables and Probability Distributions 1.1 Random Variables 1.1.1 Discrete random variables A random variable X is called discrete if the number of values that X takes is finite or countably
More informationGOOD MODELS FOR CUBIC SURFACES. 1. Introduction
GOOD MODELS FOR CUBIC SURFACES ANDREAS-STEPHAN ELSENHANS Abstract. This article describes an algorithm for finding a model of a hyersurface with small coefficients. It is shown that the aroach works in
More information2 Asymptotic density and Dirichlet density
8.785: Analytic Number Theory, MIT, sring 2007 (K.S. Kedlaya) Primes in arithmetic rogressions In this unit, we first rove Dirichlet s theorem on rimes in arithmetic rogressions. We then rove the rime
More information2 Asymptotic density and Dirichlet density
8.785: Analytic Number Theory, MIT, sring 2007 (K.S. Kedlaya) Primes in arithmetic rogressions In this unit, we first rove Dirichlet s theorem on rimes in arithmetic rogressions. We then rove the rime
More informationMATH 250: THE DISTRIBUTION OF PRIMES. ζ(s) = n s,
MATH 50: THE DISTRIBUTION OF PRIMES ROBERT J. LEMKE OLIVER For s R, define the function ζs) by. Euler s work on rimes ζs) = which converges if s > and diverges if s. In fact, though we will not exloit
More informationRobustness of classifiers to uniform l p and Gaussian noise Supplementary material
Robustness of classifiers to uniform l and Gaussian noise Sulementary material Jean-Yves Franceschi Ecole Normale Suérieure de Lyon LIP UMR 5668 Omar Fawzi Ecole Normale Suérieure de Lyon LIP UMR 5668
More informationPositive and null recurrent-branching Process
December 15, 2011 In last discussion we studied the transience and recurrence of Markov chains There are 2 other closely related issues about Markov chains that we address Is there an invariant distribution?
More informationHarmonic Functions and Brownian motion
Harmonic Functions and Brownian motion Steven P. Lalley April 25, 211 1 Dynkin s Formula Denote by W t = (W 1 t, W 2 t,..., W d t ) a standard d dimensional Wiener process on (Ω, F, P ), and let F = (F
More informationReal Analysis 1 Fall Homework 3. a n.
eal Analysis Fall 06 Homework 3. Let and consider the measure sace N, P, µ, where µ is counting measure. That is, if N, then µ equals the number of elements in if is finite; µ = otherwise. One usually
More informationOutline. Markov Chains and Markov Models. Outline. Markov Chains. Markov Chains Definitions Huizhen Yu
and Markov Models Huizhen Yu janey.yu@cs.helsinki.fi Det. Comuter Science, Univ. of Helsinki Some Proerties of Probabilistic Models, Sring, 200 Huizhen Yu (U.H.) and Markov Models Jan. 2 / 32 Huizhen Yu
More informationPETER J. GRABNER AND ARNOLD KNOPFMACHER
ARITHMETIC AND METRIC PROPERTIES OF -ADIC ENGEL SERIES EXPANSIONS PETER J. GRABNER AND ARNOLD KNOPFMACHER Abstract. We derive a characterization of rational numbers in terms of their unique -adic Engel
More informationLecture 12. F o s, (1.1) F t := s>t
Lecture 12 1 Brownian motion: the Markov property Let C := C(0, ), R) be the space of continuous functions mapping from 0, ) to R, in which a Brownian motion (B t ) t 0 almost surely takes its value. Let
More informationMS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10
MS&E 321 Spring 12-13 Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 Section 3: Regenerative Processes Contents 3.1 Regeneration: The Basic Idea............................... 1 3.2
More informationChapter 7: Special Distributions
This chater first resents some imortant distributions, and then develos the largesamle distribution theory which is crucial in estimation and statistical inference Discrete distributions The Bernoulli
More informationOn Doob s Maximal Inequality for Brownian Motion
Stochastic Process. Al. Vol. 69, No., 997, (-5) Research Reort No. 337, 995, Det. Theoret. Statist. Aarhus On Doob s Maximal Inequality for Brownian Motion S. E. GRAVERSEN and G. PESKIR If B = (B t ) t
More information1 Riesz Potential and Enbeddings Theorems
Riesz Potential and Enbeddings Theorems Given 0 < < and a function u L loc R, the Riesz otential of u is defined by u y I u x := R x y dy, x R We begin by finding an exonent such that I u L R c u L R for
More informationB8.1 Martingales Through Measure Theory. Concept of independence
B8.1 Martingales Through Measure Theory Concet of indeendence Motivated by the notion of indeendent events in relims robability, we have generalized the concet of indeendence to families of σ-algebras.
More informationp-adic Measures and Bernoulli Numbers
-Adic Measures and Bernoulli Numbers Adam Bowers Introduction The constants B k in the Taylor series exansion t e t = t k B k k! k=0 are known as the Bernoulli numbers. The first few are,, 6, 0, 30, 0,
More informationSECTION 5: FIBRATIONS AND HOMOTOPY FIBERS
SECTION 5: FIBRATIONS AND HOMOTOPY FIBERS In this section we will introduce two imortant classes of mas of saces, namely the Hurewicz fibrations and the more general Serre fibrations, which are both obtained
More informationVarious Proofs for the Decrease Monotonicity of the Schatten s Power Norm, Various Families of R n Norms and Some Open Problems
Int. J. Oen Problems Comt. Math., Vol. 3, No. 2, June 2010 ISSN 1998-6262; Coyright c ICSRS Publication, 2010 www.i-csrs.org Various Proofs for the Decrease Monotonicity of the Schatten s Power Norm, Various
More informationP i [B k ] = lim. n=1 p(n) ii <. n=1. V i :=
2.7. Recurrence and transience Consider a Markov chain {X n : n N 0 } on state space E with transition matrix P. Definition 2.7.1. A state i E is called recurrent if P i [X n = i for infinitely many n]
More informationSums of independent random variables
3 Sums of indeendent random variables This lecture collects a number of estimates for sums of indeendent random variables with values in a Banach sace E. We concentrate on sums of the form N γ nx n, where
More informationElements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
More informationON FREIMAN S 2.4-THEOREM
ON FREIMAN S 2.4-THEOREM ØYSTEIN J. RØDSETH Abstract. Gregory Freiman s celebrated 2.4-Theorem says that if A is a set of residue classes modulo a rime satisfying 2A 2.4 A 3 and A < /35, then A is contained
More informationOn Wald-Type Optimal Stopping for Brownian Motion
J Al Probab Vol 34, No 1, 1997, (66-73) Prerint Ser No 1, 1994, Math Inst Aarhus On Wald-Tye Otimal Stoing for Brownian Motion S RAVRSN and PSKIR The solution is resented to all otimal stoing roblems of
More informationClassification of Countable State Markov Chains
Classification of Countable State Markov Chains Friday, March 21, 2014 2:01 PM How can we determine whether a communication class in a countable state Markov chain is: transient null recurrent positive
More informationP -PARTITIONS AND RELATED POLYNOMIALS
P -PARTITIONS AND RELATED POLYNOMIALS 1. P -artitions A labelled oset is a oset P whose ground set is a subset of the integers, see Fig. 1. We will denote by < the usual order on the integers and < P the
More informationGENERICITY OF INFINITE-ORDER ELEMENTS IN HYPERBOLIC GROUPS
GENERICITY OF INFINITE-ORDER ELEMENTS IN HYPERBOLIC GROUPS PALLAVI DANI 1. Introduction Let Γ be a finitely generated grou and let S be a finite set of generators for Γ. This determines a word metric on
More informationElements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
More informationQuaternionic Projective Space (Lecture 34)
Quaternionic Projective Sace (Lecture 34) July 11, 2008 The three-shere S 3 can be identified with SU(2), and therefore has the structure of a toological grou. In this lecture, we will address the question
More informationMATH342 Practice Exam
MATH342 Practice Exam This exam is intended to be in a similar style to the examination in May/June 2012. It is not imlied that all questions on the real examination will follow the content of the ractice
More informationArithmetic and Metric Properties of p-adic Alternating Engel Series Expansions
International Journal of Algebra, Vol 2, 2008, no 8, 383-393 Arithmetic and Metric Proerties of -Adic Alternating Engel Series Exansions Yue-Hua Liu and Lu-Ming Shen Science College of Hunan Agriculture
More informationTopic 7: Using identity types
Toic 7: Using identity tyes June 10, 2014 Now we would like to learn how to use identity tyes and how to do some actual mathematics with them. By now we have essentially introduced all inference rules
More information1 Extremum Estimators
FINC 9311-21 Financial Econometrics Handout Jialin Yu 1 Extremum Estimators Let θ 0 be a vector of k 1 unknown arameters. Extremum estimators: estimators obtained by maximizing or minimizing some objective
More informationA note on the random greedy triangle-packing algorithm
A note on the random greedy triangle-acking algorithm Tom Bohman Alan Frieze Eyal Lubetzky Abstract The random greedy algorithm for constructing a large artial Steiner-Trile-System is defined as follows.
More informationLectures on Markov Chains
Lectures on Markov Chains David M. McClendon Department of Mathematics Ferris State University 2016 edition 1 Contents Contents 2 1 Markov chains 4 1.1 The definition of a Markov chain.....................
More informationMATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression
1/9 MATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression Dominique Guillot Deartments of Mathematical Sciences University of Delaware February 15, 2016 Distribution of regression
More informationMath Homework 5 Solutions
Math 45 - Homework 5 Solutions. Exercise.3., textbook. The stochastic matrix for the gambler problem has the following form, where the states are ordered as (,, 4, 6, 8, ): P = The corresponding diagram
More informationMultiplicity of weak solutions for a class of nonuniformly elliptic equations of p-laplacian type
Nonlinear Analysis 7 29 536 546 www.elsevier.com/locate/na Multilicity of weak solutions for a class of nonuniformly ellitic equations of -Lalacian tye Hoang Quoc Toan, Quô c-anh Ngô Deartment of Mathematics,
More informationBirth-death chain models (countable state)
Countable State Birth-Death Chains and Branching Processes Tuesday, March 25, 2014 1:59 PM Homework 3 posted, due Friday, April 18. Birth-death chain models (countable state) S = We'll characterize the
More informationThe inverse Goldbach problem
1 The inverse Goldbach roblem by Christian Elsholtz Submission Setember 7, 2000 (this version includes galley corrections). Aeared in Mathematika 2001. Abstract We imrove the uer and lower bounds of the
More informationt 0 Xt sup X t p c p inf t 0
SHARP MAXIMAL L -ESTIMATES FOR MARTINGALES RODRIGO BAÑUELOS AND ADAM OSȨKOWSKI ABSTRACT. Let X be a suermartingale starting from 0 which has only nonnegative jums. For each 0 < < we determine the best
More informationModule 8. Lecture 3: Markov chain
Lecture 3: Markov chain A Markov chain is a stochastic rocess having the roerty that the value of the rocess X t at time t, deends only on its value at time t-1, X t-1 and not on the sequence X t-2, X
More informationConvergence of random variables, and the Borel-Cantelli lemmas
Stat 205A Setember, 12, 2002 Convergence of ranom variables, an the Borel-Cantelli lemmas Lecturer: James W. Pitman Scribes: Jin Kim (jin@eecs) 1 Convergence of ranom variables Recall that, given a sequence
More informationMath 4400/6400 Homework #8 solutions. 1. Let P be an odd integer (not necessarily prime). Show that modulo 2,
MATH 4400 roblems. Math 4400/6400 Homework # solutions 1. Let P be an odd integer not necessarily rime. Show that modulo, { P 1 0 if P 1, 7 mod, 1 if P 3, mod. Proof. Suose that P 1 mod. Then we can write
More informationLecture 2. We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales.
Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal
More informationDiscrete Calderón s Identity, Atomic Decomposition and Boundedness Criterion of Operators on Multiparameter Hardy Spaces
J Geom Anal (010) 0: 670 689 DOI 10.1007/s10-010-913-6 Discrete Calderón s Identity, Atomic Decomosition and Boundedness Criterion of Oerators on Multiarameter Hardy Saces Y. Han G. Lu K. Zhao Received:
More informationA D VA N C E D P R O B A B I L - I T Y
A N D R E W T U L L O C H A D VA N C E D P R O B A B I L - I T Y T R I N I T Y C O L L E G E T H E U N I V E R S I T Y O F C A M B R I D G E Contents 1 Conditional Expectation 5 1.1 Discrete Case 6 1.2
More informationIntroduction to Banach Spaces
CHAPTER 8 Introduction to Banach Saces 1. Uniform and Absolute Convergence As a rearation we begin by reviewing some familiar roerties of Cauchy sequences and uniform limits in the setting of metric saces.
More informationFOURIER SERIES PART III: APPLICATIONS
FOURIER SERIES PART III: APPLICATIONS We extend the construction of Fourier series to functions with arbitrary eriods, then we associate to functions defined on an interval [, L] Fourier sine and Fourier
More informationThe Hasse Minkowski Theorem Lee Dicker University of Minnesota, REU Summer 2001
The Hasse Minkowski Theorem Lee Dicker University of Minnesota, REU Summer 2001 The Hasse-Minkowski Theorem rovides a characterization of the rational quadratic forms. What follows is a roof of the Hasse-Minkowski
More informationSharp gradient estimate and spectral rigidity for p-laplacian
Shar gradient estimate and sectral rigidity for -Lalacian Chiung-Jue Anna Sung and Jiaing Wang To aear in ath. Research Letters. Abstract We derive a shar gradient estimate for ositive eigenfunctions of
More informationOn Z p -norms of random vectors
On Z -norms of random vectors Rafa l Lata la Abstract To any n-dimensional random vector X we may associate its L -centroid body Z X and the corresonding norm. We formulate a conjecture concerning the
More informationLecture 7. 1 Notations. Tel Aviv University Spring 2011
Random Walks and Brownian Motion Tel Aviv University Spring 2011 Lecture date: Apr 11, 2011 Lecture 7 Instructor: Ron Peled Scribe: Yoav Ram The following lecture (and the next one) will be an introduction
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationOn Isoperimetric Functions of Probability Measures Having Log-Concave Densities with Respect to the Standard Normal Law
On Isoerimetric Functions of Probability Measures Having Log-Concave Densities with Resect to the Standard Normal Law Sergey G. Bobkov Abstract Isoerimetric inequalities are discussed for one-dimensional
More informationCHAPTER 2: SMOOTH MAPS. 1. Introduction In this chapter we introduce smooth maps between manifolds, and some important
CHAPTER 2: SMOOTH MAPS DAVID GLICKENSTEIN 1. Introduction In this chater we introduce smooth mas between manifolds, and some imortant concets. De nition 1. A function f : M! R k is a smooth function if
More informationOn a Markov Game with Incomplete Information
On a Markov Game with Incomlete Information Johannes Hörner, Dinah Rosenberg y, Eilon Solan z and Nicolas Vieille x{ January 24, 26 Abstract We consider an examle of a Markov game with lack of information
More informationPractice Final Solutions
Practice Final Solutions 1. True or false: (a) If a is a sum of three squares, and b is a sum of three squares, then so is ab. False: Consider a 14, b 2. (b) No number of the form 4 m (8n + 7) can be written
More informationIntroduction to Group Theory Note 1
Introduction to Grou Theory Note July 7, 009 Contents INTRODUCTION. Examles OF Symmetry Grous in Physics................................. ELEMENT OF GROUP THEORY. De nition of Grou................................................
More informationPRIME NUMBERS YANKI LEKILI
PRIME NUMBERS YANKI LEKILI We denote by N the set of natural numbers:,2,..., These are constructed using Peano axioms. We will not get into the hilosohical questions related to this and simly assume the
More informationALGEBRAIC TOPOLOGY MASTERMATH (FALL 2014) Written exam, 21/01/2015, 3 hours Outline of solutions
ALGERAIC TOPOLOGY MASTERMATH FALL 014) Written exam, 1/01/015, 3 hours Outline of solutions Exercise 1. i) There are various definitions in the literature. ased on the discussion on. 5 of Lecture 3, as
More informationStochastic Processes (Week 6)
Stochastic Processes (Week 6) October 30th, 2014 1 Discrete-time Finite Markov Chains 2 Countable Markov Chains 3 Continuous-Time Markov Chains 3.1 Poisson Process 3.2 Finite State Space 3.2.1 Kolmogrov
More informationLecture 10: Hypercontractivity
CS 880: Advanced Comlexity Theory /15/008 Lecture 10: Hyercontractivity Instructor: Dieter van Melkebeek Scribe: Baris Aydinlioglu This is a technical lecture throughout which we rove the hyercontractivity
More informationExtension of Minimax to Infinite Matrices
Extension of Minimax to Infinite Matrices Chris Calabro June 21, 2004 Abstract Von Neumann s minimax theorem is tyically alied to a finite ayoff matrix A R m n. Here we show that (i) if m, n are both inite,
More informationMath 104B: Number Theory II (Winter 2012)
Math 104B: Number Theory II (Winter 01) Alina Bucur Contents 1 Review 11 Prime numbers 1 Euclidean algorithm 13 Multilicative functions 14 Linear diohantine equations 3 15 Congruences 3 Primes as sums
More informationMartingale transforms and their projection operators on manifolds
artingale transforms and their rojection oerators on manifolds Rodrigo Bañuelos, Fabrice Baudoin Deartment of athematics Purdue University West Lafayette, 4796 Setember 9, Abstract We rove the boundedness
More informationSolutions to In Class Problems Week 15, Wed.
Massachusetts Institute of Technology 6.04J/18.06J, Fall 05: Mathematics for Comuter Science December 14 Prof. Albert R. Meyer and Prof. Ronitt Rubinfeld revised December 14, 005, 1404 minutes Solutions
More informationConvex Optimization methods for Computing Channel Capacity
Convex Otimization methods for Comuting Channel Caacity Abhishek Sinha Laboratory for Information and Decision Systems (LIDS), MIT sinhaa@mit.edu May 15, 2014 We consider a classical comutational roblem
More informationCOMMUNICATION BETWEEN SHAREHOLDERS 1
COMMUNICATION BTWN SHARHOLDRS 1 A B. O A : A D Lemma B.1. U to µ Z r 2 σ2 Z + σ2 X 2r ω 2 an additive constant that does not deend on a or θ, the agents ayoffs can be written as: 2r rθa ω2 + θ µ Y rcov
More informationA Note on Guaranteed Sparse Recovery via l 1 -Minimization
A Note on Guaranteed Sarse Recovery via l -Minimization Simon Foucart, Université Pierre et Marie Curie Abstract It is roved that every s-sarse vector x C N can be recovered from the measurement vector
More informationAsymptotically Optimal Simulation Allocation under Dependent Sampling
Asymtotically Otimal Simulation Allocation under Deendent Samling Xiaoing Xiong The Robert H. Smith School of Business, University of Maryland, College Park, MD 20742-1815, USA, xiaoingx@yahoo.com Sandee
More informationBEST POSSIBLE DENSITIES OF DICKSON m-tuples, AS A CONSEQUENCE OF ZHANG-MAYNARD-TAO
BEST POSSIBLE DENSITIES OF DICKSON m-tuples, AS A CONSEQUENCE OF ZHANG-MAYNARD-TAO ANDREW GRANVILLE, DANIEL M. KANE, DIMITRIS KOUKOULOPOULOS, AND ROBERT J. LEMKE OLIVER Abstract. We determine for what
More information