MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 7
|
|
- Duane Hampton
- 5 years ago
- Views:
Transcription
1 MS&E 321 Spring Stochastic Systems June 1, 213 Prof. Peter W. Glynn Page 1 of 7 Section 9: Renewal Theory Contents 9.1 Renewal Equations Solving the Renewal Equation Asymptotic Behavior of the Solution of the Renewal Equation Renewal Equations Let X = (X(t) : t ) be a non-delayed regenerative process and suppose that f : S R +. Then, if a (t) = Ef(X(t)), a (t) can be expressed as follows: a (t) = Ef(X(t))I(τ 1 > t) + Ef(X(t))I(τ 1 ds). But Ef(X(t))I(τ 1 ds) = E[f(X(τ 1 + t s)) τ 1 = s]p(τ 1 ds) = Ef(X(t s))p(τ 1 ds) = a (t s)p(τ 1 ds). In other words, (a (t) : t ) satisfies the linear integral equation a(t) = b(t) + (F a)(t), for t, and F (t) = P(τ 1 t), b(t) Ef(X(t))I(τ 1 > t), (G h)(t) h(t s)g(ds) is the convolution operation (that is well-defined for any non-decreasing function G and non-negative h). 1
2 Definition Given a function b and a non-negative G, a linear integral equation of the form a = b + G a is called a renewal equation for a non-decreasing function G. Define the n-fold convolution of G via G (n) (t) = I(t ) for n = and G (n) (t) = G(t s)g (n 1) (ds) for n 1. Note that when G is the distribution function F of a positive rv τ 1, F (n) (t) = P(T (n) t), T (n) = τ 1 + τ τ n, and the τ i s are iid copies of τ 1. Proposition The function (a (t) : t ) = (Ef(X(t)) : t ) is the minimal non-negative solution of a = b + F a and is given by F (n) b. Here are some additional examples of renewal equations: Example Let a (t) = P(T (N(t) + 1) t > x) be the tail probability of the residual life process T (N(t)+1) t corresponding to a non-delayed renewal counting process N = (N(t) : t ). Note that a satisfies the renewal equation and a = b + F a, (9.1.1) b(t) = P(τ 1 > t + x) F (t) = P(τ 1 t). The function a is the minimal non-negative solution of (9.1.1) and is given by F (n) b. Example Let X = (X(t) : t ) be a non-delayed regenerative process and let a (t) = P(t T (N(t)) x) be the distribution of the current age process t T (N(t). Here, a satisfies a = b + F a, (9.1.2) b(t) = I(t x), 2
3 F (t) = P(τ 1 t). The function a is the minimal non-negative solution of (9.1.2) and is given by F (n) b. Example Let X = (X(t) : t ) be a non-delayed S-valued regenerative process and let T = inf{t : X(t) A} be the first hitting time if some subset A S. Put a (t) = P(T > t). Then, a satisfies a = b + G a, (9.1.3) b(t) = P(T τ 1 > t), G(t) = P(τ 1 t, T > τ 1 ). The function a is the minimal non-negative solution of (9.1.3) and is given by G (n) b. Note that G is not a probability distribution function here (since G( ) = P(T < τ 1 ) < 1 in general). Definition A renewal equation is said to be a proper renewal equation if G is a probability distribution function; otherwise, the renewal equation is said to be an improper renewal equation. In particular, if G() = and G( ) < 1, the improper renewal equation is said to be defective; if G() = and G( ) > 1, the improper renewal equation is said to be excessive. Improper renewal equations can frequently be transformed into proper renewal equations via use of the following trick. Note that if a is given by G (n) b, then ã(t) = ( G (n) b)(t), ã(t) = e γt a(t) b(t) = e γt b(t) G(dt) = e γt G(dt), and ã satisfies the renewal equation So if we can find γ such that ã = b + G ã. (9.1.4) G(dt) = e γ t G(dt) is a probability distribution function, (9.1.4) becomes a proper renewal equation. 3
4 Exercise a.) Prove that if a renewal equation is excessive, there always exists a unique γ such that G is a probability distribution function. b.) Prove that if a renewal equation is defective, there need not exist a γ such that G is a probability distribution function. 9.2 Solving the Renewal Equation so Hence, If F is the distribution of an exponential rv with parameter λ >, then F (n) (dt) = λ (λt)n 1 (n 1)! exp( λt)dt F (n) (dt) = δ (dt) + λdt. (F (n) b)(t) = b(t) + λ t b(s) ds. So, the renewal equation can be explicitly solved when F is exponential. It can also be solved in closed form in a limited number of other cases. Let ã(γ) = e γt a(t) dt, b(γ) = e γt b(t) dt, G(γ) = e γt G(dt), and note that ã, b, and G are the Laplace transforms of a, b, and G, respectively. If a satisfies a = b + G a, it follows that so that ã = b + Gã, ã = b 1 G. If b and G can be computed in closed form, then a can potentially be computed in closed form by calculating the inverse Laplace transform corresponding to b(1 G) 1. For example, this can generally be done when G has a rational Laplace transform (i.e. is the ratio of two polynomials in γ). 4
5 9.3 Asymptotic Behavior of the Solution of the Renewal Equation Typically, the solution to the renewal equation can not be computed in closed form. In such settings, one usually must be satisfied with computing the limiting behavior of when t tends to infinity. Definition The function (F (n) b)(t) U(t) = F (n) (t) is called the renewal function and the corresponding distribution/measure is called the renewal measure. If F has a density f, u(t) = f (n) (t) is called the renewal density. If F is the distribution function of an integer-valued rv τ 1, then ( p (k) n u n = n=1 k=1 p (k) n = P (τ τ k = n)) is called the renewal mass function. We focus first on the case F is the distribution function of an integer-valued rv τ 1. Note that u n = = E P (τ τ k = n) k=1 I(T (k) = n) k=1 = EI(regeneration at time n). It turns out that the asymptotic behavior of u n as n can be analyzed via consideration of an appropriate Markov chain X = (X n : n ). In particular, let X n = inf{t (k) n : T (k) n} be the residual life process corresponding to the sequence of event times T (1), T (2),.... Note that u n = P (X n = ), X = (X n : n ) is an Z + -valued Markov chain with transition probabilities given by P (i, i 1) = 1 for i 1, and P (, i) = P (τ 1 = i + 1). 5
6 Note that the state is always recurrent for X. Furthermore, X is positive recurrent if and only if <, in which case π is given by j>i π(i) = P (τ 1 < j) = P (τ 1 > i). Finally, X is aperiodic if and only if If X is aperiodic and <, it follows that as n. It follows that if gcd{k 1 : P (τ 1 = k) > } = 1. u n = P (X n = ) π() = 1 b k <, (9.3.1) k= then the Dominated Convergence Theorem implies that as n. (F (k) b)(n) = k= k= u n k b k 1 k= b k Theorem (Discrete Renewal Theorem) Suppose < and If (9.3.1) holds, then gcd{k 1 : P (τ 1 = k) > } = 1. k= as n. If = and (9.3.1) holds, then u n k b k 1 k= b k as n. u n k b k k= Let X = (X n : n ) be a S-valued non-delayed regenerative sequence and suppose f : S R is a bounded function (i.e. sup{ f(x) : x S}). As noted earlier, satisfies a renewal equation, namely a n = b n + [,n] a n = Ef(X n ) a (t s)p (τ 1 ds) = b n + 6 a n jp j, j=1
7 so a n = u n j b j, b j = Ef(X j )I(τ 1 > j). The discrete renewal theorem guarantees that if τ 1 is aperiodic and has finite mean, then j= a n j= b j as n. In other words, as n. Ef(X n ) j= Ef(X j)i(τ 1 > j) = E τ1 1 j= f(x j) Exercise Prove that if X = (X n : n ) is a delayed S-valued regenerative sequence with < and f : S R is a bounded function, then as n, provided that τ 1 is aperiodic. Ef(X n ) E τ 1 1 j= f(x j) All the above discussion for discrete-time renewal theory extends to the continuous-time setting. 7
MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10. x n+1 = f(x n ),
MS&E 321 Spring 12-13 Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 Section 4: Steady-State Theory Contents 4.1 The Concept of Stochastic Equilibrium.......................... 1 4.2
More informationMS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10
MS&E 321 Spring 12-13 Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 Section 3: Regenerative Processes Contents 3.1 Regeneration: The Basic Idea............................... 1 3.2
More information9.5 The Transfer Function
Lecture Notes on Control Systems/D. Ghose/2012 0 9.5 The Transfer Function Consider the n-th order linear, time-invariant dynamical system. dy a 0 y + a 1 dt + a d 2 y 2 dt + + a d n y 2 n dt b du 0u +
More informationStochastic process. X, a series of random variables indexed by t
Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,
More informationRenewal theory and its applications
Renewal theory and its applications Stella Kapodistria and Jacques Resing September 11th, 212 ISP Definition of a Renewal process Renewal theory and its applications If we substitute the Exponentially
More informationLecture 10: Semi-Markov Type Processes
Lecture 1: Semi-Markov Type Processes 1. Semi-Markov processes (SMP) 1.1 Definition of SMP 1.2 Transition probabilities for SMP 1.3 Hitting times and semi-markov renewal equations 2. Processes with semi-markov
More informationAdventures in Stochastic Processes
Sidney Resnick Adventures in Stochastic Processes with Illustrations Birkhäuser Boston Basel Berlin Table of Contents Preface ix CHAPTER 1. PRELIMINARIES: DISCRETE INDEX SETS AND/OR DISCRETE STATE SPACES
More informationMarkov chains. Randomness and Computation. Markov chains. Markov processes
Markov chains Randomness and Computation or, Randomized Algorithms Mary Cryan School of Informatics University of Edinburgh Definition (Definition 7) A discrete-time stochastic process on the state space
More informationSelected Exercises on Expectations and Some Probability Inequalities
Selected Exercises on Expectations and Some Probability Inequalities # If E(X 2 ) = and E X a > 0, then P( X λa) ( λ) 2 a 2 for 0 < λ
More informationLet (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t
2.2 Filtrations Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of σ algebras {F t } such that F t F and F t F t+1 for all t = 0, 1,.... In continuous time, the second condition
More informationChapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 2. Poisson Processes Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Outline Introduction to Poisson Processes Definition of arrival process Definition
More informationMath 2C03 - Differential Equations. Slides shown in class - Winter Laplace Transforms. March 4, 5, 9, 11, 12, 16,
Math 2C03 - Differential Equations Slides shown in class - Winter 2015 Laplace Transforms March 4, 5, 9, 11, 12, 16, 18... 2015 Laplace Transform used to solve linear ODEs and systems of linear ODEs with
More informationSolutions to Homework Discrete Stochastic Processes MIT, Spring 2011
Exercise 1 Solutions to Homework 6 6.262 Discrete Stochastic Processes MIT, Spring 2011 Let {Y n ; n 1} be a sequence of rv s and assume that lim n E[ Y n ] = 0. Show that {Y n ; n 1} converges to 0 in
More informationThe Laplace Transform
The Laplace Transform Introduction There are two common approaches to the developing and understanding the Laplace transform It can be viewed as a generalization of the CTFT to include some signals with
More informationIrreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1
Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate
More informationISyE 6761 (Fall 2016) Stochastic Processes I
Fall 216 TABLE OF CONTENTS ISyE 6761 (Fall 216) Stochastic Processes I Prof. H. Ayhan Georgia Institute of Technology L A TEXer: W. KONG http://wwong.github.io Last Revision: May 25, 217 Table of Contents
More informationIEOR 6711, HMWK 5, Professor Sigman
IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.
More information2. Transience and Recurrence
Virtual Laboratories > 15. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 2. Transience and Recurrence The study of Markov chains, particularly the limiting behavior, depends critically on the random times
More informationTheory and Applications of Stochastic Systems Lecture Exponential Martingale for Random Walk
Instructor: Victor F. Araman December 4, 2003 Theory and Applications of Stochastic Systems Lecture 0 B60.432.0 Exponential Martingale for Random Walk Let (S n : n 0) be a random walk with i.i.d. increments
More informationNotes on uniform convergence
Notes on uniform convergence Erik Wahlén erik.wahlen@math.lu.se January 17, 2012 1 Numerical sequences We begin by recalling some properties of numerical sequences. By a numerical sequence we simply mean
More informationMARKOV PROCESSES. Valerio Di Valerio
MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some
More information1 Stationary point processes
Copyright c 22 by Karl Sigman Stationary point processes We present here a brief introduction to stationay point processes on the real line.. Basic notation for point processes We consider simple point
More informationStatistics 150: Spring 2007
Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities
More information6. Brownian Motion. Q(A) = P [ ω : x(, ω) A )
6. Brownian Motion. stochastic process can be thought of in one of many equivalent ways. We can begin with an underlying probability space (Ω, Σ, P) and a real valued stochastic process can be defined
More information1 Delayed Renewal Processes: Exploiting Laplace Transforms
IEOR 6711: Stochastic Models I Professor Whitt, Tuesday, October 22, 213 Renewal Theory: Proof of Blackwell s theorem 1 Delayed Renewal Processes: Exploiting Laplace Transforms The proof of Blackwell s
More information(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?
IEOR 3106: Introduction to Operations Research: Stochastic Models Fall 2006, Professor Whitt SOLUTIONS to Final Exam Chapters 4-7 and 10 in Ross, Tuesday, December 19, 4:10pm-7:00pm Open Book: but only
More informationPoisson Processes. Stochastic Processes. Feb UC3M
Poisson Processes Stochastic Processes UC3M Feb. 2012 Exponential random variables A random variable T has exponential distribution with rate λ > 0 if its probability density function can been written
More informationPath Decomposition of Markov Processes. Götz Kersting. University of Frankfurt/Main
Path Decomposition of Markov Processes Götz Kersting University of Frankfurt/Main joint work with Kaya Memisoglu, Jim Pitman 1 A Brownian path with positive drift 50 40 30 20 10 0 0 200 400 600 800 1000-10
More informationMATH 56A: STOCHASTIC PROCESSES CHAPTER 6
MATH 56A: STOCHASTIC PROCESSES CHAPTER 6 6. Renewal Mathematically, renewal refers to a continuous time stochastic process with states,, 2,. N t {,, 2, 3, } so that you only have jumps from x to x + and
More informationTHE VARIANCE CONSTANT FOR THE ACTUAL WAITING TIME OF THE PH/PH/1 QUEUE. By Mogens Bladt National University of Mexico
The Annals of Applied Probability 1996, Vol. 6, No. 3, 766 777 THE VARIANCE CONSTANT FOR THE ACTUAL WAITING TIME OF THE PH/PH/1 QUEUE By Mogens Bladt National University of Mexico In this paper we consider
More informationMath 180C, Spring Supplement on the Renewal Equation
Math 18C Spring 218 Supplement on the Renewal Equation. These remarks supplement our text and set down some of the material discussed in my lectures. Unexplained notation is as in the text or in lecture.
More information1 Some basic renewal theory: The Renewal Reward Theorem
Copyright c 27 by Karl Sigman Some basic renewal theory: The Renewal Reward Theorem Here, we will present some basic results in renewal theory such as the elementary renewal theorem, and then the very
More information6 Markov Chain Monte Carlo (MCMC)
6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution
More informationBranching Processes II: Convergence of critical branching to Feller s CSB
Chapter 4 Branching Processes II: Convergence of critical branching to Feller s CSB Figure 4.1: Feller 4.1 Birth and Death Processes 4.1.1 Linear birth and death processes Branching processes can be studied
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 7 9/25/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 7 9/5/013 The Reflection Principle. The Distribution of the Maximum. Brownian motion with drift Content. 1. Quick intro to stopping times.
More informationI forgot to mention last time: in the Ito formula for two standard processes, putting
I forgot to mention last time: in the Ito formula for two standard processes, putting dx t = a t dt + b t db t dy t = α t dt + β t db t, and taking f(x, y = xy, one has f x = y, f y = x, and f xx = f yy
More informationBIBO STABILITY AND ASYMPTOTIC STABILITY
BIBO STABILITY AND ASYMPTOTIC STABILITY FRANCESCO NORI Abstract. In this report with discuss the concepts of bounded-input boundedoutput stability (BIBO) and of Lyapunov stability. Examples are given to
More informationNONLINEARLY PERTURBED STOCHASTIC PROCESSES
Theory of Stochastic Processes Vol. 14 (30), no. 3-4, 2008, pp.129-164 DMITRII SILVESTROV NONLINEARLY PERTURBED STOCHASTIC PROCESSES This paper is a survey of results presented in the recent book [25]
More informationTRANSIENT SIMULATION VIA EMPIRICALLY BASED COUPLING
Probability in the Engineering and Informational Sciences, 13, 1999, 147 167+ Printed in the U+S+A+ TRANSIENT SIMULATION VIA EMPIRICALLY BASED COUPLING EUGENE W. WONG, PETER W. GLYNN, AND DONALD L. IGLEHART*
More informationSpecial Mathematics Laplace Transform
Special Mathematics Laplace Transform March 28 ii Nature laughs at the difficulties of integration. Pierre-Simon Laplace 4 Laplace Transform Motivation Properties of the Laplace transform the Laplace transform
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationStochastic Calculus and Black-Scholes Theory MTH772P Exercises Sheet 1
Stochastic Calculus and Black-Scholes Theory MTH772P Exercises Sheet. For ξ, ξ 2, i.i.d. with P(ξ i = ± = /2 define the discrete-time random walk W =, W n = ξ +... + ξ n. (i Formulate and prove the property
More informationLaplace Transform Introduction
Laplace Transform Introduction In many problems, a function is transformed to another function through a relation of the type: where is a known function. Here, is called integral transform of. Thus, an
More informationThe Laplace transform
The Laplace transform Samy Tindel Purdue University Differential equations - MA 266 Taken from Elementary differential equations by Boyce and DiPrima Samy T. Laplace transform Differential equations 1
More informationManual for SOA Exam MLC.
Chapter 10. Poisson processes. Section 10.5. Nonhomogenous Poisson processes Extract from: Arcones Fall 2009 Edition, available at http://www.actexmadriver.com/ 1/14 Nonhomogenous Poisson processes Definition
More informationMultivariate Risk Processes with Interacting Intensities
Multivariate Risk Processes with Interacting Intensities Nicole Bäuerle (joint work with Rudolf Grübel) Luminy, April 2010 Outline Multivariate pure birth processes Multivariate Risk Processes Fluid Limits
More informationSMSTC (2007/08) Probability.
SMSTC (27/8) Probability www.smstc.ac.uk Contents 12 Markov chains in continuous time 12 1 12.1 Markov property and the Kolmogorov equations.................... 12 2 12.1.1 Finite state space.................................
More informationMarkov processes and queueing networks
Inria September 22, 2015 Outline Poisson processes Markov jump processes Some queueing networks The Poisson distribution (Siméon-Denis Poisson, 1781-1840) { } e λ λ n n! As prevalent as Gaussian distribution
More informationPoint Process Control
Point Process Control The following note is based on Chapters I, II and VII in Brémaud s book Point Processes and Queues (1981). 1 Basic Definitions Consider some probability space (Ω, F, P). A real-valued
More information20. The pole diagram and the Laplace transform
95 0. The pole diagram and the Laplace transform When working with the Laplace transform, it is best to think of the variable s in F (s) as ranging over the complex numbers. In the first section below
More informationExact Simulation of the Stationary Distribution of M/G/c Queues
1/36 Exact Simulation of the Stationary Distribution of M/G/c Queues Professor Karl Sigman Columbia University New York City USA Conference in Honor of Søren Asmussen Monday, August 1, 2011 Sandbjerg Estate
More informationDisjointness and Additivity
Midterm 2: Format Midterm 2 Review CS70 Summer 2016 - Lecture 6D David Dinh 28 July 2016 UC Berkeley 8 questions, 190 points, 110 minutes (same as MT1). Two pages (one double-sided sheet) of handwritten
More informationFunctional Limit theorems for the quadratic variation of a continuous time random walk and for certain stochastic integrals
Functional Limit theorems for the quadratic variation of a continuous time random walk and for certain stochastic integrals Noèlia Viles Cuadros BCAM- Basque Center of Applied Mathematics with Prof. Enrico
More informationDynamical systems with Gaussian and Levy noise: analytical and stochastic approaches
Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches Noise is often considered as some disturbing component of the system. In particular physical situations, noise becomes
More informationMidterm 2 Review. CS70 Summer Lecture 6D. David Dinh 28 July UC Berkeley
Midterm 2 Review CS70 Summer 2016 - Lecture 6D David Dinh 28 July 2016 UC Berkeley Midterm 2: Format 8 questions, 190 points, 110 minutes (same as MT1). Two pages (one double-sided sheet) of handwritten
More information1. Stochastic Processes and filtrations
1. Stochastic Processes and 1. Stoch. pr., A stochastic process (X t ) t T is a collection of random variables on (Ω, F) with values in a measurable space (S, S), i.e., for all t, In our case X t : Ω S
More informationPrime numbers and Gaussian random walks
Prime numbers and Gaussian random walks K. Bruce Erickson Department of Mathematics University of Washington Seattle, WA 9895-4350 March 24, 205 Introduction Consider a symmetric aperiodic random walk
More informationUpper and lower bounds for ruin probability
Upper and lower bounds for ruin probability E. Pancheva,Z.Volkovich and L.Morozensky 3 Institute of Mathematics and Informatics, the Bulgarian Academy of Sciences, 3 Sofia, Bulgaria pancheva@math.bas.bg
More informationFourier Transform for Continuous Functions
Fourier Transform for Continuous Functions Central goal: representing a signal by a set of orthogonal bases that are corresponding to frequencies or spectrum. Fourier series allows to find the spectrum
More information2.161 Signal Processing: Continuous and Discrete Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 2.6 Signal Processing: Continuous and Discrete Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. MASSACHUSETTS
More informationIEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.
IEOR 6711: Stochastic Models I, Fall 23, Professor Whitt Solutions to Final Exam: Thursday, December 18. Below are six questions with several parts. Do as much as you can. Show your work. 1. Two-Pump Gas
More information1 Sequences of events and their limits
O.H. Probability II (MATH 2647 M15 1 Sequences of events and their limits 1.1 Monotone sequences of events Sequences of events arise naturally when a probabilistic experiment is repeated many times. For
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca November 26th, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationRENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO. X i
RENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO 1. RENEWAL PROCESSES A renewal process is the increasing sequence of random nonnegative numbers S 0,S 1,S 2,... gotten by adding i.i.d. positive random
More informationECE 3620: Laplace Transforms: Chapter 3:
ECE 3620: Laplace Transforms: Chapter 3: 3.1-3.4 Prof. K. Chandra ECE, UMASS Lowell September 21, 2016 1 Analysis of LTI Systems in the Frequency Domain Thus far we have understood the relationship between
More informationImportance Sampling Methodology for Multidimensional Heavy-tailed Random Walks
Importance Sampling Methodology for Multidimensional Heavy-tailed Random Walks Jose Blanchet (joint work with Jingchen Liu) Columbia IEOR Department RESIM Blanchet (Columbia) IS for Heavy-tailed Walks
More informationLecturer: Olga Galinina
Renewal models Lecturer: Olga Galinina E-mail: olga.galinina@tut.fi Outline Reminder. Exponential models definition of renewal processes exponential interval distribution Erlang distribution hyperexponential
More informationRENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO. X i
RENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO 1. RENEWAL PROCESSES A renewal process is the increasing sequence of random nonnegative numbers S 0,S 1,S 2,... gotten by adding i.i.d. positive random
More informationINHOMOGENEOUS LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS
INHOMOGENEOUS LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS Definitions and a general fact If A is an n n matrix and f(t) is some given vector function, then the system of differential equations () x (t) Ax(t)
More informationThe multidimensional Ito Integral and the multidimensional Ito Formula. Eric Mu ller June 1, 2015 Seminar on Stochastic Geometry and its applications
The multidimensional Ito Integral and the multidimensional Ito Formula Eric Mu ller June 1, 215 Seminar on Stochastic Geometry and its applications page 2 Seminar on Stochastic Geometry and its applications
More informationLecture 17 Brownian motion as a Markov process
Lecture 17: Brownian motion as a Markov process 1 of 14 Course: Theory of Probability II Term: Spring 2015 Instructor: Gordan Zitkovic Lecture 17 Brownian motion as a Markov process Brownian motion is
More informationHomework 4 due on Thursday, December 15 at 5 PM (hard deadline).
Large-Time Behavior for Continuous-Time Markov Chains Friday, December 02, 2011 10:58 AM Homework 4 due on Thursday, December 15 at 5 PM (hard deadline). How are formulas for large-time behavior of discrete-time
More informationMañé s Conjecture from the control viewpoint
Mañé s Conjecture from the control viewpoint Université de Nice - Sophia Antipolis Setting Let M be a smooth compact manifold of dimension n 2 be fixed. Let H : T M R be a Hamiltonian of class C k, with
More informationUniversal examples. Chapter The Bernoulli process
Chapter 1 Universal examples 1.1 The Bernoulli process First description: Bernoulli random variables Y i for i = 1, 2, 3,... independent with P [Y i = 1] = p and P [Y i = ] = 1 p. Second description: Binomial
More informationlim n C1/n n := ρ. [f(y) f(x)], y x =1 [f(x) f(y)] [g(x) g(y)]. (x,y) E A E(f, f),
1 Part I Exercise 1.1. Let C n denote the number of self-avoiding random walks starting at the origin in Z of length n. 1. Show that (Hint: Use C n+m C n C m.) lim n C1/n n = inf n C1/n n := ρ.. Show that
More informationRandom Locations of Periodic Stationary Processes
Random Locations of Periodic Stationary Processes Yi Shen Department of Statistics and Actuarial Science University of Waterloo Joint work with Jie Shen and Ruodu Wang May 3, 2016 The Fields Institute
More informationChap 4. State-Space Solutions and
Chap 4. State-Space Solutions and Realizations Outlines 1. Introduction 2. Solution of LTI State Equation 3. Equivalent State Equations 4. Realizations 5. Solution of Linear Time-Varying (LTV) Equations
More informationStochastic Modelling Unit 1: Markov chain models
Stochastic Modelling Unit 1: Markov chain models Russell Gerrard and Douglas Wright Cass Business School, City University, London June 2004 Contents of Unit 1 1 Stochastic Processes 2 Markov Chains 3 Poisson
More informationMalliavin Calculus in Finance
Malliavin Calculus in Finance Peter K. Friz 1 Greeks and the logarithmic derivative trick Model an underlying assent by a Markov process with values in R m with dynamics described by the SDE dx t = b(x
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationRENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO. X i
RENEWAL THEORY STEVEN P. LALLEY UNIVERSITY OF CHICAGO 1. RENEWAL PROCESSES A renewal process is the increasing sequence of random nonnegative numbers S 0,S 1,S 2,... gotten by adding i.i.d. positive random
More informationStochastic Processes
Stochastic Processes 8.445 MIT, fall 20 Mid Term Exam Solutions October 27, 20 Your Name: Alberto De Sole Exercise Max Grade Grade 5 5 2 5 5 3 5 5 4 5 5 5 5 5 6 5 5 Total 30 30 Problem :. True / False
More informationExponential functionals of Lévy processes
Exponential functionals of Lévy processes Víctor Rivero Centro de Investigación en Matemáticas, México. 1/ 28 Outline of the talk Introduction Exponential functionals of spectrally positive Lévy processes
More informationMath Ordinary Differential Equations
Math 411 - Ordinary Differential Equations Review Notes - 1 1 - Basic Theory A first order ordinary differential equation has the form x = f(t, x) (11) Here x = dx/dt Given an initial data x(t 0 ) = x
More informationECEN 420 LINEAR CONTROL SYSTEMS. Lecture 2 Laplace Transform I 1/52
1/52 ECEN 420 LINEAR CONTROL SYSTEMS Lecture 2 Laplace Transform I Linear Time Invariant Systems A general LTI system may be described by the linear constant coefficient differential equation: a n d n
More informationBrownian Motion. 1 Definition Brownian Motion Wiener measure... 3
Brownian Motion Contents 1 Definition 2 1.1 Brownian Motion................................. 2 1.2 Wiener measure.................................. 3 2 Construction 4 2.1 Gaussian process.................................
More informationSTAT 380 Markov Chains
STAT 380 Markov Chains Richard Lockhart Simon Fraser University Spring 2016 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring 2016 1 / 38 1/41 PoissonProcesses.pdf (#2) Poisson Processes
More informationHarmonic Functions and Brownian motion
Harmonic Functions and Brownian motion Steven P. Lalley April 25, 211 1 Dynkin s Formula Denote by W t = (W 1 t, W 2 t,..., W d t ) a standard d dimensional Wiener process on (Ω, F, P ), and let F = (F
More informationRecap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks
Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 15. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationELEMENTS OF PROBABILITY THEORY
ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable
More informationLiquidation in Limit Order Books. LOBs with Controlled Intensity
Limit Order Book Model Power-Law Intensity Law Exponential Decay Order Books Extensions Liquidation in Limit Order Books with Controlled Intensity Erhan and Mike Ludkovski University of Michigan and UCSB
More informationStability of Stochastic Differential Equations
Lyapunov stability theory for ODEs s Stability of Stochastic Differential Equations Part 1: Introduction Department of Mathematics and Statistics University of Strathclyde Glasgow, G1 1XH December 2010
More informationExercises with solutions (Set D)
Exercises with solutions Set D. A fair die is rolled at the same time as a fair coin is tossed. Let A be the number on the upper surface of the die and let B describe the outcome of the coin toss, where
More informationDefinition of the Laplace transform. 0 x(t)e st dt
Definition of the Laplace transform Bilateral Laplace Transform: X(s) = x(t)e st dt Unilateral (or one-sided) Laplace Transform: X(s) = 0 x(t)e st dt ECE352 1 Definition of the Laplace transform (cont.)
More informationCauchy Problems in Bounded Domains and Iterated Processes
Cauchy Problems in Bounded Domains and Iterated Processes Erkan Nane DEPARTMENT OF MATHEMATICS AND STATISTICS AUBURN UNIVERSITY August 18, 21 Continuous time random walks R d X 2 X 3 X 1 J 1 J 2 J 3 J
More informationExercises. T 2T. e ita φ(t)dt.
Exercises. Set #. Construct an example of a sequence of probability measures P n on R which converge weakly to a probability measure P but so that the first moments m,n = xdp n do not converge to m = xdp.
More informationOutlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC)
Markov Chains (2) Outlines Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) 2 pj ( n) denotes the pmf of the random variable p ( n) P( X j) j We will only be concerned with homogenous
More informationCentral limit theorems for ergodic continuous-time Markov chains with applications to single birth processes
Front. Math. China 215, 1(4): 933 947 DOI 1.17/s11464-15-488-5 Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes Yuanyuan LIU 1, Yuhui ZHANG 2
More information