Markov processes and queueing networks
|
|
- Eugenia Watts
- 5 years ago
- Views:
Transcription
1 Inria September 22, 2015
2 Outline Poisson processes Markov jump processes Some queueing networks
3 The Poisson distribution (Siméon-Denis Poisson, ) { } e λ λ n n! As prevalent as Gaussian distribution n N
4 The Poisson distribution (Siméon-Denis Poisson, ) { } e λ λ n n! As prevalent as Gaussian distribution n N Law of rare events (a.k.a. law of small numbers) p n,i 0 such that lim n sup i p n,i = 0, lim n i p n,i = λ > 0
5 The Poisson distribution (Siméon-Denis Poisson, ) { } e λ λ n n! As prevalent as Gaussian distribution n N Law of rare events (a.k.a. law of small numbers) p n,i 0 such that lim n sup i p n,i = 0, lim n i p n,i = λ > 0 Then X n = i Z n,i with Z n,i : independent Bernoulli(p n,i ) verifies X n D Poisson(λ)
6 Point process on R + Definition Point process on R + : Collection of random times {T n } n>0 with 0 < T 1 < T 2... Alternative description Collection {N t } t R+ with N t := n>0 I T n [0,t] Yet another description Collection {N(C)} for all measurable C R + where N(C) := n>0 I Tn C
7 Poisson process on R + Definition Point process such that for all s 0 = 0 < s 1 < s 2 <... < s n, 1 Increments {N si N si 1 } 1 i n independent 2 Law of N t+s N s only depends on t 3 for some λ > 0, N t Poisson(λt)
8 Poisson process on R + Definition Point process such that for all s 0 = 0 < s 1 < s 2 <... < s n, 1 Increments {N si N si 1 } 1 i n independent 2 Law of N t+s N s only depends on t 3 for some λ > 0, N t Poisson(λt) In fact, (3) follows from (1) (2)
9 Poisson process on R + Definition Point process such that for all s 0 = 0 < s 1 < s 2 <... < s n, 1 Increments {N si N si 1 } 1 i n independent 2 Law of N t+s N s only depends on t 3 for some λ > 0, N t Poisson(λt) In fact, (3) follows from (1) (2) λ is called the intensity of the process
10 A first construction Given λn i.i.d. numbers U n,i, uniform on [0, n], let N (n) t := n i=1 I U n,i t Then for any k N, s 0 = 0 < s 1 < s 2 <... < s n, {N (n) s i N (n) s i 1 } 1 i k D 1 i k Poisson(λ(s i s i 1 ))
11 A first construction Given λn i.i.d. numbers U n,i, uniform on [0, n], let N (n) t := n i=1 I U n,i t Then for any k N, s 0 = 0 < s 1 < s 2 <... < s n, {N (n) s i N (n) s i 1 } 1 i k D 1 i k Poisson(λ(s i s i 1 )) Proof: Multinomial distribution of {N (n) s i N (n) s i 1 } 1 i k Convergence of Laplace transform E exp( k i=1 α i(n s (n) i N s (n) i 1 )) for all α1 k Rk +
12 A first construction Given λn i.i.d. numbers U n,i, uniform on [0, n], let N (n) t := n i=1 I U n,i t Then for any k N, s 0 = 0 < s 1 < s 2 <... < s n, {N (n) s i N (n) s i 1 } 1 i k D 1 i k Poisson(λ(s i s i 1 )) Proof: Multinomial distribution of {N (n) s i N (n) s i 1 } 1 i k Convergence of Laplace transform E exp( k i=1 α i(n s (n) i N s (n) i 1 )) for all α1 k Rk + Suggests Poisson processes exist and are limits of this construction
13 A second characterization Proposition For Poisson process {T n } n>0 of intensity λ, its interarrival times τ i = T i+1 T i, where T 0 = 0, verify {τ n } n 0 i.i.d. with common distribution Exp(λ)
14 A second characterization Proposition For Poisson process {T n } n>0 of intensity λ, its interarrival times τ i = T i+1 T i, where T 0 = 0, verify {τ n } n 0 i.i.d. with common distribution Exp(λ) Density of Exp(λ): λe λx I x>0
15 A second characterization Proposition For Poisson process {T n } n>0 of intensity λ, its interarrival times τ i = T i+1 T i, where T 0 = 0, verify {τ n } n 0 i.i.d. with common distribution Exp(λ) Density of Exp(λ): λe λx I x>0 Key property: Exponential random variable τ is memoryless, i.e. t > 0, P(τ t τ > t) = P(τ )
16 A third characterization Proposition Process with i.i.d., Exp(λ) interarrival times {τ i } i 0 can be constructed on [0, t] by 1) Drawing N t Poisson(λt) 2) Putting N t points U 1,..., U Nt on [0, t] where U i : i.i.d. uniform on [0, t]
17 A third characterization Proposition Process with i.i.d., Exp(λ) interarrival times {τ i } i 0 can be constructed on [0, t] by 1) Drawing N t Poisson(λt) 2) Putting N t points U 1,..., U Nt on [0, t] where U i : i.i.d. uniform on [0, t] Proof. Establish identity for all n N, φ : R n + R: where S n 1 E[φ(τ 0, τ 0 + τ 1,..., τ τ n 1 )I Nt=n] = e λt (λt) n n! n! (0,t] n φ(s 1, s 2,..., s n )I s1 <s 2 <...<s n n i=1 ds i = P(Poisson(λt) = n) E[φ(S 1,..., S n )] : sorted version of i.i.d. variables uniform on [0, t]
18 Laplace transform of Poisson processes Definition The Laplace transform of point process N {T n } n>0 is the functional whose evaluation at f : R + R + is L N (f ) := E exp( N(f )) = E(exp( n>0 f (T n ))).
19 Laplace transform of Poisson processes Definition The Laplace transform of point process N {T n } n>0 is the functional whose evaluation at f : R + R + is L N (f ) := E exp( N(f )) = E(exp( n>0 f (T n ))). Proposition: Knowledge of L N (f ) on sufficiently rich class of functions f : R + R + (e.g. piecewise continuous with compact support) characterizes law of point process N.
20 Laplace transform of Poisson processes Definition The Laplace transform of point process N {T n } n>0 is the functional whose evaluation at f : R + R + is L N (f ) := E exp( N(f )) = E(exp( n>0 f (T n ))). Proposition: Knowledge of L N (f ) on sufficiently rich class of functions f : R + R + (e.g. piecewise continuous with compact support) characterizes law of point process N. Laplace transform of Poisson process with intensity λ: L N (f ) = exp( λ(1 e f (x) )dx) R +
21 Laplace transform of Poisson processes Definition The Laplace transform of point process N {T n } n>0 is the functional whose evaluation at f : R + R + is L N (f ) := E exp( N(f )) = E(exp( n>0 f (T n ))). Proposition: Knowledge of L N (f ) on sufficiently rich class of functions f : R + R + (e.g. piecewise continuous with compact support) characterizes law of point process N. Laplace transform of Poisson process with intensity λ: L N (f ) = exp( λ(1 e f (x) )dx) R + (i) Previous construction yields expression for L N (f ) (ii) For f = i α i I Ci N(C i ) Poisson(λ C i dx), with independence for disjoint C i. Hence existence of Poisson process...
22 Poisson process with general space and intensity Definition For λ : R d R + locally integrable function, N {T n } n>0 point process on R d is Poisson with intensity function λ if and only if for measurable, disjoint C i R d, i = 1,..., n, N(C i ) independent, Poisson( C i λ(x)dx)
23 Poisson process with general space and intensity Definition For λ : R d R + locally integrable function, N {T n } n>0 point process on R d is Poisson with intensity function λ if and only if for measurable, disjoint C i R d, i = 1,..., n, N(C i ) independent, Poisson( C i λ(x)dx) Proposition Such a process exists and admits Laplace transform ( ) L N (f ) = exp λ(x)(1 e f (x) )dx R d
24 Further properties Strong Markov property: Poisson process N with intensity λ, stopping time T (i.e. t 0, {T t} σ(n s, s t)) then on {T < + }, {N T +t N T } t 0 : Poisson with intensity λ and independent of {N s } s T
25 Further properties Strong Markov property: Poisson process N with intensity λ, stopping time T (i.e. t 0, {T t} σ(n s, s t)) then on {T < + }, {N T +t N T } t 0 : Poisson with intensity λ and independent of {N s } s T Superposition: For independent Poisson processes N i with intensities λ i, i = 1,..., n then N = i N i: Poisson with intensity i λ i
26 Further properties Strong Markov property: Poisson process N with intensity λ, stopping time T (i.e. t 0, {T t} σ(n s, s t)) then on {T < + }, {N T +t N T } t 0 : Poisson with intensity λ and independent of {N s } s T Superposition: For independent Poisson processes N i with intensities λ i, i = 1,..., n then N = i N i: Poisson with intensity i λ i Thinning: For Poisson process N {T n } n>0 with intensity λ, {Z n } n>0 independent of N, i.i.d., valued in [k], processes N i : N i (C) = n>0 I T n C I Zn=i are independent, Poisson with intensities λ i = λp(z n = i)
27 Markov jump processes Process {X t } t R+ with values in E, countable or finite, is Markov if P(X tn = x n X tn 1 = x n 1,... X t1 = x 1 ) = P(X tn = x n X tn 1 = x n 1 ), t n 1 Rn +, t 1 < < t n, x n 1 E n Homogeneous if P(X t+s = y X s = x) =: p xy (t) independent of s, s, t R +, x, y E
28 Markov jump processes Process {X t } t R+ with values in E, countable or finite, is Markov if P(X tn = x n X tn 1 = x n 1,... X t1 = x 1 ) = P(X tn = x n X tn 1 = x n 1 ), t n 1 Rn +, t 1 < < t n, x n 1 E n Homogeneous if P(X t+s = y X s = x) =: p xy (t) independent of s, s, t R +, x, y E Semi-group property p xy (t + s) = z E p xz(t)p zy (s), or P(t + s) = P(t)P(s) with P(t) = {p xy (t)} x,y E
29 Markov jump processes Process {X t } t R+ with values in E, countable or finite, is Markov if P(X tn = x n X tn 1 = x n 1,... X t1 = x 1 ) = P(X tn = x n X tn 1 = x n 1 ), t n 1 Rn +, t 1 < < t n, x n 1 E n Homogeneous if P(X t+s = y X s = x) =: p xy (t) independent of s, s, t R +, x, y E Semi-group property p xy (t + s) = z E p xz(t)p zy (s), or P(t + s) = P(t)P(s) with P(t) = {p xy (t)} x,y E Definition {X t } t R+ is a pure jump Markov process if in addition (i) It spends with probability 1 a strictly positive time in each state (ii) Trajectories t X t are right-continuous
30 Markov jump processes: examples Poisson process {N t } t R+ : then Markov jump process with p xy (t) = P(Poisson(λt) = y x)
31 Markov jump processes: examples Poisson process {N t } t R+ : then Markov jump process with p xy (t) = P(Poisson(λt) = y x) Single-server queue, FIFO ( First-in-first-out ) discipline, arrival times: N Poisson (λ), service times: i.i.d. Exp(µ) independent of N X t = number of customers present at time t: Markov jump process by Memoryless property of Exponential distribution + Markov property of Poisson process (the M/M/1/ queue)
32 Markov jump processes: examples Poisson process {N t } t R+ : then Markov jump process with p xy (t) = P(Poisson(λt) = y x) Single-server queue, FIFO ( First-in-first-out ) discipline, arrival times: N Poisson (λ), service times: i.i.d. Exp(µ) independent of N X t = number of customers present at time t: Markov jump process by Memoryless property of Exponential distribution + Markov property of Poisson process (the M/M/1/ queue) Infinite server queue with Poisson arrivals and Exponential service times: customer arrived at T n stays in system till T n + σ n, where σ n : service time X t = number of customers present at time t: Markov jump process (the M/M/ / queue)
33 Structure of Markov jump processes Infinitesimal Generator 1 p x, y, y x E, limits q x := lim xx (t) p t 0 t, q xy = lim xy (t) t 0 t exist in R + and satisfy y x q xy = q x q xy : Jump rate from x to y Q := {q xy } x,y E where q xx = q x : Infinitesimal Generator of process {X t } t R+ 1 Formally: Q = lim h 0 h [P(h) I ] where I : identity matrix
34 Structure of Markov jump processes Infinitesimal Generator 1 p x, y, y x E, limits q x := lim xx (t) p t 0 t, q xy = lim xy (t) t 0 t exist in R + and satisfy y x q xy = q x q xy : Jump rate from x to y Q := {q xy } x,y E where q xx = q x : Infinitesimal Generator of process {X t } t R+ 1 Formally: Q = lim h 0 h [P(h) I ] where I : identity matrix Structure of Markov jump processes Sequence {Y n } n N of visited states: Markov chain with transition q matrix p xy = I xy x y q x Conditionally on {Y n } n N, sojourn times {τ n } n N in successive states Y n : independent, with distributions Exp(q Yn )
35 Examples for Poisson process (λ): only non-zero jump rate q x,x+1 = λ = q x, x N
36 Examples for Poisson process (λ): only non-zero jump rate q x,x+1 = λ = q x, x N For FIFO M/M/1/ queue, non-zero rates: q x,x+1 = λ, q x,x 1 = µi x>0, x N hence q x = λ + µi x>0
37 Examples for Poisson process (λ): only non-zero jump rate q x,x+1 = λ = q x, x N For FIFO M/M/1/ queue, non-zero rates: q x,x+1 = λ, q x,x 1 = µi x>0, x N hence q x = λ + µi x>0 For M/M/ / queue, non-zero rates: q x,x+1 = λ, q x,x 1 = µx, x N hence q x = λ + µx
38 Structure of Markov jump processes (continued) Let T n := n 1 k=0 τ k: time of n-th jump. If T = + almost surely: trajectory determined on R +, hence generator Q determines law of process {X t } t R+
39 Structure of Markov jump processes (continued) Let T n := n 1 k=0 τ k: time of n-th jump. If T = + almost surely: trajectory determined on R +, hence generator Q determines law of process {X t } t R+ Process is called explosive if instead T < + with positive probability. Then process not completely characterized by generator
40 Structure of Markov jump processes (continued) Let T n := n 1 k=0 τ k: time of n-th jump. If T = + almost surely: trajectory determined on R +, hence generator Q determines law of process {X t } t R+ Process is called explosive if instead T < + with positive probability. Then process not completely characterized by generator Sufficient conditions for non-explosiveness: sup x E q x < + Recurrence of induced chain {Y n } n N For Birth and Death processes (i.e. E = N, only non-zero rates: β n = q n,n+1, birth rate; δ n = q n,n 1, death rate), non-explosiveness holds if 1 = + β n + δ n n>0
41 Kolmogorov s forward and backward equations Formal differentiation of P(t + h) = P(t)P(h) = P(h)P(t) yields d dt P(t) = P(t)Q d dt p xy(t) = z E p xz(t)q zy d dt d P(t) = QP(t) dt p xy(t) = z E q xzp zy (t) Kolmogorov s forward equation Kolmogorov s backward equation
42 Kolmogorov s forward and backward equations Formal differentiation of P(t + h) = P(t)P(h) = P(h)P(t) yields d dt P(t) = P(t)Q d dt p xy(t) = z E p xz(t)q zy d dt d P(t) = QP(t) dt p xy(t) = z E q xzp zy (t) Kolmogorov s forward equation Kolmogorov s backward equation 1 Follow directly from Q = lim h 0 h [P(h) I ] for finite E, in which case P(t) = exp(tq), t 0 Hold more generally in particular for non-explosive processes with a more involved proof (justifying exchange of summation and differentiation)
43 Stationary distributions and measures Definition Measure {π x } x E is stationary if it satisfies π T Q = 0, or equivalently the global balance equations x E, π x y x q xy = y x π y q yx flow out of x flow into x,
44 Stationary distributions and measures Definition Measure {π x } x E is stationary if it satisfies π T Q = 0, or equivalently the global balance equations x E, π x y x q xy = y x π y q yx flow out of x flow into x Kolmogorov s equations suggest that, if X 0 π for stationary π then X t π for all t 0,
45 Stationary distributions and measures Definition Measure {π x } x E is stationary if it satisfies π T Q = 0, or equivalently the global balance equations x E, π x y x q xy = y x π y q yx flow out of x flow into x Kolmogorov s equations suggest that, if X 0 π for stationary π then X t π for all t 0, Example: stationarity for birth and death processes π 0 β 0 = π 1 δ 1, π x (β x + δ x ) = π x 1 β x 1 + π x+1 δ x+1, x 1
46 Irreducibility, recurrence, invariance Definition Process {X t } t R+ is irreducible (respectively, irreducible recurrent) if induced chain {Y n } n N is. State x is positive recurrent if E x (R x ) < +, where R x = inf{t > τ 0 : X t = x}. Measure π is invariant for process {X t } t R+ if for all t > 0, π T P(t) = π T, i.e. x E, y E π y p yx (t) = π x.
47 Limit theorems 1 Theorem For irreducible recurrent {X t } t R+, invariant measure π, unique up to some scalar factor. It can be defined as, for any x E: Rx y E, π y = E x I Xt=y dt, or alternatively with T x := inf{n > 0 : Y n = x}, y E, π y = 1 T x E x I Yn=y. q y 0 n=1
48 Limit theorems 1 Theorem For irreducible recurrent {X t } t R+, invariant measure π, unique up to some scalar factor. It can be defined as, for any x E: Rx y E, π y = E x I Xt=y dt, or alternatively with T x := inf{n > 0 : Y n = x}, y E, π y = 1 T x E x I Yn=y. q y 0 n=1 Corollaries {ˆπ y } invariant for {Y n } n N {ˆπ y /q y } invariant for {X t } t R+. For irreducible recurrent {X t } t R+, either all or no state x E is positive recurrent.
49 Limit theorems 2 Theorem {X t } t R+ is ergodic (i.e. irreducible, positive recurrent) iff it is irreducible, non-explosive and such that π satisfying global balance equations. Then π is also the unique invariant probability distribution.
50 Limit theorems 2 Theorem {X t } t R+ is ergodic (i.e. irreducible, positive recurrent) iff it is irreducible, non-explosive and such that π satisfying global balance equations. Then π is also the unique invariant probability distribution. Theorem For ergodic {X t } t R+ with stationary distribution π, any initial distribution for X 0 and π-integrable f, 1 t almost surely lim f (X s )ds = π x f (x) (ergodic theorem) t t 0 x E and in distribution X t D π as t.
51 Limit theorems 3 Theorem For irreducible, non-ergodic {X t } t R+, any initial distribution for X 0, then for all x E, lim P(X t = x) = 0. t
52 Time reversal and reversibility For stationary ergodic {X t } t R with stationary distribution π, time-reversed process X t = X t : πy qyx Markov with transition rates q xy = π x
53 Time reversal and reversibility For stationary ergodic {X t } t R with stationary distribution π, time-reversed process X t = X t : πy qyx Markov with transition rates q xy = π x Definition Stationary ergodic {X t } t R with stationary distribution π reversible iff distributed as time-reversal { X t } t R, i.e. x y E, π x q xy = π y q yx, flow from x to y flow from y to x detailed balance equations.
54 Time reversal and reversibility For stationary ergodic {X t } t R with stationary distribution π, time-reversed process X t = X t : πy qyx Markov with transition rates q xy = π x Definition Stationary ergodic {X t } t R with stationary distribution π reversible iff distributed as time-reversal { X t } t R, i.e. x y E, π x q xy = π y q yx, flow from x to y flow from y to x detailed balance equations. Detailed balance, i.e. reversibility for π implies global balance for π. Example: for birth and death processes, detailed balance always holds for stationary measure.
55 Reversibility and truncation Proposition Let generator Q on E admit reversible measure π. Then for subset F E, truncated generator ˆQ: ˆQ xy = Q xy, x y F, ˆQ xx = ˆQ y x xy, x F admits {π x } x F as reversible measure.
56 Erlang s model of telephone network Call types s S: type-s calls arrive at instants of Poisson (λ s ) process, last (if accepted) for duration Exponential (µ s ) type-s calls require one circuit (unit of capacity) per link l s Link l has capacity C l circuits
57 Erlang s model of telephone network Call types s S: type-s calls arrive at instants of Poisson (λ s ) process, last (if accepted) for duration Exponential (µ s ) type-s calls require one circuit (unit of capacity) per link l s Link l has capacity C l circuits Stationary probability distribution: π x = 1 Z s S ρ xs s x s! I s l xs C, l where: ρ s = λ s /µ s, Z: normalizing constant. l
58 Erlang s model of telephone network Call types s S: type-s calls arrive at instants of Poisson (λ s ) process, last (if accepted) for duration Exponential (µ s ) type-s calls require one circuit (unit of capacity) per link l s Link l has capacity C l circuits Stationary probability distribution: π x = 1 Z s S ρ xs s x s! I s l xs C, l where: ρ s = λ s /µ s, Z: normalizing constant. Basis for dimensioning studies of telephone networks (prediction of call rejection probabilities) More recent application: performance analysis of peer-to-peer systems for video streaming. l
59 Jackson networks Stations i I receive external arrivals at Poisson rate λ i Station i when processing x i customers completes service at rate µ i φ i (x i ) (e.g.: φ i (x) = min(x i, n i ): queue with n i servers and service times Exponential (µ i )) After completing service at station i, customer joins station j with probability p ij, j I, and leaves system with probability 1 j I p ij Matrix P = (p ij ): sub-stochastic, such that (I P) 1
60 Jackson networks Stations i I receive external arrivals at Poisson rate λ i Station i when processing x i customers completes service at rate µ i φ i (x i ) (e.g.: φ i (x) = min(x i, n i ): queue with n i servers and service times Exponential (µ i )) After completing service at station i, customer joins station j with probability p ij, j I, and leaves system with probability 1 j I p ij Matrix P = (p ij ): sub-stochastic, such that (I P) 1 Traffic equations i I, λ i = λ i + j I λ j p ji or λ = (I P T ) 1 λ
61 Jackson networks (continued) Stationary measure: π x = i I xi ρ x i i m=1 φ i(m), where ρ i = λ i /µ i, and λ i : solutions of traffic equations
62 Jackson networks (continued) Stationary measure: π x = i I xi ρ x i i m=1 φ i(m), where ρ i = λ i /µ i, and λ i : solutions of traffic equations Application: process ergodic when π has finite mass. e.g. for φ i (x) = min(x, n i ), ergodicity iff i I, ρ i < n i.
63 Jackson networks (continued) Stationary measure: π x = i I xi ρ x i i m=1 φ i(m), where ρ i = λ i /µ i, and λ i : solutions of traffic equations Application: process ergodic when π has finite mass. e.g. for φ i (x) = min(x, n i ), ergodicity iff i I, ρ i < n i. Proof: verify partial balance equations for all x N I : i I, π x [ j i q x,x e i +e j + q x,x ei ] = j i π x e i +e j q x ei +e j,x + π x ei q x ei,x, π x i I q x,x+e i = i I π x+e i q x+ei,x, which imply global balance equations π x [ i I (q x,x e i + q x,x+ei + j i q x,x e i +e j )] = i I (π x e i q x ei,x + π x+ei q x+ei,x + j i π x e i +e j q x ei +e j,x)
64 Takeaway messages Poisson process a fundamental continuous-time process, adequate model for aggregate of infrequent independent events Markov jump processes: i) generator Q characterizes distribution if not explosive ii) Balance equation characterizes invariant distribution if irreducible non-explosive iii) Limit theorems: stationary distribution reflects long-term performance Exactly solvable models include reversible processes, plus several other important classes (e.g. Jackson networks)
Part I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationIEOR 6711, HMWK 5, Professor Sigman
IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.
More informationLecture 20: Reversible Processes and Queues
Lecture 20: Reversible Processes and Queues 1 Examples of reversible processes 11 Birth-death processes We define two non-negative sequences birth and death rates denoted by {λ n : n N 0 } and {µ n : n
More informationStochastic process. X, a series of random variables indexed by t
Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,
More informationMarkov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015
Markov chains c A. J. Ganesh, University of Bristol, 2015 1 Discrete time Markov chains Example: A drunkard is walking home from the pub. There are n lampposts between the pub and his home, at each of
More informationRecap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks
Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution
More informationLecture 7: Simulation of Markov Processes. Pasi Lassila Department of Communications and Networking
Lecture 7: Simulation of Markov Processes Pasi Lassila Department of Communications and Networking Contents Markov processes theory recap Elementary queuing models for data networks Simulation of Markov
More informationStatistics 150: Spring 2007
Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities
More informationData analysis and stochastic modeling
Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt
More informationSTAT 380 Continuous Time Markov Chains
STAT 380 Continuous Time Markov Chains Richard Lockhart Simon Fraser University Spring 2018 Richard Lockhart (Simon Fraser University)STAT 380 Continuous Time Markov Chains Spring 2018 1 / 35 Continuous
More informationFigure 10.1: Recording when the event E occurs
10 Poisson Processes Let T R be an interval. A family of random variables {X(t) ; t T} is called a continuous time stochastic process. We often consider T = [0, 1] and T = [0, ). As X(t) is a random variable
More informationPerformance Evaluation of Queuing Systems
Performance Evaluation of Queuing Systems Introduction to Queuing Systems System Performance Measures & Little s Law Equilibrium Solution of Birth-Death Processes Analysis of Single-Station Queuing Systems
More informationIntroduction to Markov Chains, Queuing Theory, and Network Performance
Introduction to Markov Chains, Queuing Theory, and Network Performance Marceau Coupechoux Telecom ParisTech, departement Informatique et Réseaux marceau.coupechoux@telecom-paristech.fr IT.2403 Modélisation
More informationChapter 5. Continuous-Time Markov Chains. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Continuous-Time Markov Chains Consider a continuous-time stochastic process
More informationThe Transition Probability Function P ij (t)
The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it
More information1 Continuous-time chains, finite state space
Université Paris Diderot 208 Markov chains Exercises 3 Continuous-time chains, finite state space Exercise Consider a continuous-time taking values in {, 2, 3}, with generator 2 2. 2 2 0. Draw the diagramm
More informationELEMENTS OF PROBABILITY THEORY
ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable
More informationGI/M/1 and GI/M/m queuing systems
GI/M/1 and GI/M/m queuing systems Dmitri A. Moltchanov moltchan@cs.tut.fi http://www.cs.tut.fi/kurssit/tlt-2716/ OUTLINE: GI/M/1 queuing system; Methods of analysis; Imbedded Markov chain approach; Waiting
More informationA review of Continuous Time MC STA 624, Spring 2015
A review of Continuous Time MC STA 624, Spring 2015 Ruriko Yoshida Dept. of Statistics University of Kentucky polytopes.net STA 624 1 Continuous Time Markov chains Definition A continuous time stochastic
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationIntroduction to Queuing Networks Solutions to Problem Sheet 3
Introduction to Queuing Networks Solutions to Problem Sheet 3 1. (a) The state space is the whole numbers {, 1, 2,...}. The transition rates are q i,i+1 λ for all i and q i, for all i 1 since, when a bus
More informationChapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 2. Poisson Processes Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Outline Introduction to Poisson Processes Definition of arrival process Definition
More informationContinuous Time Processes
page 102 Chapter 7 Continuous Time Processes 7.1 Introduction In a continuous time stochastic process (with discrete state space), a change of state can occur at any time instant. The associated point
More informationSTATS 3U03. Sang Woo Park. March 29, Textbook: Inroduction to stochastic processes. Requirement: 5 assignments, 2 tests, and 1 final
STATS 3U03 Sang Woo Park March 29, 2017 Course Outline Textbook: Inroduction to stochastic processes Requirement: 5 assignments, 2 tests, and 1 final Test 1: Friday, February 10th Test 2: Friday, March
More informationStability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk
Stability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk ANSAPW University of Queensland 8-11 July, 2013 1 Outline (I) Fluid
More informationCDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical
CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one
More informationContents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory
Contents Preface... v 1 The Exponential Distribution and the Poisson Process... 1 1.1 Introduction... 1 1.2 The Density, the Distribution, the Tail, and the Hazard Functions... 2 1.2.1 The Hazard Function
More informationLECTURE #6 BIRTH-DEATH PROCESS
LECTURE #6 BIRTH-DEATH PROCESS 204528 Queueing Theory and Applications in Networks Assoc. Prof., Ph.D. (รศ.ดร. อน นต ผลเพ ม) Computer Engineering Department, Kasetsart University Outline 2 Birth-Death
More informationIrreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1
Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate
More informationContinuous-Time Markov Chain
Continuous-Time Markov Chain Consider the process {X(t),t 0} with state space {0, 1, 2,...}. The process {X(t),t 0} is a continuous-time Markov chain if for all s, t 0 and nonnegative integers i, j, x(u),
More informationCDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes
CDA6530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic process X = {X(t), t2 T} is a collection of random variables (rvs); one rv
More informationQueueing Networks and Insensitivity
Lukáš Adam 29. 10. 2012 1 / 40 Table of contents 1 Jackson networks 2 Insensitivity in Erlang s Loss System 3 Quasi-Reversibility and Single-Node Symmetric Queues 4 Quasi-Reversibility in Networks 5 The
More information1 IEOR 4701: Continuous-Time Markov Chains
Copyright c 2006 by Karl Sigman 1 IEOR 4701: Continuous-Time Markov Chains A Markov chain in discrete time, {X n : n 0}, remains in any state for exactly one unit of time before making a transition (change
More informationQuantitative Model Checking (QMC) - SS12
Quantitative Model Checking (QMC) - SS12 Lecture 06 David Spieler Saarland University, Germany June 4, 2012 1 / 34 Deciding Bisimulations 2 / 34 Partition Refinement Algorithm Notation: A partition P over
More informationM/G/1 and M/G/1/K systems
M/G/1 and M/G/1/K systems Dmitri A. Moltchanov dmitri.moltchanov@tut.fi http://www.cs.tut.fi/kurssit/elt-53606/ OUTLINE: Description of M/G/1 system; Methods of analysis; Residual life approach; Imbedded
More informationIEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.
IEOR 6711: Stochastic Models I, Fall 23, Professor Whitt Solutions to Final Exam: Thursday, December 18. Below are six questions with several parts. Do as much as you can. Show your work. 1. Two-Pump Gas
More informationChapter 1. Introduction. 1.1 Stochastic process
Chapter 1 Introduction Process is a phenomenon that takes place in time. In many practical situations, the result of a process at any time may not be certain. Such a process is called a stochastic process.
More informationExact Simulation of the Stationary Distribution of M/G/c Queues
1/36 Exact Simulation of the Stationary Distribution of M/G/c Queues Professor Karl Sigman Columbia University New York City USA Conference in Honor of Søren Asmussen Monday, August 1, 2011 Sandbjerg Estate
More informationTCOM 501: Networking Theory & Fundamentals. Lecture 6 February 19, 2003 Prof. Yannis A. Korilis
TCOM 50: Networking Theory & Fundamentals Lecture 6 February 9, 003 Prof. Yannis A. Korilis 6- Topics Time-Reversal of Markov Chains Reversibility Truncating a Reversible Markov Chain Burke s Theorem Queues
More informationIntro Refresher Reversibility Open networks Closed networks Multiclass networks Other networks. Queuing Networks. Florence Perronnin
Queuing Networks Florence Perronnin Polytech Grenoble - UGA March 23, 27 F. Perronnin (UGA) Queuing Networks March 23, 27 / 46 Outline Introduction to Queuing Networks 2 Refresher: M/M/ queue 3 Reversibility
More informationSTAT STOCHASTIC PROCESSES. Contents
STAT 3911 - STOCHASTIC PROCESSES ANDREW TULLOCH Contents 1. Stochastic Processes 2 2. Classification of states 2 3. Limit theorems for Markov chains 4 4. First step analysis 5 5. Branching processes 5
More informationLecture 4a: Continuous-Time Markov Chain Models
Lecture 4a: Continuous-Time Markov Chain Models Continuous-time Markov chains are stochastic processes whose time is continuous, t [0, ), but the random variables are discrete. Prominent examples of continuous-time
More informationClass 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis.
Service Engineering Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis. G/G/1 Queue: Virtual Waiting Time (Unfinished Work). GI/GI/1: Lindley s Equations
More informationPoisson Processes. Stochastic Processes. Feb UC3M
Poisson Processes Stochastic Processes UC3M Feb. 2012 Exponential random variables A random variable T has exponential distribution with rate λ > 0 if its probability density function can been written
More informationSTA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008
Name STA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008 There are five questions on this test. DO use calculators if you need them. And then a miracle occurs is not a valid answer. There
More informationSMSTC (2007/08) Probability.
SMSTC (27/8) Probability www.smstc.ac.uk Contents 12 Markov chains in continuous time 12 1 12.1 Markov property and the Kolmogorov equations.................... 12 2 12.1.1 Finite state space.................................
More informationBirth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes
DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains
More informationStatistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014
Statistics 253/317 Introduction to Probability Models Winter 2014 - Midterm Exam Monday, Feb 10, 2014 Student Name (print): (a) Do not sit directly next to another student. (b) This is a closed-book, closed-note
More informationSTOCHASTIC PROCESSES Basic notions
J. Virtamo 38.3143 Queueing Theory / Stochastic processes 1 STOCHASTIC PROCESSES Basic notions Often the systems we consider evolve in time and we are interested in their dynamic behaviour, usually involving
More informationSimple queueing models
Simple queueing models c University of Bristol, 2012 1 M/M/1 queue This model describes a queue with a single server which serves customers in the order in which they arrive. Customer arrivals constitute
More informationContinuous-time Markov Chains
Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 23, 2017
More informationContinuous time Markov chains
Continuous time Markov chains Alejandro Ribeiro Dept. of Electrical and Systems Engineering University of Pennsylvania aribeiro@seas.upenn.edu http://www.seas.upenn.edu/users/~aribeiro/ October 16, 2017
More informationMARKOV PROCESSES. Valerio Di Valerio
MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some
More informationStochastic Processes (Week 6)
Stochastic Processes (Week 6) October 30th, 2014 1 Discrete-time Finite Markov Chains 2 Countable Markov Chains 3 Continuous-Time Markov Chains 3.1 Poisson Process 3.2 Finite State Space 3.2.1 Kolmogrov
More informationSolution: The process is a compound Poisson Process with E[N (t)] = λt/p by Wald's equation.
Solutions Stochastic Processes and Simulation II, May 18, 217 Problem 1: Poisson Processes Let {N(t), t } be a homogeneous Poisson Process on (, ) with rate λ. Let {S i, i = 1, 2, } be the points of the
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 15. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationExponential Distribution and Poisson Process
Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential
More informationOther properties of M M 1
Other properties of M M 1 Přemysl Bejda premyslbejda@gmail.com 2012 Contents 1 Reflected Lévy Process 2 Time dependent properties of M M 1 3 Waiting times and queue disciplines in M M 1 Contents 1 Reflected
More informationPerformance Modelling of Computer Systems
Performance Modelling of Computer Systems Mirco Tribastone Institut für Informatik Ludwig-Maximilians-Universität München Fundamentals of Queueing Theory Tribastone (IFI LMU) Performance Modelling of Computer
More informationQueuing Networks: Burke s Theorem, Kleinrock s Approximation, and Jackson s Theorem. Wade Trappe
Queuing Networks: Burke s Theorem, Kleinrock s Approximation, and Jackson s Theorem Wade Trappe Lecture Overview Network of Queues Introduction Queues in Tandem roduct Form Solutions Burke s Theorem What
More informationTime Reversibility and Burke s Theorem
Queuing Analysis: Time Reversibility and Burke s Theorem Hongwei Zhang http://www.cs.wayne.edu/~hzhang Acknowledgement: this lecture is partially based on the slides of Dr. Yannis A. Korilis. Outline Time-Reversal
More informationECE-517: Reinforcement Learning in Artificial Intelligence. Lecture 4: Discrete-Time Markov Chains
ECE-517: Reinforcement Learning in Artificial Intelligence Lecture 4: Discrete-Time Markov Chains September 1, 215 Dr. Itamar Arel College of Engineering Department of Electrical Engineering & Computer
More informationThe exponential distribution and the Poisson process
The exponential distribution and the Poisson process 1-1 Exponential Distribution: Basic Facts PDF f(t) = { λe λt, t 0 0, t < 0 CDF Pr{T t) = 0 t λe λu du = 1 e λt (t 0) Mean E[T] = 1 λ Variance Var[T]
More informationPart II: continuous time Markov chain (CTMC)
Part II: continuous time Markov chain (CTMC) Continuous time discrete state Markov process Definition (Markovian property) X(t) is a CTMC, if for any n and any sequence t 1
More informationLectures 16-17: Poisson Approximation. Using Lemma (2.4.3) with θ = 1 and then Lemma (2.4.4), which is valid when max m p n,m 1/2, we have
Lectures 16-17: Poisson Approximation 1. The Law of Rare Events Theorem 2.6.1: For each n 1, let X n,m, 1 m n be a collection of independent random variables with PX n,m = 1 = p n,m and PX n,m = = 1 p
More informationQueueing Theory. VK Room: M Last updated: October 17, 2013.
Queueing Theory VK Room: M1.30 knightva@cf.ac.uk www.vincent-knight.com Last updated: October 17, 2013. 1 / 63 Overview Description of Queueing Processes The Single Server Markovian Queue Multi Server
More informationStochastic Modelling Unit 1: Markov chain models
Stochastic Modelling Unit 1: Markov chain models Russell Gerrard and Douglas Wright Cass Business School, City University, London June 2004 Contents of Unit 1 1 Stochastic Processes 2 Markov Chains 3 Poisson
More informationContinuous Time Markov Chains
Continuous Time Markov Chains Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 2015 Outline Introduction Continuous-Time Markov
More informationLecture 7. We can regard (p(i, j)) as defining a (maybe infinite) matrix P. Then a basic fact is
MARKOV CHAINS What I will talk about in class is pretty close to Durrett Chapter 5 sections 1-5. We stick to the countable state case, except where otherwise mentioned. Lecture 7. We can regard (p(i, j))
More informationQueueing. Chapter Continuous Time Markov Chains 2 CHAPTER 5. QUEUEING
2 CHAPTER 5. QUEUEING Chapter 5 Queueing Systems are often modeled by automata, and discrete events are transitions from one state to another. In this chapter we want to analyze such discrete events systems.
More informationreversed chain is ergodic and has the same equilibrium probabilities (check that π j =
Lecture 10 Networks of queues In this lecture we shall finally get around to consider what happens when queues are part of networks (which, after all, is the topic of the course). Firstly we shall need
More informationT. Liggett Mathematics 171 Final Exam June 8, 2011
T. Liggett Mathematics 171 Final Exam June 8, 2011 1. The continuous time renewal chain X t has state space S = {0, 1, 2,...} and transition rates (i.e., Q matrix) given by q(n, n 1) = δ n and q(0, n)
More informationTMA4265 Stochastic processes ST2101 Stochastic simulation and modelling
Norwegian University of Science and Technology Department of Mathematical Sciences Page of 7 English Contact during examination: Øyvind Bakke Telephone: 73 9 8 26, 99 4 673 TMA426 Stochastic processes
More informationName of the Student:
SUBJECT NAME : Probability & Queueing Theory SUBJECT CODE : MA 6453 MATERIAL NAME : Part A questions REGULATION : R2013 UPDATED ON : November 2017 (Upto N/D 2017 QP) (Scan the above QR code for the direct
More informationStatistics 992 Continuous-time Markov Chains Spring 2004
Summary Continuous-time finite-state-space Markov chains are stochastic processes that are widely used to model the process of nucleotide substitution. This chapter aims to present much of the mathematics
More informationMarkov Processes Cont d. Kolmogorov Differential Equations
Markov Processes Cont d Kolmogorov Differential Equations The Kolmogorov Differential Equations characterize the transition functions {P ij (t)} of a Markov process. The time-dependent behavior of the
More informationQuality of Real-Time Streaming in Wireless Cellular Networks : Stochastic Modeling and Analysis
Quality of Real-Time Streaming in Wireless Cellular Networs : Stochastic Modeling and Analysis B. Blaszczyszyn, M. Jovanovic and M. K. Karray Based on paper [1] WiOpt/WiVid Mai 16th, 2014 Outline Introduction
More informationMulti Stage Queuing Model in Level Dependent Quasi Birth Death Process
International Journal of Statistics and Systems ISSN 973-2675 Volume 12, Number 2 (217, pp. 293-31 Research India Publications http://www.ripublication.com Multi Stage Queuing Model in Level Dependent
More informationQUEUING SYSTEM. Yetunde Folajimi, PhD
QUEUING SYSTEM Yetunde Folajimi, PhD Part 2 Queuing Models Queueing models are constructed so that queue lengths and waiting times can be predicted They help us to understand and quantify the effect of
More informationStochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS
Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review
More informationQueueing Theory I Summary! Little s Law! Queueing System Notation! Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K "
Queueing Theory I Summary Little s Law Queueing System Notation Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K " Little s Law a(t): the process that counts the number of arrivals
More informationIntroduction to queuing theory
Introduction to queuing theory Claude Rigault ENST claude.rigault@enst.fr Introduction to Queuing theory 1 Outline The problem The number of clients in a system The client process Delay processes Loss
More informationLecture 10: Semi-Markov Type Processes
Lecture 1: Semi-Markov Type Processes 1. Semi-Markov processes (SMP) 1.1 Definition of SMP 1.2 Transition probabilities for SMP 1.3 Hitting times and semi-markov renewal equations 2. Processes with semi-markov
More informationLIMITS FOR QUEUES AS THE WAITING ROOM GROWS. Bell Communications Research AT&T Bell Laboratories Red Bank, NJ Murray Hill, NJ 07974
LIMITS FOR QUEUES AS THE WAITING ROOM GROWS by Daniel P. Heyman Ward Whitt Bell Communications Research AT&T Bell Laboratories Red Bank, NJ 07701 Murray Hill, NJ 07974 May 11, 1988 ABSTRACT We study the
More informationExample: physical systems. If the state space. Example: speech recognition. Context can be. Example: epidemics. Suppose each infected
4. Markov Chains A discrete time process {X n,n = 0,1,2,...} with discrete state space X n {0,1,2,...} is a Markov chain if it has the Markov property: P[X n+1 =j X n =i,x n 1 =i n 1,...,X 0 =i 0 ] = P[X
More informationStochastic Proceses Poisson Process
Stochastic Proceses 2014 1. Poisson Process Giovanni Pistone Revised June 9, 2014 1 / 31 Plan 1. Poisson Process (Formal construction) 2. Wiener Process (Formal construction) 3. Infinitely divisible distributions
More informationChapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationQueueing Review. Christos Alexopoulos and Dave Goldsman 10/6/16. (mostly from BCNN) Georgia Institute of Technology, Atlanta, GA, USA
1 / 24 Queueing Review (mostly from BCNN) Christos Alexopoulos and Dave Goldsman Georgia Institute of Technology, Atlanta, GA, USA 10/6/16 2 / 24 Outline 1 Introduction 2 Queueing Notation 3 Transient
More informationExamination paper for TMA4265 Stochastic Processes
Department of Mathematical Sciences Examination paper for TMA4265 Stochastic Processes Academic contact during examination: Andrea Riebler Phone: 456 89 592 Examination date: December 14th, 2015 Examination
More informationPoisson Processes. Particles arriving over time at a particle detector. Several ways to describe most common model.
Poisson Processes Particles arriving over time at a particle detector. Several ways to describe most common model. Approach 1: a) numbers of particles arriving in an interval has Poisson distribution,
More informationNon Markovian Queues (contd.)
MODULE 7: RENEWAL PROCESSES 29 Lecture 5 Non Markovian Queues (contd) For the case where the service time is constant, V ar(b) = 0, then the P-K formula for M/D/ queue reduces to L s = ρ + ρ 2 2( ρ) where
More informationQueueing Theory II. Summary. ! M/M/1 Output process. ! Networks of Queue! Method of Stages. ! General Distributions
Queueing Theory II Summary! M/M/1 Output process! Networks of Queue! Method of Stages " Erlang Distribution " Hyperexponential Distribution! General Distributions " Embedded Markov Chains M/M/1 Output
More informationNetworking = Plumbing. Queueing Analysis: I. Last Lecture. Lecture Outline. Jeremiah Deng. 29 July 2013
Networking = Plumbing TELE302 Lecture 7 Queueing Analysis: I Jeremiah Deng University of Otago 29 July 2013 Jeremiah Deng (University of Otago) TELE302 Lecture 7 29 July 2013 1 / 33 Lecture Outline Jeremiah
More informationarxiv: v2 [math.pr] 24 Mar 2018
Exact sampling for some multi-dimensional queueing models with renewal input arxiv:1512.07284v2 [math.pr] 24 Mar 2018 Jose Blanchet Yanan Pei Karl Sigman October 9, 2018 Abstract Using a recent result
More informationExercises Stochastic Performance Modelling. Hamilton Institute, Summer 2010
Exercises Stochastic Performance Modelling Hamilton Institute, Summer Instruction Exercise Let X be a non-negative random variable with E[X ]
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationThe story of the film so far... Mathematics for Informatics 4a. Continuous-time Markov processes. Counting processes
The story of the film so far... Mathematics for Informatics 4a José Figueroa-O Farrill Lecture 19 28 March 2012 We have been studying stochastic processes; i.e., systems whose time evolution has an element
More informationContinuous time Markov chains
Chapter 2 Continuous time Markov chains As before we assume that we have a finite or countable statespace I, but now the Markov chains X {X(t) : t } have a continuous time parameter t [, ). In some cases,
More informationSTAT 380 Markov Chains
STAT 380 Markov Chains Richard Lockhart Simon Fraser University Spring 2016 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring 2016 1 / 38 1/41 PoissonProcesses.pdf (#2) Poisson Processes
More information