Introduction to Queueing Theory

Similar documents
Introduction to Queueing Theory

Introduction to Queueing Theory

QUEUING SYSTEM. Yetunde Folajimi, PhD

CPSC 531: System Modeling and Simulation. Carey Williamson Department of Computer Science University of Calgary Fall 2017

2 k Factorial Designs Raj Jain Washington University in Saint Louis Saint Louis, MO These slides are available on-line at:

Analysis of A Single Queue

Chapter 10. Queuing Systems. D (Queuing Theory) Queuing theory is the branch of operations research concerned with waiting lines.

Performance Evaluation of Queuing Systems

Queueing Theory I Summary! Little s Law! Queueing System Notation! Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K "

Convolution Algorithm

Queueing Theory and Simulation. Introduction

Queueing systems. Renato Lo Cigno. Simulation and Performance Evaluation Queueing systems - Renato Lo Cigno 1

Queues and Queueing Networks

Chapter 2 Queueing Theory and Simulation

Kendall notation. PASTA theorem Basics of M/M/1 queue

Elementary queueing system

Operational Laws 33-1

Analysis of Software Artifacts

M/G/1 and M/G/1/K systems

Networking = Plumbing. Queueing Analysis: I. Last Lecture. Lecture Outline. Jeremiah Deng. 29 July 2013

Probability and Statistics Concepts

Part II: continuous time Markov chain (CTMC)

Introduction to Queuing Theory. Mathematical Modelling

Queueing Theory II. Summary. ! M/M/1 Output process. ! Networks of Queue! Method of Stages. ! General Distributions

Queueing Systems: Lecture 3. Amedeo R. Odoni October 18, Announcements

Part I Stochastic variables and Markov chains

Introduction to queuing theory

Operational Laws. Operational Laws. Overview. Operational Quantities

Lecture 7: Simulation of Markov Processes. Pasi Lassila Department of Communications and Networking

Systems Simulation Chapter 6: Queuing Models

Non Markovian Queues (contd.)

Queueing Theory. VK Room: M Last updated: October 17, 2013.

BIRTH DEATH PROCESSES AND QUEUEING SYSTEMS

Data analysis and stochastic modeling

M/G/1 and Priority Queueing

Exercises Stochastic Performance Modelling. Hamilton Institute, Summer 2010

GI/M/1 and GI/M/m queuing systems

Solutions to Homework Discrete Stochastic Processes MIT, Spring 2011

Queuing Networks: Burke s Theorem, Kleinrock s Approximation, and Jackson s Theorem. Wade Trappe

PBW 654 Applied Statistics - I Urban Operations Research

CDA5530: Performance Models of Computers and Networks. Chapter 4: Elementary Queuing Theory

Introduction to Queueing Theory with Applications to Air Transportation Systems

NATCOR: Stochastic Modelling

Intro to Queueing Theory

PREPARED BY: INSTITUTE OF AERONAUTICAL ENGINEERING (Autonomous) Dundigal, Hyderabad

Chapter 5: Special Types of Queuing Models

Lecture 20: Reversible Processes and Queues

Introduction to Markov Chains, Queuing Theory, and Network Performance

LECTURE #6 BIRTH-DEATH PROCESS

Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis.

Birth-Death Processes

2 k Factorial Designs Raj Jain

Contents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory

Chapter 6 Queueing Models. Banks, Carson, Nelson & Nicol Discrete-Event System Simulation

Queueing Review. Christos Alexopoulos and Dave Goldsman 10/6/16. (mostly from BCNN) Georgia Institute of Technology, Atlanta, GA, USA

Name of the Student:

Základy teorie front

Queuing Theory. Using the Math. Management Science

Chapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan

Queuing Theory and Stochas St t ochas ic Service Syste y ms Li Xia

Queueing Review. Christos Alexopoulos and Dave Goldsman 10/25/17. (mostly from BCNN) Georgia Institute of Technology, Atlanta, GA, USA

7 Variance Reduction Techniques

Basic Concepts in Number Theory and Finite Fields

Assignment 3 with Reference Solutions

Computer Systems Modelling

Chapter 11. Performance Modeling

Link Models for Circuit Switching

Classical Queueing Models.

Stochastic Models of Manufacturing Systems

Slides 9: Queuing Models

Základy teorie front II

Review of Queuing Models

Figure 10.1: Recording when the event E occurs

Since D has an exponential distribution, E[D] = 0.09 years. Since {A(t) : t 0} is a Poisson process with rate λ = 10, 000, A(0.

Computer Networks More general queuing systems

Chapter 3 Balance equations, birth-death processes, continuous Markov Chains

NICTA Short Course. Network Analysis. Vijay Sivaraman. Day 1 Queueing Systems and Markov Chains. Network Analysis, 2008s2 1-1

Chapter 1. Introduction. 1.1 Stochastic process

Operational Laws Raj Jain

CHAPTER 4. Networks of queues. 1. Open networks Suppose that we have a network of queues as given in Figure 4.1. Arrivals

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks

Module 5: CPU Scheduling

Queuing Theory. Queuing Theory. Fatih Cavdur April 27, 2015

Chapter 6: CPU Scheduling

Two Factor Full Factorial Design with Replications

Queuing Theory. Richard Lockhart. Simon Fraser University. STAT 870 Summer 2011

6 Solving Queueing Models

Advanced Computer Networks Lecture 3. Models of Queuing

2905 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES

CPU scheduling. CPU Scheduling

Chapter 11. Performance Modeling

IEOR 6711, HMWK 5, Professor Sigman

The exponential distribution and the Poisson process

Photo: US National Archives

IEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.

Stochastic Models in Computer Science A Tutorial

Glossary availability cellular manufacturing closed queueing network coefficient of variation (CV) conditional probability CONWIP

All models are wrong / inaccurate, but some are useful. George Box (Wikipedia). wkc/course/part2.pdf

CPU Scheduling. CPU Scheduler

Exact Simulation of the Stationary Distribution of M/G/c Queues

Transcription:

Introduction to Queueing Theory Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: 30-1

Overview Queueing Notation Rules for All Queues Little's Law Types of Stochastic Processes 30-2

Basic Components of a Queue 1. Arrival process 6. Service discipline 2. Service time distribution 5. Population Size 4. Waiting positions 3. Number of servers 30-3

Kendall Notation A/S/m/B/K/SD A: Arrival process S: Service time distribution m: Number of servers B: Number of buffers (system capacity) K: Population size, and SD: Service discipline 30-4

Arrival Process Arrival times: Interarrival times: j form a sequence of Independent and Identically Distributed (IID) random variables Notation: M = Memoryless Exponential E = Erlang H = Hyper-exponential D = Deterministic constant G = General Results valid for all distributions 30-5

Service Time Distribution Time each student spends at the terminal. Service times are IID. Distribution: M, E, H, D, or G Device = Service center = Queue Buffer = Waiting positions 30-6

Service Disciplines First-Come-First-Served (FCFS) Last-Come-First-Served (LCFS) = Stack (used in 9-1-1 calls) Last-Come-First-Served with Preempt and Resume (LCFS-PR) Round-Robin (RR) with a fixed quantum. Small Quantum Processor Sharing (PS) Infinite Server: (IS) = fixed delay Shortest Processing Time first (SPT) Shortest Remaining Processing Time first (SRPT) Shortest Expected Processing Time first (SEPT) Shortest Expected Remaining Processing Time first (SERPT). Biggest-In-First-Served (BIFS) Loudest-Voice-First-Served (LVFS) 30-7

Example M/M/3/20/1500/FCFS Time between successive arrivals is exponentially distributed. Service times are exponentially distributed. Three servers 20 Buffers = 3 service + 17 waiting After 20, all arriving jobs are lost Total of 1500 jobs that can be serviced. Service discipline is first-come-first-served. Defaults: Infinite buffer capacity Infinite population size FCFS service discipline. G/G/1 = G/G/1/ / /FCFS 30-8

Key: A/S/m/B/K/SD T F Quiz 30A The number of servers in a M/M/1/3 queue is 3 G/G/1/30/300/LCFS queue is like a stack M/D/3/30 queue has 30 buffers G/G/1 queue has population size D/D/1 queue has FCFS discipline 30-9

Key: A/S/m/B/K/SD T F Solution to Quiz 30A The number of servers in a M/M/1/3 queue is 3 G/G/1/30/300/LCFS queue is like a stack M/D/3/30 queue has 30 buffers G/G/1 queue has population size D/D/1 queue has FCFS discipline 30-10

Exponential Distribution Probability Density Function (pdf): f(x) = 1 a e x/a Cumulative Distribution Function (cdf): F (x) =P (X <x)= Mean: a Variance: a 2 Z x 0 f(x)dx =1 e x/a Coefficient of Variation = (Std Deviation)/mean = 1 Memoryless: Expected time to the next arrival is always a regardless of the time since the last arrival Remembering the past history does not help. 1 a f(x) 1 F(x) x x 30-11

Erlang Distribution Sum of k exponential random variables Series of k servers with exponential service times kx X = x i where x i exponential Probability Density Function (pdf): Expected Value: ak Variance: a 2 k CoV: 1/ k i=1 f(x) = xk 1 e x/a (k 1)!a k 1 k 30-12

Hyper-Exponential Distribution The variable takes i th value with probability p i x 1 x x 2 x m x i is exponentially distributed with mean a i Higher variance than exponential Coefficient of variation > 1 30-13

Bulk arrivals/service Group Arrivals/Service M [x] :xrepresents the group size G [x] : a bulk arrival or service process with general inter-group times. Examples: M [x] /M/1 : Single server queue with bulk Poisson arrivals and exponential service times M/G [x] /m: Poisson arrival process, bulk service with general service time distribution, and m servers. 30-14

Quiz 30B Exponential distribution is denoted as distribution represents a set of parallel exponential servers Erlang distribution E k with k=1 is same as distribution 30-15

Solution to Quiz 30B Exponential distribution is denoted as M Hyperexponential distribution represents a set of parallel exponential servers Erlang distribution E k with k=1 is same as Exponential distribution 30-16

Key Variables 1 2 m Time Previous Arrival Arrival 30-17 n q n Begin Service n s w s r End Service

Key Variables (cont) Inter-arrival time = time between two successive arrivals. Mean arrival rate = 1/E[ ] May be a function of the state of the system, e.g., number of jobs already in the system. s = Service time per job. = Mean service rate per server = 1/E[s] Total service rate for m servers is m n = Number of jobs in the system. This is also called queue length. Note: Queue length includes jobs currently receiving service as well as those waiting in the queue. 30-18

n q = Number of jobs waiting Key Variables (cont) n s = Number of jobs receiving service r = Response time or the time in the system = time waiting + time receiving service w = Waiting time = Time between arrival and beginning of service 30-19

Rules for All Queues Rules: The following apply to G/G/m queues 1. Stability Condition: Arrival rate must be less than service rate < m Finite-population or finite-buffer systems are always stable. Instability = infinite queue Sufficient but not necessary. D/D/1 queue is stable at λ=μ 2. Number in System versus Number in Queue: n = n q + n s Notice that n, n q, and n s are random variables. E[n]=E[n q ]+E[n s ] 30-20

Rules for All Queues (cont) 3. Number versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate Mean response time 4. Similarly, Mean number of jobs in the queue = Arrival rate Mean waiting time This is known as Little's law. 5. Time in System versus Time in Queue r = w + s r, w, and s are random variables. E[r] = E[w] + E[s] 30-21

Rules for All Queues(cont) 6. If the service rate is independent of the number of jobs in the queue, Cov(w,s)=0 30-22

Quiz 30C If a queue has 2 jobs waiting for service, the number is system is If the arrival rate is 2 jobs/second, the mean interarrival time is second. In a 3 server queue, the jobs arrive at the rate of 1 jobs/second, the service time should be less than second/job for the queue to be stable. 30-23

Solution to Quiz 30C If a queue has 2 jobs waiting for service, the number is system is m+2. If the arrival rate is 2 jobs/second, the mean interarrival time is 0.5 second. In a 3 server queue, the jobs arrive at the rate of 1 jobs/second, the service time should be less than 3 second/job for the queue to be stable. 30-24

Little's Law Mean number in the system = Arrival rate Mean response time This relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing service. Named after Little (1961) Based on a black-box view of the system: Arrivals Black Box Departures In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions 30-25

Proof of Little's Law Job number 4 3 2 1 Arrival If T is large, arrivals = departures = N Arrival rate = Total arrivals/total time= N/T Hatched areas = total time spent inside the system by all jobs = J Mean time in the system= J/N Mean Number in the system = J/T = = Arrival rate Mean time in the system 1 2 3 4 5 6 7 8 Time Departure 30-26 Number in System 4 3 2 1 Time in System 1 2 3 4 5 6 7 8 Time 4 3 2 1 1 2 3 Job number

Application of Little's Law Arrivals Departures Applying to just the waiting facility of a service center Mean number in the queue = Arrival rate Mean waiting time Similarly, for those currently receiving the service, we have: Mean number in service = Arrival rate Mean service time 30-27

Example 30.3 A monitor on a disk server showed that the average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server? Using Little's law: Mean number in the disk server = Arrival rate Response time = 100 (requests/second) (0.1 seconds) = 10 requests 30-28

Key: n = λ R Quiz 30D During a 1 minute observation, a server received 120 requests. The mean response time was 1 second. The mean number of queries in the server is 30-29

Key: n = λ R Solution to Quiz 30D During a 1 minute observation, a server received 120 requests. The mean response time was 1 second. The mean number of queries in the server is 2. λ = 120/60 = 2 R = 1 n = 2 30-30

Stochastic Processes Process: Function of time Stochastic Process: Random variables, which are functions of time x t Example 1: n(t) = number of jobs at the CPU Observe n(t) at several identical systems The number n(t) is a random variable. Find the probability distribution functions for n(t) at each t. Example 2: w(t) = waiting time in a queue t 30-31

Types of Stochastic Processes Discrete or Continuous State Processes Markov Processes Birth-death Processes Poisson Processes 30-32

Discrete/Continuous State Processes Discrete = Finite or Countable Number of jobs in a system n(t) = 0, 1, 2,... n(t) is a discrete state process The waiting time w(t) is a continuous state process. Stochastic Chain: discrete state stochastic process Note: Time can also be discrete or continuous Discrete/continuous time processes Here we will consider only continuous time processes State Time 30-33

Markov Processes Future states are independent of the past and depend only on the present. Named after A. A. Markov who defined and analyzed them in 1907. Markov Chain: discrete state Markov process Markov It is not necessary to history of the previous states of the process Future depends upon the current state only M/M/m queues can be modeled using Markov processes. The time spent by a job in such a queue is a Markov process and the number of jobs in the queue is a Markov chain. 30-34

Birth-Death Processes j j j 0 1 2 j-1 j j+1 j j j The discrete space Markov processes in which the transitions are restricted to neighboring states Process in state n can change only to state n+1 or n-1. Example: the number of jobs in a queue with a single server and individual arrivals (not bulk arrivals) 30-35

Poisson Distribution If the inter-arrival times are exponentially distributed, number of arrivals in any given interval are Poisson distributed τ ~ Exponential time n ~ Poisson f(τ) =λe λτ E[τ] = 1 λ P (n arrivals in t) =(λt) n e λt n! E[n] =λt M = Memoryless arrival = Poisson arrivals Example: λ=4 4 jobs/sec or 0.25 sec between jobs on average 30-36

Poisson Processes Interarrival time s = IID and exponential number of arrivals n over a given interval (t, t+x) has a Poisson distribution arrival = Poisson process or Poisson stream Properties: 1.Merging: λ 1 λ i λ k 30-37...... λ = kx i=1 2.Splitting: If the probability of a job going to ith substream is p i, each substream is also Poisson with a mean rate of p i p 1. p 1 λ. p i. λ. p k p i λ.. p k λ λ i

Poisson Processes (Cont) 3.If the arrivals to a single server with exponential service time are Poisson with mean rate, the departures are also Poisson with the same rate provided <. 30-38

Poisson Process(cont) 4. If the arrivals to a service facility with m service centers are Poisson with a mean rate, the departures also constitute a Poisson stream with the same rate, provided < i i. Here, the servers are assumed to have exponentially distributed service times. 30-39

PASTA Property Poisson Arrivals See Time Averages Poisson arrivals Random arrivals from a large number of independent sources If an external observer samples a system at a random instant: P(System state = x) = P(State as seen by a Poisson arrival is x) Example: D/D/1 Queue: Arrivals = 1 job/sec, Service =2 jobs/sec 0 1 2 3 4 5 All customers see an empty system. M/D/1 Queue: Arrivals = 1 job/sec (avg), Service = 2 jobs/sec Randomly sample the system. System is busy half of the time. 0 1 2 3 4 5 30-40

Relationship Among Stochastic Processes 30-41

Quiz 30E T F Birth-death process can have bulk service Merger of Poisson processes results in a Process The number of jobs in a M/M/1 queue is Markov T F A discrete time process is also called a chain 30-42

Solution to Quiz 30E T F Birth-death process can have bulk service Merger of Poisson processes results in a Poisson Process The number of jobs in a M/M/1 queue is Markov Chain T F A discrete time process is also called a chain 30-43

Summary Kendall Notation: A/S/m/B/k/SD, M/M/1 Number in system, queue, waiting, service Service rate, arrival rate, response time, waiting time, service time Little s Law: Mean number in system = Arrival rate Mean time in system Processes: Markov Only one state required, Birth-death Adjacent states Poisson IID and exponential inter-arrival 30-44

Homework 30 Updated Exercise 30.4 During a one-hour observation interval, the name server of a distributed system received 12,960 requests. The mean response time of these requests was observed to be one-third of a second. a. What is the mean number of queries in the server? b. What assumptions have you made about the system? c. Would the mean number of queries be different if the service time was not exponentially distributed? 30-45

Reading List If you need to refresh your probability concepts, read chapter 12 Read Chapter 30 Refer to Chapter 29 for various distributions 30-46

Scan This to Download These Slides Raj Jain http://rajjain.com 30-47

Related Modules CSE567M: Computer Systems Analysis (Spring 2013), https://www.youtube.com/playlist?list=pljgg94etkypjekjnaa1n_1x0bwwnyzcof CSE473S: Introduction to Computer Networks (Fall 2011), https://www.youtube.com/playlist?list=pljgg94etkypjwospmh8azcgy5e_10tidw Wireless and Mobile Networking (Spring 2016), https://www.youtube.com/playlist?list=pljgg94etkypkeb0nzyn9tss_hcd5c4wxf CSE571S: Network Security (Fall 2011), https://www.youtube.com/playlist?list=pljgg94etkypkvzfvtuthcpfjxumyyg93u Video Podcasts of Prof. Raj Jain's Lectures, https://www.youtube.com/channel/ucn4-5wznp9-ruozqms-8nuw 30-48