Introduction to queuing theory

Introduction to queuing theory Claude Rigault ENST claude.rigault@enst.fr Introduction to Queuing theory 1

Outline The problem The number of clients in a system The client process Delay processes Loss processes Introduction to Queuing theory 2

The problem Introduction to Queuing theory 3

Connections The problem : how many servers? 1 1 2 1 1 2 2 2 J a j trunks J a l L lines L a l lines Signaling Introduction to Queuing theory 4

How many servers? n(t) N Standard deviation A σ Number of servers Lost calls Mean value Observation period T t Introduction to Queuing theory 5

Lost call processes 1 L clients L Reorder tone 1 2 N N servers Introduction to Queuing theory 6

Delay processes 1 L clients L 1 2 N N servers Introduction to Queuing theory 7

Interesting parameters Loss systems : Loss Probability : B n n B < ε Delay systems : Probability of delay : n t> P n ( τ ) ε < Introduction to Queuing theory 8

The number of clients in a system Introduction to Queuing theory 9

The traffic for one machine ρ is the time probability (fraction of time) of the machine being busy. It is also called the utilization ratio of the machine t1 t2 t3 t4 ti tn Observation period T ρ T ti T t Introduction to Queuing theory 10

T t ρ 1 Traffic Units When we have the machine busy all the time This is called a 1 Erlang Traffic, in honor to Agner Krarup Erlang (1878, 1929), a Danish Engineer, Pioneer of traffic engineering The Americans use the CCS (Cent Call Second) : 1 CCS 100 seconds of work per hour 1 Erlang 36 CCS Introduction to Queuing theory 11

The traffic for a group of machines The traffic for the group is given by : Or : A n τ λ T µ A M i T ti M ρm In this formula λ is the sum of the arrival rates of all the machines and µ is the service rate for one machine M L M 3 M 2 M 1 Observation period T Introduction to Queuing theory 12

Remarks on the traffic : The traffic is also the number of arrivals per service time : if we take as unit of time the service time, then µ 1 and A λ The Traffic of a group of machines is not a probability For N machines, we have A N Introduction to Queuing theory 13

Delay box Average time spent in the box : Number of clients in the box a time t : x(t) Number of clients arrived between 0 and T : n(t) τ Offered Clients λ Carried Clients λc buffer Server Server Lost Clients λb Delay box Server c servers Introduction to Queuing theory 14

Loss model : Offered Traffic and Throughput Number of customers arriving during T n : offered arrivals n B : lost arrivals n C : carried arrivals Fraction of clients lost or loss probability : τ T n nc + nb A AC + The lost traffic is : The Throughput is the carried traffic : τ T τ T AB A B A B AC n nc + nb B n n B A 1 ( B) Introduction to Queuing theory 15

Averages The average number of clients in the box is : x T 1 T T 0 x()dt t The average carried arrival rate is : The average time spent in the box is : lim λc τ lim T nc n ( T) T ( T ) C i lim 0 T nc t i ( T) Introduction to Queuing theory 16

Arrivals and departures The shaded surface S is : 0 It is also, at the condition to stop at a place where x(t) 0 : S S ti i n(t) () T x t dt Arrival process n(t) Departure process d(t) d(t) t 1 t 2 t 4 t 3 x(t) T t Introduction to Queuing theory 17

Little s Formula From the 2 calculations of the shaded surface : Or S ti i T 0 () x t dt () dt nc() t T 0 i x t T T n C and Ti () t T S ti i T 0 T x() t dt T By taking the limit when T Little s formula : X λ C τ Introduction to Queuing theory 18

Average number of clients in the box From Little s Formula : X λ C τ But we also have A C nc T τ X The Carried traffic is also the average number of simultaneous clients in the box Introduction to Queuing theory 19

Ergodicity Ergodicity means that time probabilities equal space probabilities : In case of Ergodic traffic : P T PS Therefore, the average number of clients simultaneously in the system is also the traffic M L M 3 M 2 PT ρ P X S X Lρ L Α A M 1 Observation period T Introduction to Queuing theory 20

The Client s processes Introduction to Queuing theory 21

Probability of x simultaneous clients The probability of having x clients simultaneously in the system is : x P() x t T n(t) t x Elapsed time when x clients are in the delay box x Average Α Observation period T Introduction to Queuing theory 22 t

Statistical Equilibrium For large T, transitions from x to x-1equal transitions from x-1 to x (the edge effect becomes negligible) n(t) x 1 service 1 service 1 arrival 1 arrival 1 arrival T Introduction to Queuing theory 23

The steady state equation : x N The balance equation λ x 1t x 1 µ x If : where N is the number of servers (no wait), then µ x x where τ is the average service time τ t x λ t x-1 x-1 x - 1 x µ x t x Introduction to Queuing theory 24

Case of a large number of clients λ x is independent of x : When this is the case we talk of a Markovian arrival process If there is no delay µ is : The balance equation : becomes : Or : t T n x 1 tx λ T n x τ AP x 1 x µ x x τ ( ) ( ) P x x 1t x 1 µ xtx λ Introduction to Queuing theory 25

The Poisson distribution For a large number of clients : We compute P(0) by : Ax P( x) P(0) x! Ax P( x) P(0) 1! x 0 x 0 x Or : A P( x) x! x e A X A We get and σ A Introduction to Queuing theory 26

Consequences of the Poisson distribution The standard deviation is not linear with the traffic : σ A A network is not dimensioned for the average number of clients in the system, but for this number plus the fluctuation. The fluctuation varies like the square root of the traffic The bigger the traffic, the less the fluctuation (relatively) Networks dimensioning is not linear Introduction to Queuing theory 27

The full availibility rule Service rate µ Service rate 4µ Large fluctuations felt by the client! Bad solution Small fluctuations felt by the client! Good solution Introduction to Queuing theory 28

Case of a small number of clients λ x depends of x : if there are L clients and x are in the box, then only L ( x 1 ) L x+ 1 clients may produce arrivals A client has traffic ρ T ντ A free client produces λ ν f arrivals per unit of time T ντ The balance equation becomes : ( L x+ 1) ν t x x 1 tx T ντ τ Or And : L x+ 1 ρ x 1 ρ x P x Cx ρ L P 1 ρ () ( ) P( x 1 ) P x () () 0 Introduction to Queuing theory 29

The Bernouilli distribution We get P(0) by : Or : And : L x ( ) (0) C ρ P x P x L 0 x 0 1 x ρ L ( 0) ρ P 1 + 1 ρ ( ρ) L P(0) 1 1 1 Therefore : ( ) x( ) L C x Lρ 1 ρ x P x X Lρ A σ A( 1 ρ ) We get : and : Introduction to Queuing theory 30

Comparison of Bernouilli and Poisson For a given mean, Poisson has largest entropy Poisson is a worst case model Bernouilli fluctuation is slightly smaller than Poisson fluctuation The Bernouilli model is more accurate for a small number of clients When Bernouilli Poisson () P x And L x x L x ( ) x L L! ρ ρ 1 ρ x x L L! 1!!! x L A! ( ) L L! L, 1 L A ( ) ( ) L x L x ( ) x A ( x)! e P () x x x ( Lρ) A A A x! e x! Introduction to Queuing theory 31 e

Delay processes Introduction to Queuing theory 32

Delay probability in Delay systems We call Delay probability D the fraction of arrivals that suffer delay : Delayed arrivals are arrivals occurring when all the servers are busy : Or : And : n D λ n D x N n D t D x x N x N P n n T n D ( x) ( ) P x x N t x Introduction to Queuing theory 33

Two different waiting times Let W be the waiting time (not including server time) averaged on all arrivals (even on arrivals that are immediately served) Let W be the average waiting time, averadged only on the fraction of arrivals that experience delay Or : n ( τ + W ) ( n nd ) τ + nd( τ + W' ) nw n D W' And : W DW' Introduction to Queuing theory 34

Carried traffic and delayed traffic Let δ be the delay averaged on all arrivals n The carried traffic is now : AC ( W ) τ + T The offered traffic is this fictitious traffic that would exist if there was no delay nτ A The delay traffic is the traffic of the waiting clients. The delay traffic is n D T W A D T Introduction to Queuing theory 35

Queuing systems classification Queues are classified by the Kendall notation : Client s process/service process/servers/max occupancy M exponential D Deterministic G General When maximum occupancy is not mentioned it means that the occupancy of the system is unlimited Introduction to Queuing theory 36

The M/M/1queue Clients are Poisson λ is independent of x λ There is no loss λ C λ Only one server of rate µ independent of x: Offered Clients λ Server infinite buffer Introduction to Queuing theory 37

Balance equation in the M/M/1queue The balance equation becomes : λ t x 1 µ tx or and P ( x) ρ P( x 1 ) P ( x) ρ x P( 0) Introduction to Queuing theory 38

Probability of x clients in the M/M/1 queue We get P(0) by : x 0 P( x) P(0) ρ x x 0 1 or And P( 0) 1-ρ ( 1 ) P ( x) ρ x -ρ Introduction to Queuing theory 39

Average total number of clients in the M/M/1 system (buffer + server) The average number of clients in the queue and the server is : X xp( x) ρ ρ d d ρ 1 1 ρ ( 1-ρ) x x ( 1-ρ) x 0 x 0 1 ρ -ρ And X ρ 1 ρ λ- λ µ Introduction to Queuing theory 40

Average delay in the M/M/1queue By Little s Formula the average delay in the system, including service time is : T X τ λ 1- ρ µ 1 -λ The average wait in the queue is W T τ ρ τ 1 ρ Introduction to Queuing theory 41

Average number of clients in the M/M/1 queue and in the server From Little s formula the average number of clients in the server is ns λτ ρ So, the average number of clients waiting in the queue is : Or : nd nd ρ - ρ 1 ρ ρ2 1 ρ ρ ρ 1- ρ ( 1 ρ) And the average time in the queue is : Introduction to Queuing theory 42

Application to statistical multiplexing Multiple access uses all the channel capacity for transmission. Delay is : TS µ λ 1 TDM using N slots has a slot capacity µ µ T N Arrival rate is λ T λ N So delay with TDM is T NTS µ 1 λ N N So delay is N times shorter with statistical multiplexing! Introduction to Queuing theory 43

The M/M/N queue Clients are Poisson λ is independent of x λ There is no loss λ C λ N servers of rate µ independent of x Server Offered Clients λ infinite buffer Server Server N servers Introduction to Queuing theory 44

Service rate in the M/M/N queue There are 2 cases : x N x xµ 1) if then µx where µ is the service τ rate for one server x > N 2) If then µ x µx N τ c x Introduction to Queuing theory 45

Balance equations in the M/M/N queue 1) if then x tx and or x N () () P x λ t 1 xµ Ax x! P 0 A x () P( x 1 ) P x x x N+ j > N λ 1 t Nµ t 2) If then x x 1 x and ( + j) P( N+ j 1) P N N A or P ( ) ( ) N+ j j P( N) N A Introduction to Queuing theory 46

P(0) in the M/M/N queue We get P(0) by : This gives : or x 0 P( x) N Ax P( x) P(0) + P( N) x! N A x 0 x 0 j 1 P(0) N Ax x! + P( N x 0 A AN N 1 But P( N) P(0) so : (0) Ax AN P + N! x 0 x! N N A N! P( 0) 1 N and : 1 Ax AN + x 0 x! N N A N! Introduction to Queuing theory 47 1 )( ) 1 N A ( ) j 1 1

State probabilities in the M/M/N queue 1) if x N () P x Ax x! N 1 Ax AN + x 0 x! N N A N! 2) If x N+ j > N ( j) P N A ( ) j N A N + N! N 1 Ax AN + x 0 x! N N A N! Introduction to Queuing theory 48

Delay probability in the M/M/N queue We call Delay probability D the fraction of arrivals that suffer delay : Delayed arrivals are arrivals occurring when all the servers are busy : Or : And : n D λ n D x N n D t D x x N x N P n n T n D ( x) ( ) P x x N t x Introduction to Queuing theory 49

The Erlang C law The delay probability D is : and We get : Or : x N Also called the second law of Erlang ( ) D P x ( ) ( )( ) P N+ j P N ( ) ( ) ( ) + j P N P( N) D P N D j 0 j 0 ( A) N A AN N N A N! E2,N N 1 Ax AN + x 0 x! N N A N j N N A! N A j Introduction to Queuing theory 50

Average number of waiting clients in the M/M/N queue The average number of clients Waiting in the queue buffer is : j From previous slide : ( ) ( ) j j jp( N+ j) P N so : and : N A j 0 j 0 1 D 1 1 N A ( ) P( N) N A ( ) N A j ( ) D 1 N A j D N A A N A 2 P( N) Introduction to Queuing theory 51

Average delay and waiting time in the M/M/N queue From Little s formula : j λw with : j N A A D W j τ λ N A D D N τ T W+ τ τ D+ A N A τ so : and W Also W' N τ A Introduction to Queuing theory 52

Loss processes Introduction to Queuing theory 53

Loss probability in loss systems Fraction of clients lost or loss probability : n nc + nb The lost traffic is : B n n B The Throughput is the carried traffic : τ T τ τ T T AB A B n nc + nb A AC + AC A 1 ( B) A B Introduction to Queuing theory 54

Time Congestion and Call Congestion λ N t N n ( L N) N ν t ν T L n N,, νc ν Cτ B n n B ( L N) ( 1 L 1 nc τ ) B L L A N L T P( N) t T N nc L C Introduction to Queuing theory 55

Poisson Clients B L L N A PN C ( ) Clients are Poisson L is infinite and : B P( N) λ T n An other way to see it : is independent of N n B λt N T n t N nb tn B n T P ( N) Introduction to Queuing theory 56

The M/M/N/N system Clients are Poisson λ is independent of x λ N servers of rate µ independent of x Only N clients are admitted in the system (no queue) Server Offered Clients λ no buffer Server Server N servers Introduction to Queuing theory 57

Probability of x active clients Clients are Poisson λ is independent of x λ Only N clients are admitted in the system (no queue) P N A i () 0 1 P() 0 N i 0 i! () N i P x Ax x! A i i 0! i 0 1 Ai i! Ax x! P 0 () () P x Introduction to Queuing theory 58

The Erlang B law Clients are Poisson L is infinite BP( N) We get : B AN E,N( A) N! 1 N Ax x! x 0 This law is also called the first law of Erlang Introduction to Queuing theory 59

Erlang B law inversion E 1 N 1 1+ A E A ( ) ( ) A 1, N 1, N 1 ( ) E10, A 1 Introduction to Queuing theory 60

Rigault s rule N A B E A N 1, N ( )! N i A i! i 0 If B 10 k then N A+ k A Introduction to Queuing theory 61

Traffic carried by the N th trunk The N th trunk carries the traffic lost by the group of N-1 previous trunks and loses the traffic lost by a group of N trunks N-1 trunks A ρn N th trunk [ EN ( A) EN( A) ] A 1 Introduction to Queuing theory 62

Number of seizures per hour and traffic If we have n seizures of average duration τ during time T: ti nτ i We get a very important relation between the traffic and the number of seizures per hour (T) : ρ T nτ Introduction to Queuing theory 63

Arrival rate and service rate The arrival rate is the number of new machine seizures per unit of time : T n λ The service rate is the maximum number of services a machine can do per unit of time : µ 1 τ The traffic is the ratio of the arrival rate over the service rate : ρ n τ T µ λ Introduction to Queuing theory 64