COMP9334: Capacity Planning of Computer Systems and Networks Week 2: Operational analysis Lecturer: Prof. Sanjay Jha NETWORKS RESEARCH GROUP, CSE, UNSW
Operational analysis Operational: Collect performance data during day-today operation Operational laws: A set of basic relationships between measured performance data, or operational variables Very simple, but useful because they are: Based on readily available measurement data Can be used to obtain helpful performance metrics Can be used for performance analysis Bottleneck analysis (this lecture) Mean value analysis (Week 8 lecture) Page 1
Motivating problem System with one single-server resource, e.g. one CPU Observation period = 1 minute CPU busy for 36 sec 1800 customers arrived & served Customer: a generic term, represents a database transaction, an HTTP request, a batch job, etc Find Arrival rate Mean service time per customer Utilization of CPU System throughput Page 2
Notations for measured quantities τ A i B i C i A 0 C 0 Observation period Total number of arrivals to resource i in τ Total busy time of resource i in τ Total number of completions from resource i in τ Total number of arrivals to the system in τ Total number of completions from the system in τ Page 3
Notations for derived quantities λ i S i U i X i X 0 Arrival rate at resource i Mean service time per customer at resource i Utilization of resource i Throughput of resource i Throughput of the system Page 4
Solution to the motivating problem The measured quantities are: τ = 60 sec A 1 = A 0 = 1800 arrivals B 1 = 36 sec C 1 = C 0 = 1800 completions The derived quantities are: λ 1 = A 1 /τ = 1800/60 = 30 customers per sec S 1 = B 1 /C 1 = 36/1800 = 0.02 sec per customer U 1 = B 1 /τ = 36/60 = 60% X 0 = C 0 /τ = 1800/60 = 30 customers per sec Page 5
Utilization Law Utilization: average fraction of time a resource is busy on By definition, U = B/τ U 1 High utilization High response time What if U 1? Devices with high utilization cause bottlenecks Utilization Law: U = C τ B C = X S With flow equilibrium assumption, i.e. X = λ, then U = λ S For a m-server resource, B C = S m U = X S m Page 6
Example A network segment transmits 1000 packets per second with average packet transmission time of 0.15 msec. What is the utilization of this network segment? U = 1000 0.15 10 3 = 0.15 or 15% Page 7
Visit count A customer may require multiple visits to a resource in the system E.g. If on average a customer arriving at the system will require 3 visits to resource i, what is the ratio C i /C 0? We expect C i /C 0 = 3 V i : average number of times a customer visits resource i V i = C i /C 0 Page 8
Example A database server with three resources: CPU, Disk 1, Disk 2 On average, a transaction makes 25 visits to Disk 1 and 6 visits to Disk 2 What is the visit count of CPU? V CPU = 25 + 6 + 1 = 32 Page 9
Forced Flow Law V i = C i C 0 = C iτ C 0 τ = X i X 0 Throughput of resource i is equal to the average number of visits made by a customer to resource i multiplied by the system throughput X i = V i X 0 Page 10
Service Demand Law Service demand: total average service time required by a customer at resource i, denoted D i On average, a customer visits resource i for V i times On average, service time per visit of a customer at resource i is S i So, D i = V i S i Service Demand Law D i = V i S i = X i X 0 S i = U i X 0 Page 11
Little s Law This very general law is remarkably useful For a resource with throughput X and response time R, average number of customers in the system is N = X R = λ R Page 12
Intuitive explanation of Little s Law Consider the customer Bob who has just left the system (completed service) Bob sees behind his back on average N customers. Who are these customers? They are the customers that had been arriving during the time that Bob was in the system. Their average number is λr Page 13
Example 1 A router processes 200 packets per second, average response time 0.1 second. How many buffers are needed? Average number of packets in router: N = 200 0.1 So 20 buffers required on average At least three times of that to cope with peaks, four would be better Page 14
Example 2 http://www.cse.unsw.edu.au/ jguo/tcomm05.pdf Find mean waiting time of admitted customers in a finite buffer TDMA system Mean Waiting Time (timeslot) Running Time (sec) Little s Law 1.9649 0.03 Queueing analysis 1.9649 16 Simulation 1.9648 6886 Page 15
Interactive system Closed system Each user alternates between thinking (i.e., composing a request) and waiting for a response Think time: time since a user receives a response until a subsequent request is submitted Response time: time since the computer system receives a request until the response is sent Page 16
Interactive system (cont.) M users Z = mean think time R = mean response time of the computer system X 0 = system throughput Page 17
Interactive system (cont.) M = average number of users thinking N = average number of users waiting for a response A user is either thinking or waiting for a response M = M + N Page 18
Interactive Response Time Law Applying Little s Law, we have M = X 0 Z N = X 0 R So, M + N = M = X0 (Z + R) Interactive Response Time Law R = M X 0 Z Page 19
Example 7,200 requests are processed during one hour by an interactive computer system with 40 users and an average think time of 15 sec. The average response time is 40 R = 15 = 5 sec 7200/3600 Page 20
Summary of operational laws Utilization Law: U i = X i S i = λ i S i Forced Flow Law: X i = V i X 0 Service Demand Law: D i = V i S i = U i /X 0 Little s Law: N = X R = λ R Interactive Response Time Law: R = M/X 0 Z Page 21
Bottleneck analysis Also known as bounding analysis The bottleneck resource determines the best possible performance of the system Upper bounds on throughput Lower bounds on response time The bounds are valid with the load-independent assumption of service demands: The service demand of a customer at a resource is constant, independent of the number of concurrent customers in the system Page 22
Motivating example 1 Open system with three resources: CPU, Disk 1, Disk 2 Each customer makes on average 25 visits to Disk 1, 6 visits to Disk 2, and hence 32 visits to CPU Mean service time (per visit) is 37.5 msec at CPU, 20 msec at Disk 1, 50 msec at Disk 2 What is the service demand of each resource? Page 23
Bottleneck 1 0.9 0.8 Resource 1 Resource 2 Resource 3 Utilization 0.7 0.6 0.5 0.4 0.3 0.2 0.1 D 1 =1.2 sec D 2 =0.5 sec D 3 =0.3 sec 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 System throughput What is the bottleneck of the system? The resource with the highest utilization, or equivalently, the one with the largest service demand Service Demand Law: D i = U i /X 0 The bottleneck resource in this example is resource 1 (CPU) U 1 1 X 0 1/D 1 = 0.833 completions per sec upper bound on system throughput Page 24
Asymptotic bounds for example 1 X 0 1 max i D i R i D i Upper bound on system throughput = Throughput of the bottleneck resource when its utilization reaches one Lower bound on system response time = Sum of service demands At least, when there is no queueing Page 25
Motivating example 2 Closed system with three resources, same as example 1 Each customer makes on average 25 visits to Disk 1, 6 visits to Disk 2, and hence 32 visits to CPU Mean service time (per visit) is 37.5 msec at CPU, 20 msec at Disk 1, 50 msec at Disk 2 Service demand is 1.2 sec at CPU, 0.5 sec at Disk 1, 0.3 sec at Disk 2 Page 26
Asymptotic bounds for example 2 Same as example 1, we have the two bounds: X 0 1 max i D i and R i D i We have another bound in this example on X 0 and R In closed systems, bounded and known number of jobs N is the number of jobs in the system By Little s Law, N = X 0 R 1 X 0 R N max i D i max i D i R i D i X 0 N i D i Page 27
Asymptotic bounds for example 2 (cont.) X 0 min ( 1 max i D i, ) N i D i R max ( i D i, N max i D i ) Page 28
What about an interactive system? Interactive Response Time Law: R = M X 0 Z M: Bounded and known number of jobs Z: Mean think time, independent on M What is the upper bound on X 0? What is the lower bound on R? Solution to be discussed in sample problems Page 29
Why bottleneck analysis? Performance bounds are obtained by considering the service demands only Simple, very little computation, useful Bounds are conservative If the bounds don t meet requirements, the system certainly won t! A quick way to test system designs Work out what s the best the system can do If that s not good enough, fix the bottleneck resource (e.g. upgrade it with a faster one), and try again Page 30
Limitations of operational analysis Mean values for both input parameters and output variables More detailed analytical results of queueing models require queueing analysis E.g. standard deviation, probability distribution, etc Page 31
Reference Textbook Chapter 3 P. J. Denning and J. P. Buzen, The operational analysis of queueing network models, ACM Computing Surveys, Vol. 10, No. 3, September 1978, pp. 225 261. Page 32