Tuan V. Dinh, Lachlan Andrew and Philip Branch

Transcription

1 Predicting supercomputing workload using per user information Tuan V. Dinh, Lachlan Andrew and Philip Branch 13 th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. Delft, 14 th -16 th May, 213 Motivation Load BUT not too often Load Turn some servers OFF time Turn some servers ON time Another Application: valley filling July 213 Slide 2 Swinburne University of Technology 1

2 Outline How user generate workload? Model of users Prediction technique Expected load Model of cluster of users Question: Expected total requested CPUs of incoming jobs? July 213 Slide 3 Related work 1. Characterising supercomputer job arrival process: Feitelson, 1999: Poisson process time Exponential distribution Lublin et al., 23: Gamma distributions Li et al., 25: Heavy-tail distributions (Log-normal or Pareto) i.i.d Squilante et al., 1999: studied dependence between arrivals primarily descriptive, developing synthetic workload models July 213 Slide 4 Swinburne University of Technology 2

3 Related work 2. Estimating future CPU utilisation: CPU Load? arrivals ~ derivative of the CPU load time Auto-regression (AR) models: use historical correlation Wu et al., 21: successful for Grid computing Liang et al., 213: use AR model, but employ data filters (Kalman). July 213 Slide 5 Outline How user generate workload? Model of users Prediction technique Expected load Model of cluster of users Question: Expected total requested CPUs of incoming jobs? July 213 Slide 6 Swinburne University of Technology 3

4 Swinburne supercomputer The rest of users 14% 16% 86% Top 15% users 84% Year Users & July 213 Slide 7 User submission behaviors No. of CPUs No. of CPUs Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov No. of CPUs Jan Mar May Jul Sep Nov user 1 user 2 No. of CPUs Jan Mar May Jul Sep Nov user 3 user 4 BIG USER: HETEROGENOUS July 213 Slide 8 Swinburne University of Technology 4

5 Outline How user generate workload? Model of users Prediction technique Expected load Model of cluster of users Question: Expected total requested CPUs of incoming jobs? July 213 Slide 9 User s model Use job inter-arrival time distribution of each user Hazard rate function = Probability of he/she about to submit one Given not submitted for t unit of time: = pdf(inter-arrival time) 1 Pr[inter-arrival time < t ] Can be obtained from system July 213 Slide 1 Swinburne University of Technology 5

6 Measured hazard curves user user user 3 user 4 July 213 Slide 11 Parametric fitting Fitting family Key properties b h ( t) t W (Weibull) monotonic h ( t) t e C b ct July 213 Slide 12 Swinburne University of Technology 6

7 Fitting quality user 1 user user 3 user 4 Weibull proposed July 213 Slide 13 Outline How user generate workload? Model of users Prediction technique Expected load Model of cluster of users Question: Expected total requested CPUs of incoming jobs? July 213 Slide 14 Swinburne University of Technology 7

8 User clustering mean sq. inter-arrival std. dev. sq. inter-arrival mean inter-arrival std. dev. interarrival cluster 1 (21) cluster 2 (8) cluster 3 (42) User clustering for 21 (except for the top 15 users). C = 3. Algorithm: k-means algorithm. Software: WEKA [*] [*]: July 213 Slide 15 Outline How user generate workload? Model of users Prediction technique Expected load Model of cluster of users Question: Expected total requested CPUs of incoming jobs? July 213 Slide 16 Swinburne University of Technology 8

9 Schemes HR, aggregate HR, cluster: the rest is further clustered in to C clusters. Each is treated as a user HR, per-user ˆ N N < N < number of big users total number of users July 213 Slide 17 Performance evaluation HR models (21 traces) last submission time (of each user) HR, aggregate or HR, cluster or HR, per user Prediction of total requesting CPUs Performance metrics: MSE, MAE Jan-211 = slot width, is small ~ order of minutes Dec-211 measured prediction July 213 Slide 18 Swinburne University of Technology 9

10 Performance evaluation Schemes no smoothing smoothing MSE MAE MSE MAE AR(p=35) HR, per-user HR, aggregate HR, clustered 4,3 2,749 2,738 2, , No. of individual users: 15 Slot time = 2 minutes C = 3, clustering algorithm: k-means algorithm. Attributes: mean and std. dev. inter-arrival time; mean and std. dev. squared inter-arrival time July 213 Slide 19 Degree of clustering? 385 big users: 84% load MSE with smoothing C = number of big users big users: 93% load July 213 Slide 2 Swinburne University of Technology 1

11 Remarks and future work Limitations: experiment with only ONE workload assumptions on user correlations Future work: conduct experiment with more workload optimising user clustering process July 213 Slide 21 Conclusions User: heterogeneous Big users dominate User s submission behaviours are different Cluster small users HR, cluster works best Modelling all users may give worse results Choosing right number of big users Next (or remaining) questions? Other traces? Optimise the clustering process July 213 Slide 22 Swinburne University of Technology 11