2004 Journal of Software 软件学报

Transcription

1 /2004/5(07) Journal of Software 软件学报 Vol5, No7 如何测量 SMP 机群可扩放性 何家华 +, 陈国良, 单久龙 ( 中国科学技术大学计算机科学技术系国家高性能计算中心 ( 合肥 ), 安徽合肥 ) How to Measure Scalability of SMP Cluster HE Jia-Hua +, CHEN Guo-Liang, SHAN Jiu-Long (National High Performance Comuting Center (Hefei), Deartment of Comuter Science and echnology, University of Science and echnology of China, Hefei , China) + Corresonding author: Phn: , jiahua@mailustceducn, htt://wwwustceducn Received ; Acceted He JH, Chen GL, Shan JL How to measure scalability of SMP cluster Journal of Software, 2004,5(7): 977~986 htt://wwwjosorgcn/ /5/977htm Abstract: Scalability is an imortant erformance criterion of arallel comuting However the conventional scalability metrics are not suitable for SMP(symmetric multirocessor) cluster How to measure scalability of SMP cluster? his aer rooses a solution to the roblem It first finds out and verifies the reason, nonequivalence of the rocessor sets and then it adots the viewoint of rocessor set to observe the behaviors of the system correctly and comrehensively instead of using only the number of rocessors to describe the arallel system By introducing the concet of erformance erence factor, it extends the conventional metrics to fit the SMP cluster architecture As the exeriments indicate, the extended metrics are alicable to the SMP cluster and have high recision Key words: arallel comuting; erformance evaluation; SMP cluster; scalability 摘要 : 可扩放性是并行计算的一个重要性能标准, 但是传统的可扩放性准则并不适用于 SMP 机群 如何测量 SMP 机群的可扩放性? 试图提出该问题的一个解决方案 首先找出并验证问题的根源 处理器集合不等价性 然后, 采用处理器集合的观点来全面 正确地观察系统的行为, 而并非像传统的做法那样仅仅使用处理器数来描述并行系统 通过引入性能参考因子的概念, 扩展了传统的准则以适应 SMP 机群体系结构 实验结果显示, 扩展后的度量准则适用于 SMP 机群, 且具有较高的准确性 关键词 : 并行计算 ; 性能评测 ;SMP 机群 ; 可扩放性 Suorted by the National High-ech Research and Develoment Plan of China under Grant No200AA04 ( 国家高技术研究发展计划 (863)) HE Jia-Hua was born in 976 He is a master candidate at the Deartment of Comuter Science and echnology, the University of Science and echnology of China His current research areas include arallel and distributed comuting CHEN Guo-Liang was born in 938 He is a rofessor and doctoral suervisor at the Deartment of Comuter Science and echnology, the University of Science and echnology of China and is an academician of Chinese Academy of Sciences His research areas are arallel and distributed comuting SHAN Jiu-Long was born in 977 He is a PhD candidate at the Deartment of Comuter Science and echnology, the University of Science and echnology of China His current research interests include arallel and distributed comuting

2 978 Journal of Software 软件学报 2004,5(7) 中图法分类号 : P338 文献标识码 : A Introduction here are many different kinds of architectures in high erformance comuting SMP(symmetric multirocessor) and cluster are two imortant ones among them he shared-memory rogramming model of SMP derived from its symmetric and shared memory makes it easy to rogram and exloit high DoP(degree of arallelism), and its shared resources bring it high erformance-cost ratio However, it is the symmetry that limits the scale of the SMP In contrast, cluster is famous for its excellent scalability here seems to be an inherent comlementarity and otential advantages in combining the two architectures into a SMP cluster, that is, a cluster with SMP nodes Johnston [] had redicted its success in the last century and now we can see many examles of the architecture on the list of o500 [2] including the second fastest comuter, HP ASCI Q Scalability analysis lays an imortant role in erformance evaluation of large arallel systems One key issue in these systems is how to make an effective use of the increasing rocessors because their average erformance will usually dro while more rocessors are used Scalability is just such a metric that measures the caability of the system, including hardware and software, to utilize effectively the scaling-u rocessors Conventional aroaches to scalability analysis include Isoefficiency metric [3], Isoseed metric [4], Latency metric [5], and ime-scale metric [6], etc However, we cannot directly aly the conventional aroaches to the SMP cluster because of the nonequivalence of rocessor sets hat is, a articular arallel rogram will take different times on different rocessor sets with the same number of rocessors We will solve this roblem by two concets, rocessor-set viewoint and erformance erence factor, and give a solution for scalability measurement of the SMP cluster Before beginning our discussion, we must make the following two assumtions, based on the consideration that most of the SMP cluster systems used ractically in large-scale scientific comuting will adot homogeneous or almost homogeneous rocessors and nodes in order to achieve high erformance () Homogeneity of rocessors: all the rocessors are identified in frequency, cache and architecture etc; (2) Homogeneity of nodes: the features of all the nodes, such as number of rocessors, caability of main memory and bandwidth of system bus, are the same he rest of the aer is arranged as follows In Section 2, conventional scalability metrics are reviewed briefly, and then we verify the existence of nonequivalence of rocessor sets and introduce the concet rocessor-set viewoint in Section 3 On the base of the concets erformance erence rogram and erformance erence factor defined in Section 4, we extend the conventional scalability metrics in Section 5 and illustrate them by the examle algorithm of arallel matrix multilication in Section 6 Some conclusions and suggestions for future work are given in the last section 2 Conventional Scalability Metrics In this section, we will give a brief review of the conventional aroaches to scalability analysis, including Isoefficiency metric, Isoseed metric, Latency metric, and ime-scale metric Isoefficiency metric: Let be the comutation time, namely, the sum of the time sent by all the rocessors in useful comutation, comm be the communication time, namely, the sum of the time sent by all rocessors in communicating with neighboring rocessors, waiting for messages, time in starvation, etc, and n be the arallel execution time on n rocessors, then we can infer that n n + comm and the efficiency E is given as:

3 何家华等 : 如何测量 SMP 机群可扩放性 979 E n n + comm + According to the above formula, Kumar et al roosed in 987 that the roblem size W needs to grow as a function of n, the number of the increasing rocessors, to maintain the efficiency E hey named the function Isoefficiency Function and use the function curve, Isoefficiency Curve, to lect the algorithm s scalability Isoseed metric: Sun et al resented the metric in 994 hey first defined the average seed V as the achieved seed of the given comuting system divided by the number of rocessors, and then gave a definition: Let W be the amount of work of an algorithm when n rocessors are emloyed in a machine, and let W be the amount of work of the algorithm when >n rocessors are emloyed to maintain the average seed, then we define the scalability form system size n to the system size of the algorithm-machine combination as follows W ψ ( n, ) nw Latency metric: Zhang et al also gave a definition in 994 For a given algorithm imlementation on a given machine, let L e (W,n) be the average latency when the algorithm for solving a roblem of size W on n rocessors, and L e (W,) be the average latency when the algorithm for solving the roblem of size of W on >n rocessors If the system size changes from n to, and the efficiency is ket to a constant E [0,], the scalability latency metric is defined as comm Le ( W, n) scale( E,( n, )) L ( W, ) ime-scale metric: his is our revious work in 2000 For a given algorithm imlementation on a given machine, let (W,n) be the arallel execution time with the roblem workload W on n rocessors, and (W,) the arallel execution time with the roblem workload W on (>n) rocessors If the number of rocessors increases from n to, and the efficiency E kees conserved, then the ime-scale scalability metric is defined as: ( W, n) time _ scale( E,( n, )) ( W, ) e 3 Nonequivalence of Processor Set and Processor-Set Viewoint We have surveyed the conventional metrics of scalability hey can do a good job in most of the arallel and distributed comuting architectures, but in fact they do not work unless all the n-rocessor sets are equivalent because they use only one arameter n to describe the arallel rogram It is a ity that the SMP cluster does not satisfy the assumtion Namely, different n-rocessor sets for a given n may have different comuting abilities in the SMP cluster We call that nonequivalence of n-rocessors sets o verify the existence of the nonequivalence, we imlemented the algorithms of arallel matrix multilication and arallel longest common subsequence on Dawning2000-II [7] at NHPCC (Hefei) [8] he execution times of different 4-rocessor sets with the same roblem size are listed in able, where *4 means the set includes 4 nodes with rocessor er node, *2+2* means the set includes 2 nodes with rocessor er node and node with 2 rocessors, and so on What we must exlain is the sets 4* and *+3* do not exist because the nodes used in the exeriment are 2-way nodes he exeriment data demonstrate the existence of the nonequivalence One question is what causes the nonequivalence? By analyzing the architecture features of the SMP cluster caully, we can see that the nonequivalence derives from the following two heterogeneities: () Communication heterogeneity: there are two communication methods in the SMP cluster; shared memory inside one node and message assing between nodes

4 980 Journal of Software 软件学报 2004,5(7) (2) Resource heterogeneity: the rocessors inside one node share resources, and the ones in resective nodes have their dedicated resources hese two factors affect the things in oosite directions he communication heterogeneity ers the rocessors in one node to those in different nodes In contrast, the resource heterogeneity will make the erformance of latter condition better than that of the former able Parallel execution times on different 4-rocessor sets Processor sets *4 *2+2* 2*2 Execution time of matrix multilication (s) Execution time of longest common subsequence (s) Since the way using only one arameter n to describe the arallel rogram is invalid for the SMP cluster, we need a new standing oint to observe the behaviors of the system We will adot the rocessor-set viewoint in the rest of the aer hat is, no matter when and where we mention the size of the machine, we are always erring to a certain rocessor set, which imlies not only the number but also the allocation of the rocessors 4 Performance Reference Program and Performance Reference Factor o fulfill the rocessor-set viewoint is not easy because no matter which scalability metric we adot, Isoefficiency metric, Isoseed metric, Latency metric or ime-scale metric, the arameter n will aear in the definition of efficiency E or average seed V We have to find a new arameter according to rocessor-set viewoint to relace n In the last section, we have discussed that the reason for the invalidation of the arameter n is that different n-rocessor sets for a given n have different comuting abilities It imlies that the original urose of n is to describe the comuting ability of the rocessor set, so the new arameter must be able to finish this task he interesting issue is how to find the new one he current way is using the amount of resources to describe the comuting ability indirectly Now that the indirect way is not feasible in the SMP cluster, the new arameter may describe the comuting ability directly Seed V is one of the candidates What rogram should be used to measure the seed? Just like the way we select a frame of erence in mechanics, we need to choose a rogram as erence for indicating their relative erformance to those of others, and theore name it erformance erence rogram Such a relative concet gives us a flexibility to choose different erformance erence rograms according to demands and conditions, esecially in different alication fields However, the erformance erence rogram should satisfy the following requirements for racticability and high recision () he workload should be in roortion to the number of the rocessors We set the requirement instead of the fixed workload to avoid too heavy load on single rocessor and too light load on n-rocessor set when n is large Let W () be the workload on single rocessor and W (n_set) be the workload on a given n-rocessor set, then W (n_set)nw () (2) he comutation to communication ratio should be constant when the workload increases Or the erformance erence factor will vary with the size of workload In ractical oeration, this requirement will not be satisfied until the size of workload is large enough because of the initialization affects As for the erformance erence rogram used in this aer, we adot the simlest way to construct it, that is, let each rocessor roceed a certain amount of comutational and communicational oerations round and round In addition to the advantage of simleness, such a erformance erence rogram makes it ossible for us to adjust the

5 何家华等 : 如何测量 SMP 机群可扩放性 98 comutation to communication ratio as needed Since we have finished the discussion of erformance erence rogram, it is time to come back to the new arameter we are looking for Seed (not average seed) V is the very thing to describe the comuting ability of the machine, but we cannot use it to relace n directly because the latter is a nondimensional arameter, and we have to introduce a new concet erformance erence factor Let V () be the seed on single rocessor and V (n_set) be the seed on a given n-rocessor set, then we define the erformance erence factor of the n-rocessor set as: V rf V () In ractical oeration, seed is not easy to measure directly We will measure execution time instead and ulate out the result of erformance erence factor Let () be the execution time on single rocessor and (n_set) be the time on n-rocessor set, then W V n () rf V () W () W () () () 5 Scalability Metrics of SMP Cluster On the base of the discussion in the last section, let s see how to extend the scalability metrics of the SMP cluster relacing n with erformance erence factor Extended Isoefficiency metric: When the arameter n is relaced by the erformance erence factor, the efficiency E looks like: nw () E rf n () + n comm () + comm From the formula, we can see that there is still a functional relation between W and the rocessor set (not n) when E is fixed But to work out the relation between () and (n_set) is not easy We will leave the roblem as a future work Extended Isoseed metric: Let (W,n_set) be the arallel execution time with roblem workload W on the n-rocessor set, the average seed is defined as: W V rf ( W, n _ set) Let W be the amount of work of an algorithm when an n-rocessor set is emloyed in a machine, and W be the amount of work of the algorithm when an -rocessor set is emloyed to maintain the average seed, then we define the scalability from n-rocessor set to -rocessor set of the algorithm-machine combination as follows rf ( _ set) W ( W, n _ set) ψ ( n _ set, _ set) rf W ( W, _ set) Extended Latency metric: Let (W,) be the arallel execution time with roblem workload W on single rocessor, and let (W,n_set) be the arallel execution time with roblem workload W on the n-rocessor set, the efficiency is defined as:

6 982 Journal of Software 软件学报 2004,5(7) ( W,) E rf ( W, n _ set) For a given algorithm imlementation on a given machine, let L e (W,n_set) be the average latency when the algorithm for solving a roblem of size W on an n-rocessor set, and L e (W,_set) be the average latency when the algorithm for solving the roblem of size of W on an -rocessor set If the system size changes from n-rocessor set to -rocessor set, and the efficiency is ket to a constant, the scalability latency metric is defined as: Le ( W, n _ set) scale( E, ( n _ set, _ set)) L ( W, _ set) Extended ime-scale metric: he definition of E is the same as that in Latency metric For a given algorithm imlementation on a given machine, let (W,n_set) be the arallel execution time with roblem workload W on an n-rocessor set, and (W,_set) the arallel execution time with roblem workload W on an -rocessors set If the system size changes from n-rocessor set to -rocessors set, and the efficiency E kees conserved, then the ime-scale scalability metric is defined as: ( W, n _ set) time _ scale( E,( n _ set, _ set)) ( W, _ set) e 6 Case Study In this section, let s see how the two extended metrics, Isoseed metric and ime-scale metric, work in the SMP cluster Before imlementing the metrics, we run the erformance erence rogram, measure the execution times and then ulate the erformance erence factors as follows able 2 Execution times of erformance erence rogram and erformance erence factors Processor set ime (s) Rf(n_set)

7 何家华等 : 如何测量 SMP 机群可扩放性 983 Extended Isoseed Metric: By the aroach 2 described in Ref[4], we get the time variation and the comuted scalability of the matrix multilication algorithm as follows able 3 ime variation under extended Isoseed metric Processor set ime (s) Processor set ime (s) able 4 Comuted scalability of extended Isoseed metric Ψ(n_set, _set) Ψ(n_set, _set) In fact, the deendant variable (Ψ) and the two indeendent variables (n_set and _set) form a three dimensional sace he two sub-tables of able 4 lect only two cross sections of the sace hese tabular data can also be reresented by the following ictorial diagrams

8 984 Journal of Software 软件学报 2004,5(7) Ψ (n _set,n' _set ) n' _set 2 Ψ (n _set,n' _set ) n'_set Fig Scalability curves of extended Isoseed metric Extended ime-scale Metric: By the aroach described in Ref[6], we get the time variation and the comuted scalability of the matrix multilication algorithm as follows able 5 ime variation under extended ime-scale metric Processor set ime (s) Processor set ime (s) As you may have noticed, most of the rocesses are very similar to that of the conventional metrics he only difference is that the erformance erence factor must be measured and ulated first, and then relace n through the entire rocess of comuting scalability he solution is simle and easy to oerate, but it does solve the roblem how to measure the scalability of the SMP cluster In addition, the fact that the two extended metrics resent the very similar scalability trends of the same roblem indicates that they have high recision

9 何家华等 : 如何测量 SMP 机群可扩放性 985 able 6 Comuted scalability of extended ime-scale metric time_scale(e, (n_set, _set)) time_scale(e, (n_set, _set)) Ψ (n _set,n' _set ) n'_set 2 Ψ (n _set,n' _set ) n' _set Fig2 Scalability curves of extended ime-scale metric

10 986 Journal of Software 软件学报 2004,5(7) 7 Conclusions and Future Directions In this aer, we review the conventional scalability metrics, including Isoefficiency metric, Isoseed metric, Latency metric and ime-scale metric, resent their unsuitability for the SMP cluster and verify the reason, nonequivalence of the rocessor sets o solve the roblem, we adot the viewoint of rocessor-set to observe the behaviors of the system, introduce the concets of erformance erence rogram and erformance erence factor and then extend the scalability metrics to fit the architecture of SMP cluster With the examle of matrix multilication algorithm, we illustrate the validity of the extended metrics As for the future work, we will ut an emhasis on how to construct erformance erence rogram Now we adot the simlest way to construct and believe that it is the best way to begin We will try other real rograms such as a benchmark to comare their effects in future At the same time, how to use the extended Isoefficiency metric to analyze the scalability of SMP cluster in ractice is another roblem References: [] Johnston WE he case for using commercial symmetric multirocessors as suercomuters htt://www-itglblgov/~johnston/ ScientificComArch/ScientificComArchfmhtml [2] o500 htt://wwwto500org [3] Kumar V, Rao VN Parallel deth first search, Part Ⅱ: Analysis International Journal of Parallel Programming, 987,6(6): 50~59 [4] Sun XH, Rover D Scalability of arallel algorithm-machine combinations IEEE rans on Parallel Distributed Systems, 994, 5(6):599~63 [5] Zhang XD, Yan Y, He K Latency metric: An exerimental method for measuring and evaluating arallel rogram and architecture scalability Journal of Parallel and Distributed Comuting, 994,22(3):392~40 [6] Ji YC, An H, Ding WQ, Chen GL A scalability metric for algorithm-machine on NOW and MPP In: Proc of the 4th Int l Conf on High-Performance Comuting in Asia-Pacific Region ~407 [7] Dawning-2000 htt:// /dawn2000 [8] NHPCC(Hefei) htt://nhccustceducn