HPC and High-end Data Science

Transcription

1 HPC and High-end Data Science for the Power Grid Alex Pothen August 3, 2018

2 Outline High-end Data Science 1 3 Data Anonymization Contingency Analysis 2 4 Parallel Oscillation Monitoring 2 / 22

3 PMUs in the U.S. Phasor Measurement Units in the North American Grid as of March 2015 [1] 3 / 22 [1] NASPI website:

4 Emerging Data Logistics 4 / 22

5 Challenges in High-end Data Analysis Computing is a major consumer of electricity, recent estimates about 10%. Data movement within a processor consumes more energy than arithmetic. Data movement from the instrument to the curation and archival sites, visualizations, researcher s work spaces, etc. represents a significant use of energy. Hence energy is an overarching challenge for sustainable High-end Data Analysis (HDA). Solutions include: Reduce computational costs by matching platforms to each stage of the scientific method Reduce data movement costs by collocation, compression and caching Reuse data through sharing, metadata, and catalogs 5 / 22

6 Challenges in High-end Data Analysis Data reduction is a fundamental pattern in the convergence of HPC and HDA. need new algorithms for loss-less and lossy compression of scientific data, and need algorithms that can work with the compressed data. New algorithms and software for HPC, data science, analytics, Machine learning, etc. are needed here. A major challenge is the convergence in the software ecosystem for HPC and HDA. 6 / 22

7 Challenges in High-end Data Analysis High volume data needs high performance computing Streaming data needs on-line algorithms Integration of heterogeneous data e.g.., consumption forecasting in smart grids: smart meters, utility databases, weather data, microgrid sensor networks, etc. Anonymization of consumer data 7 / 22

8 Outline High-end Data Science 1 3 Data Anonymization Contingency Analysis 2 4 Parallel Oscillation Monitoring 8 / 22

9 Contingency Analysis N x contingency analysis on power flow evaluates the stability of a power system by simulating the failures of x out of N transmission lines or generators (components) gen-2 5 load 6 gen-3 load gen-1 load / 22

10 Contingency Analysis N x contingency analysis on power flow evaluates the stability of a power system by simulating the failures of x out of N transmission lines or generators (components) gen-2 5 load 6 gen-3 load gen-1 load / 22

11 Contingency Analysis N x contingency analysis on power flow evaluates the stability of a power system by simulating the failures of x out of N transmission lines or generators (components) gen-2 5 load 6 gen-3 load gen-1 load / 22

12 Contingency Analysis Currently each N x contingency analysis is performed independently, but since the number of cases increases exponentially with x, this is computationally impractical. (For example, if N = 3120 and x = 5, there are in total cases.) Heuristics are needed for choosing the cases to be analyzed. To be able to analyze more cases, we have to speed up the solution time for each case. We propose new algorithms for the problem by observing that only a principal submatrix of the system is changed when a component is removed upon failure. 10 / 22

13 Augmented Formulation The augmented matrix system can be expressed as: n{ A AH H m{ H A H ÂH x 1 b 0 x 2 = H b. m{ H 0 0 x 3 0 A is the original matrix; H is a submatrix of the identity matrix; Â = A HEH T updated matrix; m n; b is the new right-hand-side of the updated rows. 11 / 22

14 Runtimes Power Grid Contingency Analysis of 777,646 Nodes 10 1 time (second) PARDISO LUSOL CHOLMOD augmented (Direct) augmented (GMRES) x (number of components removed) 12 / 22

15 Outline High-end Data Science 1 3 Data Anonymization Contingency Analysis 2 4 Parallel Oscillation Monitoring 13 / 22

16 Adaptive Anonymity Users Features F 1 F 2 F 3 F 4 F 5 F 6 U U U U U U / 22

17 Adaptive Anonymity Users Features F 1 F 2 F 3 F 4 F 5 F 6 U U U U U U Users Features F 1 F 2 F 3 F 4 F 5 F 6 U U U U U U / 22

18 Adaptive Anonymity with Approximation Algorithms Prob. Inst. Feat. b EdgeCover Time Util. Census K 68 9m 90% PokerHands 500K 95 2h 17m 84% CMS 1M h 33m 81% Earlier Algorithms Belief propagation: used exact b-matching algorithm, could solve problems with few thousand instances only. The approximate edge cover approach is two to three orders of magnitude faster on those smaller problems. 16 / 22

19 Adaptive Anonymity Computation on Cori 17 / 22

20 Outline High-end Data Science 1 3 Data Anonymization Contingency Analysis 2 4 Parallel Oscillation Monitoring 18 / 22

21 PMUs in the U.S. Stochastic Subspace Identification Covariance State Space Model xk 1 Axk wk, yk Cxk vk Past & Future Output Matrices y y1 y 0 J 1 y1 y2 y J Yp, y y y I I I J 1 2 y y y I I 1 I J 1 yi 1 yi 2 y I J Yf, y y y I I I J Covariance Matrix T H Y Y O G, I f P I 2 I 1 T T, G Exkyk, O C CA CA CA Singular Value Decomposition (SVD) T H O G USV, O U S I I 1/2 1 1, Fast SVD Approaches State & Output Matrices Mode Identification A OI OI, C OI 1: l,:, A c = 1 log A, C T c = C, S Mode Freq. and Damping: eigenvalues of Aˆ c, s ModeShape: ˆ m C. i c i 19 / 22

22 PMUs in the U.S. Case Voltage Phase Angles Computational Time Comparison Mac. No. Full SVD (sec) Strat. 1 sec (speedup) Randomized SVD h = 20, q = 1 Strat. 2 sec (speedup) Strat. 3 sec (speedup) Strat. 4 sec (speedup) Strat. 1 sec (speedup) Lanczos SVD k = 16 Strat. 2 sec (speedup) Strat. 3 sec (speedup) Strat. 4 sec (speedup) 1 112, ,826 (1.06x) (542x) (828x) (1060x) 106,234 (1.05x) 81.2 (1380x) (1119x) 96.0 (1167x) 2 125, ,007 (1.04x) (529x) (723x) (887x) 115,458 (1.09x) 87.4 (1439x) (850x) (1031x) 3 138, ,304 (1.02x) (463x) (600x) (1030x) 133,961 (1.03x) (1357x) (781x) (1202x) 4 170,518 Theoretical speedup 168,617 (1.01x) 58x (560x) (754x) (1218x) 163,028 (1.05x) 140x (1627x) (1079x) (1233x) 20 / 22

23 References HDA and HPC M. Asch, I. Moore, R. Badia, J. Dongarra et al, Big Data and Extreme-Scale Computing: Pathways to Convergence, Internat. J. High Performance Computing Applications, 2018, 32(4), pp Data Anonymization A. Khan, K. Choromanski, A. Pothen, SM Ferdous, M. Halappanavar and A. Tumeo, Adaptive Anonymization of Data using b-edge cover, ACM/IEEE Proc. Supercomputing, Nov Data Reduction for Oscillation Monitoring T. Wu, V. Venkatasubramanian, and A. Pothen, Fast parallel stochastic subspace algorithms for large-scale ambient oscillation monitoring, IEEE Trans. Smart Grid, 8(3), pp , / 22

24 References Contingency Analysis Y. Yeung, A. Pothen, M. Halappanavar and Z. Huang, AMPS: An augmented matrix formulation for principal submatrix updates with application to power grids, SIAM J. Sci. Comput., 39 (3), S809-S827, / 22