Marla Meehl Manager of NCAR/UCAR Networking and Front Range GigaPoP (FRGP)

Transcription

1 Big Data at the National Center for Atmospheric Research (NCAR) & expanding network bandwidth to NCAR over Pacific Wave and Western Regional Network (WRN) Marla Meehl Manager of NCAR/UCAR Networking and Front Range GigaPoP (FRGP) 1

2 CyberInfrastructure (CI) for Climate Science Focus on Big Data Create the data Move the data Store the data Analyze the data To accelerate the rate of scientific discovery, researchers must get the data they need, where they need it, and when they need it, said UC San Diego computer science and engineering professor Larry Smarr 2

3 Climate science Grand challenge problem only one example of big data science Expanding scope higher and higher resolution models - Climate Earth System Model (CESM) Atmosphere multi-level Ocean size, depth Clouds and radiation Precipitation Land surface: soil moisture, land use/land cover (crops, deserts, etc.) Ecosystems: forests, grasslands Chemistry: CO2, methane, ozone, chemical interactions that produce aerosols (visible air pollution) Sea ice Ice sheets on land (Greenland, Antarctica) that influence sea level rise Ocean biogeochemistry (ocean acidification, ocean oxygen, ocean ecosystems) 3

4 Context IT for climate science Big, fast computers - NCAR s latest Cheyenne High Performance Computing SGI ICE XA cluster with 18-core Intel Xeon E5-2697v4 processors 5.34 PFLOPS 4,032 nodes 145,152 Broadwell cores 313 TB total memory Mellanox EDR InfiniBand interconnect, 9-D enhanced hypercube >2.45 Yellowstone equivalents Cheyenne currently ranks as the 20th fastest supercomputer in the world and the fastest in the Mountain West 4

5 Context IT for climate science Big, fast networks In 2017, 100Gbps end to end from the NCAR-Wyoming Supercomputing Center (NWSC) to the FRGP to WRN to CENIC, which reaches research campuses and institutions in California PRP runs over WRN and provides access to the FIONA data node at NCAR at 40Gbps Disk to disk performance measurement to verify path performance Lessons learned can applied be applied to operational systems NCAR s Globally Accessible Data Environment (GLADE) system using Globus is highly optimized for data transfer bypassing firewalls 5

6 Context IT for climate science Big data storage - NCAR s HPSS Data Archive 160 PB capacity Current total holdings: 60 PB Current growth rate: ~12 PB/year increasing to ~30 PB/year with Cheyenne online 6

7 Context IT for climate science Big data analysis - NCAR s latest GLADE Centralized File systems and Data Storage 56 PB usable after planned 20 PB capacity increase 4x DDN SFA14KX each with 20x 84-slot drive chassis 3,360 8-TB disk drives; expandable to 6,720 drives >200 GB/s aggregate I/O bandwidth for new capacity Integrates with existing 16 PB, 90 GB/s GLADE file systems Uses Globus 7

8 Coupled Model Intercomparison Project (CMIP) 6 CMIP TB ~17 modeling groups CMIP PB ~20 modeling groups CMIP6 Simulations starting now, Estimate 10PB of data ~30 modeling groups 8

9 NCAR Research Data Archive During the past two years ( ) 481 users from California have accessed 127 datasets from the Research Data Archive at NCAR, transferring 209 TBytes of data. For example 75 users transferred data as whole files or through subsetting processes that produce customized data packages from the National Centers for Environmental Prediction (NCEP) Final Operational Tropospheric Analyses ( The top 10 users by data volume (TB) are: 28 - UC Irvine 26 - UC Davis 26 - Weather Underground 14 - UC Davis 10 - UC Davis 9 - LBL 7 - UC Davis 7 - Stanford 7 - PNNL 6 - UC Davis 9

10 Example research initiatives using NWSC and WRN/PWAVE between CA and NCAR/CO 325 active users from CA 940 active projects 10

11 Collaborative Research with Climate Models to Narrow Uncertainties in Projections of Future Floods and Droughts CMIP5 multi-model ensemble data and the CESM Large Ensemble climate model simulations Moved terabytes of high-time-resolution global precipitation data from NWSC to the Livermore Computing center The process is so quick and easy that I just let it run in a background window after starting it with a few simple commands. I am most pleasantly surprised at the lack of problems! - Curt Covey This work is part of a DOE-funded collaboration between LLNL, NCAR, SUNY) 11

12 Global warming: heavy precipitation increases above the current-climate UCLA - David Neelin, Sandeep Sahany, Samuel N. Stechmann, and Diana N. Bernstein Precipitation accumulations, integrated over rainfall events, can be affected by both intensity and duration of the storm event Thus, although precipitation intensity is widely projected to increase under global warming, a clear framework for predicting accumulation changes has been lacking, despite the importance of accumulations for societal impacts Analysis of a set of climate model simulations run at the NWSC with CESM show that there is an exponential increase in the frequency of the very largest precipitation accumulations in a future warmer climate ~10 TB of model data 12

13 13

14 Summary Climate Science is a grand challenge problem Requires integrated and scaled compute, storage, analysis, and networks to enable the research In the future, may have to selectively produce, save, and move big data CMIP Analysis Platform NWSC, PRP, WRN, and CENIC are enabling climate science in the west 14