Paralleliza(on and Performance of the NIM Weather Model on CPU, GPU and MIC Architectures

Size: px
Start display at page:

Download "Paralleliza(on and Performance of the NIM Weather Model on CPU, GPU and MIC Architectures"

Transcription

1 Paralleliza(on and Performance of the NIM Weather Model on CPU, GPU and MIC Architectures Mark Gove? NOAA Earth System Research Laboratory

2 We Need Be?er Numerical Weather Predic(on Superstorm Sandy Hurricane Sandy Second most destruc(ve in U.S. History $75B in October damages 28, 2012 Over 200 deaths A European forecast that closely predicted Hurricane Sandy's onslaught days ahead of U.S. and other models is raising complaints in the meteorological community. "The U.S. does not lead the world; we are not No. 1 in weather forecascng, I'm very sorry to say that," says AccuWeather's Mike Smith Source: USA Today, October 30, 2012 Congressional Response: High Impact Weather Predic(on Program (HIWPP) Next Genera(on Weather Predic(on Program (NGGPS)

3 Three Years Later Hurricane Joaquin October 2, 2015 Some improvement NOAA s Hurricane Weather Research & Forecast Model intensity forecasts were accurate US research models had 20 precipita(on forecasts in South Carolina 36 hours in advance (verified) NY Times: Why U.S weather model has fallen behind WashingtonPost: Why the forecast cone of uncertainty is inadequate But European models predicted Joaquin would not make landfall (verified) All U..S models incorrectly predicted landfall The Na(onal Hurricane Center correctly never issued any hurricane watches or warnings for the mainland Forecasters relied on the European model for guidance

4 Weather Predic(on: Forecast Process Opera(onal weather predic(on models at NWS are required to run in about 1 percent of real-(me A one hour forecast produced in 8.5 minutes Data assimila(on, post processing are similarly constrained HPC Data NWP Assimila<on Forecaster PostProcessing Accelerators can speed up Assimila(on and Numerical Weather Predic(on (NWP) Stakeholders

5 Why Does NWP Need Accelerators? Increasing computer power has provided linear forecast improvement for decades CPU clock speeds have stalled Increased number of processing cores: MIC, GPU Lower energy requirements IBM IBM NCEP Operational Forecast Skill 36 and 72 Hour 500 MB over North America [100 * (1-S1/70) Method] IBM IBM CDC Hour Forecast 72 Hour Forecast IBM 360/ Years CYBER 205 CRAY Y-MP CRAY C90 IBM SP IBM P690 IBM P655+ IBM Power NCEP Central Operations January 2015

6 Resolu(on Ma?ers: Large Scale Ocean-Land-Atmosphere Interac(ons Global opera(onal weather models: 13KM

7 Resolu(on Ma?ers: Fine-Scale SimulaCon of a Tornado-Producing Super-Cell Storm Produces a Tornado 4-km More Intense UpdraGs 1-km Simula(ons with GFDL s variable-resolu(on FV 3, non-hydrosta(c (aka cloud-permijng) model. Courtesy of Lin and Harris (2015 manuscript)

8 Be?er Data Assimila(on = Be?er Forecasts Hurricane Joaquin 00Z October 1, N 00Z October 1, N Hurricane Joaquin Track Forecast 40 N US model w/old data assimila(on US model w/new data assimila(on Actual track (through 03Z 07 October) 35 N European model 30 N 25 N 80 W 70 W 60 W Source: Corey Guas(ni EMC s Model EvaluaCon Group

9 Formula to Radically Improve U.S. Weather Predic(on (and be #1) Increase resolu(on of global models to 3KM or finer Capture moisture, storm scale features Coupling atmosphere, ocean, chemistry, land surface Improve data assimila(on Use ensemble and (me-based varia(onal methods Massive increase in number of observa(ons handled Increase scalability to thousands of cores Increase in compu(ng (mes more than current models use

10 Non-hydrosta(c Icosahedral Model (NIM) Experimental global weather forecast model began in 2008 Uniform Icosahedral grid Designed for GPU, MIC Run on 10K GPUs, 600 MIC, 250K CPU cores Tested at 3KM resolu(on Single source code (Fortran) Serial, parallel execu(on on CPU, GPU, MIC Paralleliza(on direc(ves GPU OpenACC, F2C-ACC CPU OpenMP MIC OpenMP MPI SMS Useful for evalua(ng compilers, GPU & MIC hardware Fine-Grained Parallelism GPU Blocks in horizontal Threads in ver(cal CPU, MIC Threading in horizontal Vectoriza(on of ver(cal

11 Hardware Comparisons Performance comparisons in literature, presenta(ons can be misleading Ideally want: Same source code Op(mized for all architectures Standard, high volume chips Comparisons in terms of: Device Single node Mul(-node Cost benefit Programmability

12 Device Performance run(me (sec) NIM DYNAMICS 110 KM RESOLUTION 96 VERTICAL LEVELS Intel CPU NVIDIA GPU Intel MIC / Year Intel CPU (cores) NVIDIA GPU (cores) Intel MIC (cores) 2010/11 Westmere (12) Fermi (448) 2012 SandyBridge (16) Kepler K20x (2688) 2013 IvyBridge (20) Kepler K40 (2880) Knights Corner (61) 2014 Haswell (24) Kepler K80 (4992)

13 Single Node Performance Results from: NOAA / ESRL - August 2014 Run-<me (sec) Numeric values represent node run-(mes for each configura(on KM Resolu<on 40,968 Columns, 96 Ver<cal Levels 100 <me steps 58 Symmetric Mode Execu<on CPU run(me MIC run(me GPU run(me using F2C-ACC Node Type: 0 IB20 only IB24 only MIC only GPU only IB24 + MIC IB20 + GPU IB GPU IB20: Intel IvyBridge, 20 cores, 3.0GHz IB24: Intel IvyBridge 24 cores, 2.70 GHz GPU: Kepler K cores, 745 MHz MIC: KNC cores, 1.23GHZ

14 Single Node Performance - Strong Scaling - Intel IvyBridge with up to 4 NVIDIA K80s As the work per GPU decreases: inter-gpu communica(ons increases slightly efficiency decreases At least 10,000 columns per GPU is best 50 NIM Single Node Performance 40,962 Columns, 100 <mesteps Run<me (seconds) GPUs 0.95 Run(me 0.90 Communica(ons Parallel Efficiency Cols/GPU

15 Dynamics only CPU GPU Cost-Benefit Different CPUs and GPU configura(ons 40 Haswell CPUs, 20 K80 GPUs incorporate off-node MPI communica(ons All runs executed in the same (me Meets a ~1% opera(onal (me constraint for a 3KM resolu(on model 20K columns / GPU used which equates to 95% GPU strong scaling efficiency

16 Cost-Benefit NIM Dynamics 30KM resolu(on runs in same execu(on (me with: - 40 Intel Haswell CPU Nodes (list price: $6,500) - 20 NVIDIA K80 GPUs (list price: $5,000) Execu(on (me represents ~1.5% of real-(me for 3KM resolu(on ~2.75% of real-(me when model physics is included CPU versus GPU Cost-Benefit NIM 30 km resolu(on 230 Cost (thousands) CPU only CPU & GPU numcpus: K80s per CPU:

17 Lessons Learned: Code Design Avoid language constructs that are less well supported or difficult for compilers to op(mize Pointers, derived types Separate rou(nes for fine-grain (GPU, MIC) and coarse grain (MIC) Avoid single loop kernels High cost of kernel startup, synchroniza(on Avoid large kernels (GPU) Limited fast register, cache / shared memory Use scien(fic formula(ons that are highly parallel

18 Lessons Learned: Inter-Process Communica(ons Use of icosahedral grid gave flexibility in how columns could be distributed among MPI ranks MPI regions should be square to minimize points to be communicated Spiral ordering to eliminate MPI message packing and unpacking helped CPU, GPU, MIC GPUDirect gave 30% performance improvement CUDA Mul(-Process Service (MPS) sped up NIM by 35% on Titan Not reflected in the results shown

19 Lessons Learned: Fine-Grain Choice of innermost dimension important Vectoriza(on on CPU, MIC SIMD, Coalesced memory on GPU For NIM, ver(cal dimension used for dynamics Horizontal dimension for physics Innermost dimension should be mul(ple of 32 for GPU, bigger is be?er Mul(ple of 8 is sufficient for MIC Minimize branching Very few special cases in NIM

20 Improved OpenACC Compilers Performance of PGI nearly matches F2C-ACC Was 2.1X slower in 2014 Cray was 1.7X slower PGI does good job with analysis, data movement Use!$acc kernels to get the applica(on running 800 line MPAS kernel running on GPU in 10 minutes Use!$acc parallel to op(mize performance Use!$acc data to handle data movement Diagnos(c output to guide paralleliza(on, op(miza(on Cray, IBM comparisons planned

21 Summary Goal to radically improve U.S. Weather Predic(on (and be #1) Develop and run NGGPS model at 3KM MPAS or FV3 selec(on in May 2016 Lessons learned with NIM will guide paralleliza(on Significant improvement in data assimila(on Algorithms, techniques must be scalable to tens of thousands of compute cores Fine-grain compu(ng OpenACC, OpenMP compilers

Advancing Weather Prediction at NOAA. 18 November 2015 Tom Henderson NOAA / ESRL / GSD

Advancing Weather Prediction at NOAA. 18 November 2015 Tom Henderson NOAA / ESRL / GSD Advancing Weather Prediction at NOAA 18 November 2015 Tom Henderson NOAA / ESRL / GSD The U. S. Needs Better Global Numerical Weather Prediction Hurricane Sandy October 28, 2012 A European forecast that

More information

NOAA Research and Development High Performance Compu3ng Office Craig Tierney, U. of Colorado at Boulder Leslie Hart, NOAA CIO Office

NOAA Research and Development High Performance Compu3ng Office Craig Tierney, U. of Colorado at Boulder Leslie Hart, NOAA CIO Office A survey of performance characteris3cs of NOAA s weather and climate codes across our HPC systems NOAA Research and Development High Performance Compu3ng Office Craig Tierney, U. of Colorado at Boulder

More information

Recent advances in the GFDL Flexible Modeling System

Recent advances in the GFDL Flexible Modeling System Recent advances in the GFDL Flexible Modeling System 4th ENES HPC Workshop Toulouse, FRANCE V. Balaji and many others NOAA/GFDL and Princeton University 6 April 2016 V. Balaji (balaji@princeton.edu) GFDL

More information

Exascale challenges for Numerical Weather Prediction : the ESCAPE project

Exascale challenges for Numerical Weather Prediction : the ESCAPE project Exascale challenges for Numerical Weather Prediction : the ESCAPE project O Olivier Marsden This project has received funding from the European Union s Horizon 2020 research and innovation programme under

More information

Julian Merten. GPU Computing and Alternative Architecture

Julian Merten. GPU Computing and Alternative Architecture Future Directions of Cosmological Simulations / Edinburgh 1 / 16 Julian Merten GPU Computing and Alternative Architecture Institut für Theoretische Astrophysik Zentrum für Astronomie Universität Heidelberg

More information

HYCOM and Navy ESPC Future High Performance Computing Needs. Alan J. Wallcraft. COAPS Short Seminar November 6, 2017

HYCOM and Navy ESPC Future High Performance Computing Needs. Alan J. Wallcraft. COAPS Short Seminar November 6, 2017 HYCOM and Navy ESPC Future High Performance Computing Needs Alan J. Wallcraft COAPS Short Seminar November 6, 2017 Forecasting Architectural Trends 3 NAVY OPERATIONAL GLOBAL OCEAN PREDICTION Trend is higher

More information

Scalability Programme at ECMWF

Scalability Programme at ECMWF Scalability Programme at ECMWF Picture: Stan Tomov, ICL, University of Tennessee, Knoxville Peter Bauer, Mike Hawkins, George Mozdzynski, Tiago Quintino, Deborah Salmond, Stephan Siemen, Yannick Trémolet

More information

Scalability Ini,a,ve at ECMWF

Scalability Ini,a,ve at ECMWF Scalability Ini,a,ve at ECMWF Picture: Stan Tomov, ICL, University of Tennessee, Knoxville Peter Bauer, Mike Hawkins, George Mozdzynski, Deborah Salmond, Stephan Siemen, Peter Towers, Yannick Trémolet,

More information

Improvement of MPAS on the Integration Speed and the Accuracy

Improvement of MPAS on the Integration Speed and the Accuracy ICAS2017 Annecy, France Improvement of MPAS on the Integration Speed and the Accuracy Wonsu Kim, Ji-Sun Kang, Jae Youp Kim, and Minsu Joh Disaster Management HPC Technology Research Center, Korea Institute

More information

Future Improvements of Weather and Climate Prediction

Future Improvements of Weather and Climate Prediction Future Improvements of Weather and Climate Prediction Unidata Policy Committee October 21, 2010 Alexander E. MacDonald, Ph.D. Deputy Assistant Administrator for Labs and Cooperative Institutes & Director,

More information

The Panel: What does the future look like for NPW application development? 17 th ECMWF Workshop on High Performance Computing in Meteorology

The Panel: What does the future look like for NPW application development? 17 th ECMWF Workshop on High Performance Computing in Meteorology The Panel: What does the future look like for NPW application development? 17 th ECMWF Workshop on High Performance Computing in Meteorology 16:00-17:30 27 October 2016 Panelists John Michalakes (UCAR,

More information

WRF- Hydro Development and Performance Tes9ng

WRF- Hydro Development and Performance Tes9ng WRF- Hydro Development and Performance Tes9ng Wei Yu, David Gochis, David Yates Research Applica9ons Laboratory Na9onal Center for Atmospheric Research Boulder, CO USA Scien9fic Mo9va9on How does terrain

More information

ACCELERATING WEATHER PREDICTION WITH NVIDIA GPUS

ACCELERATING WEATHER PREDICTION WITH NVIDIA GPUS ACCELERATING WEATHER PREDICTION WITH NVIDIA GPUS Alan Gray, Developer Technology Engineer, NVIDIA ECMWF 18th Workshop on high performance computing in meteorology, 28 th September 2018 ESCAPE NVIDIA s

More information

Moving to a simpler NCEP production suite

Moving to a simpler NCEP production suite Moving to a simpler NCEP production suite Unified coupled global modeling Hendrik L. Tolman Director, Environmental Modeling Center NOAA / NWS / NCEP Hendrik.Tolman@NOAA.gov page 1 of 14 Content The suite

More information

Scaling the Software and Advancing the Science of Global Modeling and Assimilation Systems at NASA. Bill Putman

Scaling the Software and Advancing the Science of Global Modeling and Assimilation Systems at NASA. Bill Putman Global Modeling and Assimilation Office Scaling the Software and Advancing the Science of Global Modeling and Assimilation Systems at NASA Bill Putman Max Suarez, Lawrence Takacs, Atanas Trayanov and Hamid

More information

Improving weather prediction via advancing model initialization

Improving weather prediction via advancing model initialization Improving weather prediction via advancing model initialization Brian Etherton, with Christopher W. Harrop, Lidia Trailovic, and Mark W. Govett NOAA/ESRL/GSD 15 November 2016 The HPC group at NOAA/ESRL/GSD

More information

Weather and Climate Modeling on GPU and Xeon Phi Accelerated Systems

Weather and Climate Modeling on GPU and Xeon Phi Accelerated Systems Weather and Climate Modeling on GPU and Xeon Phi Accelerated Systems Mike Ashworth, Rupert Ford, Graham Riley, Stephen Pickles Scientific Computing Department & STFC Hartree Centre STFC Daresbury Laboratory

More information

Piz Daint & Piz Kesch : from general purpose supercomputing to an appliance for weather forecasting. Thomas C. Schulthess

Piz Daint & Piz Kesch : from general purpose supercomputing to an appliance for weather forecasting. Thomas C. Schulthess Piz Daint & Piz Kesch : from general purpose supercomputing to an appliance for weather forecasting Thomas C. Schulthess 1 Cray XC30 with 5272 hybrid, GPU accelerated compute nodes Piz Daint Compute node:

More information

Parallelizing Gaussian Process Calcula1ons in R

Parallelizing Gaussian Process Calcula1ons in R Parallelizing Gaussian Process Calcula1ons in R Christopher Paciorek UC Berkeley Sta1s1cs Joint work with: Benjamin Lipshitz Wei Zhuo Prabhat Cari Kaufman Rollin Thomas UC Berkeley EECS (formerly) IBM

More information

A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters

A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters ANTONINO TUMEO, ORESTE VILLA Collaborators: Karol Kowalski, Sriram Krishnamoorthy, Wenjing Ma, Simone Secchi May 15, 2012 1 Outline!

More information

A model leading to self-consistent iteration computation with need for HP LA (e.g, diagonalization and orthogonalization)

A model leading to self-consistent iteration computation with need for HP LA (e.g, diagonalization and orthogonalization) A model leading to self-consistent iteration computation with need for HP LA (e.g, diagonalization and orthogonalization) Schodinger equation: Hψ = Eψ Choose a basis set of wave functions Two cases: Orthonormal

More information

Cosmological N-Body Simulations and Galaxy Surveys

Cosmological N-Body Simulations and Galaxy Surveys Cosmological N-Body Simulations and Galaxy Surveys Adrian Pope, High Energy Physics, Argonne Na3onal Laboratory, apope@anl.gov CScADS: Scien3fic Data and Analy3cs for Extreme- scale Compu3ng, 30 July 2012

More information

GPU Acceleration of Weather Forecasting and Meteorological Satellite Data Assimilation, Processing and Applications http://www.tempoquest.com Allen Huang, Ph.D. allen@tempoquest.com CTO, Tempo Quest Inc.

More information

NOAA Supercomputing Directions and Challenges. Frank Indiviglio GFDL MRC Workshop June 1, 2017

NOAA Supercomputing Directions and Challenges. Frank Indiviglio GFDL MRC Workshop June 1, 2017 NOAA Supercomputing Directions and Challenges Frank Indiviglio GFDL frank.indiviglio@noaa.gov MRC Workshop June 1, 2017 2 NOAA Is Vital to American Economy A quarter of the GDP ($4 trillion) is reliant

More information

Nonhydrostatic Icosahedral Model (NIM) A 3-D finite-volume NIM Jin Lee (+ other contributors)

Nonhydrostatic Icosahedral Model (NIM) A 3-D finite-volume NIM Jin Lee (+ other contributors) Nonhydrostatic Icosahedral Model (NIM) A 3-D finite-volume NIM Jin Lee (+ other contributors) Earth System Research Laboratory (ESRL) NOAA/OAR GFDL,NSSL,ARL,AOML,GLERL,PMEL Aeronomy Lab. Climate Diagnostic

More information

First, a look at using OpenACC on WRF subroutine advance_w dynamics routine

First, a look at using OpenACC on WRF subroutine advance_w dynamics routine First, a look at using OpenACC on WRF subroutine advance_w dynamics routine Second, an estimate of WRF multi-node performance on Cray XK6 with GPU accelerators Based on performance of WRF kernels, what

More information

Welcome to MCS 572. content and organization expectations of the course. definition and classification

Welcome to MCS 572. content and organization expectations of the course. definition and classification Welcome to MCS 572 1 About the Course content and organization expectations of the course 2 Supercomputing definition and classification 3 Measuring Performance speedup and efficiency Amdahl s Law Gustafson

More information

Weather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012

Weather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012 Weather Research and Forecasting (WRF) Performance Benchmark and Profiling July 2012 Note The following research was performed under the HPC Advisory Council activities Participating vendors: Intel, Dell,

More information

ECMWF Computing & Forecasting System

ECMWF Computing & Forecasting System ECMWF Computing & Forecasting System icas 2015, Annecy, Sept 2015 Isabella Weger, Deputy Director of Computing ECMWF September 17, 2015 October 29, 2014 ATMOSPHERE MONITORING SERVICE CLIMATE CHANGE SERVICE

More information

Performance and Application of Observation Sensitivity to Global Forecasts on the KMA Cray XE6

Performance and Application of Observation Sensitivity to Global Forecasts on the KMA Cray XE6 Performance and Application of Observation Sensitivity to Global Forecasts on the KMA Cray XE6 Sangwon Joo, Yoonjae Kim, Hyuncheol Shin, Eunhee Lee, Eunjung Kim (Korea Meteorological Administration) Tae-Hun

More information

Mul$- model ensemble challenge ini$al/model uncertain$es

Mul$- model ensemble challenge ini$al/model uncertain$es Mul$- model ensemble challenge ini$al/model uncertain$es Yuejian Zhu Ensemble team leader Environmental Modeling Center NCEP/NWS/NOAA Acknowledgments: EMC ensemble team staffs Presenta$on for WMO/WWRP

More information

Introduction The Nature of High-Performance Computation

Introduction The Nature of High-Performance Computation 1 Introduction The Nature of High-Performance Computation The need for speed. Since the beginning of the era of the modern digital computer in the early 1940s, computing power has increased at an exponential

More information

1.2 DEVELOPMENT OF THE NWS PROBABILISTIC EXTRA-TROPICAL STORM SURGE MODEL AND POST PROCESSING METHODOLOGY

1.2 DEVELOPMENT OF THE NWS PROBABILISTIC EXTRA-TROPICAL STORM SURGE MODEL AND POST PROCESSING METHODOLOGY 1.2 DEVELOPMENT OF THE NWS PROBABILISTIC EXTRA-TROPICAL STORM SURGE MODEL AND POST PROCESSING METHODOLOGY Huiqing Liu 1 and Arthur Taylor 2* 1. Ace Info Solutions, Reston, VA 2. NOAA / NWS / Science and

More information

Real-time signal detection for pulsars and radio transients using GPUs

Real-time signal detection for pulsars and radio transients using GPUs Real-time signal detection for pulsars and radio transients using GPUs W. Armour, M. Giles, A. Karastergiou and C. Williams. University of Oxford. 15 th July 2013 1 Background of GPUs Why use GPUs? Influence

More information

Heterogeneous programming for hybrid CPU-GPU systems: Lessons learned from computational chemistry

Heterogeneous programming for hybrid CPU-GPU systems: Lessons learned from computational chemistry Heterogeneous programming for hybrid CPU-GPU systems: Lessons learned from computational chemistry and Eugene DePrince Argonne National Laboratory (LCF and CNM) (Eugene moved to Georgia Tech last week)

More information

Next Genera*on Compu*ng: Needs and Opportuni*es for Weather, Climate, and Atmospheric Sciences. David Randall

Next Genera*on Compu*ng: Needs and Opportuni*es for Weather, Climate, and Atmospheric Sciences. David Randall Next Genera*on Compu*ng: Needs and Opportuni*es for Weather, Climate, and Atmospheric Sciences David Randall Way back I first modified, ran, and analyzed results from an atmospheric GCM in 1972. The model

More information

Scalable and Power-Efficient Data Mining Kernels

Scalable and Power-Efficient Data Mining Kernels Scalable and Power-Efficient Data Mining Kernels Alok Choudhary, John G. Searle Professor Dept. of Electrical Engineering and Computer Science and Professor, Kellogg School of Management Director of the

More information

Operational and research activities at ECMWF now and in the future

Operational and research activities at ECMWF now and in the future Operational and research activities at ECMWF now and in the future Sarah Keeley Education Officer Erland Källén Director of Research ECMWF An independent intergovernmental organisation established in 1975

More information

Section 3. Computational studies including new techniques, the effect of varying model resolution, parallel processing

Section 3. Computational studies including new techniques, the effect of varying model resolution, parallel processing Section 3 Computational studies including new techniques, the effect of varying model resolution, parallel processing Eta vs sigma: Precipitation scores, monotonic and unconditionally stable horizontal

More information

arxiv: v1 [hep-lat] 7 Oct 2010

arxiv: v1 [hep-lat] 7 Oct 2010 arxiv:.486v [hep-lat] 7 Oct 2 Nuno Cardoso CFTP, Instituto Superior Técnico E-mail: nunocardoso@cftp.ist.utl.pt Pedro Bicudo CFTP, Instituto Superior Técnico E-mail: bicudo@ist.utl.pt We discuss the CUDA

More information

Parallelization of Molecular Dynamics (with focus on Gromacs) SeSE 2014 p.1/29

Parallelization of Molecular Dynamics (with focus on Gromacs) SeSE 2014 p.1/29 Parallelization of Molecular Dynamics (with focus on Gromacs) SeSE 2014 p.1/29 Outline A few words on MD applications and the GROMACS package The main work in an MD simulation Parallelization Stream computing

More information

Perm State University Research-Education Center Parallel and Distributed Computing

Perm State University Research-Education Center Parallel and Distributed Computing Perm State University Research-Education Center Parallel and Distributed Computing A 25-minute Talk (S4493) at the GPU Technology Conference (GTC) 2014 MARCH 24-27, 2014 SAN JOSE, CA GPU-accelerated modeling

More information

Targeting Extreme Scale Computational Challenges with Heterogeneous Systems

Targeting Extreme Scale Computational Challenges with Heterogeneous Systems Targeting Extreme Scale Computational Challenges with Heterogeneous Systems Oreste Villa, Antonino Tumeo Pacific Northwest Na/onal Laboratory (PNNL) 1 Introduction! PNNL Laboratory Directed Research &

More information

Reflecting on the Goal and Baseline of Exascale Computing

Reflecting on the Goal and Baseline of Exascale Computing Reflecting on the Goal and Baseline of Exascale Computing Thomas C. Schulthess!1 Tracking supercomputer performance over time? Linpack benchmark solves: Ax = b!2 Tracking supercomputer performance over

More information

Deterministic vs. Ensemble Forecasts: The Case from Sandy

Deterministic vs. Ensemble Forecasts: The Case from Sandy Deterministic vs. Ensemble Forecasts: The Case from Sandy Robert Gall, David McCarren and Fred Toepfer Hurricane Forecast Improvement Program (HFIP); National Weather Service (NWS); National Oceanic and

More information

11 Parallel programming models

11 Parallel programming models 237 // Program Design 10.3 Assessing parallel programs 11 Parallel programming models Many different models for expressing parallelism in programming languages Actor model Erlang Scala Coordination languages

More information

Exascale I/O challenges for Numerical Weather Prediction

Exascale I/O challenges for Numerical Weather Prediction Exascale I/O challenges for Numerical Weather Prediction A view from ECMWF Tiago Quintino, B. Raoult, S. Smart, A. Bonanni, F. Rathgeber, P. Bauer, N. Wedi ECMWF tiago.quintino@ecmwf.int SuperComputing

More information

From Piz Daint to Piz Kesch : the making of a GPU-based weather forecasting system. Oliver Fuhrer and Thomas C. Schulthess

From Piz Daint to Piz Kesch : the making of a GPU-based weather forecasting system. Oliver Fuhrer and Thomas C. Schulthess From Piz Daint to Piz Kesch : the making of a GPU-based weather forecasting system Oliver Fuhrer and Thomas C. Schulthess 1 Piz Daint Cray XC30 with 5272 hybrid, GPU accelerated compute nodes Compute node:

More information

ECMWF Scalability Programme

ECMWF Scalability Programme ECMWF Scalability Programme Picture: Stan Tomov, ICL, University of Tennessee, Knoxville Peter Bauer, Mike Hawkins, Deborah Salmond, Stephan Siemen, Yannick Trémolet, and Nils Wedi Next generation science

More information

HRRR-AK: Status and Future of a High- Resolu8on Forecast Model for Alaska

HRRR-AK: Status and Future of a High- Resolu8on Forecast Model for Alaska HRRR-AK: Status and Future of a High- Resolu8on Forecast Model for Alaska Trevor Alco* 1, Jiang Zhu 2, Don Morton 3, Ming Hu 4, Cur8s Alexander 1 1 ESRL Global Systems Division, Boulder, CO 2 GINA/UAF,

More information

Research on GPU-accelerated algorithm in 3D finite difference neutron diffusion calculation method

Research on GPU-accelerated algorithm in 3D finite difference neutron diffusion calculation method NUCLEAR SCIENCE AND TECHNIQUES 25, 0501 (14) Research on GPU-accelerated algorithm in 3D finite difference neutron diffusion calculation method XU Qi ( 徐琪 ), 1, YU Gang-Lin ( 余纲林 ), 1 WANG Kan ( 王侃 ),

More information

Observa(on- Driven Studies Using GEOS- 5 Earth System Modeling and Analysis: Some Examples

Observa(on- Driven Studies Using GEOS- 5 Earth System Modeling and Analysis: Some Examples Observa(on- Driven Studies Using GEOS- 5 Earth System Modeling and Analysis: Some Examples Steven Pawson Global Modeling and Assimila(on Office Earth Sciences Division, NASA GSFC Transla(ng Process Understanding

More information

NOAA s Hurricane Forecast Improvement Project: Framework for Addressing the Weather Research Forecasting Innovation Act of 2017

NOAA s Hurricane Forecast Improvement Project: Framework for Addressing the Weather Research Forecasting Innovation Act of 2017 NOAA s Hurricane Forecast Improvement Project: Framework for Addressing the Weather Research Forecasting Innovation Act of 2017 Frank Marks (NOAA/AOML/HRD) November 7, 2018 NOAA Hurricane Forecast Improvement

More information

HFIP- Supported Improvements to Storm Surge Forecas6ng in 2012

HFIP- Supported Improvements to Storm Surge Forecas6ng in 2012 HFIP- Supported Improvements to Storm Surge Forecas6ng in 2012 Jesse C. Feyen (NOS/OCS), Jamie Rhome (NWS/NHC), Rick LueJch (UNC- CH), Jason Fleming (Seahorse Consul6ng), Brian Blanton (RENCI), Yuji Funakoshi

More information

Efficient Molecular Dynamics on Heterogeneous Architectures in GROMACS

Efficient Molecular Dynamics on Heterogeneous Architectures in GROMACS Efficient Molecular Dynamics on Heterogeneous Architectures in GROMACS Berk Hess, Szilárd Páll KTH Royal Institute of Technology GTC 2012 GROMACS: fast, scalable, free Classical molecular dynamics package

More information

An Overview of HPC at the Met Office

An Overview of HPC at the Met Office An Overview of HPC at the Met Office Paul Selwood Crown copyright 2006 Page 1 Introduction The Met Office National Weather Service for the UK Climate Prediction (Hadley Centre) Operational and Research

More information

HMON (HNMMB): Development of a new Hurricane model for NWS/NCEP operations

HMON (HNMMB): Development of a new Hurricane model for NWS/NCEP operations 1 HMON (HNMMB): Development of a new Hurricane model for NWS/NCEP operations Avichal Mehra, EMC Hurricane and Mesoscale Teams Environmental Modeling Center NOAA / NWS / NCEP HMON: A New Operational Hurricane

More information

WRF Modeling System Overview

WRF Modeling System Overview WRF Modeling System Overview Jimy Dudhia What is WRF? WRF: Weather Research and Forecasting Model Used for both research and operational forecasting It is a supported community model, i.e. a free and shared

More information

Deutscher Wetterdienst

Deutscher Wetterdienst Deutscher Wetterdienst The Enhanced DWD-RAPS Suite Testing Computers, Compilers and More? Ulrich Schättler, Florian Prill, Harald Anlauf Deutscher Wetterdienst Research and Development Deutscher Wetterdienst

More information

NOAA s FV3 based Unified Modeling System Development Strategies

NOAA s FV3 based Unified Modeling System Development Strategies NOAA s FV3 based Unified Modeling System Development Strategies HFIP Annual Meeting, 8-9 Nov. 2017 HFIP Annual Meeting, Miami, FL; 8-9 Nov. 2017 1 NOAA s Modeling capabilities (Hurricane related) Global

More information

A Massively Parallel Eigenvalue Solver for Small Matrices on Multicore and Manycore Architectures

A Massively Parallel Eigenvalue Solver for Small Matrices on Multicore and Manycore Architectures A Massively Parallel Eigenvalue Solver for Small Matrices on Multicore and Manycore Architectures Manfred Liebmann Technische Universität München Chair of Optimal Control Center for Mathematical Sciences,

More information

Climate Program Office Research Transition Activities

Climate Program Office Research Transition Activities Climate Program Office Research Transition Activities Dan Barrie, Annarita Mariotti, Jin Huang, Monika Kopacz, Sandy Lucas, Ken Mooney Building a Weather-Ready Nation by Transitioning Academic Research

More information

Weather Permitting/Meteorology. North Carolina Science Olympiad Coaches Clinic October 6, 2018 Michelle Hafey

Weather Permitting/Meteorology. North Carolina Science Olympiad Coaches Clinic October 6, 2018 Michelle Hafey Weather Permitting/Meteorology North Carolina Science Olympiad Coaches Clinic October 6, 2018 Michelle Hafey hafeym@uncw.edu Read the Rules Weather Permitting Division A Team Of Up To: 2 Bring writing

More information

An Investigation of Reforecasting Applications for NGGPS Aviation Weather Prediction: An Initial Study of Ceiling and Visibility Prediction

An Investigation of Reforecasting Applications for NGGPS Aviation Weather Prediction: An Initial Study of Ceiling and Visibility Prediction An Investigation of Reforecasting Applications for NGGPS Aviation Weather Prediction: An Initial Study of Ceiling and Visibility Prediction Kathryn L. Verlinden, Oregon State University David Bright, WFO

More information

Dense Arithmetic over Finite Fields with CUMODP

Dense Arithmetic over Finite Fields with CUMODP Dense Arithmetic over Finite Fields with CUMODP Sardar Anisul Haque 1 Xin Li 2 Farnam Mansouri 1 Marc Moreno Maza 1 Wei Pan 3 Ning Xie 1 1 University of Western Ontario, Canada 2 Universidad Carlos III,

More information

Funding Realities at NOAA Climate Program. NOAA Climate Goal

Funding Realities at NOAA Climate Program. NOAA Climate Goal Funding Realities at NOAA Climate Program Jin Huang NOAA Climate Program Office June 2, 2009 NOAA Climate Goal Understand Climate Variability and Change to Enhance Society s Ability to Plan and Respond

More information

NATS 101 Section 13: Lecture 25. Weather Forecasting Part II

NATS 101 Section 13: Lecture 25. Weather Forecasting Part II NATS 101 Section 13: Lecture 25 Weather Forecasting Part II NWP s First Baby Steps: Mid-Twentieth Century It wasn t until the development of computers in the 1940s and 1950s that NWP could be even attempted.

More information

S8241 VERSIONING GPU- ACCLERATED WRF TO Jeff Adie, 26 March, 2018 (Presented by Stan Posey, NVIDIA)

S8241 VERSIONING GPU- ACCLERATED WRF TO Jeff Adie, 26 March, 2018 (Presented by Stan Posey, NVIDIA) S8241 VERSIONING GPU- ACCLERATED WRF TO 3.7.1 Jeff Adie, 26 March, 2018 (Presented by Stan Posey, NVIDIA) 1 ACKNOWLEDGEMENT The work presented here today would not have been possible without the efforts

More information

Supercomputer Programme

Supercomputer Programme Supercomputer Programme A seven-year programme to enhance the computational and numerical prediction capabilities of the Bureau s forecast and warning services. Tim Pugh, Lesley Seebeck, Tennessee Leeuwenburg,

More information

DART Tutorial Sec'on 19: Making DART-Compliant Models

DART Tutorial Sec'on 19: Making DART-Compliant Models DART Tutorial Sec'on 19: Making DART-Compliant Models UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons

More information

Na#onal Hurricane Center Official Intensity Errors

Na#onal Hurricane Center Official Intensity Errors Na#onal Hurricane Center Official Intensity Errors Assimilate Airborne Doppler Winds with WRF-EnKF Available for 20+ years but never used in operational models due to the lack of resolution and/or the

More information

SPARSE SOLVERS POISSON EQUATION. Margreet Nool. November 9, 2015 FOR THE. CWI, Multiscale Dynamics

SPARSE SOLVERS POISSON EQUATION. Margreet Nool. November 9, 2015 FOR THE. CWI, Multiscale Dynamics SPARSE SOLVERS FOR THE POISSON EQUATION Margreet Nool CWI, Multiscale Dynamics November 9, 2015 OUTLINE OF THIS TALK 1 FISHPACK, LAPACK, PARDISO 2 SYSTEM OVERVIEW OF CARTESIUS 3 POISSON EQUATION 4 SOLVERS

More information

How to shape future met-services: a seamless perspective

How to shape future met-services: a seamless perspective How to shape future met-services: a seamless perspective Paolo Ruti, Chief World Weather Research Division Sarah Jones, Chair Scientific Steering Committee Improving the skill big resources ECMWF s forecast

More information

Optimization strategy for MASNUM surface wave model

Optimization strategy for MASNUM surface wave model Hillsboro, September 27, 2018 Optimization strategy for MASNUM surface wave model Zhenya Song *, + * First Institute of Oceanography (FIO), State Oceanic Administrative (SOA), China + Intel Parallel Computing

More information

SP-CNN: A Scalable and Programmable CNN-based Accelerator. Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay

SP-CNN: A Scalable and Programmable CNN-based Accelerator. Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay SP-CNN: A Scalable and Programmable CNN-based Accelerator Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay Motivation Power is a first-order design constraint, especially for embedded devices. Certain

More information

Accelerated Prediction of the Polar Ice and Global Ocean (APPIGO)

Accelerated Prediction of the Polar Ice and Global Ocean (APPIGO) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. Accelerated Prediction of the Polar Ice and Global Ocean (APPIGO) Eric Chassignet Center for Ocean-Atmosphere Prediction

More information

Outlook 2008 Atlantic Hurricane Season. Kevin Lipton, Ingrid Amberger National Weather Service Albany, New York

Outlook 2008 Atlantic Hurricane Season. Kevin Lipton, Ingrid Amberger National Weather Service Albany, New York Outlook 2008 Atlantic Hurricane Season Kevin Lipton, Ingrid Amberger National Weather Service Albany, New York Summary 2007 Hurricane Season Two hurricanes made landfall in the Atlantic Basin at category-5

More information

Mesoscale meteorological models. Claire L. Vincent, Caroline Draxl and Joakim R. Nielsen

Mesoscale meteorological models. Claire L. Vincent, Caroline Draxl and Joakim R. Nielsen Mesoscale meteorological models Claire L. Vincent, Caroline Draxl and Joakim R. Nielsen Outline Mesoscale and synoptic scale meteorology Meteorological models Dynamics Parametrizations and interactions

More information

Donna J. Kain, PhD and Catherine F. Smith, PhD East Carolina University

Donna J. Kain, PhD and Catherine F. Smith, PhD East Carolina University Risk Perceptions and Emergency Communication Effectiveness in Coastal Zones Preliminary Findings on Interpretations of Weather Related Messages and Maps Donna J. Kain, PhD (kaind@ecu.edu), and Catherine

More information

Acceleration of WRF on the GPU

Acceleration of WRF on the GPU Acceleration of WRF on the GPU Daniel Abdi, Sam Elliott, Iman Gohari Don Berchoff, Gene Pache, John Manobianco TempoQuest 1434 Spruce Street Boulder, CO 80302 720 726 9032 TempoQuest.com THE WORLD S FASTEST

More information

GEOG 401 Climate Change

GEOG 401 Climate Change GEOG 401 Climate Change Climate Downscaling GCMs have coarse resolu/on Spa

More information

WEATHER FORECASTING Acquisition of Weather Information WFO Regions Weather Forecasting Tools Weather Forecasting Tools Weather Forecasting Methods

WEATHER FORECASTING Acquisition of Weather Information WFO Regions Weather Forecasting Tools Weather Forecasting Tools Weather Forecasting Methods 1 2 3 4 5 6 7 8 WEATHER FORECASTING Chapter 13 Acquisition of Weather Information 10,000 land-based stations, hundreds of ships and buoys; four times a day, airports hourly Upper level: radiosonde, aircraft,

More information

The next-generation supercomputer and NWP system of the JMA

The next-generation supercomputer and NWP system of the JMA The next-generation supercomputer and NWP system of the JMA Masami NARITA m_narita@naps.kishou.go.jp Numerical Prediction Division (NPD), Japan Meteorological Agency (JMA) Purpose of supercomputer & NWP

More information

Performance Evaluation of MPI on Weather and Hydrological Models

Performance Evaluation of MPI on Weather and Hydrological Models NCAR/RAL Performance Evaluation of MPI on Weather and Hydrological Models Alessandro Fanfarillo elfanfa@ucar.edu August 8th 2018 Cheyenne - NCAR Supercomputer Cheyenne is a 5.34-petaflops, high-performance

More information

Performance evaluation of scalable optoelectronics application on large-scale Knights Landing cluster

Performance evaluation of scalable optoelectronics application on large-scale Knights Landing cluster Performance evaluation of scalable optoelectronics application on large-scale Knights Landing cluster Yuta Hirokawa Graduate School of Systems and Information Engineering, University of Tsukuba hirokawa@hpcs.cs.tsukuba.ac.jp

More information

Scalable Tools for Debugging Non-Deterministic MPI Applications

Scalable Tools for Debugging Non-Deterministic MPI Applications Scalable Tools for Debugging Non-Deterministic MPI Applications ReMPI: MPI Record-and-Replay tool Scalable Tools Workshop August 2nd, 2016 Kento Sato, Dong H. Ahn, Ignacio Laguna, Gregory L. Lee, Mar>n

More information

MAGMA. Matrix Algebra on GPU and Multicore Architectures. Mark Gates. February 2012

MAGMA. Matrix Algebra on GPU and Multicore Architectures. Mark Gates. February 2012 MAGMA Matrix Algebra on GPU and Multicore Architectures Mark Gates February 2012 1 Hardware trends Scale # cores instead of clock speed Hardware issue became software issue Multicore Hybrid 1.E+07 1e7

More information

Dynamic Scheduling within MAGMA

Dynamic Scheduling within MAGMA Dynamic Scheduling within MAGMA Emmanuel Agullo, Cedric Augonnet, Jack Dongarra, Mathieu Faverge, Julien Langou, Hatem Ltaief, Samuel Thibault and Stanimire Tomov April 5, 2012 Innovative and Computing

More information

The benefits and developments in ensemble wind forecasting

The benefits and developments in ensemble wind forecasting The benefits and developments in ensemble wind forecasting Erik Andersson Slide 1 ECMWF European Centre for Medium-Range Weather Forecasts Slide 1 ECMWF s global forecasting system High resolution forecast

More information

Canes on Canes: Keeping South Florida Prepared During the Calm Before the Storm. Matt Onderlinde and Pete Finocchio

Canes on Canes: Keeping South Florida Prepared During the Calm Before the Storm. Matt Onderlinde and Pete Finocchio Canes on Canes: Keeping South Florida Prepared During the Calm Before the Storm Matt Onderlinde and Pete Finocchio Outline The Science of Hurricanes Why and When South Floridians Must Be Weather-Ready

More information

Estimating Atmospheric Water Vapor with Groundbased. Lecture 12

Estimating Atmospheric Water Vapor with Groundbased. Lecture 12 Estimating Atmospheric Water Vapor with Groundbased GPS Lecture 12 Overview This lecture covers metrological applica4ons of GPS Some of the material has already been presented and is shown here for completeness.

More information

Leigh Orf 1 Robert Wilhelmson 2,3 Roberto Sisneros 3 Brian Jewett 2 George Bryan 4 Mark Straka 3 Paul Woodward 5

Leigh Orf 1 Robert Wilhelmson 2,3 Roberto Sisneros 3 Brian Jewett 2 George Bryan 4 Mark Straka 3 Paul Woodward 5 Simulation and Visualization of Tornadic Supercells on Blue Waters PRAC: Understanding Tornadoes and Their Parent Supercells Through Ultra-High Resolution Simulation/Analysis Leigh Orf 1 Robert Wilhelmson

More information

THE WEATHER RESEARCH AND FORECAST MODEL VERSION 2.0

THE WEATHER RESEARCH AND FORECAST MODEL VERSION 2.0 THE WEATHER RESEARCH AND FORECAST MODEL VERSION 2.0 J. MICHALAKES, J. DUDHIA, D. GILL J. KLEMP, W. SKAMAROCK, W. WANG Mesoscale and Microscale Meteorology National Center for Atmospheric Research Boulder,

More information

Potential Operational Capability for S2S Prediction

Potential Operational Capability for S2S Prediction Potential Operational Capability for S2S Prediction Yuejian Zhu Environmental Modeling Center Acknowledgements: Brian Gross and Vijay Tallapragada Staffs of EMC and ESRL Present for Metrics, Post-processing,

More information

Figure 1 - Resources trade-off. Image of Jim Kinter (COLA)

Figure 1 - Resources trade-off. Image of Jim Kinter (COLA) CLIMATE CHANGE RESEARCH AT THE EXASCALE Giovanni Aloisio *,, Italo Epicoco *,, Silvia Mocavero and Mark Taylor^ (*) University of Salento, Lecce, Italy ( ) Euro-Mediterranean Centre for Climate Change

More information

Observing System Simulation Experiments to link Research and Operation

Observing System Simulation Experiments to link Research and Operation Observing System Simulation Experiments to link Research and Operation Masutani Michiko University of Maryland Earth System Science Interdisciplinary Center Cooperative Institute for Climate & Satellite-Maryland

More information

DART Ini)al Condi)ons for a Refined Grid CAM- SE Forecast of Hurricane Katrina. Kevin Raeder (IMAGe) Colin Zarzycki (ASP)

DART Ini)al Condi)ons for a Refined Grid CAM- SE Forecast of Hurricane Katrina. Kevin Raeder (IMAGe) Colin Zarzycki (ASP) DART Ini)al Condi)ons for a Refined Grid CAM- SE Forecast of Hurricane Katrina Kevin Raeder (IMAGe) Colin Zarzycki (ASP) 1 Mo)va)on Thousands of processors on current supercomputers. - > new CAM dynamical

More information

On Portability, Performance and Scalability of a MPI OpenCL Lattice Boltzmann Code

On Portability, Performance and Scalability of a MPI OpenCL Lattice Boltzmann Code On Portability, Performance and Scalability of a MPI OpenCL Lattice Boltzmann Code E Calore, S F Schifano, R Tripiccione Enrico Calore INFN Ferrara, Italy 7 th Workshop on UnConventional High Performance

More information

Crossing the Chasm. On the Paths to Exascale: Presented by Mike Rezny, Monash University, Australia

Crossing the Chasm. On the Paths to Exascale: Presented by Mike Rezny, Monash University, Australia On the Paths to Exascale: Crossing the Chasm Presented by Mike Rezny, Monash University, Australia michael.rezny@monash.edu Crossing the Chasm meeting Reading, 24 th October 2016 Version 0.1 In collaboration

More information

Enhancing Weather Information with Probability Forecasts. An Information Statement of the American Meteorological Society

Enhancing Weather Information with Probability Forecasts. An Information Statement of the American Meteorological Society Enhancing Weather Information with Probability Forecasts An Information Statement of the American Meteorological Society (Adopted by AMS Council on 12 May 2008) Bull. Amer. Meteor. Soc., 89 Summary This

More information