Climate Applications Experiments Simulations on the GRID

Size: px

Start display at page:

Download "Climate Applications Experiments Simulations on the GRID"

Irma Norris
5 years ago
Views:

Climate Applications Experiments Simulations on the GRID meteo@unican.es Antonio S.

1 Climate Applications Experiments Simulations on the GRID Antonio S. Cofiño Department of Applied Mathematics & Computer Sciences University of Cantabria, Spain Santander Meteorology Group International Symposioum on Grid Computing 2010 Taipei, March 2010

Latin America (Floodings in the coast areas, problems with fishing, ).

2 Climate application GOAL: Develop a simulation and analysis tool to predict local impacts of El Niño in Latin America A challenging problem with huge socio-economical impact in Latin America (Floodings in the coast areas, problems with fishing, ). El Niño events are characterized by having anomalous Sea Surface Temperature (SST) over the equatorial eastern.

SST-Precipitation patterns Sensitivity study SST-Precipitation Hundreds of simulations with different perturbed SST (1997 SST used as pattern) What would it happen if the

3 SST-Precipitation patterns Sensitivity study SST-Precipitation Hundreds of simulations with different perturbed SST (1997 SST used as pattern) What would it happen if the water was warmer/colder? Oceanic Niño Index (ONI)

CAM also serves as the atmospheric component of the Community Climate System Model (CCSM). (www.ccsm.ucar.

4 Global Circulation Model CAM: Global Atmospheric Model. The Community Atmosphere Model (CAM) is the latest in a series of global atmosphere models developed at NCAR for the weather and climate research communities. CAM also serves as the atmospheric component of the Community Climate System Model (CCSM). ( Computing and storage requirements: - Intensive use of CPU - Supports a variety of parallel programming paradigms (OpenMP, MPI, serial,...).. - Large input, output and checkpointing data. - Typical experiments last for days (even in parallel). - Application with a complex workflow (preprocessing, ( postprocessing execution &

5 Application description: CAM4G Experiment: a given scientific challenge to be solved by running an ensemble of independent CAM simulations. (E.g. Niño with 1000 perturbed SST as boundary condition.) Realization: Each of independent CAM simulations. Each realization takes more than 50 CPU/hours. Job: a realization requires several GRID jobs to complete 1 year of simulation: 50 CPUhours in a Xeon(R) CPU 2.00GHz Mhz 400 MB memory An experiment with realizations: 50K CPUhours, 1 TB 1 GB data output EXPERIMENT REALIZATIONS CAM (SST1) JOB1 (SST1) JOBS JOB2(SST1) SST PDF CAM (SST2) CAM (SSTn) JOB1(SSTn) JOB1(SST2) JOB2(SST2) JOB2(SSTn)

6 Santander Meteorology Group Downscaling Methodologies GCM Emission Scenarios Dynamical Downscaling runs regional climate models in reduced domains with boundary conditions given by the GCMs. RCM A2 Historical Records Statistical Downscaling techniques are based on empirical models fitted to data using historical records. Global Predictions B2 A2 Climatology ( ) Y = f (X;θ) The form and parameters of the model depend of the different tecniques used. A2

WRF Model ( regional ) Numerical Weather Prediction model.( Fortran90 ) Open source A world community with 6000+ registered users.

7 WRF Model ( regional ) Numerical Weather Prediction model.( Fortran90 ) Open source A world community with registered users. Application in a wide variety of studies: ( NCEP - Weather forecasting (operational at - Data assimilation studies - Regional climate studies - Idealized simulations - Educational applications,...

8 WRF Requirements Computing and storage requirements: - Intensive use of CPU - Supports a variety of parallel programming paradigms (OpenMP, MPI, serial,...). It's a common benchmarking application for HPC. - Large amount of input, output and restart data. - Typical experiments last for days (even in parallel). - Application with a complex workflow (preprocessing, ( postprocessing execution & WRF WORKFLOW ungrib geogrid metgrid namelist.wps WRF Preprocessing System real namelist.input WRF ARW WRFGEL wrf

Santander Meteorology Group Reanalisys/Reforecasting Scientific experiments WRF - High number (~10

projections for climate change - Contiguous simulations (~10), weeks of walltime each.

- QoS and optimal resources: deadline for delivering Sensitive studies for Climate and Weather -

9 Santander Meteorology Group Reanalisys/Reforecasting Scientific experiments WRF - High number (~10 4 ) of independent simulations - High volume of output-data (>TB) - Requires scalability Regional projections for climate change - Contiguous simulations (~10), weeks of walltime each. - High volume of output data (>10TB) - Recovering system for simulation restart Weather Forecasting - QoS and optimal resources: deadline for delivering Sensitive studies for Climate and Weather - Physical schemes, initial conditions and boundary conditions uncertainties sampling - Resource demanding experiments

10 GOALS Santander Meteorology Group - Make easier for the user the process of design, execution and monitoring of the experiments with WRF. - Develop a framework that allow the user to use at the same time different computing resources in a transparent way: Local clusters (PBS, SGE, LoadLeveler, ) Grid infrastructures (glite, Globus, ) Local resources (SSH, fork, ). - Develop a set of command line tools and a Web portal for the WRF users. WRF4G: Goals

Role of WRF4G in CORDEX http://wcrp.ipsl.jussieu.

11 Role of WRF4G in CORDEX /wiki/cordexwrf CORDEX (COordinated Regional climate Downscaling EXperiment) is a framework to improve coordination of international efforts in regional climate downscaling research. CORDEX was initiated as a result of the Task Force on Regional Climate Downscaling, formed by the World Climate Research Program (WCRP). A set of target regions has been proposed and modeling groups willing to contribute must comply with simulations specifications. CORDEX will produce an ensemble of simulations sampling uncertainties related to: (i) varying Global Climate Model (GCM) simulations; (ii) varying greenhouse gas (GHG) concentration scenarios; (iii) natural climate variability; and (iv) different downscaling methods. There are currently 15 groups planning to contribute to CORDEX with WRF, 5 groups planning to contribute the African domain (key region for the AR5). All of them could benefit from WRF4G app.

Santander Meteorology Group EGEE: Example of big Grid infrastructure 150.

12 Santander Meteorology Group EGEE: Example of big Grid infrastructure CPUs 70 PB 260 sites worldwide Arquitectures: i386,x86_64 LRMS: torque, sge, lsf, bqs Shared and not shared Home Computing resources: - OS: Debian, SL/Centos 4 and 5. - Memory: 250MB to 16GB - Processor: P4 to i7 Different queues limitations: walltime, memory & disk quotas Bandwidth in some sites is very small.

13 GRID Challenges GRID Challenges: - Develop an application adapted to run in different arquitectures, OS, LRMS and parallel environments. - Repository of binaries adapted to those environments - Detect sites that not accomplish application requirements: Memory, Processor, queue time limitations, - Optimizing the data transfers and replication between resources distributed geographically. - Develop a monitoring tool for experiments status. - Create a checkpointing management system that allow to restart simulations. - Failure detection and recovery.

14 WRF for GRID: WRF4G WRF4G is a port of the WRF Modeling System prepared to run in GRID environments. WRF has been splitted in 2 layers to separate the scientific experiment from the infrastructure details (wrf.input & wrf4g.conf). WRF4G supports several data transfer services (gsiftp, rsync, ) and execution systems (glite, globus, PBS, SGE, ). It can be run in the user s machine. No deployment required on sites. The application is deployed by a pilot job.

WRF4G Workflow WRF4G_DOMAINS geo_em.nc namelist.wps wrf4g.conf WRF4G wrf.input UI WRF4G_APPS wrf4g_submitter.sh WRF4G-0.0.2.tgz WRF4Gbin-3.1r83.tgz Job submission abstraction layer WRF4G_ini.

15 WRF4G Workflow WRF4G_DOMAINS geo_em.nc namelist.wps wrf4g.conf WRF4G wrf.input UI WRF4G_APPS wrf4g_submitter.sh WRF4G tgz WRF4Gbin-3.1r83.tgz Job submission abstraction layer WRF4G_ini.sh WRF4G_INPUT Gridded Data: NAM, GFS, RUC, AGRMET, etc. Input data abstraction layer ungrib WRF4G.sh WRFGEL metgrid real wrf WRF4G_BASEPATH wrfout_* wrfrst_* wrfinput_* wrfbdy_* Static Geographical Data geogrid namelist.wps namelist.input WN Processed locally WRF Preprocessing System WRF ARW

16 WRF4G structure ${WRF4G_ROOT} examples ui wn scripts bin lib WRFGEL WPS WRFV3 cccantabria wrf.input wrf4g.conf domain_partition wrf.input.in wrf4g.conf.in nino50 wrf.input wrf4g.conf operativo operativo.sh post_operatorio.sh wrf.input.in wrf4g.conf WRF4G_ini.sh wrf4g_submitter.sh wrf4g_make_tarball.sh wrf4g_make_tarball_bin.sh wrf4g_submit.eela_grid_job wrf4g_submit.mdmcluster wrf4g_submit.mdmclusterifb wrf4g_wrfgel_environment vcp fortnml ncdump preprocessor.* WRF4G.sh create_output_structure download_file exist_wps get_date_restart register_file

17 exp rea1 Create and run an experiment Three realizations split into two chunks each: exp rea2 exp rea time Storage Element - METEO4G/WRF/experiments/exp exp rea1 exp rea2 exp rea3 output restart wpsout output restart wpsout output restart wpsout wrfout_t0 wrfout_t1 wrfout_t2 wrfout_t3 wrfout_t4 wrfout_t5 wrfout_t6 wrfout_t7 wrfout_t8 wrfrst_t2 wrfrst_t4 wrfrst_t6 wrfrst_t8 wrfinput_t0 wrfbdy_t0 wrfinput_t4 wrfbdy_t4 wrfout_t0 wrfout_t1 wrfout_t2 wrfout_t3 wrfout_t4 wrfout_t5 wrfout_t6 wrfout_t7 wrfout_t8 wrfrst_t2 wrfrst_t4 wrfrst_t6 wrfrst_t8 wrfinput_t0 wrfbdy_t0 wrfinput_t4 wrfbdy_t4 wrfout_t0 wrfout_t1 wrfout_t2 wrfout_t3 wrfout_t4 wrfout_t5 wrfout_t6 wrfout_t7 wrfout_t8 wrfrst_t2 wrfrst_t4 wrfrst_t6 wrfrst_t8 wrfinput_t0 wrfbdy_t0 wrfinput_t4 wrfbdy_t4

18 management File To hide the complexity of the GRID to the application developer, several abstraction layers which encapsulate the routine tasks in generic function calls: - Data Management vcp: Provides transparent copy between any of the following protocols: gsiftp, LFC, rsync, local copies vcp -r /local/dir gridftp://server:port/remote/path vcp gsiftp://srv1:port/rmt/file gsiftp://srv2:port/other/file vcp /local/file1 rsync://server/other/local/path/ vcp gsiftp://srv1:port/rmt/file ln:link

19 CLIMATE-G: Data and metadata management Climate-G is a distributed testbed for climate change data and metadata management issues at a very large scale. The main scope of Climate-G is to allow scientists to carry out geographical and cross-institutional data discovery, access, visualization and sharing of climate data. The Climate-G testbed consist 2TB of data provided by IPSL,University of Cantabria IPCC website (AR4). Developed by Centro Euro-Mediterraneo per i Cambiamenti Climatici (CMCC, Italy) Participating institutions: Institut Pierre-Simon Laplace (IPSL), France Rensselaer Polytechnic Institute (RPI) and High Altitude Observatory (HAO) at the National Center for Atmospheric Research (NCAR),USA Fraunhofer-SCAI, Germany University of Reading, UK University of Salento, Italy University of Cantabria, Spain

20 Conclusions & Current Work The CAM4G and WRF4G applications has been ported to EELA-2, EGEE & NGI-ES infrastructures and GT2.4 & GT4. Collaborations with end-users in LA (CETA-CIIFEN- UPS, SENAMHI), Europe (CESGA-MeteoGalicia) for feedback and Asia (HAII, Thailand) Currently we are testing MPI support in the GRID. Daily operational version of WRF4G running on the GRID for ensemble of weather forecasts.

21 Santander Meteorology Group Develop a monitoring system oriented to app status. Execution management works on Globus and glite CE based Gridway plugins. PBS y SGE are used, but plugins will developed for better integration with Gridway. Develop an scalable replica management integrated with Gridway. (bottleneck in GRID infrastructures). Create a user portal that allow the users managing and monitoring their experiments. Collaborate with EGI and other International GRID Initiatives like EUAsiaGrid Incorporate users feedback Future Work

22 Thanks!!!

Grid implementation of the Weather Research and Forecasting model

Grid implementation of the Weather Research and Forecasting model -Prtenjak Centre for informatics and computing Zagreb, Croatia davor.davidovic@irb.hr Department of Geophysics Faculty of Science, University