Climate Modeling and IPCC Data Management. DCERC Data Curation Workshop Gary Strand

Size: px

Start display at page:

Download "Climate Modeling and IPCC Data Management. DCERC Data Curation Workshop Gary Strand"

Pierce Jefferson
5 years ago
Views:

1 Climate Modeling and IPCC Data Management DCERC Data Curation Workshop Gary Strand

2 A brief history of climate modeling s 195s 196s 197s 198s 199s Vilhelm Bjerknes Lewis Fry Richardson ENIAC NWP and climate NCAR; ocean modeling Land and sea ice modeling Early coupled models First IPCC report IPCC AR4, Nobel Prize IPCC AR5, petascale

Mid-196s 197s-198s 199s 2-28 21-212 ATMOSPHERE/ LAND SURFACE ATMOSPHERE/ LAND SURFACE/ VEGETATION ATMOSPHERE/ LAND SURFACE/ VEGETATION ATMOSPHERE ATMOSPHERE OCEAN OCEAN OCEAN OCEAN OCEAN SEA ICE SEA

3 Mid-196s 197s-198s 199s ATMOSPHERE/ LAND SURFACE ATMOSPHERE/ LAND SURFACE/ VEGETATION ATMOSPHERE/ LAND SURFACE/ VEGETATION ATMOSPHERE ATMOSPHERE OCEAN OCEAN OCEAN OCEAN OCEAN SEA ICE SEA ICE SEA ICE SEA ICE SULFATE AEROSOLS SULFATE AEROSOLS SULFATE AEROSOLS SOLAR FORCING VOLCANIC AEROSOLS SOLAR FORCING VOLCANIC AEROSOLS CARBON CYCLE CARBON/ NITROGEN CYCLE DUST/ SEA SPRAY/ MINERAL AEROSOLS DUST/ SEA SPRAY/ MINERAL AEROSOLS VEGETATION INTERACTIVE VEGETATION The Development of Climate Models Past, Present, and Future BIOGEOCHEMICAL CYCLES ICE SHEET

4 Schematically Grid Processes

5 supercomputer ˈsoōpərkəmˌpyoōtər A machine that takes one problem (computation) and turns it into more - I/O, storage, and management.

6 What is CESM? Community Earth System Model CESM belongs to an elite category of computer-based simulations known as earth system models. Such models use mathematical formulas to recreate the chemical and physical processes that drive Earth s climate. Extraordinarily sophisticated, they incorporate phenomena ranging from the effect that volcanic eruptions have on temperature patterns to the impact of shifting sea ice on sunlight in the atmosphere. What emerges from trillions of computer calculations is a picture of the world s climate in all its complexity.

7 Improved geographic resolution 198s 199s (~8 km) (~3 km) 2s 21s (~15 km) (~3 km)

8 A brief history of the IPCC & data First Assessment (FAR) 5 modeling groups, 8 models, 7 simulations Second Assessment (SAR) 8 modeling groups, 1 models, 6 IS92a simulations 21 - Third Assessment (TAR) 7 modeling groups, 8 models, 6 SRES simulations 27 - Fourth Assessment (AR4) 16 modeling groups, 24 models, 12 simulation types Fifth Assessment (AR5) 21 modeling groups, 25+ models, 45 simulation types (decadal prediction and long-term)

9 Impact of IPCC on data volume 2,, 6,, CMIP5 (IPCC AR5) cumulative files cumulative volume 1,6, 4,8, CMIP3 (IPCC AR4) , ,2, , ,4, 2-7 1,2, 2-1 3,6,

10 IPCC AR5 data format standard Quite detailed (PDF is 167 pages): Single variable per netcdf-3 file ~ File sizes must be < 4 GB (as practical) Stringent file naming convention Defined temporal, horizontal and vertical resolutions Specific fields (can be the same as model output but also derived)

11 IPCC AR5 data format standard

12 IPCC AR5 data format standard CMOR2: Climate Model Output Rewriter (v2) Each file contains a single primary output variable, with coordinate/grid variables, attributes and other metadata, with flexibility in how many time slices are stored Metadata is defined in MIP-specific tables For IPCC AR5, CMOR2 uses netcdf-3 CMOR uses udunits2 to verify (and possibly convert) to the units attribute specified by the MIP table CMOR uses uuid to produce a unique tracking number for each file

13 IPCC AR5 variable counts 1 hour 3 hour 6 hour daily month annual totals land land ice ocean biogeochemistry sea ice aerosol atmosphere totals

14 IPCC AR5 metadata standard Standard CESM output for a specific variable float TS(time, lat, lon) TS:units = "K" TS:long_name = "Surface temperature (radiative)" TS:cell_method = "time: mean"

15 IPCC AR5 metadata standard Same variable as required by IPCC AR5 float ts(time, lat, lon) ts:standard_name = "surface_temperature" ts:long_name = "Surface Temperature" ts:comment = "skin" temperature (i.e., SST for open ocean)" ts:units = "K" ts:original_name = "TS" ts:cell_methods = "time: mean (interval: 3 days)" ts:cell_measures = "area: areacella" ts:history = " T:5:32Z altered by CMOR: replaced missing value flag (-1e+32) with standard missing value (1e+2)." ts:missing_value = 1.e+2f ts:_fillvalue = 1.e+2f ts:associated_files = "baseurl: CMIP5/dataLocation gridspecfile: gridspec_atmos_fx_ccsm4_historical_rip.nc areacella: areacella_fx_ccsm4_historical_rip.nc"

16 IPCC AR5 metadata standard Standard CESM global attributes :Conventions = "CF-1." :source = "CAM" :case = "b4.2th.track1.1deg.6" :title = "UNSET" :logname = "mai" :host = "be89en.ucar.ed" :Version = "$Name$" :revision_id = "$Id$" :initial_file = "b4.185.track1.1deg.6.cam2.i nc" :topography_file = "/fis/cgd/cseg/csm/inputdata/atm/cam/ topo/usgs-gtopo3_.9x1.25_remap_c5127.nc" :nco_openmp_thread_number = 1

17 Global attributes as required by IPCC AR5 :project_id :product :frequency :modeling_realm = = = = "CMIP5" "output" "mon" "atmos" :institution = "NCAR (National Center for Atmospheric Research), Boulder, CO, USA" :institute_id = "NCAR" :model_id = "CCSM4" :source = "CCSM4 (tag: ccsm4 beta43 compset: B2TRCN)" :references = "Gent P. R., et.al. 211: The Community Climate System Model version 4. J. Climate, doi: /211JCLI483.1" :resolution = "f9_g16 (.9x1.25_gx1v6)" :title = "CCSM4 model output prepared for CMIP5 historical" :contact = "cesm_data@ucar.edu" :acknowledgements = "The CESM project is supported by the National Science Foundation and the Office of Science (BER) of the U.S. Department of Energy. NCAR is sponsored by the National Science Foundation. Computing resources were provided by the Climate Simulation Laboratory at the NCAR Computational and Information Systems Laboratory (CISL), sponsored by the National Science Foundation and other agencies."

18 Global attributes as required by IPCC AR5 :experiment = "historical" :experiment_id = "historical" :forcing = "Sl GHG Vl SS Ds SD BC MD OC Oz AA LU" :realization = 1 :initialization_method = 1 :physics_version = 1 :parent_experiment :parent_experiment_id :parent_experiment_rip :branch_time = = = = "pre-industrial control" "picontrol" "r1i1p1" 937. :tracking_id = "d33ccf77-a73c-4f55-8f2-3a734d51151" :creation_date = " T:5:32Z" :history = " T:5:32Z CMOR rewrote data to comply with CF standards and CMIP5 requirements." :Conventions = "CF-1.4" :table_id = "Table Amon (27 April 211) a5a1c518f52ae34313baaada3f862" :cmor_version = "2.7.1"

19 Additional global attributes :cesm_casename = "b4.2th.aero.1deg.8" ; :cesm_repotag = "ccsm4 beta56" ; :cesm_compset = "B185CN" ; :resolution = "f9_g16 (.9x1.25_gx1v6)" ; :forcing_note = "Additional information on the external forcings used in this experiment can be found at forcing_information" ; :processed_by = "strandwg on silver.cgd.ucar.edu at " ; :processing_code_information = "Last Changed Rev: 525 Last Changed Date: :11:55-7 Repository UUID: d2181dbe dc7f-cbd98591f93d" ;

20 IPCC AR5 metadata standard CESM filename b4.2th.track1.1deg.6.cam2.h.ts.185-1_cat_25-12.nc CESM archival location /CCSM/csm/b4.2th.track1.1deg.6/atm/proc/tseries/monthly IPCC AR5 filename ts_amon_ccsm4_historical_r1i1p1_ nc IPCC AR5 location.../cmip5/output/ncar/ccsm4/historical/mon/atmos/ts/r1i1p1

21 AR4 data volumes by group 9, IPCC AR4 by group 8, total: 35 TB 7, 6, 5, 4, 3, 2, 1, UKMO U Reading NorClim NCAR MRI MPI MIUB MIROC3 METRI IPSL INMCM3 INGV IAP GISS GFDL EC CSIRO CNRM CCCMA CAWCR BCCR

22 IPCC AR4 distribution Modeling centers (16) Gateway (1) Users (1s)

23 IPCC AR5 volumes by group 9, 8, 7, IPCC AR4 by group total: 35 TB IPCC AR5 by group total: 2,2 TB 6, 5, 4, 3, 2, 1, UKMO U Reading NorClim NCAR MRI MPI MIUB MIROC3 METRI IPSL INMCM3 INGV IAP GISS GFDL EC CSIRO CNRM CCCMA CAWCR BCCR

24 IPCC AR5 distribution Modeling centers (24) Gateways (9) Nodes (14) Users (1s)

25 Distribution: The ESG federation!

26 METAFOR Common Metadata for Climate Modelling Digital Repositories Develop a Common Information Model (CIM) to describe climate data and the models that produce it in a standard way Ensure the wide adoption of the CIM Addresses the fragmentation and gaps in availability of metadata as well as duplication of information collection and problems of identifying, accessing or using climate data that are currently found in existing repositories.

27 METAFOR Common Information Model (CIM)

28 METAFOR - CIM The CIM (Common Information Model) is a domain model of the concepts and relationships used in climate modeling: It includes descriptions not only of the data, but also of the models that generated that data, the simulations those models implemented, the experiments for which those simulations were run, the people and institutions involved, and why they bothered. It tries to describe the full provenance of climate modeling artifacts It is an emerging standard It s the core of a related set of tools and services It s the infrastructure around which IPCC AR5 metadata is based

29 Data QC

30 Data QC QC Level 1 QC Level 2 QC Level 3 Description CMOR2 and ESG publisher Data consistency checks conformance checks Double- and cross-checks of data and metadata and data publication as DataCite DOI Data preliminary; no user notification about changes; performed for all data; metadata may not be complete no user notification about changes; performed for CMIP5 requested metadata and data published and persistent data with version and unique DOI as persistent identifier; user notification about changes; performed for replicated data constrained to non-commercial research and educational purposes constrained to non-commercial research and educational purposes, or open for unrestricted use (as specified by the modeling centers) Access constrained to CMIP5 modeling centers Access Control PCMDI, BADC, WDCC/DKRZ core PCMDI on behalf of WMO/ data archives on behalf of WMO/ WGCM WGCM IPCC-DDC on behalf of TGICA Citation no citation reference informal citation reference formal citation reference "subjective quality control passed" "approved by author" (in case of newer DOI available: "approved by author, but suspended") "automated conformance Quality Flag checks passed"

31 Possible futures Assuming there is an IPCC AR6... The current model, even with federated distribution, will likely not work Model resolution will preclude access of global fields Little big iron More-exotic solutions

32 Websites CESM website CESM Data Management Plan CMIP5 website

NetCDF, NCAR s climate model data, and the IPCC. Gary Strand NCAR/NESL/CGD

NetCDF, NCAR s climate model data, and the IPCC Gary Strand NCAR/NESL/CGD NCAR s climate model data A bit of history... 1960s - 1990s Self-designed self-implemented binary formats 1990s-2000s netcdf-3