Guidelines: Numerical Data CAMS

Transcription

1 Guidelines: Numerical Data CAMS Issued by: METEO FRANCE Date: 07/09/2017

2 Summary : The aim of the user guide of Regional Air Quality data production is to give a general description of CAMS production and Data Service and to explain the users : what are the new file naming conventions implemented and what kind of data are available with the Data Service. The different types of products are described and elements about availability of the products are given. Table of contents 1. Site of access Production CAMS (portfolio) Description of the files NEW File Naming Convention used according to the WMO standard More about files : Online data Archived data Note: File format and concentration unit Access through data download service...10 This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view. Guidelines: Numerical Data CAMS 1

3 1. Site of access Direct data access througth the CAMS website Access through data download service Requires the acceptance of the data license: CAMS data license (document available on the website). 2. Production CAMS (portfolio) NRT DATA AVAILABILITY: Grib2 or Netcdf files of the ENSEMBLE and of all the 7 models CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM The files are available for 10 pollutants + 3 varieties for pollens (Birch Pollen, Olive Pollen and Grass). Pollen data are available during a specific period depending on the type of pollen. The Grib2/Netcdf files are available for 4 days of Forecast and one day of Analysis. 7 Models : CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM The ENSEMBLE is the median of the 7 models. FILES Availability of data at HH:MM UTC Forecast (0 24h) for 7 Models 05:15 Forecast (25 48h) for 7 Models 05:30 Forecast (49 72h) for 7 Models 07:15 Forecast (73 96h) for 7 Models 07:30 Forecast (0 24h) for ENSEMBLE 06:30 Forecast (25 48h) for ENSEMBLE 06:30 Forecast (49 72h) for ENSEMBLE 08:30 Forecast (73 96h) for ENSEMBLE 08:30 Analysis (-24H-1H) for 7 Models 11:45 Analysis (-24H-1H) for ENSEMBLE 12:00 Guidelines: Numerical Data CAMS 2

4 3. Description of the files Files are created as follows 1 file per Day (24H) per Pollutant per Model at Surface for Forecast and Analysis. 1 file per Day (24H) per Pollutant per Model at All Levels for Forecast. Days: Forecast: (0H-24H),(25H-48H),(49H-72H),(73H-96H) Analysis: (-24H-1H) Yesterday Pollutants: O3, CO, NO2, SO2, PM25, PM10, PANS, NMVOC, NO, NH3, BIRCH POLLEN, OLIVE POLLEN, GRASS POLLEN. Pollen data are available during a specific period depending on the type of pollen. Models : ENSEMBLE, CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM Levels: Surface (0m), 50, 250, 500, 1000, 2000, 3000, 5000 m Files are available in Grib2 format and Netcdf format for model ENSEMBLE for both Forecast (surface as well as other levels ) & Analysis (surface as well as other levels). Birch pollen forecasts, olive pollen and grass pollen are available at surface level only. Files are available only in Netcdf format for models :CHIMERE, EMEP, EURAD, LOTOS- EUROS, MATCH, MOCAGE, SILAM for both Forecast (surface as well as other levels) & Analysis (surface as well as other levels). Birch pollen forecasts, olive pollen and grass pollen are available at surface level only NEW File Naming Convention used according to the WMO standard Reference document: WMO Manual on the Global Telecommunication System, Vol. 1 (See in document: paragraph procedures and file naming convention) ( Example : For the pollutant NMVOC forecast issued by model SILAM for the reference date T00:00:00Z for the data of first 24 Hours [0H to 24H] for all the levels Surface to 5000m the new filename is : W_fr-meteofrance,MODEL,SILAM+FORECAST+ALLLEVELS+NMVOC+0H24H_C_LFPW_ nc Guidelines: Numerical Data CAMS 3

5 3.1.1 NOMENCLATURE of Filename: W _ fr Symbol WMO product identifier pflag delimiter of fields location indicator (fr= FRANCE) - delimiter within the field meteofrance production center Meaning, delimiter used in the product identifier MODEL data designator, delimiter used in the product identifier Name of model + Type of data composite Type of data + Type of level composite Type of level Possible values: ENSEMBLE, CHIMERE, EMEP, EURAD, LOTOSEUROS, MATCH, MOCAGE, SILAM Possible Values: FORECAST/ANALYSIS/FORECASTREPORT/ANALYSISREPORT Possible values: ALLLEVELS (0, 50, 250, 500, 1000, 2000, 3000, 5000 m) SURFACE (0m) + Type of Pollutant composite Type of Pollutant Possible values: O3, CO, NO2, SO2, PM25, PM10, PANS, NMVOC, NO, NH3, BIRCHPOLLEN, OLIVEPOLLEN, GRASSPOLLEN + Type of Timeline composite Type of Timeline _ C _ LFPW _ Timeline of Forecast data of 24H:0H24H,25H48H,49H72H, 73H96H Timeline of Analysis data of 24H : -24H-1H) delimiter of fields Pflag C delimiter of fields Country code delimiter of fields Reference Date & Time Fixed length Date & Time in the format YYYYMMDDHHMMSS. delimiter of format Type of File Format File extension of format: nc (for Netcdf format) / grib2 (for GRIB2 format) / txt (for reports) Guidelines: Numerical Data CAMS 4

6 3.1.2 NETCDF+ GRIB2 File Names examples : ENSEMBLE : NETCDF+ GRIB2 FORECAST 1 file per DAY [24H] per pollutant, ALLLEVELS or SURFACE ALLLEVELS : 0, 50, 250, 500, 1000, 2000, 3000, 5000 meters. POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3 TIMELINE of Forecast data:0h24h,25h48h,49h72h, 73H96H Examples: W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+ALLLEVELS+O3+0H24H_C_LFPW_ nc W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+ALLLEVELS+O3+0H24H_C_LFPW_ grib2 SURFACE: 0 m POLLUTANTS provided : O3, CO, NO2, SO2,PM25, PM10, BIRCHPOLLEN, OLIVEPOL- LEN, GRASSPOLLEN. Pollen data are available during a specific period depending on the type of pollen. TIMELINE of Forecast data:0h24h,25h48h,49h72h, 73H96H Examples: W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+SURFACE+O3+0H24H_C_LFPW_ nc W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+SURFACE+O3+0H24H_C_LFPW_ grib2 Report Of Ensemble Forecast Production: Reports in the form of text files gives the information of number of models involved in the production of Forecast of Model Ensemble Example of Filename: W_fr-meteofrance,MODEL,ENSEMBLE+FORECASTREPORT+0H24H_C_LFPW_ txt W_fr-meteofrance,MODEL,ENSEMBLE+FORECASTREPORT+49H72H_C_LFPW_ txt Guidelines: Numerical Data CAMS 5

7 ANALYSIS : 1 file per DAY [24 H] per pollutant, ALLLEVELS or SURFACE for D-1 ALLLEVELS : 0, 50, 250, 500, 1000, 2000, 3000, 5000 meters. POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3. TIMELINE of Analysis data: -24H-1H Example of Filename: W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+ALLLEVELS+O3+-24H-1H_C_LFPW_ nc W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+ALLLEVELS+O3+-24H-1H_C_LFPW_ grib2 SURFACE : 0m POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3. TIMELINE of Analysis data: -24H-1H Example of Filename: W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+SURFACE+O3+-24H- 1H_C_LFPW_ nc W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSIS+SURFACE+O3+-24H-1H_C_LFPW_ grib2 Report Of Ensemble Analysis Production: Reports in the form of text files gives the information of number of models involved in the production of Analysis of Model Ensemble Example of Filename: W_fr-meteofrance,MODEL,ENSEMBLE+ANALYSISREPORT+-24H-1H_C_LFPW_ txt Guidelines: Numerical Data CAMS 6

8 MODELS (CHIMERE, EMEP, EURAD-IM, LOTOS-EUROS, MATCH, MOCAGE, SILAM): only NETCDF FORECAST : 1 file per DAY [24 H] per pollutant per model for ALLEVELS / SURFACE ALLLEVELS : 0, 50, 250, 500, 1000, 2000, 3000, 5000 meters. POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3 TIMELINE of Forecast data:0h24h,25h48h,49h72h, 73H96H Examples: W_fr-meteofrance,MODEL,EMEP+FORECAST+SURFACE+SO2+0H24H_C_LFPW_ nc SURFACE : 0m POLLUTANTS provided : O3, CO, NO2, SO2, PM25, PM10, BIRCHPOLLEN, OLIVEPOLLEN, GRASSPOLLEN. Pollen date are available during a specific period depending on the type of pollen. TIMELINE of Forecast data:0h24h,25h48h,49h72h, 73H96H Examples: W_fr-meteofrance,MODEL,SILAM+FORECAST+SURFACE+O3+73H96H_C_LFPW_ nc ANALYSIS : 1 file per DAY [24 H] per pollutant per model, ALLLEVELS or SURFACE for DAY -1 ALLLEVELS : 0, 50, 250, 500, 1000, 2000, 3000, 5000 meters. POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3 TIMELINE of Analysis data: -24H-1H Examples: W_fr-meteofrance,MODEL,EURAD+ANALYSIS+ALLLEVELS+CO+-24H-1H_C_LFPW_ nc W_fr-meteofrance,MODEL,MATCH+ANALYSIS+ALLLEVELS+PM10+-24H-1H_C_LFPW_ nc Guidelines: Numerical Data CAMS 7

9 SURFACE : 0meters POLLUTANTS provided :O3, CO, NO2, SO2,PM25, PM10, PANS, NMVOC, NO,NH3 TIMELINE of Analysis data: -24H-1H Examples: W_fr-meteofrance,MODEL,EURAD+ANALYSIS+SURFACE+CO+-24H-1H_C_LFPW_ nc W_fr-meteofrance,MODEL,MATCH+ANALYSIS+SURFACE+PM10+-24H-1H_C_LFPW_ nc # NOTE : Netcdf files for models are not available if the model is absent that particular day. Guidelines: Numerical Data CAMS 8

10 4. More about files : 4.1. Online data Online data relate to data less than 30 days old. Forecasts and analyses are available in Grib Edition 2 or/and Netcdf Format depending on the model : Available file format for Ensemble data are Grib Edition 2 and Netcdf. Available file format for partners data are Netcdf only Archived data Archived data means data older than 30 days old. This Regional Air Quality archive is available since October 1 st, Below are a few more particularities about Regional Air Quality Archived production : Forecasts and analyses are currently available in Grib Edition2 format for archived products. Analyses issued from Ensemble model are available FOR ALL SPECIES since March Before this date, only O3 and NO2 species were produced by a sufficient number of models to generate reliable Ensemble products. So please, don't use analysis data from other pollutants than O3 and NO2 before March Note that archived analyses from partner models could be missing, occasionnaly, in case of production problems. In such a case, please use Ensemble analyses instead. In any case, in order to get long and complete series of data, please select Ensemble products. Analyses are available for levels 50, 250, 500, 1000, 2000, 3000, 5000 meters since June 22, WARNING : Should you request an all levels analysis at an earlier date, then you will get a file named : W_fr-meteofrance,MODEL,[MODEL_NAME]+ANALYSIS+ALLLEVELS+[POLLUTANT]+-24H- 1H_C_LFPW_[REQUESTED_DATE].[file_format] with the surface only. Guidelines: Numerical Data CAMS 9

11 4.3. Note: File format and concentration unit Unit used to define pollutant concentration is closely linked with the file format. Below are the list of available format and the concentration unit used for each species. Data format Unit Species GRIB Edition 2 kg/m3 CO NH3 NMVOC NO NO2 O3 PANS PM10 PM25 SO2 GRIB Edition 2 m-3 BIRCHPOLLEN OLIVEPOLLEN GRASSPOLLEN NetCDF micro g/m3 CO NH3 NMVOC NO NO2 O3 PANS PM10 PM25 SO2 NetCDF grains/m3 BIRCHPOLLEN OLIVEPOLLEN GRASSPOLLEN Guidelines: Numerical Data CAMS 10

12 5. Access through data download service The data download service is a REST API where users can request the desired products. Example of a request: &token=...&grid=0.1&model=ensemble&package=forecast_o3_surface&time=0h24h&referen cetime= t00:00:00z&format=grib2&licence=yes Below are the full-wording and the possible parameter values used for the API request (the character means one choice between many values). IMPORTANT: Downloading data services is a public service and no authentification is required. So all Regional Air Quality users will use a generic token (value: M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg ) Element of the request Base URL Token (unique value) Grid size (in degree) Model name Package (composite parameter) package type package species Authorized values &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg &grid=0.1 &model=chimere EMEP ENSEMBLE EURAD LOTOSEUROS MATCH MOCAGE SILAM &package=type_species_level FORECAST ANALYSIS ANALYSISREPORT FORECASTREPORT CO NH3 NMVOC NO NO2 O3 PANS PM10 PM25 SO2 BIRCHPOLLEN OLIVEPOLLEN GRASSPOLLEN package level Time Reference time (ISO 8601 format) Format of the file Licence agreement ALLSPECIES (for analysis and forecast reports only) SURFACE ALLLEVELS For analysis products: -24H-1H For forecast products: 0H24H 25H48H 49H72H 73H96H &referencetime=yyyy-mm-ddt00:00:00z Note: this is the requested date. Archive (in GRIB2 only) is available from Oct 1 st, &format=grib2 NETCDF TXT &licence=yes Guidelines: Numerical Data CAMS 11

13 Remark 1: The key-value licence=yes means that users requesting via API accept the CAMS licence (available on the website, see 1). Please read it before using this service. Remark 2: Not all combinations of the above parameters are meaningful. E.g, you can t request an ANALYSIS product with a FORECAST time value. The API manages some error cases by responding with an HTTP error code. Developpers using this API must know them and must catch them in their code: Error case Error code (HTTP) and optional message (XML) Incomplete request or missing parameter 400 : Parameter missing, parameter licence is mandatory Invalid token Erroneous parameter value Data not available Connection refused (may be due to number of requests limitation) 401 : Parameter token is invalid. Please specify &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3 oezjbnsg 403 : Values of parameters are wrong, Value of parameter licence MUST BE 'yes'. Please specify &licence=yes 404 : Not Found 503 : Please try again later Remark 3: (Reminder) The valid HTTP response code, when the request succeeds, is 200. Remark 4: When automating the download requests, it is important to consider the daily data availability hour as described in chapter 2. It means that it is of no interest to request the data sever too early, leading to large number of erroneous access requests... Guidelines: Numerical Data CAMS 12

14 Some examples of valid requests are therefore (ignore line returns): &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg &grid=0.1 &model=ensemble &package=forecast_o3_surface &time=0h24h &referencetime= t00:00:00z &format=grib2 &licence=yes &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg &grid=0.1 &model=ensemble &package=analysisreport_allspecies_surface &time=-24h-1h &referencetime= t00:00:00z &format=txt &licence=yes &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg &grid=0.1 &model=ensemble &package=forecastreport_allspecies_alllevels &time=0h24h &referencetime= t00:00:00z &format=txt &licence=yes &token= M0bChV6QsoOFqHz31VRqnpr4GhWPtcpaRy3oeZjBNSg &grid=0.1 &model=ensemble &package=analysis_pm10_alllevels &time=-24h-1h &referencetime= t00:00:00z &format=grib2 &licence=yes Guidelines: Numerical Data CAMS 13

15 CONTACT: For any questions please contact: Copernicus Support mail contact: Copernicus CAMS and C3S Support (Public Service Desk web site): Guidelines: Numerical Data CAMS 14