Regional Production Quarterly report on the daily analyses and forecasts activities, and verification of the SILAM performances December 2015 January 2016 February 2016
This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view. Qr. report on daily analyses and forecasts activities, verification of the SILAM performances
Quarterly report on the daily analyses and forecasts activities, and verification of the SILAM performances December 2015 January 2016 February 2016 FMI (M. Sofiev, J. Vira) METEO-FRANCE (M. Pithon, M. Plu, J. Parmentier, J. Arteta, S. Guidotti, N. Assar) Date: 04/05/2016 REF.: CAMS50_2015SC1_D50.3.2.SILAM-2016Q1_201605 CAMS50_2015SC1_D50.3.4.SILAM-2016Q1_201605 CAMS50_2015SC1_D50.5.1.SILAM-2016Q1_201605 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances
Contents: 1. Executive Summary...4 2. The SILAM model (FMI)...5 Product portfolio...5 Availability statistics...5 Use for observations for data assimilation...7 3. Verification report...10 Verification of NRT forecasts...10 Verification of NRT analyses...15 Analysis of SILAM performances for the quarter...19 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances
1. Executive Summary The Copernicus Atmosphere Monitoring Service (CAMS, www.copernicusatmosphere.eu) is establishing the core global and regional atmospheric environmental service delivered as a component of Europe's Copernicus programme. The regional forecasting service provides daily 4-days forecasts of the main air quality species and analyses of the day before, from 7 state-of-the-art atmospheric chemistry models and from the median ensemble calculated from the 7 model forecasts. The regional service also provides posteriori reanalyses using the latest validated observation dataset available for assimilation. This report covers the deliverables related to Near Real Time Production (NRT) for SILAM: D50.3.2-2016Q1, D50.3.4-2016Q1, D50.5.1-2016Q1, for the quarter December 2015 January 2016 February 2016. Verification is done against in-situ surface observations; they are described in the report D50.1.2-2016Q1, that will be delivered shortly. The verification of analyses is done against non assimilated observations. Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 4
2. The SILAM model (FMI) Product portfolio Name Forecast Analysis Description Forecast at surface, 50m, 250m, 500m, 1000m, 2000m, 3000m, 5000m above ground Analysis at the surface Available for users at 3:00 UTC 09:30 UTC for the day before Species O 3, NO 2, CO, SO 2, PM 2.5, PM 10, NO, NH 3, NMVOC, PANs, Birch pollen at surface during season O3, NO 2, CO, SO 2, PM 2.5, PM10, NO Time span 0-96h, hourly 0-24h for the day before, hourly Availability statistics The statistics below describe the ratio of days for which the SILAM model outputs were available on time to be included in the ENSEMBLE fields (analyses and forecasts) that are computed at METEO-FRANCE. They are based on the following schedule for the provision at METEO-FRANCE of: - forecasts data before: 05:30 UTC for D0-D1 (up to 48h), 07:30 UTC for D2-D3 (from 49h to 96h) - analyses data: before 11:00 UTC These schedules have been set to meet the IT requirements for ENSEMBLE products (no later than 8 UTC for 0-48h, 10 UTC for 49-96h and 12 UTC for analyses). Indicators: Availability_model_Forecast Quarterly basis Availability _model_analysis Quarterly basis D0: 99% D1: 99% D2: 99% D3: 99% D: 99% Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 5
During this quarter, the following issues have been encountered by the SILAM production system: Date Problem description (origin, effects) 08/12/2015 Breakdown of computational chain, technical reason at the FMI supercomputer 18/01/2016 Hindcast run was delayed at the start, reasons originating at the FMI supercomputer / SMS timing software Impact on production SILAM forecast results non available for ENSEMBLE calculation SILAM analysis results non available for ENSEMBLE calculation Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 6
Use for observations for data assimilation Day Use of observation for SILAM December O 3 NO 2 NO SO 2 CO PM 10 PM 2.5 1 All All All 2 All All All 3 All All All 4 All All All 5 All All All 6 All All All 7 All All All 8 All All All 9 All All All 10 All All All 11 All All All 12 All All All 13 All All All 14 All All All 15 All All All 16 All All All 17 All All All 18 All All All 19 All All All 20 All All All 21 All All All 22 All All All 23 All All All 24 All All All 25 All All All 26 All All All 27 All All All 28 All All All 29 All All All 30 All All All 31 All All All average Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 7
Day Use of observation for SILAM January O 3 NO 2 NO SO 2 CO PM 10 PM 2.5 1 All All All 2 All All All 3 All All All 4 All All All 5 All All All 6 All All All 7 All All All 8 All All All 9 All All All 10 All All All 11 All All All 12 All All All 13 All All All 14 All All All 15 All All All 16 All All All 17 All All All 18 All All All 19 All All All 20 All All All 21 All All All 22 All All All 23 All All All 24 All All All 25 All All All 26 All All All 27 All All All 28 All All All 29 All All All 30 All All All 31 All All All average Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 8
Day Use of observation for SILAM February average O 3 NO 2 NO SO 2 CO PM 10 PM 2.5 1 All All All 2 All All All 3 All All All 4 All All All 5 All All All 6 All All All 7 All All All 8 All All All 9 All All All 10 All All All 11 All All All 12 All All All 13 All All All 14 All All All 15 All All All 16 All All All 17 All All All 18 All All All 19 All All All 20 All All All 21 All All All 22 All All All 23 All All All 24 All All All 25 All All All 26 All All All 27 All All All 28 All All All 29 All All All 30 All All All 31 All All All NB: Daily statistics of number of assimilated values was not set. The model uses all the data available by 7:00 UTC. Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 9
3. Verification report This verification report covers the period December 2015 January 2016 February 2016. The SILAM skill scores are successively presented for four pollutants: ozone, NO 2, PM 10 and PM 2.5. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. The forecasts and the analyses cover a large European domain (25 W-45 E, 30 N-70 N). The statistical scores that are reported are the root-mean-square error, the modified mean bias and the correlation. The surface observations that are acquired by METEO-FRANCE and used for verification are described in the report D50.1.2-2016Q1, that will be delivered shortly. Verification of NRT forecasts The following figures present, for each pollutant (ozone, NO 2, PM 10, PM 2.5 ): - in the upper-left panel, the root-mean square error of daily maximum (for ozone and NO 2 ) or of daily mean (PM 10 ) for the first-day forecasts with regards to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line, - in the upper-right panel, the root-mean square error of pollutant concentration forecasts with regards to surface observations as a function of forecast term, - in the lower-left panel, the modified mean bias of pollutant concentration forecasts with regards to surface observations as a function of forecast term, - in the lower-right panel, the correlation of pollutant concentration forecasts with regards to surface observations as a function of forecast term. The graphics show the performance of SILAM (black curves) and of the ENSEMBLE (blue curves). Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 10
SILAM forecasts: ozone skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 11
SILAM forecasts: NO 2 skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 12
SILAM forecasts: PM 10 skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 13
SILAM forecasts: PM 2.5 skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 14
Verification of NRT analyses The following figures present, for each pollutant (ozone, NO 2, PM 10 ): - in the upper-left panel, the root-mean square error of daily maximum (for ozone and NO 2 ) or of daily mean (PM 10 ) for the analyses (solid line) and for the first-day forecasts (dashed line) with regards to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line, - in the upper-right panel, the root-mean square error of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regards to surface observations as a function of forecast term, - in the lower-left panel, the modified mean bias of pollutant concentration forecasts of the analyses (solid line) and of the first-day forecasts (dashed line), with regards to surface observations as a function of forecast term, - in the lower-right panel, the correlation of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regards to surface observations as a function of forecast term. The graphics show the performances of SILAM (black curves) and of the ENSEMBLE (blue curves). The superposition of the analysis scores (solid lines) and of the forecast scores (dashed lines) computed over the same observation dataset is helpful to assess the added value of data assimilation. Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 15
SILAM analyses: ozone skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 16
SILAM analyses: NO 2 skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 17
SILAM analyses: PM 10 skill scores against data from representative sites, period December 2015 - January 2016 - February 2016 Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 18
Analysis of SILAM performances for the quarter The current winter period 2015-2016 was quite successful in terms of the forecast stability and robustness: 99% of both forecasts and analysis fields were delivered in time. This was also the first full winter period when the model was running at 0.1 degree resolution. This posed certain challenges to the system, but measures undertaken to ensure timely delivery appeared sufficient. Computational scores were pretty close to those of the previous winter, in fact, even a bit lower. The same tendency was well seen for ENSEMBLE, so we conclude that this was rather a peculiarity of the period or features of input data that affected most models in the ENSEMBLE. Except for under-estimated PM, SILAM maintained quite low bias, which had diurnal cycle following the rush hours. The tendency noticed a year ago to reduce the negative bias of ozone concentrations continued this year too, probably again owing to a very warm winter: there were no substantial changes made in ozone chemistry or depositon modules. Certain improvement was found to NO 2, where RMSE and bias both became smaller, especially at peak hours. One possible explanations is the high resolution of the runs. The same reason is probably behind some reduction of the correlation coefficient: double-punishment is among the likely reasons. Qr. report on daily analyses and forecasts activities, verification of the SILAM performances 19