ECMWF COPERNICUS REPORT Copernicus Atmosphere Monitoring Service Regional Production, Quarterly report on the daily analyses and forecasts activities, and verification of the EURAD-IM performances March April May 2017 Issued by: METEO-FRANCE / S. Guidotti Date: 24/07/2017 Ref: CAMS50_2015SC2_D50.3.1.2.EURAD-IM_201707_Daily_Analyses_Report_v1 CAMS50_2015SC2_D50.3.2.2.EURAD-IM_201707_Daily_Forecasts_Report_v1 CAMS50_2015SC2_D50.5.1.1.EURAD-IM_201707_NRT_Verification_Report_v1
This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). Monitoring The activities Service leading (CAMS). to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, The operator activities of CAMS leading on behalf to these of results the European have been Union contracted (Delegation by the Agreement European signed on 11/11/2014). All information in this Centre document for Medium-Range is provided "as is" Weather and no Forecasts, guarantee operator or warranty of CAMS is given on that behalf the of information the is fit for any particular purpose. European The user thereof Union (Delegation uses the information Agreement at signed its sole on risk 11/11/2014). and liability. All For information the avoidance in of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.
Contributors RIUUK E. Friese H. Elbern METEO-FRANCE M. Pithon M. Plu J. Parmentier J. Arteta S. Guidotti N. Assar CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 3 of 22
Table of Contents 1. The EURAD-IM model 6 1.1 Product portfolio 6 1.2 Availability statistics 6 1.2.1 Indicators 6 1.2.2 Problems encountered 7 1.3 Use of observations for data assimilation 7 1.3.1 Use of observations March 2017 8 1.3.2 Use of observations April 2017 9 1.3.3 Use of observations May 2017 10 2. Verification report 11 2.1 Verification of NRT forecasts 11 2.1.1 EURAD-IM forecasts: ozone skill scores against data from representative sites 12 2.1.2 EURAD-IM forecasts: NO 2 skill scores against data from representative sites 13 2.1.3 EURAD-IM forecasts: PM10 skill scores against data from representative sites 14 2.1.4 EURAD-IM forecasts: PM2.5 skill scores against data from representative sites 15 2.2 Verification of NRT analyses 16 2.2.1 EURAD-IM analyses: ozone skill scores against data from representative sites 17 2.2.2 EURAD-IM analyses: NO 2 skill scores against data from representative sites 18 2.2.3 EURAD-IM analyses: PM10 skill scores against data from representative sites 19 2.2.4 EURAD-IM analyses: PM2.5 skill scores against data from representative sites 20 2.3 Analysis of the EURAD-IM performances over the quarter 21 CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 4 of 22
Executive summary The Copernicus Atmosphere Monitoring Service (CAMS, atmosphere.copernicus.eu/) is establishing the core global and regional atmospheric environmental service delivered as a component of Europe's Copernicus programme. The regional forecasting service provides daily 4-day forecasts of the main air quality species and analyses of the day before, from 7 state-of-the-art atmospheric chemistry models and from the median ensemble calculated from the 7 model forecasts. The regional service also provides posteriori reanalyses using the latest validated observation dataset available for assimilation. This report covers the D50.3.1.2, D50.3.2.2 and D50.5.1.1 deliverables related to the EURAD-IM Near Real Time Production (NRT), for the quarterly period ending May 31 st, 2017. Verification is done against in-situ surface observations; these are described in the D50.1.1.2 report covering the same period. The verification of analyses is done against non-assimilated observations. During this quarter, production reliability was excellent for forecast results with a 100% delivery score. Furthermore, the number of days for which the EURAD-IM analysis outputs were available on time to be included in the ENSEMBLE reached 96%. For the MAM2017 period, the RMSE of the EURAD-IM forecast is below the target value for the daily maximum of O 3 and NO 2 and well below the target value for the daily mean of PM10. Compared to MAM2016, the EURAD-IM prediction of the daily O 3 maximum did slightly improve (about 3 µg/m 3 ). The prediction of the daily maximum of NO 2 is worse than for the MAM2016 period (about 3 µg/m 3 ). There are no significant changes in the prediction of the daily mean of PM. The RMSE of the EURAD-IM analysis is below the target value for the daily maximum of O 3 and NO 2 and well below the target value for the daily mean of PM10. Similar to previous periods, the differences between the skill scores of the forecast and of the analysis are larger for PM than for the gaseous components. The low correlation coefficient of the EURAD-IM forecast is significantly improved by the assimilation. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 5 of 22
1. The EURAD-IM model 1.1 Product portfolio Item Forecast Analysis Description Forecast at surface, 50m, 250m, 500m, 1000m, 2000m, 3000m, 5000m above ground Analysis at the surface Available for users at 4:00 UTC 10:30 UTC for the day before Species O 3, NO 2, CO, SO 2, PM2.5, PM10, NO, NH 3, NMVOC, PANs, Birch pollen at surface during season O 3, NO 2, CO*, SO 2, PM2.5, PM10, NO*, NH 3 *, NMVOC*, PANS* Time span 0-96h, hourly 0-24h for the day before, hourly * Non-assimilated species 1.2 Availability statistics The statistics below describe the ratio of days for which the EURAD-IM model outputs were available on time to be included in the ENSEMBLE fields (analyses and forecasts) that are computed at Meteo- France They are based on the following schedule for the provision at Meteo-France of: Forecasts data before: 05:30 UTC for D0-D1 (up to 48h), 07:30 UTC for D2-D3 (from 49h to 96h); Analyses data: before 11:00 UTC. These schedules were set to meet the IT requirements for ENSEMBLE products (no later than 8 UTC for 0-48h, 10 UTC for 49-96h and 12 UTC for analyses). 1.2.1 Indicators Availability_model_Forecast Quarterly basis Availability_model_Analysis Quarterly basis D0: 100% D1: 100% D2: 100% D3: 100% D: 96% CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 6 of 22
1.2.2 Problems encountered The following issues were encountered by the EURAD-IM production system: Date Problem description Impact on production 20/04/2017 Late delivery of analysis results. EURAD-IM analysis results non available on time for ENSEMBLE calculation. 06/05/2017 Late delivery of analysis results: problem with cca in degraded mode. 08/05/2017 One step missing in analysis results. 13/05/2017 All levels provided in the operational production instead of in the e-suite production. Correction made. EURAD-IM analysis results non available on time for ENSEMBLE calculation. EURAD-IM analysis results non available for ENSEMBLE calculation. EURAD-IM analysis results non available for ENSEMBLE calculation. 1.3 Use of observations for data assimilation Please see the next three pages. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 7 of 22
1.3.1 Use of observations March 2017 Day O 3 NO 2 NO SO 2 C0 PM10 PM2.5 1 18,157 16,701 3,416 10,982 4,557 2 18,460 17,065 3,409 11,570 4,805 3 18,612 17,351 3,606 11,830 4,983 4 16,959 15,390 3,643 10,387 3,913 5 18,143 15,457 3,396 10,801 4,416 6 18,474 16,666 3,239 11,602 4,656 7 18,356 16,933 2,990 11,718 4,933 8 18,063 17,044 3,194 11,589 4,840 9 18,016 16,974 3,597 11,432 4,835 10 18,064 16,790 3,675 11,624 5,006 11 18,098 16,975 3,736 11,696 5,184 12 15,988 14,964 3,486 10,270 4,207 13 17,175 16,138 3,471 10,816 4,614 14 16,980 15,703 3,394 10,920 4,680 15 16,907 16,032 3,248 10,712 4,655 16 16,244 15,809 3,816 10,708 4,661 17 15,592 15,147 3,382 10,296 4,238 18 16,163 14,566 2,629 9,644 3,814 19 17,571 15,060 2,717 10,564 4,267 20 17,606 16,129 2,900 11,104 4,511 21 0 0 0 0 0 22 17,326 16,319 2,989 11,187 4,742 23 17,785 16,596 3,293 11,379 4,874 24 17,569 16,517 3,345 10,939 4,767 25 17,495 16,136 3,518 10,810 4,732 26 16,752 14,945 3,562 10,243 4,485 27 17,033 16,206 3,780 10,742 4,577 28 14,007 13,578 3,539 8,562 3,765 29 17,594 16,504 3,328 10,990 4,787 30 17,625 16,510 3,115 10,833 4,499 31 17,764 16,576 3,479 10,968 4,737 CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 8 of 22
1.3.2 Use of observations April 2017 Day O 3 NO 2 NO SO 2 C0 PM10 PM2.5 1 17,704 15,879 3,204 10,963 4,729 2 17,681 15,822 2,988 10,886 4,765 3 17,580 16,256 3,104 10,871 4,767 4 17,553 16,344 3,075 10,915 4,738 5 17,435 16,188 3,064 10,797 4,717 6 17,576 15,854 2,561 10,678 4,446 7 17,794 16,343 2,746 10,857 4,611 8 17,878 16,205 3,123 11,000 4,671 9 17,724 16,014 3,368 11,087 4,913 10 17,778 16,232 3,321 11,109 4,739 11 17,752 16,105 2,705 10,986 4,470 12 17,819 16,336 3,004 10,972 4,617 13 17,676 16,227 2,724 10,943 4,579 14 15,921 14,366 2,605 9,372 4,005 15 13,531 12,441 1,880 8,588 3,755 16 13,493 11,709 1,600 8,352 3,570 17 15,678 13,704 2,557 9,212 3,661 18 5,872 5,445 883 3,517 1,400 19 16,262 14,562 2,408 9,278 3,897 20 16,004 14,487 2,734 9,551 4,221 21 16,175 14,666 3,220 9,730 4,294 22 18,010 13,606 2,124 9,193 4,637 23 17,921 12,860 2,723 10,847 4,307 24 17,939 16,354 3,417 11,021 4,811 25 17,590 16,135 2,809 10,900 4,565 26 18,316 16,535 2,604 11,444 4,742 27 18,749 16,860 2,852 11,651 4,914 28 18,868 17,046 2,872 11,685 5,087 29 18,589 16,389 2,814 11,509 5,017 30 17,950 15,022 2,999 11,100 4,980 CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 9 of 22
1.3.3 Use of observations May 2017 Day O 3 NO 2 NO SO 2 C0 PM10 PM2.5 1 0 0 0 0 0 2 18,410 16,489 2,916 11,321 4,784 3 18,756 16,936 2,838 11,729 5,067 4 18,660 16,873 2,584 11,685 5,069 5 18,885 17,041 2,627 11,588 5,059 6 18,697 16,468 2,662 11,800 5,023 7 18,503 16,046 2,476 11,515 4,819 8 18,322 16,201 2,494 11,587 4,957 9 18,395 16,456 2,902 11,425 4,946 10 18,353 16,654 3,339 11,628 4,950 11 18,217 16,264 3,258 11,550 4,937 12 18,149 16,113 2,804 11,468 4,763 13 18,450 16,218 2,521 11,558 4,749 14 18,323 15,819 2,468 11,613 4,791 15 18,112 16,384 2,736 11,420 4,660 16 17,503 15,836 2,843 11,040 4,244 17 17,723 16,169 2,882 11,160 4,480 18 18,123 16,221 2,927 11,479 4,921 19 17,000 14,920 2,830 10,623 4,799 20 16,976 14,395 2,570 10,248 4,465 21 17,468 14,653 2,418 10,604 4,565 22 18,255 16,547 2,952 11,428 4,912 23 18,224 16,483 2,991 11,521 5,031 24 18,225 16,480 2,548 11,523 4,938 25 18,578 15,806 2,753 11,458 4,850 26 18,417 16,435 2,824 11,388 4,701 27 18,144 15,876 2,750 11,225 4,627 28 18,289 15,762 2,934 11,478 4,928 29 18,305 16,504 2,963 11,594 5,076 30 18,497 16,613 2,672 11,675 5,056 31 18,425 16,345 2,474 11,603 4,938 CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 10 of 22
2. Verification report This verification report covers the quarterly period ending May 31 st, 2017. The EURAD-IM skill scores are successively presented for four pollutants: ozone, NO 2, PM10 and PM2.5. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. The forecasts and the analyses cover a large European domain (25 W-45 E, 30 N-70 N). The statistical scores that are reported are the root-meansquare error, the modified mean bias and the correlation. The surface observations that are acquired by Meteo-France and used for verification are described in D50.1.1.2 covering the same period. 2.1 Verification of NRT forecasts The following figures present, for each pollutant (ozone, NO 2, PM10 and PM2.5): In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO 2 ) or of daily mean (PM10 and PM2.5) for the first-day forecasts with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line; In the upper-right panel, the root-mean square error of pollutant concentration forecasts with regard to surface observations as a function of forecast term; In the lower-left panel, the modified mean bias of pollutant concentration forecasts with regard to surface observations as a function of forecast term; In the lower-right panel, the correlation of pollutant concentration forecasts with regard to surface observations as a function of forecast term. The graphics show the performance of EURAD-IM (black curves) and of the ENSEMBLE (blue curves). CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 11 of 22
2.1.1 EURAD-IM forecasts: ozone skill scores against data from representative sites The RMSE of the daily maximum of the EURAD-IM ozone forecast is below the target value, but about 2 µg/m 3 larger than the RMSE of the Ensemble median. Compared to MAM2016, the EURAD-IM prediction of the daily ozone maximum did slightly improve (about 3 µg/m 3 ). The RMSE of the EURAD- IM ozone forecast has a maximum at about 06:00 UTC, the difference between EURAD-IM and the ENSEMBLE is largest around midnight (about 5 µg/m 3 ). Like in MAM2015, there are only slight differences between the MMB of the ENSEMBLE and of the EURAD-IM forecast. The correlation coefficient of the EURAD-IM forecast is low compared to the ENSEMBLE (about 0.2 smaller). CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 12 of 22
2.1.2 EURAD-IM forecasts: NO 2 skill scores against data from representative sites Compared to MAM2015, the RMSE of the daily maximum of the EURAD-IM NO 2 forecast is about 3 µg/m 3 larger but still below the target value. As usual the daily cycle of the RMSE of NO 2 shows a characteristic double peak, which is mainly caused by high negative model biases at the morning and afternoon rush hours. The MMB of the EURAD-IM forecast is negative in general, but the value of the EURAD-IM forecast is about 0.2 µg/m 3 smaller compared to the ENSEMBLE. The correlation coefficient decreases strongly with increasing forecast time. Compared to the ENSEMBLE, the correlation coefficient of the EURAD-IM forecast is slightly lower (about 0.05). CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 13 of 22
2.1.3 EURAD-IM forecasts: PM10 skill scores against data from representative sites The RMSE of the daily mean of the EURAD-IM PM10 forecast is well below the target value and about 8 µg/m 3 smaller than in the previous period. In January and February 2017, the models were not able to predict unusual high PM10 concentrations during several strong pollution episodes. The RMSE exhibits a maximum in the forenoon and is comparable to the ENSEMBLE with the exception between midnight and about 06:00 UTC, where the RMSE of the EURAD-IM forecast is slightly larger (about 1.5 µg/m 3 ). The small MMB of the EURAD-IM forecast at step 0 is probably caused by the initialisation with a PM analysis for the previous day. Nevertheless the MMB is generally still negative. In contrast to the ENSEMBLE, the daily cycle of the MMB has larger amplitude and shows a stronger decrease with increasing forecast time. The correlation coefficient of the EURAD-IM PM10 forecast is lower (about 0.1) than that of the Ensemble median. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 14 of 22
2.1.4 EURAD-IM forecasts: PM2.5 skill scores against data from representative sites For the same reason as for PM10, the RMSE of the daily mean PM2.5 forecast is about 6 µg/m 3 smaller than in the previous period. Compared to MAM the RMSE is slightly larger (less than 1 µg/m 3 ). The RMSE of the EURAD-IM PM10 forecast is generally about 2.5 µg/m 3 larger than that of the Ensemble median. In contrast to the ENSEMBLE, the MMB is positive and decreases slightly with increasing forecast time. The high positive MMB at the first forecast day may be caused by the initialisation with a PM analysis for the previous day, especially at stations where PM10 is measured and PM2.5 is not measured. The correlation coefficient of the EURAD-IM forecast is about 0.15 smaller compared to the ENSEMBLE and decreases strongly with increasing forecast time. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 15 of 22
2.2 Verification of NRT analyses The following figures present, for each pollutant (ozone, NO 2, PM10 and PM2.5): In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO 2 ) or of daily mean (PM10 and PM2.5) for the analyses (solid line) and for the first-day forecasts (dashed line) with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line; In the upper-right panel, the root-mean square error of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term; In the lower-left panel, the modified mean bias of pollutant concentration forecasts of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term; In the lower-right panel, the correlation of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term. The graphics show the performances of EURAD-IM (black curves) and of the ENSEMBLE (blue curves). The superposition of the analysis scores (solid lines) and of the forecast scores (dashed lines) computed over the same observation dataset is helpful to assess the added value of data assimilation. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 16 of 22
2.2.1 EURAD-IM analyses: ozone skill scores against data from representative sites The RMSE of the EURAD-IM analysis of the daily maximum of O 3 is about 3.5 µg/m 3 below the target value. Similar as for MAM2016, the skill scores of the EURAD-IM surface O 3 analysis are only slightly better than the forecast skill scores. Differences between analysis and forecast are larger for the ENSEMBLE product than for EURAD-IM results, i.e. the ENSEMBLE could benefit more from the assimilation of observations than EURAD-IM alone. There are not any large differences of the diurnal cycle of the skill scores between the EURAD-IM forecast and analysis. Also the diurnal cycles of the ENSEMBLE product and the EURAD-IM results are comparable. The small difference between the forecast and analysis skill scores at 00:00 UTC is probably caused by the initialisation of the forecast with analysis results for the previous day. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 17 of 22
2.2.2 EURAD-IM analyses: NO 2 skill scores against data from representative sites The RMSE of the EURAD-IM analysis of the daily maximum of NO 2 is under the target value but about 1 µg/m 3 larger than in MAM2016. Skill scores of the EURAD-IM NO 2 analysis are generally slightly better than the forecast skill scores: the RMSE is about 1 µg/m 3 smaller, MMB 0.1 to 0.3 µg/m 3 smaller, and the correlation coefficient 0.05 to 0.15 higher. This disagrees with the EURAD-IM NO 2 reanalysis whose skill scores are clearly better than those of the forecast. The small difference between the forecast and analysis skill scores at 00:00 UTC is probably caused by the initialisation of the forecast with analysis results for the previous day. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 18 of 22
2.2.3 EURAD-IM analyses: PM10 skill scores against data from representative sites The RMSE of the daily mean of the EURAD-IM PM10 analysis is well below the target value and about 1 µg/m 3 smaller than for MAM2016. Skill scores of the analysis are generally slightly better than the forecast skill scores with improvements comparable to the NO 2 analysis. The small difference between the forecast and analysis skill scores at 00:00 UTC is probably caused by the initialisation of the forecast with analysis results for the previous day. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 19 of 22
2.2.4 EURAD-IM analyses: PM2.5 skill scores against data from representative sites In contrast to the other pollutants, the performance gain due to the PM2.5 analysis is comparable to the ENSEMBLE for the RMSE and for the correlation coefficient. The RMSE of the analysis is 1.5 to 2 µg/m 3 smaller than the RMSE of the forecast; the correlation coefficient of the analysis is 0.15 to 0.2 larger compared to the forecast. Like for MAM2016, the difference between the MMB of the analysis and of the forecast is small. The slight difference between the forecast and analysis skill scores at 00:00 UTC is probably caused by the initialisation of the forecast with analysis results for the previous day. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 20 of 22
2.3 Analysis of the EURAD-IM performances over the quarter For the MAM2017 period, the RMSE of the EURAD-IM forecast is below the target value for the daily maximum of O 3 and NO 2 and well below the target value for the daily mean of PM10. Compared to MAM2016 the EURAD-IM prediction of the daily O 3 maximum did slightly improve (about 3 µg/m 3 ). The prediction of the daily maximum of NO 2 is worse than for the period MAM2016 (about 3 µg/m 3 ). There are no significant changes in the prediction of the daily mean of PM. For MAM2017 the daily mean of PM was significantly better predicted than for the D2016 / JF2017 period. In January and February 2017, the models were not able to predict very high PM10 concentrations during several strong pollution episodes. As usual the daily cycle of the RMSE of NO 2 shows a characteristic double peak, which is mainly caused by high negative model biases at the morning and afternoon rush hours. Similar to previous periods the afternoon maximum of O 3 is under-predicted. The small MMB of the EURAD-IM PM10 forecast at step 0 is probably caused by the initialisation with a PM analysis for the previous day. Nevertheless the MMB is generally still negative. The initialisation of the forecast with a PM analysis may lead to the positive MMB of PM2.5, especially at stations where PM10 is measured and PM2.5 is not measured. The correlation coefficient of the EURAD-IM forecast is low compared to the ENSEMBLE for all investigated pollutants. The RMSE of the EURAD-IM analysis is below the target value for the daily maximum of O 3 and NO 2 and well below the target value for the daily mean of PM10. Similar to previous periods, the differences between the skill scores of the forecast and of the analysis are larger for PM than for the gaseous components. The low correlation coefficient of the EURAD-IM forecast is significantly improved by the assimilation. The small difference between the forecast and analysis skill scores at 00:00 UTC for all investigated pollutants is probably caused by the initialisation of the forecast with an analysis for the previous day. CAMS50_2015SC2 EURAD-IM Production Report MAM2017 Page 21 of 22
ECMWF - Shinfield Park, Reading RG2 9AX, UK Contact: info@copernicus-atmosphere.eu atmosphere.copernicus.eu copernicus.eu ecmwf.int