1. INTRODUCTION. Presented at the 16 th Annual CMAS Conference, Chapel Hill, NC, October 23-25, 2017

Transcription

1 Comparing CMAQ Forecasts with a Neural Network Forecast Model for PM 2.5 in New York Samuel Lightstone*, Barry Gross, and Fred Moshary Optical Remote Sensing Lab, City College of New York, New York City, NY, USA 1. INTRODUCTION Fine particulate matter air pollution (PM 2.5 ) is an important issue of public health, particularly for the elderly and young children. The study by Pope et al. suggests that exposure to high levels of PM 2.5 is an important risk factor for cardiopulmonary and lung cancer mortality [1-2]. Furthermore, increased risk of asthma, heart attack and heart failure have been linked to exposure to high PM 2.5 concentrations [3]. PM 2.5 levels are dynamic and can fluctuate dramatically over different time scales. In addition to local emission sources, pollution events can be the result of aerosol plume transport and intrusion into the lower troposphere. When there is a potential high pollution event, the local air quality agencies must alert the public, and advise the population on proper safety measures, as well as direct the reduction of emission producing activities. Therefore, accurately measuring and predicting fine particulate levels is crucial for public safety. The U.S. Environmental Protection Agency (EPA) established the National Ambient Air Quality Standards (NAAQS), which regulate levels of pollutants such as fine particulate matter. The New York State Department of Environment Conservation (NYSDEC) operates ground stations for monitoring PM 2.5 and speciation throughout NY State [4]. However, surface sampling is expensive and existing networks are limited and sparse. This results in data gaps that can affect the ability to forecast PM 2.5 over a 24-hour period. The EPA developed the Models-3 Community Multi-scale Air Quality system (CMAQ), to provide hour air quality forecasts. CMAQ provides an investigative tool to explore proper emission control strategies. CMAQ has been the standard for modeling air pollution for nearly two decades because of its ability to independently model different pollutants while describing the atmosphere using firstprinciples [5]. In their studies, McKeen et al and Yu et al evaluate the accuracy of CMAQ forecasts [6-7]. To do so, they use the CMAQ 12UTC (Version 4.4) forecast model. They observe the midnight-tomidnight local time forecast and compare the hourly and daily average forecasts to the ground monitoring stations. McKeen et al [6] observed minimal diurnal variations of PM 2.5 at urban and suburban monitor locations, with a consistent decrease of PM values between 1 and 6 local time. However, the CMAQ model showed significant diurnal variations, leading McKeen et al to conclude that aerosol loss during the late night and early morning hours has little effect on PM 2.5 concentrations, while the CMAQ model does not account for this. Therefore, in addition to testing the hourly CMAQ forecast for a 24-hour period, we focus on the daytime window for two reasons: 1) to assess the accuracy of CMAQ when aerosols do not play a reduced roll in forecasting, 2) the forecast should predict the air quality during the time of maximum human exposure. While these studies make a distinction between rural and urban locations, they take the average results for all rural and urban locations respectively; thereby, their assessment of the CMAQ model was as at a regional scale, rather than a localized one. In addition to regional emissions, these studies also considered extreme pollution events such as the wildfires in western Canada and Alaska, which occurred during the observation period for the studies by Yu et al and McKeen et al. The results of this assessment concluded that due to insufficient representation of transport pollution associated with the burning of biomass, CMAQ significantly under predicted the PM 2.5 values for these events. In the study by Huang et al [8], the bias corrected CMAQ forecast was assessed for both the 6 and 12 UTC release times. The study revealed a general improvement of forecasting skill for the CMAQ model. However, it was observed that the bias correction was limited in predicting extreme events, such as wildfires, and new predictors must be included in the bias correction to predict these events. In this study, CMAQ was assessed as a regional forecasting tool, taking 551 sites, and evaluating the average results in six subregions. In our present assessment of the current operational CMAQ forecast model (Version 4.6), we differ from the regional studies above in the following ways: Firstly, in addition to the 12UTC forecast, we evaluated the 6UTC forecast for the same period to determine if release time affects the CMAQ forecast. Second, we focused on specific locations, both rural and urban, to assess *Corresponding author: Samuel D. Lightstone; slights1@citymail.cuny.edu 1

2 the potential of CMAQ as a localized forecasting tool. In addition, we revisited the forecast potential of CMAQ for high pollution events, to determine if these events are generally caused by transport, or by local emissions. Finally, we tailor the forecast comparisons to focus on the potential of providing next day forecasts using data prior to 5PM of the previous day, since this is an operational requirement for the state environmental agencies. In focusing on both rural and urban areas in New York State, previous studies have shown anomalies in PM 2.5 from CMAQ forecasts. For example, in [9], using CMAQ (Version 4.5) with various planetary boundary layer (PBL) parameterizations, PM 2.5 forecasts during the summer pre-dawn and post-sunset periods were often highly overestimated in New York City (NYC). Further analysis of these cases demonstrated that the most significant error was the retrieval of the PBL height, which was often compressed by the CMAQ model, and did not properly take into account the Urban Heat Island mechanisms that expand the PBL layer [1]. This study showed the importance of PBL height dynamics and meteorological factors that motivated the choice of meteorological forecast inputs used during the NN development. The objective of this paper is to determine the best method to forecast PM 2.5 by direct comparison with CMAQ output products. In particular, using the CMAQ forecast model, as a baseline, we explore the performance of a NN based data driven approach with suitable meteorological and prior PM 2.5 input factors. 2. Datasets 2.1 Models The CMAQ V4.6 (CB5 gas-phase chemistry) with 12km horizontal resolution was used for this paper. The CMAQ product for meteorology predictions used is the North American Model Non- Hydrostatic Multi-Scale Model (NAM-NMMB). This version was made available starting February 216. The CMAQ data used for this paper is from February 1, 216 until October 31, 216. The data can be accessed from reference 11, and the model description can be found in references 12 and 13. The CMAQ model used has a few different configurations: release times of 6 UTC and 12 UTC, and each release time has a standard forecast as well as a bias corrected forecast. The analog ensemble method is used for bias corrections. The idea is to look at similar weather patterns for the forecast period, and statistically correct the numerical PM 2.5 forecast based on historical errors. The analog ensemble method is described in detail in Huang, et al [8]. For each release time, CMAQ provides a 48-hour forecast. The release time of 6 UTC and 12 UTC (2AM and 8AM EDT) does not give the public enough time to react to the forecast on the same day as the release. Consequently, for the 6 UTC release time, the forecast hours were used, and for the release time of 12 UTC the forecast hours of were used. This allowed us to construct a complete 24-hour diurnal period for the forecast time window, which facilitated comparison with the field station data. 2.2 Ground-based PM 2.5 Observations PM 2.5 ground data is collected from the EPA s AirNow, which collects NYSDEC monitoring station measurements in real time. The station data used for the forecast experiments in this article are from the New York State stations listed in table A1, from January 1, 211 until December 31, 216. To assess the accuracy of CMAQ model forecasts, matching the model to the ground monitoring station is necessary. To do this, we use the ground NYSDEC stations that lay within the CMAQ grid cell only. Ground stations that are not found in a CMAQ grid cell were not used for comparison; therefore, no spatial interpolation was done on the model results while mapping the model or meteorological data to the AirNow ground stations. 2.2 Meteorological Observations The meteorological data used to create the neural network was collected from the National Centers for Environmental Prediction (NCEP) North American Regional Reanalysis (NARR). NARR has high-resolution reanalysis of the North American region,.3 degrees (32km) at the lowest latitude, including assimilated precipitation. The NARR makes available 8-times-daily and monthly means respectively. The data collected for this paper is the 8-times-daily means for the duration January 1, 211 until December 31, 216. Figure A1 shows the proximity of the meteorological data and the CMAQ model outputs to the ground stations. The NN network was created and tested using historical data. In this paper, meteorology forecast data refers to NARR data that was observed the day of the PM 2.5 forecast. Observed or measured meteorology refers to NARR data that was observed before the forecast release time. 2

3 The locational data-points for the NYSDEC stations, the CMAQ PM 2.5 forecasts, and NARR meteorological reanalysis are depicted in Figure 1. 4 N 46 N 44 N 42 N 8 W 78 W 76 W Location Match NYSDEC, MET, CMAQ New York NYSDEC Station Met Data CMAQ 74 W 72 W 7 W Fig. 1. This map shows the proximity of the ground NYSDEC stations to the NARR meteorological data, and the CMAQ forecast data. 3. Development of the Neural Network As stated above, the accurate prediction of PM 2.5 values is crucial for air quality agencies, so that they could alert the public of the severity and duration of a high pollution event. Therefore, it is imperative that the forecast predictions are released to the public the day before the event. For this paper, we chose 5PM as a target for the forecast release time. Therefore, we ensure that all the methods tested, utilize factors that are available to the state agency prior to 21 UTC (5PM EDT). The NN input includes the following NARR meteorological data: surface air temperature, surface pressure, planetary boundary layer height (PBLH), relative humidity, and horizontal wind (1m). To account for the seasonal variations, the month is also used as an input in the neural network. The PM input variables for the NN are the PM 2.5 measurements averaged over a three-hour frequency to match the meteorological dataset. The NN output is the next day PM 2.5 values. In order to optimize the performance of the neural network, preliminary tests were done to determine the optimum utilization of the meteorological input variables. These tests determined that a combination of the forecast and the observed meteorology should be used as input variables. The forecast time window is midnight-tomidnight EDT for the forecast day, while the time window with the observed data is midnight to 5PM EDT the day the forecast is released. For the PBLH, the forecast value is always used as the input. For the rest of the meteorological inputs, a combination of the forecast and the observed data was utilized by subtracting the eight observation datasets from the eight forecast datasets. This NN architecture uses metrological trends as predictors. In developing a NN PM 2.5 forecast for all of New York State (NYS), we needed to take into account the very different emission sources, and to a lesser extent the meteorological conditions, between New York City (NYC) and the other sites in NYS. We found that the best solution is to design two different neural networks. The first is trained only over NYC sites, while the second is trained for the rest of NYS. It is important to note that we do not try to build a unique NN for every station, since this is not a useful approach for local agencies. PM and Meteorological data from , were used for training. For NYC, since the stations are very close to each other, the NN was trained with spatial mean values of the ground PM monitors and NARR meteorological datasets. For NYS, all the PM and meteorological data from each site outside of NYC were used. Some site-specific information was implicitly included by using the surface pressure as inputs, which provides some indicator of surface elevation. The neural network was developed using the MATLAB Neural Network Toolbox [14]. The Levenberg-Marquardt network was deployed using 1 hidden nodes. The break down for the NN input data is: 7% training, 15% validation, and 15% testing. Because the sample set of training, validation, and testing is divided randomly over the entire dataset, accuracy of the NN was determined by testing each network over 216 data only, a time window that was not included in training. Once the NN function was created, the 216 meteorological and PM data was passed through the network, and the outputs were stored with the date-time and station location as indices. 3.1 Neural Network and CMAQ Comparison The R 2 value for CMAQ and the NN, both compared to AirNow observations, is computed for each forecast model and for each location. As a representative example of the overall performance, the R 2 value for NYC, represented by CCNY, is 3

4 compared to NYS, represented by Brookside Terrace, a non-nyc, non-urban station, and these results are displayed in Figure R 2 Analysis Full 24hrs BROOKSIDE TERRACE Hourly 3hr Average NYS Average Neural Network CMAQ 6Z 6Z Bias Corrected CMAQ 12Z 12Z Bias Corrected NN R 2 Analysis Full 24hrs CCNY Hourly 3hr Average NYS Average Neural Network Fig. 2a CMAQ 6Z 6Z Bias Corrected CMAQ 12Z 12Z Bias Corrected NN Fig 2b Fig. 2. Regression analysis is computed for the comparison between AirNow observations and the various prediction models, for the complete months of February to October 216. The R 2 value for each model is plotted in the figure above to compare CMAQ to the NN. The CMAQ model includes the different release times as well as bias compensated vs. uncompensated runs. In addition, different time and spatial averaging of CMAQ is considered at each location. (a) Brookside Terrace, representative of non-nyc; (b) CCNY, representative of NYC. From Figure 2 above, it can be seen that the most accurate forecast model is the neural network for both NYS and NYC over any of the CMAQ forecasts studied. Regarding CMAQ, we note better performance for NYC than for non-urban areas. This is in contrast to the neural network, where there is very little variation in the results for locations that are urban versus non-urban, indicating that locational inputs in the model, such as the surface pressure, improves forecasting skill. In addition, for all cases, it can be seen that taking the time average improves the CMAQ results. Furthermore, the spatial averaging over NYS (with 1-hour time sampling) shows more improvement in most NYC cases and some non- NYC cases as well. These results indicate the possibility that the best use for CMAQ forecasting is on a regional level. This is supported from the 12km grid cell resolution for CMAQ, a cell size typical for regional analysis. Figure 7 below shows the residual results for CMAQ in comparison to the neural network. For CMAQ, as noted above, there seems to exist a nonrandom bias pattern, where CMAQ generally over predicts for low and high PM values, and under predicts for medium values. This pattern seems to indicate that the CMAQ model may not capture all of the underlying variability factors. On the other hand, for the neural network, the behavior of the residuals is clearly stochastic in nature. We find that an optimized NN approach generally results in a more accurate prediction of future pollution levels, as compared to CMAQ, for a single grid cell (resolution 12km). 3.2 Heavy Pollution Transport Events Because the neural network is data-driven, the network performs better when the most up-to-date inputs are used. In the current design of the neural network, we only used five PM 2.5 inputs, instead of maximum possible in a 24-hour period, eight. In the training of the NN, there were very few extreme event cases, PM 2.5 >25μg/m 3. The lack of suitable training statistics for these events causes the NN approach to have difficultly in adjusting to the sharp contrast with the onset of the event. Therefore, a second neural network was trained with the same design as the neural network detailed above; however, this neural network produces a 24-hour forecast at 5PM for the time period, 5PM 5PM (instead of a next day 24-hour midnight-to-midnight forecast). This neural network uses all eight PM measurements, because there is no lag time between the release time and the first forecast hour. This neural network, referred to as NN Continuous, was not used in the statistical analysis for the different forecast models (because the 24-hour forecast period is different than the forecast analysis above), but is being explored in the extreme event cases. The reason for developing this continuous neural network is to determine if the continuous nature of the network produces better results in extreme pollution events. 4

5 The differences between these neural network designs can be seen in Figure 3. Fig. 3. Architecture of the two Neural Networks utilized highlight the difference in input values and time-period. PM µg/m 3 PM µg/m May 2 May 22 May 24 May 26 May 28 May 3 Jun 1 Jun 3 Jun NN and CMAQ vs Airnow (hourly) NN is released at 21UTC with a 24hr forecast NYC CMAQ Release time =6Z NN and CMAQ vs Airnow (hourly) NN is released at 21UTC with a 24hr forecast NYC CMAQ Release time =6Z CMAQ Bias Corrected (3hr AVG) NN Continuous Airnow (3hr AVG) Fig. 4a CMAQ Bias Corrected (NYS AVG) Neural Network Airnow May 2 May 22 May 24 May 26 May 28 May 3 Jun 1 Jun 3 Jun 5 Fig. 4b Fig 4. NYC surface PM 2.5 levels affected from the wildfires of Alberta Canada 216. The plots focus on the aloft plumes mixing down into the PBL on May 25. The plots show different models vs. AirNow observation (a) CMAQ Biased 3-hour average, NN continuous; (b) CMAQ Biased state average, standard Neural Network To explore the behavior of the different models under high pollution transport conditions, the forecasts coinciding with the wildfires of Fort McMurray in Alberta, Canada were analyzed. The wildfire started on May 1, 216, and was declared under control on July 5, 216. Although the wildfire lasted for over two months, evidence of increased PM 2.5 surface levels in NYC resulting from the wildfire were detected on May 9, and on May 25. On these dates, instances of aloft plume intrusions and the mixing down into the planetary-boundary layer were observed by a ceilometer and a Raman- Mie lidar [15]. In Figure 4, we plot the CMAQ and NN model forecasts, focusing on the transport intrusions into NYC on May 25. The first thing to notice in Figure 4(a), is the oscillations in the CMAQ model even after time averaging over a three-hour window, and to notice how these oscillations smooth out in 4(b) when the New York State spatial average is tested. It is logical that for heavy transport cases, domain averaging helps decrease oscillations; however, we still see significant underestimation of the event. This is the first case where we analyze the behavior of the continuous neural network. Looking at Figures 4(a) and 4(b), it is clear that the continuous neural network is able to respond to the trend of the high pollution event faster, and more accurately, then the standard neural network. 4. Conclusions Prior assessments of the CMAQ forecasting model has found significant dispersion as well as a tendency for the model to overestimate the ground truth field measurements. Even in the bias corrected case, the residuals error in the model was found to have significant bias patterns, indicating that there are predictors not included in the model that could significantly improve the results. These results motivated the development of data driven approaches such as a NN. In developing a data driven NN next day forecast model, we found a general improvement of performance when using prior PM 2.5 inputs together with the difference between present and next day meteorological parameter forecasts. This differential NN approach indicates that meteorological pattern trends are important indicators. Using this NN architecture, we then made extensive regression based comparisons between CMAQ next day forecast models and regionally trained NN next day forecasts for the NYS and NYC regions. In general, we found that the NN results are a significant improvement over the CMAQ forecasts in all cases. These comparisons were 5

6 made to be consistent with state agencies where forecasts should be available by 5PM. To improve the CMAQ forecasts, we found limited improvement when spatial averaging is extended beyond the single pixel 12km resolution to all of New York State. Even in this case, the NN results were generally more accurate. Finally, we focused on forecast performance for transported high pollution events such as Canadian wildfires. In these cases, we found that the CMAQ forecasts had large temporal fluctuations, which could hide most of the event. In this case, significant improvement was obtained when using state averaged bias corrected outputs; however, in general, the smoothed results underestimate the local PM 2.5 measurements. In this application, we found the neural network approach provides a reasonably smooth forecast, although the transition from a clean state to a polluted state is very poor. Nevertheless, the standard NN performed better than CMAQ in this scenario. Further improved results for the NN were obtained in the transition period when the forecast time of the NN was reduced (NN continuous), making the transition from training to testing continuous. More details can be found in ref [16]. 5. Future Work While the continuous NN does adjust quickly to the sharp contrast in transport events, this design limits the scope of the forecast period. Clearly, local data alone is not ideal for this application. Non-local data that can identify high pollution events and assesses their potential mixing with our region is needed. As a preliminary analysis, we explored the use of a combination of HYSPLIT Air Parcel Trajectories with GOES satellite Aerosol Optical Depth (AOD) retrievals to improve the NN. In particular, we analyzed the use of these tools to quantify the relative AOD levels for all air parcels that reach our target area. We found that by properly counting the trajectories weighted by the AOD, a good correlation was seen between the relative AOD and the PM 2.5 levels. Therefore, we believe that using the relative AOD metric as an additional input factor can make improvements in the NN approach. When GOES-R AOD retrievals, with high data latency and multispectral inversion capabilities [17,18], become available, we plan to incorporate these AOD metrics as predictors in the NN. 7. References 1. Pope, C.A., III; Dockery, D.W. Health effects of fine particulate air pollution: Lines that connect. J. Air Waste Manag. Assoc. 26, 56, Pope, C.A., III; Burnett, R.T.; Thun, M.J.; Calle, E.E.; Krewski, D.; Ito, K.; Thurston, G.D. Lung cancer, cardiopulmonary mortality, and longterm exposure to fine particulate air pollution. JAMA 22, 287, Weber, S.A.; Insaf, T.Z.; Hall, E.S.; Talbot, T.O.; Huff, A.K. Assessing the impact of fine particulate matter (PM2.5) on respiratorycardiovascular chronic diseases in the New York City Metropolitan area using Hierarchical Bayesian Model estimates. Environ. Res. 216, 151, Rattigan, O.V.; Felton, H.D.; Bae, M.; Schwab, J.J.; Demerjian, K.L. Multi-year hourly PM2.5 carbon measurements in New York: Diurnal, day of week and seasonal patterns. Atmos. Environ. 21, 44, Byun, D.; Schere, K.L. Review of the governing equations, computational algorithms, and other components of the Models-3 Community Multiscale Air Quality (CMAQ) modeling system. Appl. Mech. Rev. 26, 59, McKeen, S.; Chung, S.H.; Wilczak, J.; Grell, G.; Djalalova, I.; Peckham, S.; Gong, W.; Bouchet, V.; Moffet, R.; Tang, Y.; et al. Evaluation of several PM2.5 forecast models using data collected during the ICARTT/NEAQS 24 field study. J. Geophys. Res. Atmos. 27, 112, doi:1.129/26jd Yu, S.; Mathur, R.; Schere, K.; Kang, D.; Pleim, J.; Young, J.; Tong, D.; Pouliot, G.; McKeen, S.A.; Rao, S.T. Evaluation of real-time PM2.5 forecasts and process analysis for PM2.5 formation over the eastern United States using the Eta-CMAQ forecast model during the 24 ICARTT study. J. Geophys. Res. Atmos. 28, 113, doi:1.129/27jd Huang, J.; McQueen, J.; Wilczak, J.; Djalalova, I.; Stajner, I.; Shafran, P.; Allured, D.; Lee, P.; Pan, L.; Tong, D.; et al. Improving NOAA NAQFC PM2.5 Predictions with a Bias Correction Approach. Weather Forecast. 217, 32, Doraiswamy, P.; Hogrefe, C.; Hao, W.; Civerolo, K.; Ku, J.Y.; Sistla, G. A retrospective comparison of model-based forecasted PM2.5 concentrations with measurements. J. Air Waste Manag. Assoc. 21, 6, Gan, C.-M.; Wu, Y.; Madhavan, B.L.; Gross, B.; Moshary, F. Application of active optical sensors to probe the vertical structure of the urban boundary layer and assess anomalies in air quality model PM2.5 forecasts. Atmos. Environ. 211, 45, Files in /mmb/aq/sv/grib. Available online: (accessed on 1/12/216). 12. NCEP Operational Air Quality Forecast Change Log. Available online: (accessed on 1/5/217). 13. Lee, P.; McQueen, J.; Stajner, I.; Huang, J.; Pan, L.; Tong, D.; Kim, H.; Tang, Y.; Kondragunta, S.; Ruminski, M.; et al. NAQFC developmental forecast guidance for fine particulate matter (PM2.5). Weather Forecast. 217, 32, Demuth, H; Beale, M.. MATLAB and Neural Network Toolbox Release 216a; The MathWorks, Inc.: Natick, MA, USA, Wu, Y.; Pena, W.; Diaz, A.; Gross, B.; Moshary, F. Wildfire Smoke Transport and Impact on Air Quality Observed by a Multi-Wavelength Elastic-Raman Lidar and Ceilometer in New York City. In Proceedings of the 28th International Laser Radar Conference, Bucharest, Romania, 25 3 June Lightstone, S.D., Moshary, F. and Gross, B., 217. Comparing CMAQ Forecasts with a Neural Network Forecast Model for PM2. 5 in New York. Special Issue "Air Quality Monitoring and Forecasting Atmosphere, 8(9), p Anderson, W.; Krimchansky, A.; Birmingham, M.; Lombardi, M. The Geostationary Operational Satellite R Series SpaceWire Based Data System. In Proceedings of the 216 International SpaceWire Conference (SpaceWire), Yokohama, Japan, October Lebair, W.; Rollins, C.; Kline, J.; Todirita, M.; Kronenwetter, J. Post launch calibration and testing of the Advanced Baseline Imager on the GOES-R satellite. In Proceedings of the SPIE Asia-Pacific Remote Sensing, New Delhi, India, 3 7 April Acknowledgement: S. Lightstone would like to acknowledge the support of this work through a NYSERDA EMEP Fellowship. 6