PM 10 Forecasting Using Kernel Adaptive Filtering: An Italian Case Study

Transcription

1 PM 10 Forecasting Using Kernel Adaptive Filtering: An Italian Case Study Simone Scardapane, Danilo Comminiello, Michele Scarpiniti, Raffaele Parisi, and Aurelio Uncini Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, Via Eudossiana 18, 00184, Rome {danilo.comminiello,michele.scarpiniti, Abstract. Short term prediction of air pollution is gaining increasing attention in the research community, due to its social and economical impact. In this paper we study the application of a Kernel Adaptive Filtering (KAF) algorithm to the problem of predicting PM 10 data in the Italian province of Ancona, and we show how this predictor is able to achieve a significant low error with the inclusion of chemical data correlated with the PM 10 such as NO 2. Keywords: Air pollution, Nonlinear adaptive filtering, Kernel Adaptive Filters, Least Mean Square. Introduction Due to its adverse effects on human health, airborne particulate matter has gained over the last years increasing attention, both inside the research community and in the parliaments worldwide. Numerous studies have found a direct link between inhalation, long term exposure to particulate matter, and increase in mortality rates, particularly for lung cancers [1]. Moreover, a continuous presence of particulate matter lead to a constant decrease in visibility inside the cities and to the deposition of trace elements [2]. To deal with these aspects, the European Commission issued on 22 April 1999 the Council Directive 1999/30/EC, that set a roof for daily concentration of particulate matter with an aerodynamic diameter of up to 10μm(PM 10 ), and obliged member states to issue a warning every time this roof is reached, entering an attention state. Nevertheless, last data coming from the European Environmental Agency 1 shows that, although global pollution has decreased, some pollutants such as PM 10 have stayed more or less stable and constantly exceeds the required threshold. For this reason, the development of a solid system that is able to accurately predict the daily levels of PM 10 has been, and still is, a constant priority for local administrations. 1 air-quality-in-europe-2011 B. Apolloni et al. (Eds.): Neural Nets and Surroundings, SIST 19, pp DOI: / _10 c Springer-Verlag Berlin Heidelberg 2013

2 94 S. Scardapane et al. Since environmental data presents a great complexity and an ample deal of hidden factors, numerous researchers have investigated the possibility of automatically forecasting the daily concentration of PM 10 using learning systems. In particular, neural networks [3] have been found to achieve a moderate error with small amount of data, and thus represent a good solution for implementing a robust prediction system [4]. Recently, a moderate deal of effort has been put into applying other methodologies such as Support Vector Machines [5] to the problem. In this paper, we study the application of Kernel Adaptive Filtering (KAF) techniques [6] to the task of predicting PM 10 in the Italian area of Ancona. KAF algorithms are a recent development in the adaptive filtering field, and have been rarely put to use for real world problems. For this reason, it is interesting to see how they provide a fast and efficient solution to our problem. In addition, comparisons with other standard techniques provide similar results, but the proposed approach is characterized by a low computational cost. We also investigate the inclusion in the learning process of chemical data such as NO 2, showing how this lead to a strong decrease in the prediction error, probably compensating for hidden factors that occasionally falsify the instruments readings. Several studies, for example [7,8,9,10], have addressed the problem of PM 10 forecasting using cross-prediction with different chemical agents. Cross-prediction, in fact, can provide a robust estimate in all those cases, in which the nature of the time-series to be predicted can have several contributes. For example, because the area of Ancona is on the seaside, many external factors, such as the sea salt, can give a misunderstood contribution on PM 10 values, falsifying the results and alarming people even if the concentration is less than that measured. In order to obtain a more robust estimation of the PM 10 values we choose to use cross-prediction in this work. The paper is organized as follows: Section 1 introduces the Kernel Adaptive Filtering approach. Section 2 describes the experimental setup, while Section 3 shows the experimental results. Finally Section 4 concludes the work. 1 Kernel Adaptive Filtering Kernel Adaptive Filtering (KAF) [6] is a recent family of learning algorithms that combines the simplicity of Linear Adaptive Filtering with the nonlinear modeling capabilities of other learning techniques such as Neural Networks and Support Vector Machines. KAF are kernel methods, meaning that the original input x R s to the problem is transformed into a highly dimensional feature vector ψ(x) through the use of a positive definite kernel function: κ(x,x ) : R s R s R. Choosing an appropriate kernel function, the newly created input vector is linearly separable, and is used to train a linear adaptive filter. Practically, as long as the original linear algorithms can be formulated in terms of inner products, the use of the kernel trick [11] allows to rewrite them in terms only of kernel evaluations, thus avoiding the explicit computation of the transformed vector.

3 Algorithm 1. Summary of the KLMS algorithm Initialize: Training Set T, η, κ 1: a[1]=ηd[1] 2: while (x n,d[n]) T } available do 3: y[n]= n 1 j=1 a[n j]κ(x n,x n j ) 4: e[n]=d[n] y[n 1] 5: a[n]=ηe[n] 6: end while PM 10 Forecasting Using Kernel Adaptive Filtering 95 Starting from linear filtering theory, a number of kernel filters have been devised, such as the Kernel Least-Mean Square [12] that is used in this work. They all provide nonlinear approximation to any function and moderate complexity, with respect to other methods such as recurrent neural networks. The main problems arising in the use of a KAF are the choice of a proper kernel, the need for regularization and the fact that the network grows linearly with the number of processed inputs. This last problem is not addressed in the current work. For a review of methods to efficiently curtail the growth of the network, we recommend to the interested reader [12]. 1.1 Kernel Least Mean Square Kernel Least Mean Square (KLMS) [12] is the simplest training algorithm in the KAF family, and is derived by Least Mean Square (LMS) [3] by the use of the kernel trick. Despite its simplicity, it has been shown to own a self regularizing property. The pseudo-code of KLMS is briefly summarized in Algorithm 1. It takes as input a training set T composed of pairs {(x n,d[n])}, wherex n R s is the input vector to the system, and d[n] R the desired scalar output. There is a need to select a proper step size η and the kernel function κ : R n R n R, then, for every processed input, the algorithm stores the current input vector u n and the corresponding weight a[n]=ηe[n]. The step size η balances between speed and convergence, and it can be chosen using a reasoning like the classical LMS algorithm. The output of the filter at time n is given by n 1 y[n]= j=1 a[n j]κ(x n,x n j ). (1) In equation (1) κ(u n,u n j ) refers to the value of the kernel. In this work we used the Gaussian kernel, that, for two generic input vectors u and u becomes: κ(u,u )=exp( γ u u 2 ), (2) where γ is the kernel bandwidth or kernel parameter. In addition κ(u,u ) has the interesting property of being shift-invariant. As a measure of the error we used the mean-square error, which is defined over the training set T as: MSE T (a n )= 1 N (d[n] y[n]) 2. (3) (x n,d[n]) T

4 96 S. Scardapane et al PM Observations Fig. 1. Hourly observations of PM 10 for the year 2010 (in total 8764 observations) in Marina di Ancona 2 Experimental Setup In order to test the proposed predictor, we used hourly observations of PM 10 and NO 2 for the year 2010 in Marina di Ancona (Italy) 2. These data are interesting since, as for every maritime region, accuracy of the sensors can be falsified by the presence of external factors such as sea salt. We compensate this aspect with the inclusion of chemical data given by NO 2. Data is normalized between 0 and 1, and the resulting PM 10 series can be seen in Figure 1. We observe that the concentration of particulate matter has stayed more or less stable around the year, while some observations are clearly outliers. In the first experiment, the input vector x n is an embedding of the last 168 elements of the PM 10 {x[n 1],...,x[n τ]}, corresponding to the previous 7 days of observations, while the desired output is the concentration of PM 10 fortwelvep.m.ofnextday.inthe second experiment, the corresponding observations of the NO 2 are added to each input vector, leading to a 336 elements. From the original series we extracted randomly 700 input vectors and corresponding outputs for training, and 150 elements for testing, thus generating a statistically inde- 2 Data can be freely downloaded from

5 PM 10 Forecasting Using Kernel Adaptive Filtering 97 pendent testing set. Experiments are averaged on 50 different runs, and are conducted using an Intel i GHz processor at 64 bit, with 4 GB of RAM available. 2.1 Correlation between PM 10 and NO 2 NO 2 was chosen as a supporting time series since it has been found to be highly correlated with the particulate matter evolution. In Figure 2 we can visually see the correlation between the two time series for a brief period. To confirm this dependence, we computed the correlation coefficient ρ of the covariance matrix C = cov(x), wherex is a matrix containing all the hourly observations. The correlation coefficient ρ was found to be greater than 0.4, thus confirming our initial hypothesis. Fig. 2. Correlation between PM 10 and NO 2 for a brief period of the year 2010 in Marina di Ancona Figure 2 clearly shows that, around sample 210 (emphasized with a black cross), the values of measured PM 10 are greater than the values of NO 2, differently from other samples. This fact could be due to external factors, such as the sea salt. This latter assumption is argued by means of the hourly weather condition in that day. In fact there was a strong wind from the sea towards the city.

6 98 S. Scardapane et al. 3 Results Figure 3 shows the evolution of the MSE for the two cases, with and without the inclusion of NO2 chemical correlated data. The learning rate η was set to 0.2 forthefirst case and to 0.05 for the second, while the kernel parameter has remained constant at γ = 1. The error is also compared with a standard LMS filter with η = LMS KLMS KLMS MSE iteration Fig. 3. Evolution of the mean square error (MSE) for KLMS in PM 10 prediction with and without the inclusion of NO 2 correlated data We see that KLMS achieves a low error over the testing set in less than 700 iterations, a result that further improve in the second case, with the inclusion of the NO 2 observations. The predictions of the trained KLMS network are shown in Figure 4. In addition, Figure 4, which has a slightly different scale on the y-axis due to the normalization of the data itself, confirms the hypothesized outliers in data around sample 210. In fact, now the predicted value for PM 10 is less than the measured one, according to values of NO 2 and justifying the external nature of that sample. We can conclude that this and similar outliers are due to the sea salt. Note that the resulting network is constituted of 700 nodes, corresponding to the 700 training examples, each of which is composed of 360 elements. In a real world application, for computational requirements, there would be a need of a method for actively curtailing the growth of the resulting system, such as a pruning algorithm.

7 PM 10 Forecasting Using Kernel Adaptive Filtering Predicted Measured Predicted Measured output Observations Fig. 4. Predicted and measured data of trained KLMS network We have also compared the proposed approach with a Multilayer Perceptron (MLP) [3], a Support Vector Machine [3,13] and a Nonlinear Autoregressive (NAR) model [14], with and without the inclusion of NO 2 correlated data. The MLP has one hidden layer with 50 neurons, while the learning rate is set to 0.2 for each neuron. Results obtained with these standard architectures are numerical equivalent to those of Figures 3 and 4 in terms of MSE and predicted data (for this reason are not reported in the paper). This fact suggests that the proposed approach is able to well perform the prediction of correlated environmental data and obtains similar results found in literature, but it presents the advantage of a lower computational cost, with respect MLP and SVM approaches. 4 Conclusions We demonstrated how KAF algorithms provides a fast and efficient way of training a simple network to predict environmental data such as PM 10. Our system achieves a significant low error over a testing set with the inclusion of correlated data and shows a robust behavior with respect to outliers due to external factors. By iterated application, this system can become an handy tool for local legislators to accurately predict the evolution of particulate matter in their region, in the effort to strongly reduce the global concentration of PM 10 over the year. Although comparisons with other standard

8 100 S. Scardapane et al. techniques provide similar results, the proposed approach is characterized by a low computational cost. References 1. Pope, C.A., Burnett, R., Thun, M.J., Calle, E.E., Krewskik, D., Ito, K., Thurston, G.: Lung cancer, cardiopulmonary mortality, and long term exposure to fine particulate air pollution. Journal of the American Medical Association 287, (2002) 2. Hooyberghs, F., Mensink, C., Dumont, G., Fierens, F., Brasseur, O.: A neural network forecast for daily average PM10 concentrations in Belgium. Atmospheric Environment 39(8), (2005) 3. Haykin, S.: Neural Networks and Learning Machines, 2nd edn. Pearson Publishing (2009) 4. Gardner, M.W., Dorling, S.R.: Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment 32, (1998) 5. Arampongsanuwat, S., Meesad, P.: Prediction of PM10 using Support Vector Regression. In: 2011 International Conference on Information and Electronics Engineering IPCSIT, vol. 6, pp (2011) 6. Liu, W., Principe, J.C., Haykin, S.: Kernel Adaptive Filtering: A Comprehensive Introduction. Wiley (2010) 7. Aurangojeb, M.: Relationship between PM 10,NO 2 and particle number concentration: validity of air quality controls. Procedia Environmental Sciences 6, (2011) 8. Ito, K., Thurston, J.D., Nadas, A., Lippmann, M.: Monitor-to-monitor temporal correlation of air pollution and weather variables in the North-Central U.S. Journal of Exposure Analysis and Environmental Epidemiology 11(1), (2001) 9. Lippmann, M., Ito, K., Nadas, A., Burnett, R.T.: Association of particulate matter components with daily mortality and morbidity in urban populations. Journal of Exposure Analysis and Environmental Epidemiology 9(5), 5 72 (2000) 10. Rakesh, K., Abba, E.J.: Air pollution concentrations of PM 2.5,PM 10 and NO 2 at ambient and kerbsite and their correlation in metro city - Mumbai. Environmental Monitoring and Assestment 119(1-3), (2006) 11. Scholkopf, B.: The kernel trick for distances. In: XIII Proceedings of the 2000 Conference on Advances in Neural Information Processing Systems, vol. 13, pp (2000) 12. Liu, W., Pokharel, P.P., Principe, J.C.: The Kernel Least-Mean-Square algorithm. IEEE Transactions on Signal Processing 56(2), (2008) 13. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer (1995) 14. Conover, W.J.: Practical Nonparametric Statistics. Wiley (1971)