SHORT-TERM PREDICTION OF AIR POLLUTION USING MULTI- LAYER PERCPTERON & GAMMA NEURAL NETWORKS

Transcription

1 Control 4, Unversty of Bath, UK, September 4 ID-6 SHORT-TER PREDICTIO OF AIR POUTIO USI UTI- AYER PERCPTERO & AA EURA ETWORKS. Alyar. Shoorehdel *,. Teshnehlab, A. Kha. Sedgh * PhD student. of Elect. Eng. K.. Toos Unversty of Tech. Tehran, Iran. Fax: _Alyar@eetd. ntu. ac. r Atmospherc Scences and etrologcal Research Center. & Dept. of Elect. Eng. K.. Toos Unversty of Tech. Tehran, Iran. asmerc@rmet.net Dept. of Elect. Eng. K.. Toos Unversty of Tech. Tehran, Iran. Fax: Sedgh@eetd. ntu. ac. r Keywords: Tme Seres, eural etwor, ult-ayer Perceptron, amma emory, emory Depth Parameter. Abstract Ths paper consders the problem of ar polluton data predcton usng mult- layer perceptron and gamma memores neural networs. Ar polluton data are avalable n the format of tme seres and these real data are used to tran and predct the future ar polluton condton. Due to the fast dynamcs and complex behavor of the process governng the ar polluton dynamcs, the modelng and predcton of ths process s dffcult. Also, results are provded to gve a comparson of the two proposed predctors. Introducton Ar polluton s a very complex process affected by many dfferent factors. Thus, predctng such data wth fast dynamcs s very dffcult. Then, wth the help of these data and solvng the related equatons we can model atmospherc processes n the front data networ for each part. It should be noted that atmospherc factors such as temperature, pressure, humdty ran, wnd, etc [8] cause the equatons to become unbalanced; and the maps based on the spread of pollutants to be useless. Even when we consder the atmospherc factors, other factors such as, ncrease n the manufacture of automobles, archtecture of the ctes and many other factors would serously deterorate the model. The current methods of predctng used n the metrologcal organzaton are based on the analyss of the predetermned maps whch are prepared by dfferent centers usng data provded by the land staton, and satelltes. These data are then used by experts to predct the ar polluton. Ths method has serous shortcomng due to ts human centered nature.we have tred to predct ar polluton usng two dfferent neural networ structures. In ths paper, we used real data for Ara cty durng Oct 3, ths data was collected every half hour (Table ()). as Unt Carbon onoxde µg (CO) 3 m Partcular atter ppm (partcular per mllon) ( P-) Table : polluton parameters whch are analyzed and ther unt. emory ult-ayer Perceptron (P) eural etwors The mult-layer neural nets (s) are an mportant class of s. Typcally, the s consst of a set memory sensory neurons that consttute an nput, hdden(s) and output layer of computaton neurons. The nput sgnal(s) propagate(s) through the networ structure n a feed forward drecton. These s are commonly referred to as mult- layer perceptron (P). eural networs are used n predctng tme seres especally where statonary condtons or condtons whch should be provded for classcal technques do not wor and when the dynamcs of tme seres are fast [8, 6, 7, 5]. P networs wth one hdden layer and enough neurons n the hdden layer can estmate functon wth sutable approxmaton [5]. In general, we don t now the optmal number of neurons n hdden layers. We can, based on experence, acheve the desred structure. aybe the smplest nd of memory for a neural networ s the use of past data as nput as n fgure (). Ths memory termnal wors as a delay lne at the nput. Ths nd of termnal whch s used often and s one of the smplest and most useful forms of short-term memory termnals s called Tapped Delay ne emory. Fgure : Ordnary tapped delay lne memory of order p. Fgures (, ) llustrate the structure of delay lne termnal and how t connects to the P networ. In other words nput x along wth p nputs before t, that s, x ( t ) to x( t p) stored n a delay lne memory s of

2 Control 4, Unversty of Bath, UK, September 4 ID-6 p degree whch for nstance y ( networ output that y ( wll be the same as x ˆ ( t +). In ths way predcton s one step a head [3]. Then, we consdered the second networ (P ),.e. x ( t + ) x + x( t ) (second dervaton) as the desred output n a way that the networ usng the method of BP tranng. The results are llustrated n fgures (5, 6). (It should be noted that n all the networ layers except nput we used sgmod transfer functon.) Fgure : Focused tme lagged feed forward networ (TF). Here tranng structure s based on the mnmzaton of error wth the help of the gradent descent (Bac Propagaton (BP)). To further mprove the tranng of networ, all nputs and output have been normalzed. Frst for the networ nputs, nstead of usng delay lne memory, the subtract of two successve value are used as the networ nput,.e. nput x t ) (frst dervaton) was used as networ nput so the whole nput s: X x, x( x( t ), x( t ), x( t ) x( t ) () [ ] T Fgure 5: One step ahead predcton of CO by P. Here we have consdered four nputs and the hdden layer has fve neurons. ext, we traned the networ n a way that ts output reaches x( t + ) x,.e., the frst networ (P ) predcton results are llustrated n fgures (3, 4). Fgure 6: One step ahead predcton of P- (Partcular atter) by P. 3 amma etwor and ts Forward Calculatons Fgure 3: One step ahead predcton of CO by P. It s advantages to weght the memory and ts depth used n neural networs. So, we must move from a smple TD memory to the case where we can weght past data, by defnng a parameter that frst of all determnes the memory depth and second t can be traned through each step of neural networ parameters tranng. The structure of memory n neural networs, are to represent the human memory. For example, a person may not remember last wee s temperature but he/she remembers f t was colder or warmer than today. Fgure 4: One step ahead predcton of P- (Partcular atter) by P. Human short-term memory has varous features, for nstance, ts depth s changeable. Events wth greater nfluence stay longer n the memory and unmportant events are forgotten soon. Ths feature can easly be employed by storng a small part of the data. Fgure (7) llustrates ths nd of memory whch

3 Control 4, Unversty of Bath, UK, September 4 ID-6 s called the amma memory. The depth of ths memory s adjustable to the amount of µ. Another pont of concern s the value of µ, f < µ <, the flter s stable wth poles n the unt crcle and on the rght sde of Z plane, and f < µ < the flter s stable wth poles n unt crcle and on the left sde of the Z plane. amma synapses combne attractve propertes of a FIR synapses wth some of the general power of an IIR flter []. 3. Forward Calculaton Fgure 7: amma memory unt (-order). Varous neural archtectures have been proposed for modelng tme seres. Both IIR (Infnte Impulse Response) /recurrent and FIR (Fnte Impulse Response)/feed forward tme delay approach have numerous drawbacs whch ether lmt the modelng capactes or suffer from nstabltes durng tranng. Based on fgure (8) we can show y w x + w x + + w l w l x l x (3) Where xl ( µ ) xl ( t ) + µ x, l ( t ), l > (4) When l Then x l u. Based on fgures (7, 8) we have: [ x, x( t ),, x( t m + ] T X ) (5) Fgure 8: A amma memory synapses. Untl the early nnetes t was unfeasble to combne the stablty of the FIR type nets wth the benefts of few parameters for large tme scopes n IIR type nets. A soluton to ths problem has been found n the early nnetes by Prncpe and devres, the so called amma networs. Another advantage of amma s ts ablty n modelng the reference model where measurng I/O s on-lne. amma synapses combnes attractve propertes of a FIR synapses wth some of the general power of an IIR flter [4,]. eural networs used n ths paper have three layers wth a nonlnear sgmod functon n the hdden and output layers, as µ µ z shown n fgures (, 7, 8) where or s z + µ ( µ ) z used nstead of (z). The followng calculatons are carred out based on fgure (8): µ z x ( z) x x x( ( µ ) x( t ) ( µ ) z µ x ( t ) x ( µ ) x ( t ) + µ x ( t ) () From equaton () t s clear that f the value of x ( t ) s more mportance than x ( t ) then the value of µ, nown as the factor of memory depth, s growng up. Ths s due to the multplcaton of µ. F 3 m 3 m n n 3n mn T T. X, [ et ] [ ] m. [ X] T ( [ et ] ) F( [ ]. [ X] ) et m In the hdden layer we have: n m m, o( 3 T [ ] n. [ F( et )] These calculatons are for sngle output networ and easly generalze to mult outputs. ote: by havng one amma functon n degree n one synapse, only one µ s used to determne ts memory depth. Also each one of these amma functons has one weght that plays the role of the same weghts n the P memoryless networs. In fact, each synapse doesn t have only one weght, t has + controllable weghts. oreover, each synapse has one µ for controllng the depth of the memory. (6) (7) (8)

4 Control 4, Unversty of Bath, UK, September 4 ID-6 3. Tranng the amma etwor A mnmzng of the sum of the square errors s used for tranng as the supervse rule: E ( d ( n) o ( n)) e (9) Where, d and o are desred and real outputs, respectvely. In ths method, the neurons of the nput layer are four, the hdden layer 7 and the out put layer s one. The tranng rate of weghts ( w ) equal.5 and the rate of tranng the memory depth µ equal. and 3. The calculatons and determnng the updates of w are started from outer layer. Before ths, consderng fgure (8), we can see: y xl () w l Also concernng the equaton n (3) we see: y x l wl wl α l () l l Where xl α l ( µ ) α l ( t ) () + µ α ( t ) + x x ( t ), l, l l In equaton () t > and l >, otherwse, α l. Equaton () s derved from equaton (4), so we show: ( µ ) xl ( t ) α l xl ( t ) + ( µ ) x, l ( t ) + x, l ( t ) + µ xl ( t ) (3) xl ( t ) x, l ( t ) + ( µ ) + x, l ( t ) + µ whch gves equaton (). Fnally, as the last calculaton we see: y y w x u (4) The above equaton s easly acheved from equaton (3). Consderng equatons (), () and (4), t s obvous that all of the equatons necessary for the Bac Propagaton method s calculated (chan rule). One pont worth mentonng here s the relaton of TD wth amma. The only dfference between gamma and tapped-delay lne (TD) net s that each part of the gamma memory unt has a feedbac connecton. If we consder µ, the amma flter can easly be changed to TD. Bac Propagaton calculatons for TD can easly be obtaned from the wrtten equatons for amma. The only pont s that TD networ doesn t have the µ tranng. Fgures (9, ) llustrate the predcton by a three layer amma networ. The memory unts exst, both n the hdden and n the output layers. The output layer neurons are lnear because of the value of our target data. Of course, the output layer can also be nonlnear, but the value should be normalzed. Fgure 9: One step ahead predcton of CO by amma. Fgure : One step ahead predcton of P- (Partcular atter) by amma. 4 Comparatve Studes For evaluatng and comparng two dfferent modelng and predcton methods whch have been examned, four crtera are consdered [9, 4]. Root ean Squared Errors (RSE) ormalzed ean Squared Errors (SE) ean Absolute Error (AE) ean Bas Error (BE) The crtera are defned as: RSE ( xˆ( ) (5) Where, of x s the measured and xˆ s the predcted value. SE ( xˆ( ) ( x( ) (6) Where the value of x s the mean of the measured data. AE xˆ( (7)

5 Control 4, Unversty of Bath, UK, September 4 ID-6 BE xˆ( (8) Tables () and (3) llustrate the comparson results of the methods. Co P P amma RSE SE A B Table : Comparng of three methods for predcton of Co. [6] S. Hayns, eural etwor A Comprehensve Foundaton, cmaster Unversty Hamlton, Ontaro, Canada, Prentce Hall Internatonal Inc, (999). [7]. B. enhaj, Computatonal ntellgence, Poly Technc Unversty Publsher, vol, (). [8] P. laer and. Boznar, Percptron eural etwor Based odel Predcts Ar Polluton, IEEE, World Congress, pp , (997). [9] J. C. Prncpe, J. uo and S. Celeb, Senor ember, An Analyss of the amma emory n Dynamc eural etwors, Bref Paper, IEEE Transactons on eural etwor, Vol. 5, o, pp ,(arch 994). [] J. C. Prncpe, B. de Vres, and P.. de Olvera, The gamma flter a new class of adaptve IIR flters wth restrcted feedbac, IEEE Transacton on Sgnal processng, vol 4(), pp , (993). P - P P amma RSE SE A B Table 3: Comparng of three methods for predcton of P-. 5 Concluson: Ths paper depcted the ablty of gamma neural networs for modelng and predcton of fast dynamc data such as ar polluton the s wthout memory. It s also shown that n comparson wth gamma neural networs has hgh capablty for modelng and predctng of dynamc tme seres. Accordng to tables (, ) the proposed gamma neural networs wth memory weghts demonstrated better performance n modelng and predctons of complex systems. Also, n small scale values (when the values of data are small) the method of P wors better than P. References [] S. Celeb, J. C. Prcpe Analyss of spectral Feature Extracton Usng the amma Flter, Unv Florda, (994). [] T. J. Cholewo, J.. Zurada and A. Choc, Exact radent Calculaton n amma eural etwors, In Proceedngs of the 997 Internatonal Symposum on onlnear Theory and ts Applcatons, Honolulu, Hawa, USA, ov. (997). [3] A. B. Chelan, D.. aghate, and. Z. Hasan, Predcton of Ambent P and Toxc etals Usng Artfcal eural etwors, TECHICA PAPER, Ar & Waste anagement Assocaton, Vol 5, pp 85-8, (July ). [4] B. De Vres and J. C. Prncpe, The amma odel-a new eural odel for Temporal Processng, neural networs, vol.5 no.4, pp , (99). [5]. Dorffner, eural etwors for Tme Seres Processng, eural etwor World, pp , (996).