(Received 5 October 2010; revised manuscript received 19 January 2011)

Transcription

1 Chn. Phys. B Vol. 20, No. 6 (2011) Generalzed unscented Kalman flterng based radal bass functon neural network for the predcton of ground radoactvty tme seres wth mssng data Wu Xue-Dong( ) a), Wang Yao-Nan( ) b), Lu We-Tng( ) a), and Zhu Zh-Yu( ) a) a) School of Electroncs and Informaton, Jangsu Unversty of Scence and Technology, Zhenjang , Chna b) College of Electrcal and Informaton Engneerng, Hunan Unversty, Changsha , Chna (Receved 5 October 2010; revsed manuscrpt receved 19 January 2011) On the assumpton that random nterruptons n the observaton process are modeled by a sequence of ndependent Bernoull random varables, we frstly generalze two knds of nonlnear flterng methods wth random nterrupton falures n the observaton based on the extended Kalman flterng (EKF) and the unscented Kalman flterng (UKF), whch were shortened as GEKF and GUKF n ths paper, respectvely. Then the nonlnear flterng model s establshed by usng the radal bass functon neural network (RBFNN) prototypes and the network weghts as state equaton and the output of RBFNN to present the observaton equaton. Fnally, we take the flterng problem under mssng observed data as a specal case of nonlnear flterng wth random ntermttent falures by settng each mssng data to be zero wthout needng to pre-estmate the mssng data, and use the GEKF-based RBFNN and the GUKF-based RBFNN to predct the ground radoactvty tme seres wth mssng data. Expermental results demonstrate that the predcton results of GUKF-based RBFNN accord well wth the real ground radoactvty tme seres whle the predcton results of GEKF-based RBFNN are dvergent. Keywords: predcton of tme seres wth mssng data, random nterrupton falures n the observaton, neural network approxmaton PACS: Td, Tp, Gg DOI: / /20/6/ Introducton The world s naturally radoactve and around 90% of human radaton exposure arses from natural sources such as cosmc radaton, exposure to radon gas and terrestral radaton. [1] Snce radonucldes are not unformly dstrbuted, knowledge of ther dstrbutons n sols and rocks plays an mportant role n radaton protecton and measurement. [2] Some of the exposures are farly constant and unform for all ndvduals everywhere. Other exposures vary wdely dependng on the locaton. Therefore, the assessment and the predcton of the radaton dose from natural sources are of partcular mportance. In ths work, we study the tme seres predcton of radoactvty n the ground at 2-hour ntervals over one year (from July 2005 to June 2006). Tme seres predcton has played a fundamental role n most spheres of scentfc actvty. As a result, n the last twenty years a varety of new predcton algorthms based on the theory of dynamcal systems have been developed, such as neural networks (NNs) [3 8] and support vector machnes (SVMs), [9 11] whch possess the ablty to approxmate nonlnear systems. However, nonlnear tme seres lterature hghlghts that the good fttng results of nonlnear models do not guarantee an equally good predcton performance. [12] One man reason s that ther dynamcs and propertes are change over tme. Moreover, n the modelng of tme seres usng NNs and SVRs, another key problem s the nherent nose of the tme seres. Usng observatons wthout payng attenton to nose may lead to fttng unwanted data and may damage the approxmaton functon, and the nose n the data could lead to over-fttng or under-fttng problems. [13] Furthermore, a survey of exstng lterature reveals that there s a need to Project supported by the State Key Program of the Natonal Natural Scence of Chna (Grant No ), the Natural Scence Foundaton of Jangsu Provnce of Chna (Grant No. BK ), the Natural Scence Foundaton of Hgher Educaton Insttutons of Jangsu Provnce of Chna (Grant No. 10KJB510004), and the Natonal Natural Scence Foundaton of Chna (Grant No ). Correspondng author. E-mal: woolcn@163.com 2011 Chnese Physcal Socety and IOP Publshng Ltd

2 Chn. Phys. B Vol. 20, No. 6 (2011) develop effcent forecastng models nvolvng less computatonal load and a faster forecastng capablty. [14] Therefore, how to handle the adaptve ablty, the addtve nose and the computatonal load s a mportant but dffcult task for buldng a predcton model. Keepng these problems n mnd, some dfferent flterng methods wth a contnuous update scheme have been employed n the tme seres predcton. Ma and Ten [15] and van der Merwe [16] used the EKF and UKF for Mackey Glass tme seres predcton, Zhang et al. [17,18] proposed the partcle flterng (PF) algorthm to deal wth the non-gaussan nose n tme seres state estmaton, but these algorthms [15 18] were studed under the condton wthout mssng data. In fact, rregular observatons, mssng values and outlers are common n tme seres data, [19] and many frameworks for pre-estmatng the mssng data have been developed. [19,20] In order to solve these problems and at the same tme nclude adaptve ablty, addtve noses, computatonal load and mssng data for buldng the tme seres predcted model, n ths work we suggest the GEKF-based RBFNN and the GUKF-based RBFNN for the predcton of ground radoactvty tme seres wth mssng records. RBFNN can approxmate vrtually any measurable functon up to an arbtrary degree of accuracy and nonlnear flterng approaches wth adaptve ablty can handle addtve noses and reduce the computatonal load to some degree due to ther teratve algorthm structures. On the assumpton that random nterruptons n the observaton process are modeled by a sequence of ndependent Bernoull random varables, we generalze two knds of nonlnear flterng methods wth random nterruptons n the observaton based on the EKF and the UKF, whch are shortened as GEKF and GUKF, respectvely. When the RBFNN and the GEKF or GUKF are combned, the GEKF-based RBFNN and the GUKF-based RBFNN can be used to predct the ground radoactvty tme seres wth mssng observed data by settng each mssng observed data to be zero. The GEKF-based RBFNN and the GUKFbased RBFNN used here are formulated n the predctve correctve form whch underles the general unobserved component approach to the state space estmaton and predcton, and the predcton results are represented by the predcted observaton values of GEKF-based RBFNN and GUKF-based RBFNN. The rest of ths paper s organzed as follows. The dynamc model of tme seres predcton s ntroduced n Secton 2. In Secton 3 the GEKF and GUKF prncples are presented. Then, the GEKF-based RBFNN and the GUKF-based RBFNN appled n ths study are presented n Secton 4. Subsequently, some conclusons drawn from the present study are gven n Secton Dynamc model of tme seres predcton 2.1. Radal bass functon neural network confguraton A radal bass functon (RBF) conssts of the m- dmensonal nput u = [ I 1 I 2 I m ] passed drectly toward a hdden layer where there are assumed to be c neurons n the hdden layer. Each of the c neurons n the hdden layer uses an actvaton functon, whch s a functon of the Eucldean dstance between the nput and an m-dmensonal prototype vector. Each hdden neuron contans ts own prototype vector as a parameter. The output of each hdden neuron s then weghted and passed toward the output layer. The outputs of the network consst of the sum of the weghted hdden layer neurons. Fgure 1 shows a schematc of an RBFNN. The response of an RBFNN n the form of Fg. 1, where the hdden layer functon s n the form of g(s) = exp[ s/β 2 ] (where β s a real constant) can be wrtten as follows: y = [ w 11 w 1c ][ g u υ 1 2 g u υ c 2 ] T = W [ g u υ 1 2 g u υ c 2 ] T = h(w, υ 1,..., υ c, u). (1) Fg. 1. Radal bass functon network archtecture RBFNN based nonlnear flterng model wth random ntermttent falures n the observaton Inspred by the successful use of Kalman flterng for tranng NNs, we present the predctve nonlnear

3 model wth random ntermttent falures n the observaton n ths secton. In general, we can vew the optmzaton of the weght matrx W and the prototypes υ j as a weghted least-squares mnmzaton problem, where the error vector s the dfference between the RBFNN outputs and the target values for those outputs. Consder the RBFNN of Fg. 1 wth m nputs, c prototypes, and n outputs. In order to cast the optmzaton problem n a form suted for the nonlnear flterng methods, we let the elements of the weght matrx W and the elements of the prototypes υ j consttute the state of a nonlnear system and we let the outputs of the RBFNN consttute the output of the nonlnear system to whch the nonlnear flter s appled. Then the state of the nonlnear system can be represented as Chn. Phys. B Vol. 20, No. 6 (2011) Generalzed nonlnear flterng approaches Due to the values of the varable γ k, the densty functon of y k = γ k h(x k, u k ) + w k has the followng mxture form: g(y k ) = p k g(y k γ k = 1) + (1 p k )g(y k γ k = 0), (4) where g(y k γ k = 1) s the densty of the vector h(x k, u k ) + w k and g(y k γ k = 0) s the densty of w k. Hence, E[y k ] = p k E[h(x k, u k )], Cov[y k ] = p k Cov[h(x k, u k )] + p k (1 p k ) (5) E[h(x k, u k )] E[h T (x k, u k )] + R k, Cov[x k, y k ] = p k Cov[x k, h(x k, u k )]. x = [ W T υ T 1 υ T c ]T. (2) The vector x thus conssts of all (n(c + 1) + mc) of the RBFNN parameters arranged n a lnear array. In order to execute a stable nonlnear flterng algorthm, we need to add some artfcal process nose v k and observaton nose w k to the system model. In addton, we use a sequence of ndependent Bernoull random varables γ k (n the bnary swtchng sequence taken are the values 0 or 1) to denote whether the observaton can be observed. Accordng to Eqs. (1) and (2), the nonlnear system model to whch the nonlnear flter wth random ntermttent falures n the observaton can be appled s x k+1 = x k + v k, y k = γ k h(x k, u k ) + w k, (3) where x k s the system state vector at tme k, v k s the process nose, y k s the observaton vector, w k s the observaton nose, h( ) s the nonlnear vector functon of the state vector and nput vector. Varable γ k, whch descrbes the random ntermttent falures n the observaton, s a sequence of ndependent Bernoull random varables. After obtanng the nonlnear flterng approach wth random ntermttent falures n the observaton, we can take the flterng problem under mssng observed data as a specal case of nonlnear flterng wth random ntermttent falures by settng γ k = 0 or γ k = 1 when the k-th datum s mssed or provded and then use these flterng approaches to predct the ground radoactvty tme seres. Therefore, one way to obtan approxmatons of the above statstcs s to approxmate E[h(x k, u k )], Cov[h(x k, u k )], Cov[x k, h(x k, u k )], and to substtute the approxmatons n Eq. (5). So t s easy to derve the GEKF and GUKF based on Refs. [21] and [22] Generalzed extended Kalman flterng The EKF s a mnmum mean squared error (MSE) estmator based on the Taylor seres expanson of nonlnear functon. It s easy to derve the followng update equatons for the mean and covarance of the Gaussan approxmaton to the posteror dstrbuton of the states. () Intalze wth x 1 0 = E[x 0 ], P1 0 xx = E[(x 0 x 1 0 )(x 0 x 1 0 ) T ]. () For k = 1, 2,..., M. (a) Measurement update equatons, P yy k k 1 = p kh k Pk k 1 xx HT k + p k(1 p k ) h( x k k 1, u k )h T ( x k k 1, u k ) + R k, K k = P xy yy k k 1 [Pk k 1 ] 1 = p k Pk k 1 xx HT yy k [Pk k 1 ] 1, ˆx k = x k k 1 + K k [y k p k h( x k k 1, u k )], P xx k k = P xx k k 1 p kk k H k P xx k k 1. (b) Tme update, x k+1 k = f(ˆx k k ), Pk+1 k xx = F kpk k xx F k T + Q k, (6) (7) where Q k s the process nose covarance, R k s the measurement nose covarance, K k s the Kalman gan, F k and H k are the Jacobans of the system and the observaton equaton, respectvely

4 Chn. Phys. B Vol. 20, No. 6 (2011) Generalzed unscented Kalman flterng The GUKF algorthm that updates the mean and covarance of the Gaussan approxmaton to the posteror dstrbuton of the states s gven as follows (See Ref. [22] by Juler and Uhlmann for an n-depth dscusson and some generalzatons on UKF). ) Intalze wth x 1 0 = E[x 0 ], P xx 1 0 = E[(x 0 x 1 0 )(x 0 x 1 0 ) T ]. ) For k = 1, 2,..., M. (a) Calculate the sgma ponts and the weghts χ 0,k k 1 = x k k 1, χ,k k 1 = x k k 1 + ( (n x + λ)p k k 1 ), = 1,..., n x, χ,k k 1 = x k k 1 ( (n x + λ)p k k 1 ), = n x + 1,..., 2n x, W (m) 0 = λ/(n x + λ), W (c) 0 = λ/(n x + λ) + (1 α 2 + β), W (m) = W (c) = 1/ {2(n x + λ)}, = 1,..., 2n x, where n x s the dmenson of the state vector, α determnes the spread of the sgma ponts around x k k 1 and ( ) s usually set to be a small postve value. (nx + λ)p k k 1 s the -th row of the matrx squared root. λ s determned by λ = α 2 (n x + κ) n x wth κ beng a secondary scalng parameter whch s usually set to be 0, and β s used to ncorporate pror knowledge of the dstrbuton of x (for Gaussan dstrbutons, β = 2 s optmal). (b) Measurement update 2nx Pk k 1 zz = W (c) [ ] [ ] T h(χ,k k 1, u k ) z k k 1 h(χ,k k 1, u k ) z k k 1, (c) Tme update P yy k k 1 = p kpk k 1 zz + p k(1 p k ) z k k 1 z k k 1 T + R k, P xz k k 1 = 2nx W (c) [ χ,k k 1 x k k 1 ] [ h(χ,k k 1, u k ) z k k 1 ] T, K k = P xy yy k k 1 [Pk k 1 ] 1 = p k P xz yy k k 1 [Pk k 1 ] 1, ˆx k k = x k k 1 + K k (y k p k z k k 1 ), Pk k xx = P k k 1 xx p kk k (Pk k 1 xz )T. 2n x χ,k+1 k = f(χ,k k ), x k+1 k = W (m) χ,k+1 k, 2n a [ ] [ ] T P k+1 k = χ,k+1 k x k+1 k χ,k+1 k x k+1 k + Qk, W (c) 2n x γ,k+1 k = h(χ,k+1 k, u k ), z k+1 k = W (m) γ,k+1 k, where Q k s the process nose covarance, R k s the measurement nose covarance, and K k s the Kalman gan. (8) (9) (10) 4. Expermental results and analyss 4.1. Expermental data and setup In ths paper we explore the applcablty and the relablty of these proposed methods for the tme seres predcton of radoactvty n the ground n 2-hour ntervals over one year (from July 2005 to June 2006). A few data are mssng due to a techncal problem and the data are avalable at In order to compare the dfferent data sets analysed, we have normalzed data sets between 0 and 1 and obtaned a normalzed data y k as follows:

5 y k = Chn. Phys. B Vol. 20, No. 6 (2011) y ok mn(y ok ) max(y ok ) mn(y ok ), (11) where y ok s the orgnal data, and y k s the normalzed data. An RBFNN, whch consttutes three nputs, ffteen prototypes and one output, s traned wth GEKF and GUKF algorthms n an onlne fashon, respectvely. To afford a far predcton result, the predcton s averaged across a Monte Carlo smulaton consstng of 50 runs. In addton, generally the predcton accuracy gets better and better as the teratve steps ncrease as the proposed flterng model s convergent, so we take the last predcted 95 data as the evaluaton crteron of expermental results. Furthermore, the GEKF and the GUKF are based on the mnmum MSE prncple, so we take the average mean square error (AMSE) e av to compare the performances of the GEKF-based RBFNN and the GUKFbased RBFNN. The e av s defned as follows: Fg. 2. Predcton results of radoactvty n the ground n 2-hour ntervals over one year (from July 2005 to June 2006). e av = 1 N N y k ŷ pk 2, (12) k=1 where y k and ŷ pk are observed and predcted values respectvely, and N s the predcton samples Expermental results and dscusson The last 95 predcted data of radoactvty n the ground n 2-hour ntervals over one year are shown n Fg. 2 (where observed means the real radoactvty tme seres) and the correspondng absolute error and MSE are shown n Fgs. 3 and 4. The normalzed AMSEs of GEKF-based RBFNN and GUKF-based RBFNN for the predcted results of radoactvty n the ground are and , respectvely. Accordng to the expermental results, t turns out that the GUKF provdes much better estmaton accuracy than the GEKF, ths s our major concluson and recommendaton. Now we dscuss the reasons for the observed results. We know from Eq. (3) that our dynamc system model s nonlnear (although the state transton equaton s lnear, and the observaton equaton s nonlnear). The mproved performance of the GUKF compared wth the GEKF s due to the mproved covarance accuracy. The covarance estmatons can be qute dfferent for the two flters as shown n Eqs. (6) and (9), and ths agan makes a dfference n Kalman gan n the measurement update equaton and hence the effcency of the measurement update step. Fg. 3. Absolute errors of radoactvty tme seres predcton results. Fg. 4. Mean square errors (MSEs) of radoactvty tme seres predcton results. 5. Conclusons Integratng the GEKF and GUKF wth RBFNN correspondngly and wthout needng to pre-estmate

6 Chn. Phys. B Vol. 20, No. 6 (2011) mssng data, we drectly use the GEKF-based RBFNN and GUKF-based RBFNN to predct the ground radoactvty tme seres wth mssng observed data by settng the ndependent Bernoull random varables γ k = 0 when the k-th datum s mssed. From expermental results of our study, t follows that the predcton results of GUKF-based RBFNN accord well wth the real ground radoactvty data whle the predcton results of GEKF-based RBFNN devate from those. References [1] Ran A and Sngh S 2005 Atmos. Envron [2] Khan H M, Khan K, Atta M A and Jan F 1994 J. Chem. Soc. Pakstan [3] Lee C M and Ko C N 2009 Neurocomputng [4] Fábo A G and Leandro S C 2008 Chaos, Soltons and Fractals [5] Xao Z, Ye S J, Zhong B and Sun C X 2009 Expert Syst. Appl [6] Ca X D, Zhang N, Venayagamoorthy G K and Wunsch D C 2007 Neurocomputng [7] Ma Q L, Peng H, Qn J W, Zheng Q L and Zhong T W 2008 Chn. Phys. B [8] Dng G, L Y and Zhong S S 2008 Chn. Phys. B [9] Quan T W, Lu X M and Lu Q 2010 Appl. Soft Computng [10] Wu Q 2010 Expert Syst. Appl [11] Wang W J, Men C Q and Lu W Z 2008 Neurocomputng [12] Chatfeld C 2001 Predcton Intervals ed. Armstrong J Prncples of Forecastng: A Handbook for Researchers and Practtoners (New York: Sprnger) [13] Cao L J 2003 Neurocomputng [14] Majh R, Panda G and Sahoo G 2009 Expert Syst. Appl [15] Ma J and Ten J F 2004 Int. Conf. Machne Learnng and Cybernetcs (Shangha: Chna) [16] van der Merwe R 2004 Sgma-pont Kalman Flters for Probablstc Inference n Dynamc State-Space Models Ph. D. Thess, Oregon Health & Scence Unversty p. 1 [17] Zhang B, Chen M Y and Zhou D H 2006 Chaos, Soltons and Fractals [18] Zhang B, Chen M Y and Zhou D H 2007 Chaos, Soltons and Fractals [19] Kasahara Y, Pourahmadb M and Inoue A 2009 Stat. Probabl. Lett [20] Bermúdez J D, Vallet A C and Vercher E 2009 J. Stat. Plan. Infer [21] Chu C K and Chen G 1999 Kalman Flterng wth Real Tme Applcatons (New York: Sprnger) [22] Juler S J and Uhlmann J K 2004 P. IEEE