Kalman Filtering of Distributed Time Series

Transcription

1 Kalman Filtering of Distributed ime Series Dan Stefanoiu Janetta Culita Politehnica Universit of Bucharest Dept of Automatic Control and Computer Science 33 Splaiul Independentei 632 BucharestROMAIA s: Abstract: his paper aims to introduce an application to Kalman Filtering heor which is rather unconventional Recent experiments have shown that man natural phenomena especiall from ecolog or meteorolog could be monitored and predicted more accuratel when accounting their evolution over some geographical area hus the signals the provide are gathered together into a collection of distributed time series Despite the common sense such time series are more or less correlated each other Instead of processing each time series independentl their collection can constitute the set of measurable states provided b some open sstem Modeling and predicting the sstem states can tae benefit from the famil of Kalman filtering algorithms he article describes an adaptation of basic Kalman filter to the context of distributed signals collections and completes with an application coming from Meteorolog IRODUCIO AD PROBLEM SAEME Kalman Filtering (KF heor was originated at earl 6s b the wors of RE Kalman and RS Buc (Kalman 96 Kalman-Buc 96 One can sa that Kalman-Buc s approach acted lie a switch within the scientific communit because nowadas the literature on this topic is one of the richest concerning the theor as well as the applications Moreover new and sometimes surprising applications continue to eep the KF field alive For example one can mention the latest results from avionics (the stellar inertial navigation problem (Katon 997 fault diagnosis (Haiev 23 or robotics (egenborn 23 his paper focuses on the problem of correlated time series prediction Evolution of some natural phenomena can be monitored with higher accurac if the observation and measurement tae into account not onl time variation of some parameter but also its distribution over a geographical area ae for example the monitoring of minimum and maximum temperatures over a geographical area (see Fig suitable Sensors could thus provide several time series on different locations Such data coming from different channels are in general more or less correlated For example in Fig one can easil notice the strong correlations between the fourth temperature variations since the two cities are close to each-other It is even possible that hidden correlations (that cannot be perceived be crucial for monitoring Assume that the monitoring goal is to predict the temperature It is ver liel that better prediction results be obtained when considering the collection of all data sets rather than when building independent prediction models for each channel in isolation he problem is then to build and estimate multi-variable identification models in view of prediction he solution introduced within this paper relies on the idea that sensors provide direct nois data from the states of an open and quasi-ubiquitous sstem he sstem has in fact a continuous collection of variable states Placing a finite set of sensors at different locations in order to perform measuring is equivalent to sampling the sstem both in time and space he prediction problem of each data set (a time series in fact is actuall a problem of state prediction and can thus be solved in context of KF heor herefore an adaptation of basic KF algorithm to the context of distributed time series is presented next he article is structured as follows ext sections introduce the Marov-Kalman-Buc method and summarize the algorithm that allow the distributed prediction he simulation case stud is based on the example in Fig A conclusion and the references list complete the paper 2 DISRIBUED PREDICIO FRAMEWORK Fig emperature monitoring in 2 cities from Romania When using a stand alone sensor there is a problem with its location Obviousl the temperature varies both in time and space A small networ of sensors is seemingl more he distributed prediction relies on state representation below: x x[ + ] = Ax[ ] + Bu[ ] + Fw[ ] [ ] = Cx[ ] + Bu[ ] + Dv[ ] (

2 where: A R x nu B R n nu B R n C R n n D R and F R are matrices including all variable (but alread estimated parameters of some stochastic process; (usuall D = ; In x R is the unnown state vector; nu u R is the vector of (measurable input signals; n R is the vector of (measurable output signals; w R is the (unnown endogenous sstem noise; n v R is the (unnown exogenous noise which is usuall corrupting the measured data Whenever the stochastic process cannot be stimulated artificiall (lie in case of time series the input vector is assigned to null values herefore the noise w becomes the virtual useful input while the noise v is parasite he following noise hpotheses are assumed for model (: (a all noises are zero mean Gaussian; (b the two noises are uncorrelated each other; (c the endogenous noise is non auto-correlated but its compounds could be correlated at the same instant; (d the compounds of exogenous noise are white and uncorrelated each-other According to the last two hpotheses the covariance matrices of noises are mostl null excepting for the current instant when the are denoted b: Ψ [ ] = E{ w[ ] w [ ] } and [ ] = E{ [ ] [ ] } w Ψ v v (2 Obviousl the matrix Ψ v is diagonal his is perhaps the most restrictive condition in general case In case of distributed time series the output vector includes all the data sets (on channels whereas the state vector x encodes the invisible correlations between them aturall there is no reason to consider that the white noises corrupting the data are correlated each other he number of states is not necessaril equal to the number of time series It depends on the size of multi-dimensional ARMA(X model assigned to the global process as shown in next section wo main problems can be stated within this context he first one is to identif a rough ARMA model (b using the time series to improve its accurac and to transform it into a minimal state representation he second problem is to predict the states via an adapted version of Kalman filtering Each problem is approached next 3 FROM ARMA(X O SAE REPRESEAIO he rough ARMA model can be constructed b simpl considering that the time series are independent each other hus for each time series ( n of length the corresponding ARMA model: ( ( A q C q e v (3 (with nown notations is identified via Minimum Prediction Error Method (MPEM (Söderström and Stoica 989 After identification the estimated polnomials  and Ĉ (for n ield the evaluation of prediction error: ( ˆ ( vˆ Aˆ q C q ˆ + e (4 which actuall stands for the (approximated input overall stochastic process So nu = n in this case u of o refine the rough model one adopts a global model of ARMAX tpe: A ( q ( q ˆ ( q + B v C e (5 where the polnomial matrices n n ( q ABC R have to be identified from the time series as output data and rough prediction errors (4 as input data he two sstem functions of model (5 ie: ( q ( q = ( q & G( q = A ( q C( H A B q (6 encode moreover the correlations between time series Identification maes use of the same MPEM but applied to a multi-dimensional stochastic process Implementation of such a method is non trivial and involves man numerical problems In order to reach for a suitable tradeoff between speed and accurac some simplifications are necessar For example in MALAB environment the following general principle has been adopted: each output depends on the input signals and noises onl With another words output signals are not mixed each other which implies the matrix A ( q is diagonal Another simplification is related to inputs: since input signals are actuall noises the channel should not account twice the input u (one for input and another one for noise Consequentl the matrix B ( q has null diagonal Although the resulting model is seemingl less accurate than the one obtained b appling MPEM at once it is assumed that the accurac is acceptable he great facilit of such an approach is that the sstem functions (7 are directl computed without inverting the polnomial matrix A Once the sstem functions being estimated the MIMO model can be converted into a state representation lie ( following at least three rationales First of them is based on heorem of Division with Reminder and atomic ratios decomposition (for polnomials following the idea introduced in (Proais 996 he second one starts from the linear regression form of ARMAX model and ines the state b concatenating the regressors vector and the parameters vector (iedźwieci 2 he number of states can increase ver fast in case of MIMO models his is the

3 reason the third conversion technique introduced in (vanoverschee 996 is often preferred since it leads to the minimum state representation 4 ADAPED KALMA FILER (PREDICOR he filter is aiming to predict the states of model ( b using a numerical procedure that relies on the recursive Marov estimator (Stefanoiu 25 Hereafter the Marov estimator and the filter equations are described he main result of Gauss-Marov heorem (GM (Placet 95 can be implemented through a recursive procedure One starts from the canonic form of linear regression associated to an identification model (lie (3 or (5: Y = Φθ + V E { VV } = Ψ > (7 where Y R is the -length output (measurable data vector Φ R is the matrix of regressors (either n measurable or not θ R θ is the vector of unnown true parameters and V R is the vector of (non measurable noise data with Ψ R as covariance matrix One assumes the noise is Gaussian with zero mean but not necessaril white (ie the matrix Ψ could be non diagonal he GM states that the following Marov estimation: ( ˆ θ= Φ Ψ Φ Φ Ψ Y (8 is unbiased consistent and efficient Moreover the estimation accurac is also provided (the inverse of estimation error covariance matrix: {( ( } ˆ ˆ ˆ P = E θθ θ θ = Φ Ψ Φ (9 he problem is to design an efficient numerical procedure to compute (8 and (9 recursivel Since not onl Ψ is usuall unnown but its size could be quite big computing its inverse is non trivial o solve the problem one extends the canonical model (7 b the following virtual model (an identit also nown as random wal model: ( θ= θ + θθ where ( W θ R computation stage whereas is the Marov estimation from previous W is the virtual noise (unnown since θ is non measurable Interestingl the covariance matrix corresponding to virtual noise P is alread available from the previous computation too After fusing equations (7 and ( the extended model is: R Y Φ V = θ + ( θ I W Y Φ V with natural notations Whenever the noise V is not correlated to the virtual noise W the covariance matrix of extended noise V includes two diagonal blocs onl: { } E Ψ Ψ = V V = P (2 When computing again the Marov estimation with the extended stochastic process ( after some manipulations the direct lin between the current and previous values of estimated parameters can be revealed: ( ( ˆ θ= θ + Φ Ψ Φ+ P ΦΨ Y Φθ (3 Γ ε wo interesting terms are outlined in (3 he first one is the prediction error ε R based on previous estimation θ n he second one is the sensitivit gain Γ R θ based on previous accurac P Supplementar manipulations can lead to an equivalent expression of the gain which is more suitable for numerical evaluations (onl one matrix has to be inverted instead of 4: ( Γ = PΦ Ψ+ ΦPΦ (4 he covariance matrix of estimation error can also be updated without an matrix inversion (after computing the gain: ( ( Pˆ = P + ΦΨ Φ = I ΓΦ P (5 where I is the unit matrix of size Practicall the recursive equations (4 (5 and (3 constitute de core of implementation procedure for Marov estimator An estimation of Ψ can be obtained b using the prediction error ε instead of noise V (since actuall ε is the current estimation of V he most time consuming operation is computing the gain (4 because at ever step a matrix has to be inverted o reduce the computational effort the number of adaptation data should be used instead of whole data number So is the number of data between successive parameters upgrading (usuall no bigger than Recall the state representation ( and assume that all parameters are alread nown at current instant hen the final goal is to estimate/predict the state values at the next instant + ie x ˆ[ + ] depending on current state values

4 x ˆ[ ] and newl measured data In order to reach for this goal Marov estimator has to be emploed as main tool he main variables of Marov estimator are identified as follows from the last equation of model (: the state vector x[ ] is θ and the output matrix C is Φ Consequentl xˆ[ ] is θ while x ˆ[ + ] stands for ˆθ he covariance matrix from (9 is ined as follows: {( [ ] [ ]( [ ] [ ] Pˆ ˆ ˆ = E x x x x (6 Before starting the Marov estimation procedure it is necessar to verif whether the noise v[ ] is correlated to the error xˆ[ ] x [ ] or not Since the current state x[ ] ( onl depends on inside noise w and the estimated state xˆ[ ] is determined b the former state x ˆ[ ] (ie b the values of noises at instant at most the non correlation restriction is full verified (under the noise hpotheses of section 2 his allows the current state to roughl be estimated as follows: ( [ ] ˆ ˆ Γ = PC DΨ + v D CPC P ˆ ˆ = P ΓCP (7 x [ ] = xˆ[ ] + Γ ( [ ] ˆ Cx[ ] Bu[ ] since the exogenous noises are mixed through matrix D In (7 x [ ] is not et equal to xˆ[ + ] and P is different from Pˆ + as well because the first equation of model ( remained untouched he are onl rough approximations of the targeted terms o refine the approximations the next state is computed from the first equation of model ( with x [ ] instead of xˆ[ ] and w = hus: xˆ[ + ] = A x [ ] + B u[ ] (8 he next covariance matrix P ˆ + results then as direct consequence of equation (8: + = + w Pˆ A P A F Ψ [ ] F } (9 after some manipulations where the non correlation between w [ ] and ( x [ ] x[ ] plaed the main role After aggregating equations (7-(9 the ernel of final recursive procedure related to Kalman-Buc filtering is obtained: ( ˆ v[ ] x ( [ ] ( ˆ Γ = PC DΨ D + CPC xˆ[ + ] = A ˆ[ ] [ ] [ ] ˆ[ ] [ ] x + Bu + AΓ Cx Bu ˆ ˆ ˆ P+ = FΨ w F + A P Γ CP A (2 Obviousl the procedure (2 can be implemented onl after estimating the mixed covariance matrices of noises DΨv[ ] D and F [ ] Ψw F at an instant his operation involves 2 computation stages At the first step the mixed exogenous noise is estimated with the help of the second equation of ( Dvˆ[ n] = ε[ n] = [ n] C xˆ[ n] B u [ ] n (2 n n n n At the second step the covariance matrix is updated: D ˆ [ ] ˆ[ ] ˆ Ψv D = Dnv n v [ n] Dn = + n= = ( D ˆ [ ] ˆ ˆ Ψv D + Dv[ ] v [ ] D + (22 For the endogenous noise instead of repeating the steps above one can compute the estimation more elegantl hus it is eas to see from (2 (2 and ( that: A Γ Dvˆ[ ] = xˆ[ + ] Axˆ[ ] Bu x [ ] = Fw ˆ[ ] (23 Since the noises estimations are so correlated the estimation of F Ψ [ ] F follows straightforwardl: w F ˆ ˆ ˆ Ψw[ ] F = Fnw[ n] w [ n] Fn = + n= (25 = ( F ˆ [ ] ˆ ˆ Ψw F + AΓDv [ ] v[ ] DΓ A + Equations above can be gathered together into a numerical recipe aiming to predict the discrete stochastic states and outputs of model ( he main steps are as follows Input data: a small collection of time series values (the training set D = { [ n ] } n ielding initialization Initialization Produce the first state representation ( hen complete the initialization b setting: an arbitrar state vector ˆx the covariance matrices Pˆ = αi (with α R + ˆ [ ] F Ψw F = and DΨvˆ [ ] D = n 2 For : 2 Estimate the exogenous mixed noise: Dvˆ[ ] [ ] ˆ = Cx [ ] Bu [ ] 22 Update the covariance matrix of exogenous noise: D ˆ ˆ Ψvˆ[ ] D = D Ψvˆ[ ] D + Dv[ ] v [ ] D + 23 Compute the auxiliar matrix: Q = C Pˆ ( n n 24 Invert the matrix: R = D Ψ ˆ [ ] D + Q C R v 25 Evaluate the sensitivit gain: Γ = QR 26 Compute the auxiliar matrix S = AΓ

5 27 Update the covariance matrix of endogenous noise: F ˆ[ ] ( ˆ ˆ Ψw F = F Ψwˆ[ ] F + SDv[ ] v [ ] DS + 28 Update the covariance matrix of estimation error: Pˆ = ˆ [ ] ( ˆ + FΨ + w F A P ΓQ A 29 Predict the state: x xˆ[ + ] = A xˆ[ ] + B u[ ] + S D v ˆ [ ] 2 Predict the output: ˆ[ + ] = C xˆ[ + ] + B u[ + ] 2 Acquire new data: { } 22 Update the state model Output data: D = D + [ + ] predicted time series values { } ˆ[ ] ; estimated covariance matrices { DΨvˆ [ ] D} he most time consuming steps of algorithm above is 22 (state model matrices updating followed b 24 (matrix inversion he algorithm above can easil be adapted to multi-step prediction thans to Marov estimator In this case the algorithm has two stages he first one is concerned with the model adaptation In the second one multi-step prediction is performed he main difference between the two algorithms (one step and multi-step prediction consists of the exogenous noise estimation As long as the measured data are available equation (2 can successfull be emploed When the measured data are missing the exogenous noises have to be estimated b a different technique For example MIMO- ARMA(X models can be emploed in this aim; the estimated white noises can directl stand for mixed exogenous noises (since usuall D = Estimations of covariance matrices n In D Ψ D ˆ [ ] + v + + are necessar both to assess the prediction performance and to estimate the SRs hus the diagonal of each matrix 2 returns the set { σ } whose values pla the role of ˆ n prediction errors variances on ever channel Here is the data length and K is the current prediction step on the prediction horizon hen the following two tpes of SR can be evaluated (one for measured data and another one for predicted data: 2 2 ˆ ( 2 ( 2 ˆ K K K SR =σ / σ & SR = σ / σ n (26 where σ K σ are standard deviations of data on measuring and prediction horizons respectivel; also K ˆ is the standard deviation of prediction error he SRs (26 allow one to ine the prediction qualit ( PQ cost function below: K 2 ˆ σ = PQ = / + [%] ˆ K λe SR SR σ n (27 he bigger the norm of PQ = PQ PQ n the better the predictor performance Finding the structural indices that maximize the norm of PQ cannot be realized through an exhaustive search he structural indices are: p the degree of polnomial trend na nc the orders of ARMA model and the number of states for the linear sstem ( An evolutionar searching technique has to be emploed in this aim 5 SIMULAIO RESULS An application coming from Meteorolog has been considered Dail minimum and maximum temperatures of two cities have been monitored and predicted (as Fig suggests he cities are 6 m far each other on a plain he data bloc consists of 482 samples on 4 channels wo predictors are compared in terms of PQ: PARMA and KARMA he first one is based on ARMA prediction of each channel in isolation he second one relies on the adapted Kalman filter predictor Both algorithms have been implemented within MALAB environment In order to find optimal structural indices the technique from (Kenned 997 has been adopted here are man implementation details that cannot be described here Just one word regarding KARMA: numerical stabilit of the algorithm required special attention at step 25 For each one of the final figures (2-9 three variations are depicted: the original data with the deterministic model (trend and seasonal variation if an on top the residual noise with estimated SR in the middle and the performance on the prediction horizon at bottom Although the predictabilit varies from a channel to another KARMA succeeded to perform better than PARMA (higher PQ and SR values (his result was confirmed b other data blocs as well where correlation between channels exits However in general PARMA has superior performance on data blocs with (almost uncorrelated channels Below the PQ values and norms are shown: PQ [ ] PQ 95 ; ARMA = ARMA PQ [ ] PQ 3726 KARMA = KARMA (28 Onl 4 states were necessar to represent the linear sstem associated to data he fact = n in this case is pure coincidence On the figures corresponding to KARMA performance (Figs 6-9 the onl purpose of ARMA models is to estimate the input colored noises that stimulate the sstem he data on the first channel are seemingl the less predictable his is proven b the modest PQ values returned even b KARMA (onl slightl superior to PARMA one It seems that data from this channel are less correlated to data from the other channels which cannot be noticed b simpl inspecting the data

6 he prediction accurac has increased at the expense of computational complexit for KARMA herefore if the data are quite uncorrelated across channels PARMA should be emploed as the first option 6 COCLUSIO One can sa that KF is a new and old topic at the same time Concerning the theor KF has drawn the bottom line long time ago he applications reuvenate however this approach he KF-based algorithm introduced in this article is genuine Its maor contribution consists of noises estimation during the prediction he most KF algorithms tr to avoid this problem he simulation case stud on natural data has proven that the prediction qualit can be improved when considering correlations between channels REFERECES Haiev Ch and Calisan F (23 Fault Diagnosis and Reconfiguration in Flight Control Sstems Kluwer Academic Boston USA Kalman RE (96 A ew Approach to Linear Filtering and Prediction Problems ransactions of ASME Journal of Basic Engineering Vol 82D pp Kalman RE and Buc RS (96 ew Results in Linear Filtering and Prediction heor ransactions of ASME Journal of Basic Engineering Series D Vol 83 pp 95-8 Katon M and Fried WR (997 Avionics avigation Sstems John Wile & Sons Inc ew Yor USA Kenned J Eberhart R (995 Particle Swarm Optimization IEEE International Conference on eural etwors Piscatawa USA pp egenborn R (23 Robot Localization and Kalman Filters PhD hesis Delft Universit of echnolog Delft L iedźwieci M (2 Identification of ime-varing Processes John Wile & Sons West Sussex UK vanoverschee P and demoor B (996 Subspace Identification of Linear Sstems: heor Implementation Applications Kluwer Academic Publishers Holland Placett RL (95 Some heorems in Least Squares Biometria o 37 pp Proais JG and Manolais DG (996 Digital Signal Processing Principles Algorithms and Applications third edition Prentice Hall Upper Saddle River ew Jerse USA Söderström and Stoica P (989 Sstem Identification Prentice Hall London UK Stefanoiu D Culita J and Stoica P (25 A Foundation to Sstem Identification and Modeling PRIECH Press Bucharest Romania

7 Fig 2 PARMA performance on channel Fig 6 KARMA performance on channel Fig 3 PARMA performance on channel 2 Fig 7 KARMA performance on channel 2 Fig 4 PARMA performance on channel 3 Fig 8 KARMA performance on channel 3 Fig 5 PARMA performance on channel 4 Fig 9 KARMA performance on channel 4