A Constructive-Fuzzy System Modeling for Time Series Forecasting I. Luna 1, R. Ballini 2 and S. Soares 1 {iluna,dino}@cose.fee.unicamp.br, ballini@eco.unicamp.br 1 Department of Systems, School of Electrical and Computer Engineering, University of Campinas - Brazil 2 Institute of Economy, University of Campinas - Brazil A Constructive-Fuzzy System Modeling for Time Series Forecasting p.1/2
Summary Introduction; Time series analysis: data pre-processing; Input selection; General structure of a fuzzy rule-based model; Constructive learning; Case study: NN3 competition; Conclusions and future wors. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.2/2
Time series modeling Analysis Input selection Model Construction Adequacy Validation Trend Seasonality FNN-PMI C-FSM Residual analysis Forecasting Pre processing Figure 1: Time series modeling. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.3/2
Analysis and pre-processing Reduced data set of the NN3 competition; Stationarity: required for input selection; Seasonal and trend components: Econometric Views 2. - Quantitative Software, 1995. Trend component: series 1, 5 and 9; All series with no trend were transformed according to: z (m) = y (m) µ(m) σ(m) z : stationary version of the time series; y, = 1,...: th observation; µ(m) is the monthly average value and σ(m) is the monthly standard deviation. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.4/2
Input selection 1. FNN (False Nearest Neighbors): determines the minimum number of lags necessary to represent each pattern or state of the time series; 2. PMI (Partial Mutual Information): measure of information that each new variable x provides, taing into account an existing set of inputs Z. Given variables X e Y, PMI score between X and Y is defined by: where: PMI = 1 N N i=1 [ fx,y log (x i, y i ) ] e f X (x i )f Y (y i ) (1) x i = x i E(x i Z) e y i = y i E(y i Z) Z is the st of inputs already chosen. E( Z) is the conditional expected value; N is the number of input-output patterns. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.5/2
Input selection y = x 4 + e 1 y 1.6 1.4 1.2 1.8.6.4.2 x = sen(2πt/t) + e 2 T = 2; t = 1,...,2; e 1 and e 2 are noisy signals with normal distribution, µ = and σ =, 1. Measure Value MI.4199.2 1.5 1.5.5 1 1.5 x Correlation.32 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.6/2
Input selection Number inputs Complexity Local minimum 5 4.8.7.6 PMI score Threshold FNN 3 2 1 NN3_12 PMI.5.4.3.2.1 1 2 3 4 5 6 7 8 9 1 11 12 Lags 1 2 3 4 5 6 7 8 9 Lags Figure 2: FNN. Figure 3: PMI. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.7/2
A general structure R 1 y 1 x x. R i y i. x g 1 ŷ. R M y M x g i g M Input space partition Figure 4: A general structure of a fuzzy-rule based system, composed by a total of M fuzzy rules. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.8/2
A general structure x = [x 1, x 2,..., x p] R p is the input vector at instant, Z + ; ŷ R is the estimate output; Given centers c i R p and covariance matrices V i i = 1,...,M, membership degrees g i (x ) are defined as: with α i, M i=1 α i = 1 and: g i (x ) = gi = α i P[ i x ] M α q P[ q x ] q=1 P[ i x 1 ] = (2π) p/2 det(v i ) 1/2 { exp 1 } 2 (x c i )V 1 i (x c i ) T (2) (3) A Constructive-Fuzzy System Modeling for Time Series Forecasting p.9/2
A general structure Each local model y i, i = 1,...,M is estimated by a linear one: y i = φ θ i T (4) where φ = [1 x 1 x 2...x p] and θ i = [θ i θ i1... θ ip ]. The output model ŷ is computed as: ŷ = M g i (x ) yi (5) i=1 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.1/2
Constructive learning Initial structure M = 2, N patterns EM steps Initialization E step: g i is estimated given x and y posterior estimate h i ; for Npatterns h i = α i P(i x )P(y x, θ i ) M q=1 α qp(q x )P(y x, θ q ) No Add? Yes M = M + 1 EM steps M step: Model parameters are adjusted; No No Prune? Stop? Yes M = M 1 EM steps Adaptation Adding and pruning conditions are verified. α i = 1 N N h i (6) =1 Yes End A Constructive-Fuzzy System Modeling for Time Series Forecasting p.11/2
Second phase: Adaptation Adding a new rule: Assuming a normal input data distribution, with a confidence level equal to γ%: Ω is an i/o data set so that max i=1,...,m (P(i x Ω)) <.14 If N Ω > Then create a new rule. P(i x ) From normal distribution table: γ 73% z γ 1.1.14 (1.73)/2 γ% x x z γ diag(vi ) x + z γ diag(vi ) Pruning a new rule: α i is proportional to the sum of all h i. Thus, the more times the rule is strongly activated, the higher its α i will be. If α i < α min, then the i th rule will be pruned. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.12/2
Case study: NN3 competition Reduced data set; Table 1: Global prediction errors for series NN3_12 and NN3_14. Series In sample Out of sample Out of sample 1 step ahead 1 step ahead 1 to 18 steps ahead NN3_12 = 4,..., 18 = 19,..., 126 = 19,..., 126 NN3_14 = 4,..., 97 = 98,..., 115 = 98,..., 115 Series M smape MAE smape MAE smape MAE (%) (u) (%) (u) (%) (u) NN3_12 3 3.41 179.36 4.72 287.98 11.4 658.5 NN3_14 3 1.98 438.27 6.75 334.93 12.32 612.32 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.13/2
Case study: NN3 competition (a).2.1 r t.1.2 2 4 6 8 1 12 14 16 18 2 t 1 (b) Time series and predicted values 8 6 4 2 Real One step ahead 1 to 18 steps ahead Future 18 steps 5 1 15 Figure 5: Multiple steps ahead: (a) autocorrelation coefficients estimates, (b) predictions for series NN3_12. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.14/2
Case study: NN3 competition (a).2.1 r t.1.2 2 4 6 8 1 12 14 16 18 2 t 1 (b) Time series and predicted values 8 6 4 2 Real One step ahead 1 to 18 steps ahead Future 18 steps 2 4 6 8 1 12 14 Figure 6: Multiple steps ahead: (a) autocorrelation coefficients estimates, (b) predictions for series NN3_14. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.15/2
Case study: NN3 competition Table 2: Some characteristics of input selection and model construction. Time series Difference Num. inputs Inputs (lags) M 1 1 3 1, 2, 4 3 2 2 1, 3 3 3 2 1, 1 3 4 3 1, 2, 3 8 5 1 2 1, 2 6 6 3 2, 3, 4 3 7 1 1 2 8 1 2 2 9 1 2 2, 3 12 1 2 1, 6 5 11 1 1 2 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.16/2
Case study: NN3 competition 6 Series 1 55 5 45 4 5 1 15 1 Series 2 5 5 1 15 6 x 14 Series 3 4 2 5 1 15 1 Series 4 5 2 4 6 8 1 12 14 Figure 7: One and multi-step ahead forecasting for time series NN3_11 to NN3_14. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.17/2
Case study: NN3 competition 6 Series 5 5 4 3 5 1 15 8 Series 6 6 4 2 5 1 15 45 Series 7 4 35 3 5 1 15 1 Series 8 5 2 4 6 8 1 12 14 Figure 8: One and multi-step ahead forecasting for time series NN3_15 to NN3_18. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.18/2
Case study: NN3 competition 7 6 Series 9 5 4 3 2 5 1 15 2 15 Series 1 1 5 5 5 1 15 7 6 Series 11 5 4 3 2 5 1 15 Figure 9: One and multi-step ahead forecasting for time series NN3_19 to NN3_111. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.19/2
Conclusions and Future wors This wor presents a methodology for time series modeling. Statistical tools combined with novel methodologies provide adequate models. Objectives achieved: The study of the different tass that compose the methodology: from data pre-processing to model validation. The automatic selection of a suitable model structure; What needs to be improved: Initialization phase; Adding and pruning conditions. Thans for your attention. Ivette Luna iluna@cose.fee.unicamp.br A Constructive-Fuzzy System Modeling for Time Series Forecasting p.2/2