A Constructive-Fuzzy System Modeling for Time Series Forecasting

Transcription

1 A Constructive-Fuzzy System Modeling for Time Series Forecasting I. Luna 1, R. Ballini 2 and S. Soares 1 {iluna,dino}@cose.fee.unicamp.br, ballini@eco.unicamp.br 1 Department of Systems, School of Electrical and Computer Engineering, University of Campinas - Brazil 2 Institute of Economy, University of Campinas - Brazil A Constructive-Fuzzy System Modeling for Time Series Forecasting p.1/2

2 Summary Introduction; Time series analysis: data pre-processing; Input selection; General structure of a fuzzy rule-based model; Constructive learning; Case study: NN3 competition; Conclusions and future wors. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.2/2

3 Time series modeling Analysis Input selection Model Construction Adequacy Validation Trend Seasonality FNN-PMI C-FSM Residual analysis Forecasting Pre processing Figure 1: Time series modeling. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.3/2

4 Analysis and pre-processing Reduced data set of the NN3 competition; Stationarity: required for input selection; Seasonal and trend components: Econometric Views 2. - Quantitative Software, Trend component: series 1, 5 and 9; All series with no trend were transformed according to: z (m) = y (m) µ(m) σ(m) z : stationary version of the time series; y, = 1,...: th observation; µ(m) is the monthly average value and σ(m) is the monthly standard deviation. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.4/2

5 Input selection 1. FNN (False Nearest Neighbors): determines the minimum number of lags necessary to represent each pattern or state of the time series; 2. PMI (Partial Mutual Information): measure of information that each new variable x provides, taing into account an existing set of inputs Z. Given variables X e Y, PMI score between X and Y is defined by: where: PMI = 1 N N i=1 [ fx,y log (x i, y i ) ] e f X (x i )f Y (y i ) (1) x i = x i E(x i Z) e y i = y i E(y i Z) Z is the st of inputs already chosen. E( Z) is the conditional expected value; N is the number of input-output patterns. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.5/2

6 Input selection y = x 4 + e 1 y x = sen(2πt/t) + e 2 T = 2; t = 1,...,2; e 1 and e 2 are noisy signals with normal distribution, µ = and σ =, 1. Measure Value MI x Correlation.32 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.6/2

7 Input selection Number inputs Complexity Local minimum PMI score Threshold FNN NN3_12 PMI Lags Lags Figure 2: FNN. Figure 3: PMI. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.7/2

8 A general structure R 1 y 1 x x. R i y i. x g 1 ŷ. R M y M x g i g M Input space partition Figure 4: A general structure of a fuzzy-rule based system, composed by a total of M fuzzy rules. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.8/2

9 A general structure x = [x 1, x 2,..., x p] R p is the input vector at instant, Z + ; ŷ R is the estimate output; Given centers c i R p and covariance matrices V i i = 1,...,M, membership degrees g i (x ) are defined as: with α i, M i=1 α i = 1 and: g i (x ) = gi = α i P[ i x ] M α q P[ q x ] q=1 P[ i x 1 ] = (2π) p/2 det(v i ) 1/2 { exp 1 } 2 (x c i )V 1 i (x c i ) T (2) (3) A Constructive-Fuzzy System Modeling for Time Series Forecasting p.9/2

10 A general structure Each local model y i, i = 1,...,M is estimated by a linear one: y i = φ θ i T (4) where φ = [1 x 1 x 2...x p] and θ i = [θ i θ i1... θ ip ]. The output model ŷ is computed as: ŷ = M g i (x ) yi (5) i=1 A Constructive-Fuzzy System Modeling for Time Series Forecasting p.1/2

11 Constructive learning Initial structure M = 2, N patterns EM steps Initialization E step: g i is estimated given x and y posterior estimate h i ; for Npatterns h i = α i P(i x )P(y x, θ i ) M q=1 α qp(q x )P(y x, θ q ) No Add? Yes M = M + 1 EM steps M step: Model parameters are adjusted; No No Prune? Stop? Yes M = M 1 EM steps Adaptation Adding and pruning conditions are verified. α i = 1 N N h i (6) =1 Yes End A Constructive-Fuzzy System Modeling for Time Series Forecasting p.11/2

12 Second phase: Adaptation Adding a new rule: Assuming a normal input data distribution, with a confidence level equal to γ%: Ω is an i/o data set so that max i=1,...,m (P(i x Ω)) <.14 If N Ω > Then create a new rule. P(i x ) From normal distribution table: γ 73% z γ (1.73)/2 γ% x x z γ diag(vi ) x + z γ diag(vi ) Pruning a new rule: α i is proportional to the sum of all h i. Thus, the more times the rule is strongly activated, the higher its α i will be. If α i < α min, then the i th rule will be pruned. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.12/2

13 Case study: NN3 competition Reduced data set; Table 1: Global prediction errors for series NN3_12 and NN3_14. Series In sample Out of sample Out of sample 1 step ahead 1 step ahead 1 to 18 steps ahead NN3_12 = 4,..., 18 = 19,..., 126 = 19,..., 126 NN3_14 = 4,..., 97 = 98,..., 115 = 98,..., 115 Series M smape MAE smape MAE smape MAE (%) (u) (%) (u) (%) (u) NN3_ NN3_ A Constructive-Fuzzy System Modeling for Time Series Forecasting p.13/2

14 Case study: NN3 competition (a).2.1 r t t 1 (b) Time series and predicted values Real One step ahead 1 to 18 steps ahead Future 18 steps Figure 5: Multiple steps ahead: (a) autocorrelation coefficients estimates, (b) predictions for series NN3_12. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.14/2

15 Case study: NN3 competition (a).2.1 r t t 1 (b) Time series and predicted values Real One step ahead 1 to 18 steps ahead Future 18 steps Figure 6: Multiple steps ahead: (a) autocorrelation coefficients estimates, (b) predictions for series NN3_14. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.15/2

16 Case study: NN3 competition Table 2: Some characteristics of input selection and model construction. Time series Difference Num. inputs Inputs (lags) M , 2, , , , 2, , , 3, , , A Constructive-Fuzzy System Modeling for Time Series Forecasting p.16/2

17 Case study: NN3 competition 6 Series Series x 14 Series Series Figure 7: One and multi-step ahead forecasting for time series NN3_11 to NN3_14. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.17/2

18 Case study: NN3 competition 6 Series Series Series Series Figure 8: One and multi-step ahead forecasting for time series NN3_15 to NN3_18. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.18/2

19 Case study: NN3 competition 7 6 Series Series Series Figure 9: One and multi-step ahead forecasting for time series NN3_19 to NN3_111. A Constructive-Fuzzy System Modeling for Time Series Forecasting p.19/2

20 Conclusions and Future wors This wor presents a methodology for time series modeling. Statistical tools combined with novel methodologies provide adequate models. Objectives achieved: The study of the different tass that compose the methodology: from data pre-processing to model validation. The automatic selection of a suitable model structure; What needs to be improved: Initialization phase; Adding and pruning conditions. Thans for your attention. Ivette Luna iluna@cose.fee.unicamp.br A Constructive-Fuzzy System Modeling for Time Series Forecasting p.2/2