Nonstationary time series forecasting and functional clustering using wavelets

Transcription

1 Nonstationary time series forecasting and functional clustering using wavelets Application to electricity demand Jean-Michel POGGI Univ. Paris Sud, Lab. Maths. Orsay (LMO), France and Univ. Paris Descartes, France benbis Energy Demand Forecasting workshop Leuven, January 18, 2018 Joint works with Anestis ANTONIADIS (Univ. Grenoble, France & Univ. Cape Town, South Africa) Xavier BROSSAT (EDF R&D, France) Jairo CUGLIARI (Univ. Lyon 2, France) Yannig GOUDE (EDF R&D, France and Univ. Paris-Sud, Orsay, France)

2 Outline Industrial motivation 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

3 Outline Industrial motivation Electricity demand data Electricity demand forecasting Aim Functional time series 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

4 Electricity demand data Electricity demand data Electricity demand forecasting Aim Functional time series (a) Long term trend (b) Annual cycle (c) Weekly pattern (d) Daily pattern

5 Electricity demand data Electricity demand forecasting Aim Functional time series Electricity demand forecasting (general) Short-term electricity demand: a classical model Y t+1 = F (Y t, Y t k,..., Y t K ; X t, X t k,..., X t K ; C t) + ɛ(t) endogenous variables (instantaneous and lagged values of Y ) exogenous: meteorology (X) and calendar effects (C) As consumption habits depend hardly on the hour of the day, very often one model per instant is fitted. Data are measured every 30 minutes + forecasting the next day curve 48 models corresponding to each of the sampled instants of the day

6 Electricity demand data Electricity demand forecasting Aim Functional time series Electricity demand forecasting (general) Short-term electricity demand: an additive model Y t = K k=1 f 0,k(Y t k 48 ) + f 1(DayType t, Offset t) + 12 i=1 f2,i(tt 12)1 {M t =i} + f 3(ToY t) + f 4(t 15) + f 5(t)+ f 6(Cloud t) + f 7(T t) + f 8(W t) + f 9(θ t) + f 10(θ Min t where at time t: ) + f 11(θ Max t ) + ɛ t, Y t is the electric demand ToY t is the time of the year of observation t DayType t and Offset t are categorical variables indicating the type of day and the daylight saving time M t is the Month, t 15 = t1 {Tt 15} estimating a heating trend several lagged and smoothed variables related to temperature T t and θ t an exponential smoothing of T t, Cloud t and W t are the cloud cover and the wind

7 Electricity demand data Electricity demand forecasting Aim Functional time series Methods to design the forecasting model Short-term electricity demand forecast in literature A lot of methods are available to build prediction models [Weron (2007)] The most classical models, including those of the SARIMAX family, constitute an important baseline and are a favourite time series model They can achieve excellent results even if the price to pay is sometimes the complexity (a lot of parameters to estimate) and a certain difficulty to be adaptive. An interesting and conceptually simple extension of these models is to consider additive non linear models. Regression: Of course, today black box-type models forgetting the interpretation of the role of variables in favor of the sole objective of forecasting are particularly in vogue, thanks to the machine learning era Several types of methods coexist, but one that is most frequently used is undoubtedly neural networks [Park et al. (2011)], mainly used by engineers and computer scientists, with its last (complex) avatar: deep learning

8 Electricity demand forecast Electricity demand data Electricity demand forecasting Aim Functional time series Short-term electricity demand forecast in literature Time series analysis: sarima(x), Kalman filter [Dordonnat et al. (2009)] Machine learning [Devaine et al. (2010)] Similarity search based methods [Poggi (1994), Antoniadis et al. (2006)] Regression: edf modelisation scheme [Bruhns et al. (2005)], gam [Pierrot and Goude (2011)], Bayesian approach [Launay, Philippe and Lamarche (2012)] New challenges Market liberalization: may produce variations on clients perimeter that risk to induce nonstationarities on the signal. Development of smart grids and smart meters. But, almost all the models rely on a monoscale representation of the data.

9 Electricity demand data Electricity demand forecasting Aim Functional time series FD as slices of a continuous process [Bosq, (1990)] The prediction problem Suppose one observes a square integrable continuous-time stochastic process X = (X(t), t R) over the interval [0, T ], T > 0; We want to predict X all over the segment [T, T + δ], δ > 0 Divide the interval into n subintervals of equal size δ. Consider the functional-valued discrete time stochastic process Z = (Z k, k N), where N = {1, 2,...}, defined by X t 0 T t T + δ

10 Electricity demand data Electricity demand forecasting Aim Functional time series FD as slices of a continuous process [Bosq, (1990)] The prediction problem Suppose one observes a square integrable continuous-time stochastic process X = (X(t), t R) over the interval [0, T ], T > 0; We want to predict X all over the segment [T, T + δ], δ > 0 Divide the interval into n subintervals of equal size δ. Consider the functional-valued discrete time stochastic process Z = (Z k, k N), where N = {1, 2,...}, defined by X t Z 3(t) Z 4(t) Z 6(t) Z 1(t) Z 2(t) Z 5(t) t 0 1δ 2δ 3δ 4δ 5δ 6δ T + δ Z k (t) = X(t + (k 1)δ) k N t [0, δ) If X contents a δ seasonal component, Z is particularly fruitful.

11 Electricity demand data Electricity demand forecasting Aim Functional time series Prediction of functional time series Let (Z k, k Z) be a stationary sequence of H-valued r.v. Given Z 1,..., Z n we want to predict the future value of Z n+1. A predictor of Z n+1 using Z 1, Z 2,..., Z n is Z n+1 = E[Z n+1 Z n, Z n 1,..., Z 1]. Autoregressive Hilbertian process of order 1 The arh(1) centred process states that at each k, Z k = ρ(z k 1 ) + ɛ k (1) where ρ is a compact linear operator and {ɛ k } an H valued strong white noise. Under mild conditions, equation (1) has a unique solution which is a strictly stationary process with innovation {ɛ k } k Z. [Bosq, (1991)] When Z is a zero-mean arh(1) process, the best predictor of Z n+1 given {Z 1,..., Z n 1} is: Z n+1 = ρ(z n).

12 Overview Industrial motivation Electricity demand data Electricity demand forecasting Aim Functional time series key an appropriate distance between current and past situations. idea 1 Similar past causes produce similar future consequences. idea 2 Similar shapes form one class.

13 Wavelets to cope with fd domain-transform technique for hierarchical decomposing finite energy signals description in terms of a broad trend (approximation part), plus a set of localized changes kept in the details parts. Discrete Wavelet Transform If z L 2([0, 1]) we can write it as z(t) = 2 j 0 1 k=0 c j0,kφ j0,k(t) + 2 j 1 d j,k ψ j,k (t), where c j,k =< g, φ j,k >, d j,k =< g, ϕ j,k > are the scale coefficients and wavelet coefficients respectively, and the functions φ et ϕ are associated to a orthogonal mra of L 2([0, 1]). j=j 0 k=0

14 Outline Industrial motivation Prediction algorithm 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

15 Approximation and details Prediction algorithm In practice, we don t dispose of the whole trajectory but only with a (possibly noisy) sampling at 2 J points, for some integer J. Each approximated segment Z i,j (t) is broken up into two terms: a smooth approximation S i(t) (lower freqs) a set of details D i(t) (higher freqs) Z i,j (t) = 2 j 0 1 k=0 J 1 2 j 1 j 0,k φ j 0,k(t) + d (i) j,k ψ j,k(t) c (i) } {{ } S i (t) j=j 0 k=0 } {{ } D i (t) The parameter j 0 controls the separation. We set j 0 = 0. J 1 2 j 1 z J (t) = c 0φ 0,0(t) + d j,k ψ j,k (t). j=0 k=0

16 A two step prediction algorithm Prediction algorithm Step I: Dissimilarity between segments Search the past for segments that are similar to the last one. For two observed series of length 2 J say Z m and Z l we set for each scale j j 0: dist j(z m, Z l ) = ( 2 j 1 (d (m) j,k k=0 d (l) j,k )2 ) 1/2 Then, we aggregate over the scales taking into account the number of coefficients at each scale D(Z m, Z l ) = J 1 j=j 0 2 j/2 dist j(z m, Z l )

17 A two step prediction algorithm Prediction algorithm Step 2: Kernel regression Obtain the prediction of the wavelet coefficients Ξ n+1 = {c (n+1) J,k, d (n+1) j,k : k = 0, 1,..., 2 j 1} for Z n+1 n 1 Ξ n+1 = m=1 w m,nξ m+1 K ( D(Z n,z m) ) h w m,n = n n 1 K ( D(Z n,z m) ) m=1 h n Finally, the prediction of Z n+1 is obtained through the inverse DWT.

18 Daily prediction error Prediction algorithm Figure : Daily prediction error (in mapex100). Huge problem during the cold season Large prediction errors at quite regular frequency during warm season Need of corrections to deal with non stationarities.

19 Prediction algorithm Let us predict Saturday 10 September 2005 We use Antoniadis et al., (2006) prediction method with corrections to cope with non stationarity. Use the last observed segment (n = 9/Sept/2005) to look for similar segments in past. Construct a similarity index SimilIndex (using a kernel). Prediction can be written as n 1 Load n+1(t) = SimilIndex m,n Load m+1(t) m=1 First difference correction of the approximation part. Use of groups to anticipate calendar transitions.

20 SimilIndex date SimilIndex similar past similar future

21 Outline Industrial motivation Corrections to handle nonstationarity 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

22 Corrections to handle nonstationarity Corrections to handle nonstationarity On mean level base diff S n+1(t) = n 1 wm,nsm+1(t) m=1 S n+1(t) = S n(t) + n 1 wm,n (Sm)(t) m=2 Figure : Daily prediction error (in mapex100).

23 Corrections to handle nonstationarity Corrections to handle nonstationarity On groups by post-treatment Define new weights and renormalize. { ww,m if gr(m) = gr(n) w m,n = 0 otherwise gr(n) is the group of the n-th segment. 1 Deterministic groups: Calendar or Calendar transitions. 2 Groups coming from clustering analysis. (e.g. temperature curves) 3 Cross deterministic with clustering groups (e.g. calendar-temperature transitions). Figure : Daily prediction error (in mapex100).

24 Outline Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

25 Scale-oriented feature extraction Wavelet coherence based dissimilarity Segmentation of X may not suffice to satisfy the stationarity hypothesis. If a grouping effect exists, we may considered stationary within each group. Conditionally on the grouping, functional time series prediction methods can be applied. We propose a clustering procedure that discover the groups from a bunch of curves. We use wavelet transforms to take into account the fact that curves may present non stationary patterns. Two strategies to cluster functional time series: 1 Feature extraction (summary measures of the curves). 2 Direct similarity between curves.

26 Energy decomposition of the DWT Scale-oriented feature extraction Wavelet coherence based dissimilarity Energy conservation of the signal J 1 2 j 1 J 1 z 2 z J 2 2 = c0,0 2 + dj,k 2 = c0,0 2 + d j 2 2. For each j = 0, 1,..., J 1, we compute the absolute and relative contribution representations by j=0 k=0 j=0 cont j = d j 2 }{{} AC and rel j = d j 2 d. j j 2 }{{} RC They quantify the relative importance of the scales to the global dynamic. RC normalizes the energy of each signal to 1.

27 Schema of procedure Scale-oriented feature extraction Wavelet coherence based dissimilarity 0. Data preprocessing. Approximate sample paths of z 1 (t),..., z n(t) 1. Feature extraction. Compute either of the energetic components using absolute contribution (AC) or relative contribution (RC). 2. Feature selection. Screen irrelevant variables. [Steinley & Brusco ( 06)] 3. Determine the number of clusters. Detecting significant jumps in the transformed distortion curve. [Sugar & James ( 03)] 4. Clustering. Obtain the K clusters using PAM algorithm.

28 Toy example Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity Figure : On the left, some typical simulated trajectories of the sinus model (top panel), the far1 model (middle), and the far2 model (bottom). On the right, the mean scales energy absolute contribution by cluster.

29 Scale-oriented feature extraction Wavelet coherence based dissimilarity Clustering Feature Extraction Raw curves Abbreviation RC RAW Mean Global error (4.125) (4.834) Mean Rand Index (0.092) (0.109) Table : Indicators of the clustering quality. Mean values over the 100 replicates with standard deviation between parenthesis. Figure : Boxplots of the misclassification error (left) and the Adjusted Rand Index for the 100 replicates of the simulated data set. Clustering using PAM on the extracted features (left) and on the raw curves (right).

30 EDF data Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity Figure : French electricity power demand on autumn (top left), winter (bottom left), spring (top right) and summer (bottom right). Feature extraction: The significant scales are independent of the number of clusters. Significant scales are associated to mid-frequencies. The retained scales parametrize cycles of 1.5, 3 and 6 hours (AC).

31 Scale-oriented feature extraction Wavelet coherence based dissimilarity Figure : Number of clusters by feature extraction of the AC (top). From left to right: distortion curve, transformed distortion curve and first difference on the transformed distortion curve.

32 Scale-oriented feature extraction Wavelet coherence based dissimilarity (a) Curves (b) Calendar Figure : Curves membership of the clustering using ac based dissimilarity (a) and the corresponding calendar positioning (b).

33 Scale-oriented feature extraction Wavelet coherence based dissimilarity An alternative: function-based distance Distance based on wavelet-correlation between two time series Can be used to measure relationship between two functions The strength of the relation is hierarchically decomposed across scales without losing of time location Drawback: needs more computation time and storage (complex values)

34 CWT Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity Continuous WT Starting with a mother wavelet ψ consider ψ a,τ = a 1/2 ψ ( ) t τ a. The CWT of a function z L 2 (R) is, W z(a, τ) = z(t)ψa,τ (t)dt As for Fourier transform, a spectral approach is possible. S z(a, τ) = W z(a, τ) 2 wavelet spectrum W z,x(a, τ) = W z(a, τ)w x (a, τ) cross-wavelet transform

35 Wavelet coherence Scale-oriented feature extraction Wavelet coherence based dissimilarity R 2 z,x(a, τ) = W x,y (a, τ) 2 W x,x(a, τ) W y,y (a, τ), Based on the extended R 2 coefficient, we can construct an coefficient of determination between two wavelet spectrums ( 2 W WERz,x 2 0 z,x(a, τ) dτ) da = ( W 0 z,z(a, τ) dτ ). W x,x(a, τ) dτ da And obtain a dissimilarity based on it d(z, x) = JN(1 ŴER 2 z,x)

36 Wavelet coherence Scale-oriented feature extraction Wavelet coherence based dissimilarity We proceed as follows: Transform data z 1(t),..., z n(t) using the CWT and Morlet wavelet to obtain n matrices of size J N. Compute a dissimilarity matrix with the coherency based dissimilarity. Using PAM obtain clusters k = 8 clusters. Rand Index (AC, WER) = 0.26

37 Scale-oriented feature extraction Wavelet coherence based dissimilarity

38 Outline Industrial motivation 1 Industrial motivation Scale-oriented feature extraction Wavelet coherence based dissimilarity 5

39 Disaggregated electricity data Data set of professional customers Sampling rate: 30 minutes Period: 2009, 2010 and 2011 (only 6 month) 1 year 438 millions records 3.25 Go Figure : Aggregate demand (left) and individual demand (right) for 2010 (with infraday filtering).

40 Clients hierarchical structure and prediction Z t Z t+1 Figure : Hierarchical structure of N individual clients among K groups Z t,1 Z t,2. Z t,k Z t+1,1 Z t+1,2. Z t+1,k Z t: aggregate demand at t Z t,k : demand of group k at time t Groups can express tariffs, geographical dispersion, client class... Profiling vs Prediction We follow Misiti et al. (2010) to construct clusters of customers to better predict the aggregate.

41 2-steps strategy for building sequence of partitions 1st step: create a large number of super customers (K = 200) 2nd step: after aggregation, compute the aggregates

42 Prediction performance along sequences of partitions 1- Build sequences of consumer classes on 2009 data 2- Select by measuring the quality of daily KWF forecasts throughout 2010 Figure : Mean absolute prediction error (MAPE) as a function of the number of clusters for the baseline (without clustering) and two clustering variants

43 Aggregate signals are KWF-predictable Figure : Aggregate signals for 2 clusters Figure : Aggregate signals for 3 clusters

44 References [1] A. Antoniadis, X. Brossat, J. Cugliari, and J.-M. Poggi (2013) Functional Clustering using Wavelets. International Journal of Wavelets, Multiresolution and Information Processing, 11(1). [2] A. Antoniadis, X. Brossat, J. Cugliari, and J.-M. Poggi (2012) Prévision d un processus à valeurs fonctionnelles en présence de non stationnarités. Application à la consommation d électricité. Journal de la Société Française de Statistique, 153(2) : [3] A. Antoniadis, X. Brossat, J. Cugliari, and J.-M. Poggi (2014) Une approche fonctionnelle pour la prévision non-paramétrique de la consommation d électricité. Journal de la Société Française de Statistique, 155(2) : [4] M. Misiti, Y. Misiti, G. Oppenheim, and J. M. Poggi (2010) Optimized Clusters for Disaggregated Electricity Load Forecasting. REVSTAT Statistical Journal, 8(2): [5] J. Cugliari, Y. Goude and J.-M. Poggi (2016) Disaggregated Electricity Forecasting using Wavelet-Based Clustering of Individual Consumers. Proc. IEEE EnergyCon 2016, KU Leuven, 4-8 April 2016, 6 pages.