Introduction to Time Series (I)

Introduction to Time Series (I) ZHANG RONG Department of Social Networking Operations Social Networking Group Tencent Company November 20, 2017 ZHANG RONG Introduction to Time Series (I) 1/69

Outline 1 Time Series Algorithms 2 Control Chart Theory 3 Opprentice System 4 TSFRESH python package ZHANG RONG Introduction to Time Series (I) 2/69

Outline 1 Time Series Algorithms 2 Control Chart Theory 3 Opprentice System 4 TSFRESH python package ZHANG RONG Introduction to Time Series (I) 3/69

Time Series Definition and Methods Definition of Time Series A time series is a series of data points indexed in time order. Methods for time series analysis may be divided into two classes: Frequency-domain methods: spectral analysis and wavelet analysis; Time-domain methods: auto-correlation and cross-correlation analysis. Methods of Time Series Methods for time series analysis may be divided into another two classes: Parametic methods Non-parametic methods ZHANG RONG Introduction to Time Series (I) 4/69

Moving Average Moving Average Let {x i : i 1} be an observed data sequence. A simple moving average (SMA) is the unweighted mean of the previous w data. If the w-days values are x i, x i 1,...,x i (w 1),thentheformulais M i = 1 w wx 1 j=0 x i j = x i + x i 1 + + x i (w 1). w When calculating successive values, a new value comes into the sum and an old value drops out, that means M i = M i 1 + x i w x i w w. ZHANG RONG Introduction to Time Series (I) 5/69

Moving Average Figure: Moving Average Method for w =5 ZHANG RONG Introduction to Time Series (I) 6/69

Cumulative Moving Average Cumulative Moving Average Let {x i : i 1} be an observed data sequence. A cumulative moving average is the unweighted mean of all datas. If the w-days values are x 1,, x i,then CMA i = x 1 + + x i. i If we have a new value x i+1, then the cumulative moving average is CMA i+1 = x 1 + + x i + x i+1 i +1 = x i+1 + n CMA i i +1 = CMA i + x i+1 CMA i. i +1 ZHANG RONG Introduction to Time Series (I) 7/69

Weighted Moving Average Weighted Moving Average A weighted moving average is the weighted mean of the previous w-datas. Suppose P w 1 j=0 weight j =1withallweight j 0, then the weighted moving average is WMA i = wx 1 j=0 weight j x i j. ZHANG RONG Introduction to Time Series (I) 8/69

Weighted Moving Average ASpecialCase In particular, let {weight j :0apple j apple w 1} be a weight with w j weight j = for 0 apple j apple w 1. w +(w 1) + +1 In this situation, WMA i = wx i +(w 1)x i 1 + +2x i w+2 + x i w+1. w +(w 1) + +1 Figure: WMA weights w = 15 ZHANG RONG Introduction to Time Series (I) 9/69

Weighted Moving Average ASpecialCase Weighted Moving Average Suppose Total i = x i + + x i w+1, Numerator i = wx i +(w 1)x i 1 + + x i w+1, then the update formulas are Total i+1 = Total i + x i+1 x i w+1, Numerator i+1 = Numerator i + wx i+1 Total i, Numerator i+1 WMA i+1 = w +(w 1) + +1. ZHANG RONG Introduction to Time Series (I) 10/69

Exponential Weighted Moving Average Exponential Weighted Moving Average Suppose {Y t : t 1} is an observed data sequence, the exponential weighted moving average series {S t : t 1} is defined as ( Y 1, t =1 S t = Y t 1 +(1 ) S t 1, t 2 2 [0, 1] is a constant smoothing factor. Y t is the observed value at a time period t. S t is the value of the EMWA at any time period t. ZHANG RONG Introduction to Time Series (I) 11/69

Exponential Weighted Moving Average Moreover, from above definition, S t = [Y t 1 +(1 )Y t 2 + +(1 ) k Y t (k+1) ] +(1 ) k+1 S t (k+1) for any suitable k 2 {0, 1, 2, }. The weight of the point Y t (1 ) i 1. i is Figure: EMA weights k = 20 ZHANG RONG Introduction to Time Series (I) 12/69

Exponential Weighted Moving Average Exponential Weighted Moving Average Suppose {Y t : t 1} is an observed data sequence, the alternated exponential weighted moving average series {S t : t 1} is defined as ( Y 1, t =1 S t,alternate = Y t +(1 ) S t 1,alternate, t 2 Here, we use Y t instead of Y t 1. ZHANG RONG Introduction to Time Series (I) 13/69

Exponential Weighted Moving Average Figure: Exponential Weighted Moving Average Method for = 0.6 ZHANG RONG Introduction to Time Series (I) 14/69

Double Exponential Smoothing Double Exponential Smoothing Suppose {Y t : t 1} is an observed data sequence, there are two equations associated with double exponential smoothing: S t = Y t +(1 )(S t 1 + b t 1 ), b t = (S t S t 1 )+(1 )b t 1, where 2 [0, 1] is the data smoothing factor and trend smoothing factor. 2 [0, 1] is the ZHANG RONG Introduction to Time Series (I) 15/69

Double Exponential Smoothing Double Exponential Smoothing Here, the initial values are S 1 = Y 1 and b 1 has three possibilities: b 1 = Y 2 Y 1, b 1 = (Y 2 Y 1 )+(Y 3 Y 2 )+(Y 4 Y 3 ) 3 b 1 = Y n Y 1 n 1. = Y 4 Y 1, 3 Forecast The one-period-ahead forecast is given by F t+1 = S t + b t. The m-period-ahead forecast is given by F t+m = S t + mb t. ZHANG RONG Introduction to Time Series (I) 16/69

Double Exponential Smoothing Figure: Double Exponential Smoothing for = 0.6 and = 0.4 ZHANG RONG Introduction to Time Series (I) 17/69

Triple Exponential Smoothing Multiplicative Seasonality Triple Exponential Smoothing (Multiplicative Seasonality) Suppose {Y t : t 1} is an observed data sequence, then the triple exponential smoothing is S t = Y t +(1 )(S t 1 + b t 1 ), Overall Smoothing c t L b t = (S t S t 1 )+(1 )b t 1, Trend Smoothing Y t c t = +(1 )c t L, Seasonal Smoothing S t where 2 [0, 1] is the data smoothing factor, 2 [0, 1] is the trend smoothing factor, 2 [0, 1] is the seasonal change smoothing factor. ZHANG RONG Introduction to Time Series (I) 18/69

Triple Exponential Smoothing Multiplicative Seasonality Forcast The m-period-ahead forecast is given by F t+m =(S t + mb t )c (t L+m) mod L. Triple Exponential Smoothing Initial values are S 1 = Y 1, b 0 = (Y L+1 Y 1 )+(Y L+2 Y 2 )+ +(Y L+L Y L ), L c i = 1 NX Y L(j 1)+i, 8i 2 {1,, L}, N A j = j=1 P L i=1 Y L(j L A j 1)+i, 8j 2 {1,, N}. ZHANG RONG Introduction to Time Series (I) 19/69

Triple Exponential Smoothing Additive Seasonality Triple Exponential Smoothing (Additive Seasonality) Suppose {Y t : t 1} is an observed data sequence, then the triple exponential smoothing is S t = (Y t c t L )+(1 )(S t 1 + b t 1 ), Overall Smoothing b t = (S t S t 1 )+(1 )b t 1, Trend Smoothing c t = (Y t S t 1 b t 1 )+(1 )c t L, Seasonal Smoothing where 2 [0, 1] is the data smoothing factor, 2 [0, 1] is the trend smoothing factor, 2 [0, 1] is the seasonal change smoothing factor. The m-period-ahead forecast is given by F t+m = S t + mb t + c (t L+m) mod L. ZHANG RONG Introduction to Time Series (I) 20/69

Outline 1 Time Series Algorithms 2 Control Chart Theory 3 Opprentice System 4 TSFRESH python package ZHANG RONG Introduction to Time Series (I) 21/69

Definition of Control Chart Theory Control Chart The control chart is a graphical display of a quality characteristic that has been measured from a sample versus the sample number or time. Center Line: the average value of the quality characteristic Upper Control Limit (UCL) and Lower Control Limit (LCL): two horizontal lines. ZHANG RONG Introduction to Time Series (I) 22/69

3 Control Chart Simplest Control Chart 3 Control Chart Suppose that w is a sample statistic that measures some quality characteristic, the mean of w is µ w and the standard deviation of w is w.thenthecenterline, the upper control limit and the lower control limit becomes: UCL = µ w + L w Center line = µ w LCL = µ w L w where L is the distance of the control limits from the center line, expressed in standard deviation units. In particular, if L = 3, then it is the 3 control chart. ZHANG RONG Introduction to Time Series (I) 23/69

3 Control Chart Simplest Control Chart Figure: How the control chart works ZHANG RONG Introduction to Time Series (I) 24/69

The Cumulative Sum Control Chart CUSUM Control Chart Let x i be the i-th observation on the process {x i :1applei apple n}, {x i :1applei apple n} has a normal distribution with mean µ and standard deviation.thecumulativesumcontrol chart is calculated by, for all 1 apple i apple n, C i = ix (x j µ 0 )=C i 1 +(x i µ 0 ), j=1 where C 0 =0andµ 0 is the target for the process mean. If C i exceed the decision interval H, thentheprocessis considered to be out of control. The decision interval H is 3 or 5. ZHANG RONG Introduction to Time Series (I) 25/69

The Cumulative Sum Control Chart Data for the Cusum Example ZHANG RONG Introduction to Time Series (I) 26/69

The Cumulative Sum Control Chart The first 20 of these observations were drawn at random from a normal distribution with µ = 10 and standard deviation = 1. They are plotted on a Shewhart control chart. Figure: A Shewhart control chart for the data ZHANG RONG Introduction to Time Series (I) 27/69

The Cumulative Sum Control Chart Figure: Plot of the cumulative sum from column (c) in above table ZHANG RONG Introduction to Time Series (I) 28/69

The Cumulative Sum Control Chart Comparison to three-sigma control limit Di erence Three-sigma control limit: one or more points beyond a three-sigma control limit CUSUM control limit: it is a good choice when small shifts are important. ZHANG RONG Introduction to Time Series (I) 29/69

The Tabular or Algorithmic Cusum Let x i be the i-th observation on the process {x i :1applei apple n}, it has mean µ 0 and standard deviation.thestatisticsc + and C are computed as follows: C + i = max 0, x i (µ 0 + K)+C + i 1 C i = max 0, (µ 0 K) x i + C i 1 where C + 0 = C 0 = 0. K is the reference value, is calculated as K = µ 1 µ 0, where µ 1 = µ 0 + and =1. 2 If either C + i or C i exceed the decision interval H =5,the process is considered to be out of control. Here and H are parameters. ZHANG RONG Introduction to Time Series (I) 30/69

The Tabular or Algorithmic Cusum ZHANG RONG Introduction to Time Series (I) 31/69

The Tabular or Algorithmic Cusum Figure: CUSUM status charts for the above example ZHANG RONG Introduction to Time Series (I) 32/69

EWMA Control Chart Exponentially Weighted Moving Average Control Chart EWMA The exponentially weighted moving average is defined as z i = x i +(1 )z i 1, where 0 apple apple 1 is a constant and z 0 = µ 0.Here,µ 0 is the process target. In particular, z 0 = x. Moreover, z i = Xi 1 (1 ) j x i j +(1 ) i z 0, j=0 and the sum of their weights is one. That means Xi 1 (1 ) j +(1 ) i =1. j=0 ZHANG RONG Introduction to Time Series (I) 33/69

EWMA Control Chart Figure: Weights of past sample means ZHANG RONG Introduction to Time Series (I) 34/69

EWMA Control Chart Exponentially Weighted Moving Average Control Chart EWMA Control Chart If the observations x i are independent random variables with variance 2, then the variance of z i is 2 z i = 2 1 (1 ) [1 2i (1 ) 2i ]. The EWMA control chart is UCL = µ 0 + L Center Line = µ 0, LCL = µ 0 L s (2 ) [1 (1 )2i ], s (2 ) [1 (1 )2i ]. ZHANG RONG Introduction to Time Series (I) 35/69

EWMA Control Chart Exponentially Weighted Moving Average Control Chart EWMA Control Chart Note that the term [1 (1 ) 2i ]tendsto1asi tends to 1, then the simplified EWMA control chart is s UCL = µ 0 + L (2 ), Center Line = µ 0, s LCL = µ 0 L (2 ). ZHANG RONG Introduction to Time Series (I) 36/69

EWMA Control Chart ZHANG RONG Introduction to Time Series (I) 37/69

EWMA Control Chart Figure: The EWMA control chart for the above example ZHANG RONG Introduction to Time Series (I) 38/69

Moving Average Control Chart Moving Average Control Chart Suppose that the observations are {x 1,, x n },thenthemoving average of span w at time i is defined as M i = x i + x i 1 + + x i w+1. w The variance of the moving average M i is V (M i )= 1 w 2 ix j=i w+1 V (x j )= 1 w 2 ix j=i w+1 2 = 2 w. ZHANG RONG Introduction to Time Series (I) 39/69

Moving Average Control Chart Moving Average Control Chart Let µ 0 be the target value and the moving average control chart for M i are UCL = µ 0 + 3 p w, Center Line = µ 0, LCL = µ 0 3 pw. The control procedure would consist of calculating the new moving average M i as each observation x i becomes available, plotting M i on a control chart with upper and lower control limits, and concluding that the process is out of control if M i exceeds the limits. ZHANG RONG Introduction to Time Series (I) 40/69

Moving Average Control Chart ZHANG RONG Introduction to Time Series (I) 41/69

Moving Average Control Chart Figure: Moving average control chart with w = 5 ZHANG RONG Introduction to Time Series (I) 42/69

Multivariate Data Control Chart The Multivariate Process Monitoring and Control Multivariate Data Control Chart Hotelling T 2 Control Chart The Multivariate EWMA Control Chart Regression Adjustment Principal Components Method Partial Least Squares ZHANG RONG Introduction to Time Series (I) 43/69

Outline 1 Time Series Algorithms 2 Control Chart Theory 3 Opprentice System 4 TSFRESH python package ZHANG RONG Introduction to Time Series (I) 60/69

TSFRESH python package TSFRESH python package tsfresh is used to to extract characteristics from time series. Paper: Time Series Feature extraction based on scalable hypothesis tests Spend less time on feature engineering Automatic extraction of 100s of features ZHANG RONG Introduction to Time Series (I) 61/69

TSFRESH python package TSFRESH python package Let {x 1,, x n } be a time series, some features are max, min, median, mean µ, variance 2, standard deviation, range is maximum minus minimum skewness is the third standardized moment: nx xi µ 3 skewness =, i=1 kurtosis is the fourth standardized moment: nx xi µ 4 kurtosis =. i=1 ZHANG RONG Introduction to Time Series (I) 62/69

TSFRESH python package TSFRESH python package Let {x 1,, x n } be a time series, some features are absolute energy: E = P n i=1 x 2 i, absolute sum of changes: E = P n 1 i=1 x i+1 x i, aggregate autocorrelation: 1 n 1 nx `=1 1 (n `) 2 Xn ` (x t µ)(x t+` µ), t=1 autocorrelation: parameter is lag `, 1 (n `) 2 Xn ` (x t µ)(x t+` µ). t=1 ZHANG RONG Introduction to Time Series (I) 63/69

TSFRESH python package TSFRESH python package Let {x 1,, x n } be a time series, some features are count above mean, count below mean variance larger than standard deviation first location of maximum, first location of minimum last location of maximum, last location of minimum has duplicate, has duplicate max, has duplicate min longest strike above mean, longest strike below mean ZHANG RONG Introduction to Time Series (I) 64/69

TSFRESH python package TSFRESH python package Let {x 1,, x n } be a time series, some features are mean change: P n 1 i=1 (x i+1 x i )/n =(x n x 1 )/n mean second derivative central: 1 Xn 2 1 n 2 (x i+2 2 x i+1 + x i ) i=1 percentage of reoccurring data points to all data points percentage of reoccurring values to all values ratio value number to time series length sum of reoccurring data points sum of reoccurring values ZHANG RONG Introduction to Time Series (I) 65/69

TSFRESH python package Initialization of Time Series Let {x 1,, x n } be a time series, some initialization methods are, for 1 apple i apple n, y i = y i = y i = y i = x i mean({x i :1apple i apple n}), x i median({x i :1apple i apple n}), x i max min, x i (max min)/10, where max and min denotes the maximum and minimum value of the time series, respectively. ZHANG RONG Introduction to Time Series (I) 66/69

TSFRESH python package Example of Two Lists ZHANG RONG Introduction to Time Series (I) 67/69

TSFRESH python package Features of the Above Two Lists ZHANG RONG Introduction to Time Series (I) 68/69

Thank you for watching! ZHANG RONG zr9558@gmail.com zr9558.wordpress.com ZHANG RONG Introduction to Time Series (I) 69/69