Data Analysis, Statistics, Machine Learning

Transcription

1 Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien<st H2O.ai

2 Time Series Time series sta<s<cs invlve randm prcesses ver <me Spa<al sta<s<cs invlve randm prcesses ver space Bth invlve similar mathema<cal mdels When there is n tempral r spa<al influence, these bil dwn t rdinary sta<s<cal methds DO NOT USE OLS methds n tempral/spa<al data These require stchas<c mdels, nt OLS trend lines measurements at each <me/space pint are nt independent 1.0 Autcrrelatin Plt Sales Year Quarterly US Ecmmerce Retail Sales, Seasnally Adjusted Crrelatin Lag Cpyright 2016 Leland Wilkinsn

3 Stchas<c prcesses Up t nw, we ve been dealing with i.i.d. randm variables Independent. Iden<cally. Distributed. We assumed there was n rdering f thse randm variables Our mdels depended n randm errr plus systema<c effects Time series analy<cs deal with rdered randm variables We (usually) assume these variables are equally spaced acrss <me A variable at <me t i is predictable in part by anther variable at anther <me The simplest example f this type f behavir is called autregressive (AR) x t = φx t 1 + t 3 In this mdel each bserva<n at a given <me is a func<n f the previus bserva<n plus randm errr E[ t ]=0 E[ 2 t ]=σ 2 E[ s t ] = 0 fr all s = t Cpyright 2016 Leland Wilkinsn

4 Stchas<c prcesses Diagnsing a stchas<c prcess Crrelate a series with itself shi_ed backward by ne <me perid Crrelate the shi_ed series with itself shi_ed backward by ne <me perid And s n Here s an Autcrrela<n Func<n (ACF) Plt f white nise x t = t 4 Cpyright 2016 Leland Wilkinsn

5 Stchas<c prcesses Diagnsing an autregressive prcess Crrelate a series with itself shi_ed backward by ne <me perid Crrelate the shi_ed series with itself shi_ed backward by ne <me perid And s n Here s an Autcrrela<n Func<n Plt f an AR(1) prcess x t = φx t 1 + t 5 Cpyright 2016 Leland Wilkinsn

6 Stchas<c prcesses Mving Average (MA) prcesses In this mdel each bserva<n at a given <me is a func<n the previus errr Plus randm errr x t = θ t 1 + t Here s an Autcrrela<n Func<n Plt f an MA(1) prcess 6 Cpyright 2016 Leland Wilkinsn

7 Stchas<c prcesses Mving Average (MA) prcesses The θ parameter can be nega<ve x t = θ t 1 + t Here s an Autcrrela<n Func<n Plt f a nega<ve MA(1) prcess Nega<ve θ enhances high frequencies Psi<ve θ enhances lw frequencies 7 Cpyright 2016 Leland Wilkinsn

8 ACF Plts N<ce that withut an ACF plt, diagnsis f raw series is difficult White nise MA(1) 8 Cpyright 2016 Leland Wilkinsn

9 Stchas<c prcesses ARMA prcesses (Bx & Jenkins) We can mix these mdels An ARMA mdel lks like this x t = p φ i x t i + i=1 q θ j t j + t j=1 In mst cases, the cefficients f the terms decay expnen<ally S we d nt have t make p and q large fr mdeling mst series All the mdels we ve seen s far can include a cnstant We can als add trend t these mdels x t = α + βx t + p φ i x t i + i=1 j=1 q θ j t j + t 9 Cpyright 2016 Leland Wilkinsn

10 Stchas<c prcesses Seasnal prcesses Dependencies in the mdel can be acrss seasns A seasnal autregressive mdel lks like this x t = φx t s + t And a seasnal mving average mdel lks like this x t = θ t s + t Ecnmists lve this stuff They even mix stchas<c and classical mdels in the same equa<n Their gal is t accunt fr dependencies in the residuals in regressin mdels Here s an example f ne f their mdels Generalized Least Squares ˆβ =(X Σ 1 X) 1 (X Σ 1 Y) 10 Cpyright 2016 Leland Wilkinsn

11 11 Stchas<c prcesses Es<ma<ng ARMA mdels Are yu serius? This is a black art And usually yu want ARIMA instead f ARMA Which I haven t even tld yu abut Even a_er a semester curse in ARIMA mdels yu wn t be able t d it Yu have t learn hw t diagnse ACF plts And PACF plts, which I haven t even tld yu abut Yu have t knw when t difference yur series t achieve sta<narity Which I haven t even tld yu abut Leave this t the experts That brings us t the next tpic There s a simple mdel that des bener than fancy ARIMA fr many real frecasts It s called Expnen<al Smthing It includes seasnal effects as well Cpyright 2016 Leland Wilkinsn

12 Stchas<c prcesses Expnen<al Smthing We begin with the mving average smthing mdel Fr a pint at <me t (1 t n), a mving average smthed value is given by p ˆx t = 1 p i=1 x t i Sme cnsidera<ns: Our smthing es<mate is simply the average f the p previus values. The first p pints in the series are nt smthed. If each pint in the series has a randm cmpnent, we are averaging fixed and randm cmpnents f previus pints. In this case, the mdel smths nly p prir randm cmpnents (nt n). In ther wrds, the mdel ignres any randmness befre the previus p <me pints. If we presume nly randm errr gverns the prcess, we call the prcess a randm walk. If we believe the prcess is a randm walk, then we shuld set p = 1. If p = 1, the smth is just the previus bserva<n. If p = 1, we are assuming there is n mre infrma<n we can get ut f the data If p > 1, we are assuming we can eliminate the effects f the errrs by averaging them. 12 Cpyright 2016 Leland Wilkinsn

13 Stchas<c prcesses Expnen<al Smthing Nw g n t the weighted mving average smthing mdel ˆx t = 1 p p w i x t i i=1 Let s make these weights decline expnen<ally w i = p i And let s nrmalize them t add t 1 w i = p 1 1 p p p i This makes the expnen<ally weighted smthing mdel 13 Cpyright 2016 Leland Wilkinsn

14 Stchas<c prcesses The Expnen<ally Weighted Mving Average Mdel (EWMA) Here is the recursive frm f the expnen<ally weighted smthing mdel N<ce the ˆx t 1 n the right We assume 0 < α < 1 s things dn t explde ˆx t = αx t 1 +(1 α)ˆx t 1 This frmula gives us a recursive es<ma<n methd N fancy p<miza<n needed What we are ding here is prjec<ng frward lcal panerns in the series We culd cnsider this a sta<s<cal es<ma<n methd Or we culd just think f it as a determinis<c frward panern duplicatr 14 Cpyright 2016 Leland Wilkinsn

15 Time and Space Stchas<c prcesses The Hlt- Winters methd Nw it gets pwerful Hlt and Winters added trend and seasnality t EWMA H- W fits three types f trend mdels (nne, linear, mul<plica<ve) H- W des nt fit ther types f trend func<ns (althugh it culd be mdified t d s) H- W fits three types f seasnality (nne, addi<ve, mul<plica<ve) H- W can fit mre than simple sinusidal seasnality func<ns H- W des nt fit mre than ne type f seasnality in ne mdel (but it culd) H- W addi<ve linear mdels parallel specific ARIMA mdels H- W mul<plica<ve mdels d nt have ARIMA parallels Frecas<ng 15 Fit first half f series Extraplate t secnd half t get residuals Analyze residuals fr anmalies Frecast beynd end f series Cpyright 2016 Leland Wilkinsn

16 Stchas<c prcesses The Hlt- Winters methd 16 Cpyright 2016 Leland Wilkinsn

17 Stchas<c prcesses The Hlt- Winters methd Here is the H- W frecast fr the famus Bx- Jenkins Airline dataset hw passengers estimate / smth=.3, linear=.4,seasn=12, multiplicative=.5, frecast=10 And here is the frecast using ARIMA (0,1,0)(0,1,0) arima passengers lg difference difference / lag=12 estimate / q=1, qs = 1, seasn=12, backcast = 13,frecast=10 17 Cpyright 2016 Leland Wilkinsn

18 Stchas<c prcesses Seasnal decmpsi<n X11/12 (US Census), SABL (Cleveland, Bell Labs) Trend Seasn Residual 18 Cpyright 2016 Leland Wilkinsn

19 Crrela<ng Time Series Dn t d this I lve this site! hnp:// crrela<ns 19 Cpyright 2016 Leland Wilkinsn

20 30 Crrela<ng Time Series Series Plt Science Raw 2.5 Series Plt Science Detrended 2.0 SCIENCE SCIENCE Case Case Series Plt Suicide Raw 1000 Series Plt Suicide Detrended SUICIDE SUICIDE Case Case Cpyright 2016 Leland Wilkinsn

21 Crrela<ng Time Series CCF f Raw series vs. CCF f detrended Crss Crrelatin Plt Crss Crrelatin Plt Crrelatin Crrelatin Lag Lag Detrending desn t always get yu ut f the wds There can be secnd- rder ar<facts that influence crrela<n between series 21 Cpyright 2016 Leland Wilkinsn

22 Mul<variate analysis f <me series The cau<ns men<ned earlier apply t any analysis f <me series e.g., Clustering r Principal Cmpnents f <me series Need t difference t achieve sta<narity befre clustering First differences f a randm walk achieves sta<narity Other mdels require mre ex<c measures First 9 lags f a randm walk 22 Cpyright 2016 Leland Wilkinsn

23 Wait, there s mre But if yu insist n trying this stuff, yu d bener talk t a <me- series sta<s<cian r ecnmist But dn t ask the ecnmist t predict the ecnmy! 23 Cpyright 2016 Leland Wilkinsn