ABSTRACT. Many Intelligent Transportation System (ITS) applications under the umbrella of

Transcription

1 ABSTRACT SHEKHAR, SHASHANK. Recursive Mehods for Forecasing Shor-erm Traffic Flow Using Seasonal ARIMA Time Series Model. (Under he direcion of Dr. Billy M. Williams.) Many Inelligen Transporaion Sysem (ITS) applicaions under he umbrella of Advanced Traffic Managemen Sysems (ATMS) and Advanced Traveler Informaion Services (ATIS) call for he abiliy o anicipae fuure raffic condiions. Shor-erm raffic forecasing models play a cenral role in such applicaions. Previous research has shown ha a hree parameer SARIMA ime series model is well suied for forecasing shor-erm freeway raffic flow. However, pas applicaion has been in a saic form where he model has o be fied separaely for each locaion. This research implemens he seasonal ARIMA model in a ime-varying forma imparing plug and play capabiliy o he model. The properies of he SARIMA model for shor-erm raffic flow forecasing are discussed. Model sensiiviy o he parameers is shown. Three differen mehods (Kalman filer, recursive leas squares filer and leas mean square filer) have been invesigaed for making he model adapive. The sabiliy and robusness of he SARIMA model has been demonsraed. Resuls show ha all he hree adapive filers can be successfully used o make he model adapive. The use of Kalman filer for pracical implemenaion is recommended. Recommendaions for furher research in his regard are also presened.

2 RECURSIVE METHODS FOR FORECASTING SHORT-TERM TRAFFIC FLOW USING SEASONAL ARIMA TIME SERIES MODEL by SHASHANK SHEKHAR A hesis submied o he Graduae Faculy of Norh Carolina Sae Universiy in parial fulfillmen of he requiremens for he Degree of Maser of Science Civil, Consrucion and Environmenal Engineering Raleigh, NC 24 APPROVED BY: (Dr. Peer Bloomfield) (Dr. Nagui M. Rouphail) Chair of Advisory Commiee (Dr. Billy M. Williams)

3 Dedicaed o my parens. ii

4 BIOGRAPHY Shashank Shekhar was born on February 16 h 1979 in he ciy of Pana, India. He obained his Bachelor of Technology (Hons.) degree in Civil Engineering from Indian Insiue of Technology (IIT), Kharagpur, India in 22. He sared his maser s a NC Sae Universiy in Augus 22 and is working wih Dr. Billy Williams in he area of shor-erm raffic forecasing. He wishes o pursue a career in raffic engineering afer graduaion. iii

5 ACKNOWLEDGEMENTS I wan o express my sinceres graiude o Dr. Billy Williams for giving me he opporuniy and guiding me paienly hroughou he long course of his projec. I wan o hank him for sharing his wisdom and helping me ou whenever I was a loss. I wan o hank Dr Peer Bloomfield and Dr. Nagui Rouphail for serving on my advisory commiee and providing heir valuable suggesions. I am graeful o Dr Bloomfield for always aking he ime o answer my quesions and offer suggesions relaed o ime series analysis. I am indebed o Dr. John Sone, Dr. Joseph Hummer, Dr. Rouphail and Dr Williams for he knowledge gained in he field of ransporaion engineering hrough heir classes. My hank goes o Jianhua Guo for discussing and sharing his ideas on raffic forecasing. I am graeful o he Naional Science Foundaion for supporing his work hrough gran No I also wan o hank he Souheasern Transporaion Cener for providing financial suppor during my las semeser. Finally, I would like o hank he Unied Kingdom Highways Agency for providing he daa used in his research. iv

6 Table of conens TABLE OF CONTENTS LIST OF FIGURES...VIII LIST OF TABLES...IX 1 INTRODUCTION OVERVIEW MOTIVATION OBJECTIVES AND SCOPE ORGANIZATION SEASONAL ARIMA MODEL TIME SERIES MODELS SEASONAL ARIMA MODEL FOR TRAFFIC FLOW ARIMA MODEL ESTIMATION STATE SPACE REPRESENTATION SSF WITH PARAMETER AS STATE MAX (P,Q+1) SSF ADAPTIVE FILTERS KALMAN FILTER RECURSIVE LEAST SQUARES LEAST MEAN SQUARES SUMMARY v

7 Table of conens 5 LITERATURE REVIEW SIMPLE PREDICTION MODELS NON SEASONAL TIME SERIES NEURAL NETWORKS NON PARAMETRIC MODELS LAYERED MODELS ADAPTIVE TRAFFIC FORECASTING MODELS DATA AND METHODOLOGY BASIC DATASET ATTRIBUTES DATA AGGREGATION OUTLIER CORRECTED SERIES MOVING WINDOW METHOD PERFORMANCE MEASURES RESULTS SENSITIVITY OF THE SARIMA MODEL BLOCK PROCESSING ADAPTIVE FILTERING CONCLUSIONS RESEARCH RESULTS RECOMMENDATIONS FOR FUTURE RESEARCH REFERENCES APPENDIX A SENSITIVITY ANALYSIS vi

8 Table of conens APPENDIX B ML AND CLS ESTIMATES APPENDIX C DIFFERENT WINDOW-LENGTHS WITH CLS... 1 APPENDIX D COMPARISON OF ML AND FILTER ESTIMATES APPENDIX E MAE AND MAPE BY WEEK FOR ENTIRE SERIES APPENDIX F MEAN FLOW AND MAE BY TIME OF WEEK APPENDIX G - TESTS ON ABSOLUTE ERROR vii

9 Table of conens Lis of figures Figure 2-1: Fifeen minue flow rae agains ime for wo consecuive weeks Figure 2-2: Weekly difference of 15 minue flow Figure 2-3: Seasonal smoohing weighs for differen Θ Figure 2-4: Shor-erm smoohing weighs forφ =. 9 and differen θ Figure 2-5: Weighs for IMA (1,1) for differen Θ Figure 2-6: Weighs for ARMA (1,1) for differen θ and φ = Figure 6-1: Map of UK moorway sysem Figure 6-2: Number of non-missing values in fifeen minue inervals Figure 6-3: Flow-char of he selec-many algorihm Figure 6-4: Procedure for consrucing oulier correced series Figure 6-5: Illusraion of moving-window mehod... 6 viii

10 Table of conens Lis of Tables Table 6-1: Geomerical aspecs of sie... 5 Table 6-2: Raw daa series aribues Table 6-3: Aggregaed daa series aribues Table 6-4: Oulier correced series aribues Table 7-1: CLS esimaes for raw daa series Table 7-2: ML (and CLS) esimaes for oulier correced series Table 7-3: Comparison of MAE for differen mehods Table 7-4: MAPE for differen mehods Table 7-5: RMSE for differen mehods Table 7-6: LPI for differen mehods ix

11 Inroducion 1 INTRODUCTION Transporaion agencies around he world have made a sizeable invesmen for monioring raffic condiions on major freeway sysems. Examples of such sysems include: Freeway Performance Measuremen Sysem (PeMS) in California, Moorway Inciden Deecion and Auomaic Signaling (MIDAS) in UK, Archived Daa Managemen Sysem (ADMS) in Virginia and NaviGAor in Georgia. These sysems collec raffic sae daa in real ime using sensors like loop deecors, video cameras or radars. The daa are communicaed o a conrol cener for real ime raffic monioring and conrol purposes and hen archived for fuure use. Typical raffic sae daa consis of flow, speed and occupancy measuremens colleced over a ime inerval ranging from 2 seconds o 1 minue. Many Inelligen Transporaion Sysem (ITS) applicaions under he umbrella of Advanced Traffic Managemen Sysems (ATMS) and Advanced Traveler Informaion Services (ATIS) call for he abiliy o anicipae fuure raffic condiions. Dynamic conrol, informaion and decision suppor based on expeced fuure condiions will ransform he curren sysems from reacive o proacive ones. Under he above framework, raffic forecasing models will play a key role in supporing fuure ITS applicaions. These raffic forecasing models use real-ime and archived daa o provide he bes esimae of fuure raffic sae. 1

12 Inroducion 1.1 Overview Traffic forecasing models are needed for differen objecives and ofen hese differen objecives require informaion a differen ime-scales. For applicaions involving sysem conrol and monioring like ramp meering and inciden deecion, he predicion inerval should be in he range of 2 seconds o 1 minue. For applicaions involving sysem informaion like raveler informaion and roue guidance, a predicion inerval beween 5 minues o 3 minues is adequae. Differen modeling approaches are required for differen ime-scales. The area of shor erm raffic predicion has received considerable aenion hroughou he pas hree decades. A variey of models have been proposed for forecasing raffic. These models can be classified in many differen ways: Variable - according o he raffic variable modeled flow, speed, occupancy or heir combinaion. Predicion inerval ime scale from 3 seconds o 1 hour. Model ype - ime series, neural nework, non-parameric regression. Spaial - how daa from he adjacen locaions are used. Temporal - how daa from he pas are used. Complexiy single model o layered and class models. Applicaion inciden deecion, raveler informaion, ramp meering. Adapabiliy how models respond o change in sysem characerisics. 2

13 Inroducion The predicion problem for his work falls ino he 5 minue o 3 minue range, which can be mos useful from he sandpoin of sysem informaion disseminaion. The phrase shor-erm used hroughou his hesis refers o his ime scale. 1.2 Moivaion I is envisioned ha for implemenaion purposes, a good raffic predicion model should be a plug and play model. As more and more sysems sar using daa for real ime conrol purposes, i would be desirable ha he raffic predicion model be scalable, ineroperable and mainainable (in compliance wih he ITS guidelines). The model should be applicable anywhere wihou modificaion and should be able o adjus iself auomaically based on he changing characerisics of he sysem. I should be able o do his adjusmen in real-ime wih minimum man or machine inerference and wihou compromising is saisical accuracy. Previous research (Williams, 1999) has shown ha he seasonal auoregressive inegraed moving average (SARIMA) ime series model is a suiable model for forecasing 15 minue raffic flow. The SARIMA ime series model uses he Box-Jenkins mehod of idenificaion, esimaion and forecasing. The problem wih his mehod is ha i is a bach processing mehod in which he whole series is analyzed a once o find he model parameers. This means ha he model has o be esimaed for each sie separaely and in order o make he model parameers adjusable, bach processing of enire daa has o be repeaed. This also poses a problem when daa have ouliers and 3

14 Inroducion missing values. The main goal of his hesis is o invesigae mehods for making he SARIMA model adapive. The specific objecives of his hesis are oulined below. 1.3 Objecives and scope The cenral problem for his research is o invesigae mehods for making he seasonal ARIMA model adapable wih ime. The goal is o allow he model o be implemened in a recursive manner while mainaining is saisical opimaliy. The following broadly defined asks suppor his research goal: Invesigae he possible mehods for making he model adapive. Compare he differen bach processing mehods. Carry ou parameer sensiiviy of he model. Compare he performance of differen adapive mehods. Scope The analysis variable used in his hesis is he flow rae or raffic volume per ime inerval. However, he mehod can also be applied o model occupancy and speed. This work deals wih univariae ime series modeling (i.e. forecas a a locaion is modeled as a funcion of pas observaions a ha locaion only). Aggregaed flow rae daa a a15 minue level have been used for modeling. The daa used are from UK moorway sysem. Analysis was done using SAS sofware version 8.2. An Inel Penium IV machine wih 3.4 GHz processor was used for all he analyses. 4

15 Inroducion 1.4 Organizaion This hesis comprises of eigh chapers including his inroducion chaper. Brief summaries of he chapers follow. This hesis builds on he previous work done in he area of raffic flow forecasing. Chaper 2 presens a brief inroducion o ime series analysis and properies of he seasonal ARIMA model. The problem of parameer esimaion is discussed. The sae space represenaions used for ime series models are presened in Chaper 3. Chaper 4 gives a brief overview of adapive filers and he heory behind hem. Chaper 5 is a lieraure review of previous research done in he field raffic of forecasing. Differen modeling approaches o shor-erm raffic flow forecasing are menioned. This chaper also discusses four research works where adapive filering has been applied for raffic predicion. Chaper 6 describes he daa and mehods used in his hesis. The performance measures for comparing differen model resuls are defined. Chaper 7 gives he resuls of he differen analyses and Chaper 8 concludes wih a summary of oucomes and recommendaions for fuure work. 5

16 SARIMA model 2 SEASONAL ARIMA MODEL This chaper begins wih a brief inroducion o ime series analysis. The characerisics of he specific raffic forecasing model used in his hesis and he problem of parameer esimaion associaed wih i are discussed nex. 2.1 Time series models A univariae ime series { X } is a series of observaions of a variable over discree inervals of ime. Typically hese observaions are equally spaced in ime. { X } = x, x, x... x ) ( 1 2 3, Time series analysis involves modeling he series as a funcion of is pas observaions and errors (also referred as residuals, shocks or innovaions). Error ( e ) is he difference beween he observed value ( x ) and he forecas ( xˆ ). e = x xˆ I is generally assumed ha errors are independen and idenically disribued wih zero mean and Gaussian disribuion. The erm whie noise is used o describe such series which are normally disribued wih mean zero, varianceσ 2, and covariance (e, e -k ) = for k, i.e., {e } ~ WN (, σ 2 ). 6

17 SARIMA model The wo fundamenal building blocks of a linear univariae ime series model are he auoregressive (AR) model and moving average (MA) model. In an auoregressive model, he forecas is a funcion of is pas observaions, while in a moving average model he forecas is a funcion of is pas errors. Auoregressive model of order p, AR(p) Moving average model of order q, MA(q) xˆ φ 1 x 1 + φ2x = φ x xˆ θ 1 e 1 + θ 2e = θ e p p q q The order of he model is he highes erm presen in i. Convenionally he order of an AR polynomial is denoed by p and ha of a MA polynomial is denoed by q. I is no necessary for a model of order n o conain all he inermediae erms. For example, he model shown below has an order of 5 bu conains only wo auoregressive erms. I can be expressed as AR (5) (or AR ([2, 5]) more precisely). x + e = φ 2 x 2 + φ5x 5 In order o make noaional form simple, a special operaor called he backshif operaor (B) is used. The backshif operaor is defined by B j x = x. j Using he backshif operaor, he AR ([2,5]) model can be wrien as 5 x = φ B x + B x + e 2 2 φ5 2 5 ( 1 φ B φ B ) x = e 2 5 Or simply as, φ ( B ) x = e 7

18 SARIMA model Similarly, any moving average model of he form, 2 q x = e θ1e 1 θ 2e 2... θ qeq = (1 θ1b θ 2B... θ qb ) e can be denoed by a moving average polynomial, θ (B), as x = θ ( B) e. An auoregressive moving average (ARMA) process is a combinaion of auoregressive and moving average polynomials in a single equaion. An ARMA (p,q) model (order of AR polynomial as p and MA polynomial as q) is x = φ x + φ x... θ e φ p x p + e θ1e 1 θ2e 2 q q Or simply, φ ( B ) x = θ ( B) e An imporan concep in ime series analysis is he concep of a saionary series or saionariy. From he fundamenal heorem known as he Wold Decomposiion, i follows ha ARMA models can be applied only o ime series ha are saionary or ha can be made saionary. Wold s heorem saes ha any saionary ime series can be broken down ino wo pars - a sochasic par represened by an infinie moving average series and a deerminisic par. Saionariy (weak saionariy) is implied by he following wo condiions, expeced value of a series is he same, ( x ) cons. E = for all, covariance is dependen only on lag and no on ime, E [( x E( x ))( x E( x ))] f ( h) h + h = + for all. 8

19 SARIMA model In order o induce saionariy, classical ime series analysis employs differencing o ake care of level, rend and seasonaliy of a series. Level refers o he relaive magniude of he series which may be consan or change wih ime. If level is consan, he series is usually ransformed o a zero-mean series by subracing his consan or mean. A ime series is said o exhibi a rend if he change in level is consan wih ime. A ime series wih a rend can be made saionary by applying firs differencing (firs order ordinary differencing). A series is said o show seasonaliy if i exhibis a similar paern a regular inervals. A series wih seasonaliy can be made saionary by applying seasonal differencing. The firs order ordinary difference and seasonal difference can be expressed as ( B) x x x 1 = 1 and, x x = 1 s s ( B ) x ARMA models fied o series using ordinary differencing are called auoregressive inegraed moving average (ARIMA) models. In an ARIMA (p,d,q) model, he erm d denoes he order of differencing. An ARIMA model wih order of differencing d can be expressed as φ ( = θ d B )(1 B) x ( B) e Time series exhibiing repeaable paern are modeled hrough he use of seasonal differencing and seasonal parameers. Such models are called seasonal ARIMA or SARIMA for shor. The symbols S and D are used o denoe he lengh of seasonal cycle and he order of seasonal differencing respecively. 9

20 SARIMA model Where, A SARIMA (p,d,q)(p,d,q) S process is expressed as S d S D S ( B) Φ( B )( B) ( B ) x = θ ( B) Θ( B ) e φ 1 1, φ ( z p ) = φ z... φ 1 1, p z ( z P ) = Φ z φ Φ , θ p z ( z q ) = θ z... θ 1 1, q z Q () z = Θ z Θ Θ , and 2 e are uncorrelaed and normally disribued wih mean zero and variance σ, i.e., {e } ~ WN (, σ 2 ). Q z The ARIMA modeling approach given by Box and Jenkins is he mos popular approach in ime series analysis. I consiss of a se of analyical ools o do he hree seps of ime series analysis namely model idenificaion, model esimaion and model validaion. The model idenificaion or selecion sep consiss of making he series saionary and choosing an appropriae model form based on examinaion of he sample auocorrelaion funcion (ACF), sample parial auocorrelaion funcion (PACF) and/or sample inverse auocorrelaion funcion (IACF) of he saionary series. In he second sep, esimaion of model parameers is carried ou. In he hird sep, he errors are checked o verify he assumpion of whie noise. The hree seps are used ieraively o find he mos appropriae model. An informaion crierion such as he Akaike Informaion Crierion (AIC) or Schwarz Bayesian Crierion (SBC) is someimes uilized o selec he bes model from a se of compeing models. 1

21 SARIMA model 2.2 Seasonal ARIMA model for raffic flow A noable characerisic of raffic flow is ha i shows a very repeaable paern in ime. This paern can be seen o be repeaed over day and over week. Figure 2-1 shows he fifeen minue flow for he firs wo weeks of Sepember 22 on a locaion (deecor saion 4762A, M25 orbial) near London, UK. Week 1 15 minue flow rae Time week 2 15 minue flow rae Time Figure 2-1: Fifeen minue flow rae agains ime for wo consecuive weeks I is obvious from analyzing Figure 2-1 ha he series is no saionary. In order o ge a saionary series, differencing (weekly, daily or ordinary) or oher similar ransformaion (e.g. using daa for same ime of day) is necessary. Considering he srong weekly paern, aking seasonal differencing of a week seems more reasonable han aking a daily difference because flow on weekends differs considerably from flow 11

22 SARIMA model on weekdays. Figure 2-2 plos he weekly difference of flow for he wo week daa shown above. Weekly difference of flow 3 2 Weekly difference Time Figure 2-2: Weekly difference of 15 minue flow Time series analysis of raffic flow wih seasonal difference of a week was firs used by Williams (1999). Prior o ha, weekly differenced flow was used in combinaion wih a Kalman filer by Okuani and Sephenedes (1984). Almos all oher ime series sudies in shor-erm raffic forecasing lieraure used eiher ordinary differencing or piece-wise saionary subse of he daa. The model given by Williams (1999) conains a seasonal moving average erm along wih seasonal differencing. The model has a seasonal srucure of ARIMA(,1,1) S or IMA(1,1) S. For he non-seasonal par, ARMA (p,q) models wih order ( p + q) 5 were ried. Based on he SBC, i was found ha ARMA(1,1) model bes 12

23 SARIMA model described he non-seasonal par. The final model was hus a seasonal ARIMA (1,,1)(,1,1)s wih seasonaliy of one week. Comparison of his model (Williams, 1999) wih oher ime series models including Hol-Winers mehod and SARIMA wih seasonal differencing of one day, revealed he superioriy of he SARIMA (1,,1)(,1,1) S. The seasonal ARIMA (1,,1)(,1,1) S model can be wrien as S ( 1 φ B)( 1 B ) V = ( 1 θb)( 1 ΘB) e Where V denoes he 15 minue flow ime series e is whie noise error series S is he seasonal facor of one week for 15 minue flow series = 7 ( days ) 24 ( hrs) 6( min) /15( min) = 672 The above model can be undersood beer when is componens are wrien in erms of π-weighs. In a π-weighed form, he error is expressed as a funcion of pas observaions. Similarly, in a ψ-weighed form, he observaion is expressed in erms of he pas errors. The coefficiens used wih he observaion and he error are called π- weighs and ψ-weighs respecively. The wo represenaions for he SARIMA(1,,1)(,1,1) S model are given below: e S ( 1 φb)( 1 B ) = V = π 1 V 1 + π 2V 2 + ( 1 θb)( 1 ΘB)..., π-weighed represenaion V ( 1 θb)( 1 ΘB) = e = ψ 1 e 1 + ψ 2e 2 + S ( 1 φb)( 1 B )..., ψ-weighed represenaion 13

24 SARIMA model The SARIMA (1,,1)(,1,1) 672 model is a ype of double smoohing mehod. Boh he seasonal par, IMA(1,1) S, and he non-seasonal par, ARMA(1,1), imply a form of (exponenially weighed) smoohing. This becomes eviden when he wo processes are wrien in erms of heir π-weighs. IMA (1,1) S as exponenially weighed smoohing S S Expressed in π-weighed form, he IMA (1,1) S model, ( 1 B ) x = ( 1 ΘB ) e can be wrien as a linear combinaion of pas observaions as (1 B S ) x = e S ( 1 ΘB ) S S 2 2S Or ( 1 B )( 1 + ΘB + Θ B +...) x = e ( = 1+ r + r + r r for r < 1) Upon simplificaion, his gives an infinie series wih exponenially decaying coefficiens or weighs, 2 ( 1 Θ) x + Θ( 1 Θ) x + Θ (1 Θ) x... x ˆ = x e = S 2 S 3S + Le he expeced value of x a ime be denoed byη. Thus, s 2 ( Θ) x + Θ (1 Θ) x... = Θ 1 2 S 3S + η, The equaion can be wrien in a recursive form as: ( Θ) S + Θ S η = 1 η x 14

25 SARIMA model In order o ge he updaed smoohed value ( η ), he new observaion ( x ) is weighed by1 Θ, and he earlier value ( η S ) is weighed by Θ. The parameer Θ deermines how fas he weighs decay. If Θ is large, higher weigh is given o he pas observaions. The sum of he weighs is always equal o one. The weighing disribuion for differen values of Θis shown in Figure 2-3. Weigh Θ =.7 Θ =.8 Θ = Number of inervals Figure 2-3: Seasonal smoohing weighs for differen Θ ARMA (1,1) as a ype of exponenially weighed smoohing Expressed in π-weighed form, he ARMA (1,1) model, ( φb) x = ( 1 θb) e be wrien as linear combinaion of pas observaions as ( 1 φb) x ( ) = e 1 θb 2 2 Or ( 1 φ B )( 1 + θb + θ B +...) x = e ˆ = Or x x e = ( φ θ ) x + θ ( φ θ ) x + θ ( φ θ ) x... 1 can 15

26 SARIMA model Le he expeced value of x a ime be denoed byη. 2 Subsiuing θ ( φ θ ) x + θ φ θ ) x... 1 = 1 ( 2 + η, he equaion can be wrien in recursive form as: ( φ θ ) + 1 = η x θη In order o ge he updaed smoohed value ( η ), he new observaion ( x ) is weighed by ( ) θ φ, and he earlier value ( η 1 ) is weighed byθ. The ARMA (1,1) model has wo parameers, he auoregressive parameer, φ, and he moving average parameer, θ. The MA parameer deermines how fas he weighs decay and he AR parameer deermines he sum of he weighs. The weighing disribuion for ARMA (1,1) model is shown for differen θ (Figure 2-4). θ =.1 θ =.3 θ =.5 Weigh Number of inervals Figure 2-4: Shor-erm smoohing weighs forφ =. 9 and differen θ From Figure 2-3 and Figure 2-4, i can be seen ha he ARMA (1,1) model acs in a fashion similar o IMA(1,1), excep ha he sum of weighs is deermined by he AR coefficien (which is generally less han 1). If φ is equal o one, hen he AR operaor 16

27 SARIMA model simplifies o a differencing operaor and is equivalen o IMA(1,1). ARMA(1,1) is an economical model which can give similar weighs as some higher order AR models, despie having only wo degrees of freedom. When used in he recursive form, he wo models need only he las observaion. The recursive weighing paern for he wo processes are shown below. Weigh Θ =.7 Θ =.8 Θ = x() η() Figure 2-5: Weighs for IMA (1,1) for differen Θ θ =.1 θ =.3 θ =.5 Weigh x() η() Figure 2-6: Weighs for ARMA (1,1) for differen θ wih φ =.9 17

28 SARIMA model Inerpreaion of he SARIMA model The seasonal ARIMA model has a simple ye elegan srucural inerpreaion. The seasonal par provides an exponenially smoohed hisorical average of raffic flow series (Williams and Hoel, 23). This hisorical average can be regarded as a combinaion of flow observaions over pas several weeks for each 15 minue inerval in a week (i.e. 672 inervals). Once a new observaion comes in, his hisorical average is adjused based on Θ (higher Θ means less weigh for he new observaion). Thus he seasonal par akes care of hisorical informaion only. The informaion abou recen raffic condiions is aken ino accoun by he non-seasonal ARMA (1,1) par. I simply smoohens he difference of recen flow values from heir hisorical average. Thus raffic flow is forecased using a combinaion of seasonal smoohing and shor-erm smoohing. Parameer esimaes from differen sies (Williams, 1999) reveal ha Θ varies beween.7 and.9 (which can be inerpreed as don rus he curren values much o updae he hisorical average as hey are corruped by local noise ) φ is very sable around.9 andθ varies from.2 o.5, (which can be inerpreed as he difference of curren raffic from is hisorical average is similar o he differences observed in he las couple of inervals ) Figure 2-3 and Figure 2-4 are ploed keeping hese values in mind. The figures show ha for he shor-erm par, he weighs before wo inervals are almos negligible. These wo inervals ranslae ino 3 minues, so effecively he shor-erm par akes ino accoun condiions in las half hour only. This also explains why someimes he AR(2) model performed slighly beer han he ARMA(1,1) model (Williams, 1999). 18

29 SARIMA model 2.3 ARIMA model esimaion Once a suiable model form has been idenified, he nex sage is o ge he parameer esimaes of he model. There are basically wo differen approaches for doing parameer esimaion in ime series analysis, leas squares esimaion and maximum likelihood esimaion. In leas squares esimaion, he cos funcion o be minimized is simply he sum of squared errors. In maximum likelihood (ML) mehod he likelihood funcion is maximized in order o obain he parameer esimaes. For performing maximum likelihood esimaion, he probabiliy disribuion funcion of he model is assumed o follow a known disribuion. The likelihood of a se of daa is he probabiliy of obaining ha paricular se of daa, given is probabiliy disribuion. The philosophy behind maximum likelihood esimaion is o find a se of parameers which maximize he likelihood of observing he daa o which model is being fied. These esimaes are called maximum likelihood esimaes (MLEs). The MLEs have he desirable propery ha hey give unbiased minimum variance esimaes asympoically. In ime series analysis, he errors are generally assumed o follow a normal disribuion. The likelihood funcion is creaed using all he observaions available up o a paricular ime. Afer his a non-linear opimizaion algorihm is used o maximize he likelihood funcion wih respec o he parameer space (Shumway and Soffer, 2). Since he likelihood funcion is a monoonic increasing funcion, he logarihm or log of likelihood is more convenien o use. So in pracical applicaions, he maximum likelihood esimaes are obained by maximizing he log of likelihood funcion. 19

30 SARIMA model Based on he reamen of iniial values, esimaion mehods are ermed as condiional or uncondiional. If he iniial errors are fixed as zero and iniial observaions are also regarded as rue values, hen he esimaion is said o be condiional. If no assumpion is made abou he iniial condiions hen esimaion is said o be exac or uncondiional. Therefore if he uncondiional likelihood funcion is maximized hen he esimaes are said o be uncondiional or exac maximum likelihood esimaes. Similarly if uncondiional sum of squares is minimized, hen he esimaes are said o be uncondiional leas squares esimaes. In general, uncondiional esimaes always have lower error variance han he corresponding condiional esimaes. However he downside wih uncondiional esimaion is ha i is compuaionally more demanding because iniial values are no fixed and need o be deermined along wih he parameers. The difference beween condiional and uncondiional esimaes is more pronounced for small sample sizes. As he sample size increases, he difference beween he esimaes from wo approaches diminishes. The classical esimaion approach in ime series analysis is essenially a block esimaion approach. Esimaion is done for a finie se of archived daa and parameers remain unchanged (unil esimaion is done using anoher finie se of daa). As menioned in chaper 1, i would be useful if he model parameers could change coninuously wih ime in order o keep rack of saisical variaion in he daa. In oher words i is desirable ha he esimaion approach be an adapive or recursive one insead of he radiional block mode approach. The main goal of his sudy is o find ou ways for doing adapive esimaion of he SARIMA (1,,1)(,1,1) 672 model parameers. 2

31 SARIMA model A relaed issue is he difference beween leas squares esimaes and maximum likelihood esimaes a various lenghs of archived ime series daa. If block esimaion needs o be done, hen how much daa should be used for esimaion and in wha way do he leas squares esimaes and he maximum likelihood esimaes differ from each oher? Leas squares opimizaion is relaively simple and fas. On he oher hand, alhough exac MLE gives beer esimaes, is calculaion is very demanding compuaionally. This problem of doing exac maximum likelihood esimaion for raffic daa series is complicaed by he fac ha a) he series is very long, b) here can be missing values and c) due o he seasonal cycle being one week, a 672 lengh vecor of iniial unknowns has o be deermined. Tradiionally exac MLE is recommended in ime series lieraure because hey are mosly used in economerics where daa series are no very long. For he paricular problem of shor-erm raffic model esimaion, he series is a very long one. Never he less, here are some advanages wih his model oo, a) he parameer values lie in a narrow range, b) he model is no very sensiive if parameers lie in his range and c) a good guess of saring parameer values is available. The differen echniques in he realm of adapive filering heory are based on leas square minimizaion. Once adapive echniques can be successfully applied, one needs o compare heir performance in erms of compuaional complexiy and saisical opimaliy. The heory and algorihms behind differen adapive esimaion echniques are invesigaed in chaper 4. Before adapive echniques can be applied, i is necessary o represen a model in he general form of sae space equaions. Chaper 3 discusses how he SARIMA(1,,1)(,1,1) S model can be wrien in sae space form. 21

32 Adapive filers 3 STATE SPACE REPRESENTATION Before adapive filering echniques can be applied, i is necessary o wrie he model in a sae space form. A model is said o be in sae space form if i can be described by a se of sae space equaions. The sae space equaions consis of wo equaions, he measuremen or observaion equaion and he sae ransiion equaion. These wo equaions connec he inpu, oupu and sae of a sysem (Balakrishnan, 1987). The sae of a sysem can be hough of as variables or properies required o describe a sysem. From conrol sysems perspecive, hey are usually he parameers of he model which canno be observed or measured direcly. The variables ha can be observed direcly are he oupu. A process migh also have some inpu. The observaion equaion relaes he sae, inpu and oupu of he sysem. The sochasic variaion of he sae wih ime is described by he ransiion equaion. The general form of sae space equaions and is componens are given below. 22

33 Adapive filers The sae space equaions are Observaion equaion y = Z α + d + ε Transiion equaion = Tα 1 η α + where α sae vecor m x 1 y observaion (oupu)vecor n x 1 d inpu vecor n x 1 ε observaion noise vecor n x 1 η ransiion noise vecor m x 1 Z observaion marix n x m T ransiion marix m x m H obs. noise cov marix n x n Q sae noise cov. marix m x m and observaion noise is iid (, ) ransiion noise is iid ( ),Q, ( ) = H, ( ) = E ε Var ( ε ) = H E η Var ( η ) = Q The sysem marices Z, d and T can eiher be consan or ime-varying. If hey do no change wih ime, he sysem is said o be ime-invarian. If he marices change wih ime, he sysem is said o be ime-varian. Based on he inerpreaion of sae variable, numerous sae space represenaions are possible for he SARIMA ime series model. However, wo ypes of represenaion are mos common in lieraure. The firs represenaion can be said o be more prevalen from he sandpoin of conrol sysems and signal processing heory. In his represenaion, he saes are simply he parameers (, φ, θ,θ) µ of he SARIMA model. The second ype of represenaion is more prevalen in saisical ime series heory. This represenaion is commonly referred o as he max (p,q+1) represenaion of ARIMA process. 23

34 Adapive filers 3.1 SSF wih parameer as sae The SARIMA model wih he parameer as sae vecor can be expressed as S S ( 1 φ B)( 1 B ) V = ( 1 φ) µ + ( 1 θb)( 1 ΘB ) e Le r represen he seasonally differenced flow series r S ( B ) V = V V S = 1 Then he SARIMA model can be wrien as S ( 1 φ B) r = ( 1 φ) µ + ( 1 θb)( 1 ΘB ) e Rearranging he erms, and subsiuing c (or consan ) = ( 1 φ)µ r + e = c + φ r 1 θ e 1 Θe S + θ Θe S 1 SARIMA model can be wrien in sae space form as r + [ 1 r 1 e 1 e S ][ c φ θ Θ ] + θ Θe S 1 e = If he sae vecor is assumed o be random walk, we ge a valid sae space represenaion of he SARIMA model. y = Z α + d + ε r = [ 1 r 1 e 1 e S ][ c φ θ Θ ] + θ Θ e S 1 + e = 1 [ ] [ ] [ ] α + Tα η c φ θ Θ = c 1 φ 1 θ 1 Θ 1 + η1 η2 η3 η4 α =[ ] c Θ y = d = 1Θ 1e S 1 ε = η φ θ 4 x 1 r 1 x 1 θ 1 x 1 e 1 x 1 = η 4 x 1 Z =[ r e e ] 1 1 x S T = I 4 4 x 4 H = Var ( ) 1 x 1 e Q = ( ) Var η 4 x 4 24

35 Adapive filers One of he basic assumpions in his hesis is ha he above sae-space represenaion can be used for he SARIMA (1,,1)(,1,1) 672 model. For simpliciy, he non-linear erm has been modeled as an inpu and is direc effec on parameers has been ignored. Empirical evidence (Chaper 7) suggess ha wih he use of above represenaion, he filers can rack he parameers adequaely and do no seem o cause any problems. This may be due o he inheren sabiliy and insensiiviy of he SARIMA model form and he ype of daa used. The heoreically correc sae-space represenaion for a ime series model is presened in he following secion. 25

36 Adapive filers Max (p,q+1) SSF This sae space represenaion of an ARIMA model is used in ime series lieraure for doing exac maximum likelihood esimaion hrough predicion error decomposiion mehod. This represenaion is known as he max (p,q+1) sae space represenaion of an ARMA model (Gardner e al, 198). The name comes from he lengh of sae vecor used. The lengh of he sae vecor is he maximum of p and q+1, where p is he auoregressive order and q is he moving average order. Sae space equaions for an ARMA(p,q) process, wih ) 1, max( + = q p r q q p p e e e e x x x x = θ θ θ φ φ φ Z y α = Observaion equaion e T + R = 1 α α Transiion equaion x y = [ ] r Z = 1. 1 where α is a sae vecor of lengh r, obeying he above ransiion equaion wih r r r r T = φ φ φ φ and = r r R θ θ The above form can be applied o series wih missing values (Jones, 198) and for any general SARIMA model (Kohn and Ansley, 1986). The SARIMA model wih a

37 Adapive filers seasonal cycle of week applied o fifeen minue flow has s = 672. In his case, he lengh of he sae vecor, r is max ( + Φ, θ + Θ + 1) long vecor makes hings complicaed. φ which is equal o 674. Dealing wih such a As menioned before, his sae space represenaion is used in ime series heory for maximum likelihood esimaion and forecasing. The Kalman filer is used for he recursive consrucion of he exac likelihood funcion. Once he likelihood funcion is available, i is maximized wih respec o he parameer space by using non-linear opimizaion mehods a each ime-sep. This approach is known as he predicion error decomposiion approach (Harvey, 1993). Due o he complicaed model form and ime-consuming opimizaion procedure required a each sep, his approach was no used direcly. I was hough bes o use his mehod indirecly hrough he PROC ARIMA procedure in SAS. The maximum likelihood mehod of PROC ARIMA is based on he predicion error decomposiion approach and uses Kalman filer for his purpose inernally (SAS Insiue Inc, 1999). The moving-window approach (described in Chaper 7) can be regarded as an indirec use of his mehod. 27

38 Adapive filers 4 ADAPTIVE FILTERS A filer is anoher erm for an esimaor. In general, a filer is a daa processing algorihm used o exrac informaion abou a quaniy of ineres from a noise-corruped process. In many applicaions, esimaion of model parameers is ypically done wih a block of daa as in a bach process. Adapive filering can be hough of as a real-ime or recursive process in which model parameers are updaed wih he arrival of new observaion a each ime sep. Hereafer he erms adapive, recursive, online or real-ime processing will be used inerchangeably and in he above meaning. By applying adapive filering o he SARIMA model, i is hoped ha he parameers can be made ime-varying o rack he saisical variaion of he process and provide beer forecass. The ime series model used in his hesis is a linear model, so a linear filer can be used. A filer is said o be linear if he principle of superposiion is valid and he oupu is a linear funcion of he inpu. Typically he cos funcion which is minimized by hese filers is he error sum of squares. Before presening he individual filers, i is worhwhile o discuss he underlying concep of he linear filer heory and he Weiner- Hopf equaion. This gives a beer insigh o he problem since all filers uilize his equaion in one form or he oher. The filering problem can be regarded as a ype of opimizaion problem in which a cos funcion is minimized o ge suiable parameer esimaes. This cos funcion is he 28

39 Adapive filers error sum of squares and all he adapive filers minimize his cos funcion. This cos funcion is a quadraic funcion wih unique minima a he boom-mos poin of is convex surface in he n-dimensional space, (where n is he number of unknown parameers). In ime-varying form, his poin may be locaed a he same locaion hroughou or may change wih ime. Being a quadraic funcion, only he firs order and second order saisics are required o locae his opimal poin a each ime sep. Le he cos funcion of he sum of squared errors be denoed by C(w) (where w is he ime varying weigh or parameer vecor). The firs order differeniaion of C(w) wih respec o w a ime is called he gradien, g and he second order differeniaion of C(w) wih respec o w is called he Hessian, funcions of he correlaion marix of he inpu vecor, H. The gradien and Hessian are R and he cross-correlaion vecor beween he inpu and observaion, p as given below (Haykin, 22): C( w) g = = 2 p + 2Rw w 2 C( w) H = = 2 w R 2 R correlaion marix of he inpu vecor. p cross-correlaion marix beween he inpu vecor and he response. 29

40 Adapive filers vecor. If he lengh of he weigh vecor is n, hen R is a n x n marix and p is a n x1 Since he cos funcion is quadraic, i can be expressed using Taylor series expansion o he second order as 1 C( w) = C( w ) + ( w w ) g + ( w w ) H ( w w ) 2 Differeniaing he above equaion wih respec o w and seing i o zero gives w + 1 = w H 1 g Subsiuing he value of H and g, he updaed weigh vecor can be wrien as w p + 1 = = R R 1 p The above equaion is popularly known as he Weiner-Hopf equaion. All he linear adapive filers seek he soluion of his equaion, which resuls in he saisically opimal weigh vecor. Based on heir underlying approach, linear adapive filers can be broadly classified ino wo families (Haykin, 22). One is he recursive leas squares approach and he oher is he sochasic gradien approach. The recursive leas squares approach is based on second-order heory (boh Hessian and gradien) while he sochasic gradien approach is based on firs order heory (gradien only). In each family here are numerous varians and each mehod has is own meris and drawbacks. In his hesis, hree differen adapive filering echniques are used for SARIMA model parameer esimaion. A brief descripion of hese hree filers he Kalman filer (KF), Recursive Leas Squares (RLS) filer and leas mean squares (LMS) filer follows. 3

41 Adapive filers 4.1 Kalman filer Kalman filer is an algorihm ha provides an efficien recursive soluion o he leas squares problem. The filer reains informaion abou he firs momen (mean) and he second momen (covariance) of he sae vecor based on all he observaions up o ime. Afer each new observaion, he mean and he covariance of he sae vecor are updaed. Iniial condiions for applying Kalman filer. Suppose a process can be represened by he following sae space equaions: Observaion equaion y = Z α + d + ε E( ε ) = and Var ( ε ) = H Transiion equaion α + Rη = Tα 1 E( η ) = and Var ( η ) = Q The observaion error, ε and he sae error, η are assumed o whie noise and uncorrelaed wih each oher and wih he observaion, y. Le a denoe he known iniial esimae of he sae vecor, a = E( α ) Le P denoe he known iniial covariance marix of he sae: P = E[( α E( α))( α E( α)) ] 31

42 Adapive filers Since he filer is a recursive one, suppose a ime -1, a -1 and P -1 are given. The filer can be hough o consis of wo sages, firs one is he predicion sage (before he new observaion is in) and he second one is he correcion sage (afer he new observaion comes in). Predicor Equaions (before observaion a ime is available) Predicion of he sae: a T a 1 = 1 Predicion of he sae covariance: P T + R Q R 1 = T P 1 Once he new observaion is available, he error or innovaion is v = Z α a ) + ε ( 1 and he MSE of he innovaion is F Z + H = ZP 1 Correcor equaions (afer observaion a ime is available) Kalman gain: K = P 1 Z F 1 = Z P P 1 1 Z Z + H Updae of sae esimae: a + K v = a 1 Updae of sae covariance: P = P 1 KZP 1 32

43 Adapive filers 4.2 Recursive Leas Squares Recursive Leas Squares filer (RLS) can be derived as a special case of he more general Kalman filer (Sayed and Kailah, 1994). The difference beween he wo filers lies in he reamen of he process sae. The Kalman filer reas sae propagaion as a sochasic process while RLS models sae as a deerminisic process. The sae ransiion equaion for RLS filer can be wrien as: α λ α 1/ 2 = 1 The erm λ is referred o as he forgeing facor. The forgeing facor makes he weigh vecor adapable o recen daa by forgeing daa in he disan pas. The value of λ lies beween and 1 and generally has a value near.99. I can be seen ha he 2 ransiion equaion of RLS ( α 1/ = λ α 1 ) differs from ha of Kalman filer in wo aspecs. Firs, he sae is unforced which means ha here is no noise. Second, he 2 ransiion marix is fixed and does no vary wih ime ( T = λ 1/ ). 1 The memory of he filer is given by. The memory of he filer deermines 1 λ how much of pas daa are used for deermining he weighs. For λ =.99, he memory is 1, which means ha daa before 1 lags have no conribuion in deerminaion of he parameers. If λ is aken as 1, hen he problem reduces o ha of sandard leas squares where he memory is infinie and all daa are used. When λ is no one, he cos funcion is equivalen o he sum of exponenially weighed squared errors. 33

44 Adapive filers In order o smooh ou he flucuaions due o observaion noise, a regularizing erm is used in he cos funcion along wih exponenially weighed squared errors. The regularizing erm is especially useful when he filer has been sared and daa lengh is less han he memory. The effec of his erm decays wih ime. This erm can be wrien as: δλ α ( ) α( ), where δ is he regularizaion parameer. The cos funcion o be minimized is 2 C( ) = λ e( ) + δλ α ( ) α( ) i = 1 The equaions for he RLS filer are given below: Observaion equaion: y = Z α + d + ε 2 Sae equaion: α 1/ = λ α 1 Weigh updae: α = α 1 + P Z 1 ε λ + ZP Z 1 Covariance updae: P = P 1 P Z 1 ZP 1 λ + ZPZ λ Iniial Condiions: If α and P are unknown, hen le α = and P = δ 1 I The gain, k, of he filer a each ime sep is given by, k P Z 1 = λ + ZP Z 1 34

45 Adapive filers 4.3 Leas Mean Squares Leas mean squares (LMS) algorihm belongs o he family of sochasic gradien filers. The algorihm uses an approximae esimae of he gradien a each ime sep o ge nearer o he opimal soluion. This mehod differs from he mehod of seepes descen where rue value of he gradien is used. The key feaure of LMS algorihm is ha i uses a rough approximaion o he gradien and avoids calculaion of rue gradien (which requires considerably more compuaion). The gradien of he MSE surface, poins in he direcion of maximum increase. The algorihm changes he parameers in such a way ha he cos value moves in a direcion opposie o his gradien. The insananeous gradien based on he square of he insananeous error is (Haykin, 22). 2 e = w This use of insananeous gradien is jusified as i is an unbiased esimaor of he rue gradien. The esimae of he gradien a any ime is simply he produc of he curren inpu vecor imes he curren error vecor (Haykin, 22). = 2Z e Where Z is he observaion vecor. Hence he weighs can be updaed as w + 1 = w µ L ( ) Afer a number of seps, his evenually leads o weigh vecor which gives he minimum of he leas-square surface. Once he model converges o he minimum, i 35

46 Adapive filers execues a random moion near he opimal poin. If he saisical properies of he model vary slowly wih ime, he LMS algorihm is capable of racking such changes The advanage of his algorihm is ha i is numerically very simple (of order m only), i is very robus and numerically sable. The main disadvanage of he algorihm is is slow rae of convergence. The convergence of he algorihm depends on he learning rae parameer or he sep size parameer, µ. This parameer deermines he magniude L of he change made o he weigh vecor a each sep. If is value is close o he criical value, convergence is fas. An over-damped value of sep size parameer ensures sabiliy bu makes he convergence slower. For an under-damped value, he filer gives rise o problems of insabiliy. The sep size parameer is usually consan bu can be made ime varying. The LMS algorihm is given below Observaion equaion: y = Z α + d + ε Updae equaion: α = α + µ Z ε +1 L In words, Updaed value Learning rae = Old weigh vecor + * of Weigh vecor parameer ( Inpu vecor) * error In order o compare wih oher filers, he gain, k can be regarded as k = µ Z L 36

47 Adapive filers 4.4 Summary The Kalman filer and he RLS filer are based on he second order (gradien and Hessian) while he LMS filer is based only on he firs order (gradien) mehod. An addiional provision in KF is ha he sae process is assumed o be corruped by some known noise. The advanage of KF and RLS is ha hey have faser rae of convergence han LMS. On he oher hand LMS is compuaionally simpler han he second order filers. Kalman filer: w( + 1) = w( ) + Ke( ) = w( ) + Z ( T PT + R Q R ) ( T PT + R Q R ) Z Z + H e() Recursive Leas squares: P 1Z w( + 1) = w( ) + Ke( ) = w( ) + e( ) λ + ZP Z 1 Leas Mean Square: () + K e( ) = w( ) Z e( ) w( + 1) = w µ L The KF and RLS also updae he second order marix, P a each ime sep. LMS uses he curren error as he approximae gradien and does no require second order saisics. 37

48 Lieraure Review 5 LITERATURE REVIEW This chaper reviews he research on he opic of shor-erm raffic flow predicion. The firs secion gives a brief overview of he differen modeling approaches applied o he shor erm raffic forecasing problem. The laer par looks ino research done in he domain of adapive models for raffic forecasing. The deails have been kep o a minimum and only mehodical highlighs of differen approaches are menioned. 5.1 Simple predicion models Random walk This is he simples model for raffic in which he forecas for nex inerval is he flow observed a he curren inerval, V ˆ = V. This mehod works when he change +1 from one inerval o anoher is minimal. The model is good for benchmarking purposes, since any model worh is sal should be able o perform beer han his model. Hisorical average Hisorical average predicions simply average he hisorical flow (observed flow exacly a day before or a week before he ime in quesion). This mehod does no make use of he recen flow observaions. 38

49 Lieraure Review Exponenial smoohing models Exponenial smoohing models give he forecas as a weighed sum of he pas observaions. The weighs decay in an exponenial fashion. This model is he same as IMA(1,1). Informed hisorical average This is a combinaion of he random walk and hisorical average mehod used o forecas link ravel ime (Kyasi e al, 1993). The predicion mehod uses he raio of hisoric ravel ime o he curren ravel ime o adjus he hisorical average of he fuure inervals. UTCS models The Urban Traffic Conrol Sysems (UTCS) used mehods similar o he simple models above for forecasing shor-erm raffic flow a arerials. The firs generaion UTCS predicion relied on hisorical daa only. The second generaion UTCS used curren raffic condiions o adjus hisorical daa. Finally UTCS-3 disregarded hisorical daa alogeher and made predicions based on he curren condiions only. 39

50 Lieraure Review 5.2 Non seasonal ime series As menioned before, he model given by Williams (1999) was he firs applicaion of seasonal ime series model for raffic flow. ARIMA models applied o raffic forecasing before did no make use of he seasonal paern of raffic flow. In mos of he sudies, he invesigaed models used firs differencing o make he series saionary, and had a model form of ARIMA(p,1,q). Ahmed and Cooke (1979) proposed an ARIMA (,1,3) model. They compared i o hose obained by double exponenial smoohing, simple moving average and exponenial smoohing wih adapive response. The auhors found ha he ARIMA (,1,3) gave beer forecass han he oher hree. Levin and Tsao (198) used an ARIMA (,1,1) model, which is essenially he same as exponenial smoohing model. Their comparison wih an ARIMA (,1,), or he random walk model, shows ha ARIMA (,1,1) model performed beer. Hamed e al. (1995) invesigaed he use of ime series models for predicing arerial raffic flow. The daa consised of 1 minue raffic flow for he morning peak hour (6:3 AM o 8:15 AM). They fied several ARIMA models afer using firs order ordinary differencing. Resuls showed ha he ARIMA(,1,1) model bes described he daa. Time-series models were also used in several sudies o compare wih he resuls of oher approaches. 4

51 Lieraure Review 5.3 Neural neworks Applicaion of neural neworks (NN) o raffic flow forecasing has been invesigaed by many researchers over pas several years. The moivaion for using neural neworks is based on he fac ha hese models are capable of handling non-linear relaionships. Neural nework models are well suied for dealing wih paern recogniion problems since hey do no make any assumpion abou he srucural form of he model. They do no give he underlying relaionship beween inpu and oupu which can be described as a complex form of regression. One drawback of NN is ha his srucural form canno be obained; hey are essenially black box models. A neural nework model consiss of differen layers which are conneced o each oher by connecion weighs. Beween he exremiies of he inpu layer and he oupu layer, are he hidden layers. The nodes in each layer are conneced by flexible weighs which are adjused based on he error or bias. This process of changing he connecion weighs is called raining or learning. The wo imporan facors which conrol he final mapping funcion are he number of hidden layers and he mechanism used for raining. Clark e al. (1993) compared he performance of neural nework models o ARIMA (,1,2) models. The neural nework models were designed o include daa from oher links. Neural neworks wih wo hidden layers and back-propagaion learning approach was used. 41

52 Lieraure Review Dochy e al. (1993) used neural nework wih one hidden layer of 5 nodes and local gradien approach o updae connecion weighs. They used half-hourly and hourly daa from he French moorway sysem. The daa used ranged from July and Augus from years 1984 o 199. This neural nework model ouperformed he ATHENA model (described in secion 5.5). Smih and Demesky (1994) used a neural nework model wih one hidden layer of 1 nodes and back-propagaion learning approach. The neural nework model used univariae daa consising of curren flow, hisorical flow, average speed and pavemen condiion of 15 min duraion. The sudy used 15 minue daa from Virginia beween June and Augus 11, The model was compared wih he hisorical average model and an ARIMA (2,1,) model. The NN model was able o perform only slighly beer han he hisorical average model. The ARIMA model performance was poor. Doughery and Cobbe (1997) developed neural neworks o forecas raffic flow, occupancy, and speed using daa from Neherlands GERDIEN projec. The daa consised of 1 minue values. Their emphasis was o selec suiable parameers for he inpu layer. They concluded ha flow and occupancy can be modeled using adapive NNs. They also sugges use of elasiciy approach for selecing suiable parameers for he inpu layer. Park e al. (1998) applied radial basis funcion (RBF) neural nework for forecasing raffic flow. In his sudy, he NN wih RBF were compared wih Taylor 42

53 Lieraure Review series expansion, exponenial smoohing, double exponenial smoohing and NN wih back-propagaion. Traffic volume daa consising of 5 minue flows beween 7: am and 8: pm from February 2 o February 29, 1996 from San Anonio was used. Based on heir sudy hey recommend he exponenial smoohing mehod and RBF for use in real-ime raffic forecasing. Dia (21) proposed an objec oriened dynamic neural nework model for forecasing shor-erm raffic condiions. His mehod differs from oher NN approaches ha i gives a dynamic raher han a saic archiecure so ha he model can adap iself wih ime. The daase used consised of 2 second flow values over a 5-hour period on 2 days in April 1995 in Brisbane Ausralia. 5.4 Non parameric models The non-parameric approach locaes he sae or inpu values of he sysem a ime of ineres based on is similariy o he sae of he sysem in he pas. The k-neares neighbor (k-nn) is a non-parameric regression approach which was firs applied o raffic condiion forecasing by Davis and Nihan (1991). In he k-nn approach, pas k cases in he neighborhood are used o esimae he value of he dependen variable. The daabase is updaed afer every new observaion. For use in forecasing, similar cases need o be presen in he daabase. 43

54 Lieraure Review Smih e al (22) compare he non-parameric regression model and seasonal ARIMA model. They ouline four challenges relaed o he implemenaion of nonparameric regression mehods 1) choice of an appropriae sae space, 2) definiion of a disance meric o deermine nearness of hisorical observaions o he curren condiions, 3) selecion of a forecas generaion mehod given a collecion of neares neighbors, and 4) managemen of he poenial neighbor s daabase. 5.5 Layered models Two layered models, ATHENA and KARIMA, were proposed for forecasing raffic flow. These models firs classify daa ino differen ses or clusers and hen fi models appropriae for each cluser. Danech-Pajouh and Aron (1991) proposed he ATHENA model. The ATHENA model firs processes he daa o creae raffic volume curves of specified lengh. The raffic volume curves are hen organized by ime of day. A mahemaical classificaion algorihm is hen used o spli he raffic volume curves ino cerain classes. The crierion for classificaion is Euclidean disance from he class cenre of graviy. Der Voor e al. (1996) proposed he KARIMA model. The K in he model name sands for he mehod used for classifying he daa, a Kohonen self-organizing map. The Kohonen map classified he daa ino differen clusers. The four clusers corresponded o he four flow regimes free flow, ransiional flow, sar of congesed flow, congesed flow. A separae ARIMA (p,,q) model was fied o each cluser. 44

55 Lieraure Review 5.6 Adapive raffic forecasing models Applicaion of adapive echniques for raffic forecasing is no a new concep and has been used in pas in several sudies. In his secion, four such sudies are discussed. This hesis uses similar echniques for making he SARIMA model adapive. Kalman filering approach for mulivariae raffic flow Okuani and Sephanedes (1984) used Kalman filer for predicing raffic flow. They proposed a mulivariae model for raffic flow as a funcion of flow a nearby locaions and recen ime. The daa used consised of 15 minue flows from 8: AM 7: PM from Nagoya Ciy, Japan. They model he coefficiens as ime varying sae vecor. They proposed wo models. The firs predicion model can be wrien as z( + k) = H ( ) x( ) + H1( ) x( 1) H ( ) x( r) e( ) o r + Where z(+k) is he raffic volume k inervals ahead of ime, x(-i) is he observed raffic volume of a I inervals before ime, H() is he parameer marix which relaes he oupu o he pas values. The sae ransiion equaion (i.e. he coefficien or weigh vecor propagaion) was regarded as random walk H ( ) = H ( ) + e( ) 45

56 Lieraure Review In he second predicion model he similariy of he raffic flow paern from day o day was aken ino accoun. The parameer marix H was assumed o be saionary wih respec o day d and was represened as H ( d, ) = H ( d 1, ) + e( d 1, ) The second model can be wrien as z d, + k) = H ( d, ) x( d, ) + H ( d, ) x( d, 1) H r ( d, ) x( d, r) + e( d, ( 1 ) Thes models were applied for wo ypes of daa, one o he flow direcly and he oher o he weekly difference of he flow. The above models were esed on 15 minue flow from 4 consecuive poins and 3 previous ime periods. Kalman filer was used for updaing he parameer marix H a each ime sep. The error covariance marix of sae, Q and observaion error covariance marix, R were assumed as diagonal marices. The iniial sae esimae of H was se o zero. The iniial sae covariance, P was assumed as a diagonal marix. The wo models were compared wih each oher and wih he UTCS-2 model. Resuls indicaed ha he model wih weekly differenced flow daa consisenly performed beer han ohers. 46

57 Lieraure Review Leas Mean Square applicaion for univariae raffic flow Lu (199) used he leas mean square (LMS) algorihm for a responsive raffic flow predicion model. Hourly flow daa for hree days was used from Ausin, Texas for a single locaion. The LMS mehod was invesigaed in order o find an opimal sep size parameer and a suiable model order for applicaion o raffic flow. Resuls show ha as he order of he model increases, he rae of convergence of he LMS algorihm increases. The sep-size parameer of 5 x 1-9 was found o give bes resuls wih he chosen model order of 23. Recursive leas squares laice filering for raffic flow Kang e al (1998) used recursive leas squares and laice filering o predic shorerm raffic flow. They used he model for predicing 3 second flow raes. Their mehod is a recursive one in which he model order is also adapable along wih he weighs. A each ime sep he bes model order is chosen and hen he opimal coefficiens for his model are calculaed. Their model can be wrien as: yˆ ( ) = Where, M j= 1 λ j ω j () y( j) y() 3 second raffic volume a ime, M order of he model, ω j weigh (coefficien) for he previous raffic volumes λ - forgeing facor beween and 1 47

58 Lieraure Review n The weighs were esimaed by minimizing he sum of squared errors, λ n i= e ( i) Durbin-Levinson algorihm was used o choose an opimal model order making use of backward and forward predicion error. Kalman filer and RLS for modeling speed Yang e al. (24) used Kalman filer and recursive leas squares (RLS) for an AR (5) model in order o make he coefficiens adapable wih ime. The sudy used 5-minue speed daa for a sie from California s PeMS sysem. The speed daa used was beween 6: AM 8: PM wih a oal of 3 monhs. They discuss he problem of finding appropriae mean and covariance for he sae space vecor so ha Kalman filer can be applied o he model. 48

59 Daa and mehodology 6 DATA and METHODOLOGY This chaper describes he daa and mehods used in his hesis. The firs secion describes he sysem from which daa is used. The physical characerisics and daa aribues of he chosen sies are briefly described. The second secion describes how daa were aggregaed from one minue inerval o fifeen minue inerval. The hird secion describes he aribues of he fifeen minue flow daases. The fourh secion describes he consrucion of oulier correced series. The las secion defines he measures of performance. 6.1 Basic daase aribues Daa used in his hesis was obained from Unied Kingdom Highways Agency. Upon reques, he UK Highways Agency agreed o provide archived daa for he enire year of 22 and permission o use hese daa for research. In his projec 8 sies were chosen for analyses. The crieria were o ge sies wih differen number of lanes and differen peak hour condiions. A visual analysis of he daa was done o check for amoun of missing daa and oher disconinuiies. Based on his, hree sies in M25 orbial having 4 lanes each, 3 sies in M1 moorway wih 3 lanes each, and 2 sies in M6 moorway wih 3 lanes each were chosen. Sie aspecs are summarized in able below. 49

60 Daa and mehodology Table 6-1: Geomerical aspecs of sie Sie Moorway Lanes Type 4762A M25 4 Orbial 468B M25 4 Orbial 4565A M25 4 Orbial 6951A M6 3 Radial 6954B M6 3 Radial 2737A M1 3 Radial 288B M1 3 Radial 4897A M1 3 Radial The geographical locaions of hese hree moorways are shown in he map below. 5

61 Daa and mehodology Figure 6-1: Map of UK moorway sysem Map of UK Highways M6 M1 M25 Source: hp:// 51