Absrac Sgnal Dffuson Mappng: Opmal Forecasng wh Tme Varyng Lags. Paul Gaskell Unversy of Souhampon, WAIS Frank McGroary Unversy of Souhampon, Fnance and Bankng Group Thanasss Tropans Unversy of Souhampon, WAIS We nroduce a new mehodology for forecasng whch we call Sgnal Dffuson Mappng. Our approach accommodaes feaures of real world fnancal daa whch have been gnored hsorcally n exsng forecasng mehodologes. Our mehod bulds upon well-esablshed and acceped mehods from oher areas of sascal analyss. We develop and adap hose models for use n forecasng. We also presen ess of our model on daa n whch we demonsrae he effcacy of our approach. Inroducon Convenonal me seres mehodologes for fnancal forecasng are lmed n her ably o handle lags. They can only accommodae lags fxed neger lengh, e.g. a one-perod lag, a wo-perod lag, ec.. We argue ha he relaonshp beween real me seres can ofen have a rcher, more dynamc me srucure han has been mplcly assumed n he radonal mehodologcal approaches. We nroduce a new forecasng mehodology, Sgnal Dffuson Mappng (SDM), whch exracs he maxmum possble nformaon from he modelled relaonshp and produces he bes possble forecas a each pon n me, n crcumsances where he lag-lengh s me-varyng. The noon of me-varyng lag-lenghs may srke some researchers as odd. Indeed, we suspec ha he fac ha lags have been almos nvarably modelled as fxed-neger-lags n me-seres fnancal modellng, has probably condoned mos fnancal forecasers o beleve ha fnancal daa acually behaves n he manner ha her models seek o measure. However, we conend ha ha such a concluson would be a classc case of f all you have s a hammer, everyhng looks lke a nal. Real world emporal relaonshps are less well ordered han we mgh lke o hnk. The hsory of fnancal markes s replee wh colourful examples of convced nsder raders who exploed access o prvleged nformaon for pecunary gan. Ths can nclude nformaon abou companes' earnngs announcemens, fuure akeover arges, prce-sensve macroeconomc news, ec.. An nsder rader radng pror o a news even wll reverse he me-order ha fnancal heory convenonally assumes,.e. nformaon frs, prcereacon second. On he oher hand, nobody would accep he asseron ha nsder-radng occurs pror o each and every nformaon even. In oher words, a reasonable concluson would be ha occasonal cheang happens and hs dsors he nformaon-prce me-lag. If a fxed-neger-lag model were mposed on such a scenaro, would resuls n he measured correspondence beween nformaon and prce beng lower han f he modeller had a means of adjusng for he me-dsoron. Insder radng s no he only source of lag-lengh dsoron. Cheung e al (2004) survey he opnons of foregn exchange dealers and conclude ha news, speculave forces and bandwagon effecs are he man drvers of prce whn a day and ha fundamenals, whch mansream fnance heory assumes o be he key drvers of prce, are perceved as only relevan n deermnng reurns over he long erm (.e. over 6 monhs). These speculave forces and bandwagon effecs are perhaps bes capured n Soros' (2008) reflexvy heory whch poss a b-dreconal symboc feedback relaonshp beween nformaon and prce. Imporanly, Soros (2008) argues ha some reflexvy relaonshps are susaned for exended perods, whle ohers can fzzle ou afer a shor me. Tryng o exrac he relaonshp beween such varables wh a fxed-lengh-lag model s lke ryng o ea soup wh a fork que smply he wrong ool for he job. The key conrbuon of hs paper s he nroducon of a new model (SDM) o he forecasers' arsenal whch can handle me-varyng lag-lenghs of he ype we descrbe above, rerevng he maxmum possble nformaon abou he relaonshp beween wo me seres. Our mehod bulds upon well-esablshed, publshed mehods from he daa analyss and econophyscs leraures, whch were developed o measure opmal relaonshps n relaed seres of fxed lenghs, e.g. Keogh and Pazzan (1999), Sornee and Zhou (2005), Zhou and Sornee (2007). However, n her orgnal form none of hose mehods were approprae 1
for forecasng. We adap hose mehods o he problem of forecasng n a manner analogous o a Kalman Fler or Recursve Bayes Esmaon wh me-varyng-lags, whch we name Sgnal Dffuson Mappng. We es SDM wh synhec daa and we demonsrae ha s able o recover he rue opmal relaonshp beween seres where we have delberaely dsored he emporal relaonshp by shorenng/lenghenng he lag a dfferen pons n me. Please noe ha SDM does no rule ou he possbly ha he rue relaonshp beween wo me seres could acually be a fxed lengh lag. An mporan feaure of our proposed mehodology s ha wll denfy he opmal underlyng relaonshp beween he wo seres wheher he lag srucure s fxed of varyng n me. In oher words, f he bes possble relaonshp s obaned by laggng one of he seres by wo me perods, hen ha s wha he model wll fnd. On he oher hand, f an even beer lnk beween he wo seres could be shown by allowng seres 1 o lead seres 2 some of he me and o lag seres 2 a oher mes, our mehod wll denfy hs as he bes model. The remander of our paper s srucured as follows: Secon 1 nroduces some conceps and noaons from he relevan leraures we wll use n defnng our approach. Secon 2 wll descrbe he SDM algorhm n full. In secon 3, we presen resuls from esng he SDM algorhm on smulaed daa. Fnally, secon 4 presens our conclusons. 1. Background In he nroducon we hghlgh he work of Soros (2008) and Cheung e al (2004), as examples of work whch hghlghs he emporal complexy of he nformaon-prce relaonshp. In he case of Soros, hs complexy s capured as a consanly shfng feedback mechansm, capable of rapd changes for example perods of consan growh lasng years can shf o huge marke crashes n he space of less han a week. In he case of Cheung e al he emphass s on scale, where shor erm nformaon effecs (nra-day) are augmened wh long erm effecs of 6 monhs or more. As descrbed, hs mples he nformaon-prce relaonshp has a complex causaly srucure. Sascal causaly n he fnancal leraure s mosly consdered n erms of Granger causaly (Granger 1969), so ha where he me-sequencng of a relaonshp s unclear, Granger causaly ess are employed as he man ool n deermnng he naure of he sascal causaly. In secon 1.1 we are gong o nroduce he Granger causaly model and cases where has been appled n he sudy of nformaon-prce relaonshp. We wll also dscuss some oher approaches o deermnng sascal causaly. We are hen gong o dscuss he lmaons of hese models n erms of he dynamsm and scalng of he relaonshp we wsh o measure. In secon 1.2 we wll dscuss how he Bayesan framework s a good f for problems of hs ype. We wll brefly hghlgh oher work where Bayes Esmaors have been used for model fng n a fnancal conex and draw he lnk o oher sae space echnques used n me-seres analyss. 1.1. Tme-Sequencng n Fnancal Forecasng To gve he ssue of varable lag-lenghs a mahemacal framng, he sandard relaonshp ha s consdered n he Granger causaly approach s ha some auoregressve seres of he form. x = A x 1 +N (0,1) Where A s he auoregressve parameer, A < 1 so he process s saonary and N(0,1) ndcaes a random varable wh mean 0 and sandard devaon of 1. We hen consder anoher varable relaed o y as; y =β 1 x 1 +β 2 x 2...+β τ x τ +u where u =N (0,σ u ), (1) s he nose erm obfuscang he relaonshp. Granger causaly n he b-varae case s smply a regresson model specfed on he lagged values of x, hroughou hs paper bold-face ype wll denoe a vecor n he usual manner, where β are he coeffcens values or parameers of he model, n some cases we mgh also consder lags of y bu here we om hem for 2
brevy. The parameers of he model are hen esmaed wh he Ordnary Leas Squares algorhm. If any of he β coeffcens are sascally sgnfcan, researchers can hen sae ha here s evdence o suppor he hypohess ha x Granger causes y. Ths s by far he mos common form of analyss of he lagged dependence beween me-seres, example sudes of nformaon prce relaonshps usng hs mehodology nclude; Bollen (2011) and Sprenger (2013), boh sudyng he effec of publc senmen on sock prces, Hong e al (2009) modellng rsk spllover beween nernaonal fnancal markes and Hemsra and Jones (1994) sudyng he causaly beween sock prces and volume usng he Granger causaly es specfed n (1) and a non-lnear varan. Even n cases where more complex mehodologes are appled, usually he reamen of he emporal relaonshp beween he varables reduces o some varan of Granger causaly. An example of hs s Sharkas e al (2005), who use wavele decomposon o sudy spllover effecs from nernaonal sock ndces, hen use lnear regresson o f models on he decomposed waveles. One of he aracons of he Granger causal approach s ha gves a clear pahway o consrucng a forecasng model. Based on he resuls of (1) s possble o denfy lags wh sgnfcan p-values, hen consruc a second regresson usng hese lags. Ths second model can hen be used o deermne esmaes of fuure values of y based on he coeffcen values n he second regresson model. The suaon Soros (2008) and Cheung e al (2004) descrbe, however, s one of sgnfcan varaon n he emporal relaonshp beween varables, boh n erms of he speed by whch he relaonshp can change bu also he fac ha he relaonshp can exs a vared scales. In Soros' descrpon of reflexvy he saes how fnancal markes can swch from bull o bear perods very quckly, hese swchng momens are characersed by changes n he nformaon-prce relaonshp ha happen over he course of days - suggesng any model would have o adjus almos mmedaely o he change n crcumsance. Cheung e al repor how professonal raders consder nformaon effecs a dfferen scales, from nra-day effecs o long run effecs, >6 monhs. Now consder ryng o capure hs wh he Granger causal approach. Frsly acklng scale; Cheung e al's conclusons ndcae we would need o f a model wh a large number of parameers o capure each dfferen scale consdered. Consder daly nervals for example, hs would lead o a model wh ~180 parameers, leadng o very low sascal power for he es due o he 'curse of dmensonaly'. Secondly, assumng he parameers of (1) are me-varyng, we could employ a verson of (1) on some rollng wndow. In order o acheve reasonable resuls, however, requres specfyng a reasonable szed wndow ye he heory suggess ha change n he relaonshp wll occur almos nsanly. 1.2. Parameer Esmaon usng Bayesan Inference To be specfc; he ssue we have s ha gven he Granger causal model we beleve he model specfcaon would have oo many parameers - and ha hese parameers would change oo quckly, for us o be able o f he model we need o es he avalable fnancal heory wh he Ordnary Leas Squares (OLS) algorhm. We are no argung ha he basc srucure of he Granger causal model s defcen n 's ably o characerse he nformaon-prce relaonshp per se, jus ha we canno f he model n he way we would lke. Recenly, here have been a number of papers n he fnance and economercs leraures usng varous ypes of Recursve Bayes Esmaor (RBE) o esmae model parameers. Examples of hs work nclude, Carvalho and Lopes (2007) who presen an RBE for dynamcally paramersed sochasc volaly models and Carvalho e al (2009) who f he parameers of a dynamc, condonally lnear model (see Arulampalam e al 2002 and Lopes and Tsay 2011 for revews of he feld). As ye, however, no work exss aempng o f causaly models n a smlar fashon. From a Bayesan perspecve he parameer values for he lags n (1) can be consdered as a sae space where he parameers are represened as probables ha a gven lag could be causally nfluencng he value of y. Smplscally hs means ha we can updae he parameers based on her relave probables as hey ranson from one me-perod o anoher, raher han on her goodness of f over a large number of prevous observaons. We wll show how hs can allow he lag-srucure o vary much more dynamcally 3
han could do f fed wh OLS and crcumvens ssues of dmensonaly. Ths sae space represenaon also lnks o oher areas of he me-seres analyss leraure where varable lag pahs have been consdered. The Dynamc Tme Warpng (DTW) (see for example Keogh and Pazzan 1999, Senn 2008, Warren Lao 2005, Sakura e al 2005) and Opmal Thermal Causal Pah (OTCP) (Sorene and Zhou 2005, Zhou and Sornee 2007) leraures boh sudy hsorcal lagged relaonshps usng smlar sae space echnques. Much of secon 3.2 concerns negrang hese deas no he RBE framework - so ha we can forecas wh varable lag-srucures, raher han vew hem hsorcally. The key conrbuon we make n hs paper, s o show s possble o dynamcally f he model (1) usng a RBE, we wll begn by fng he smple lnear model and expand o more exoc cases n laer secons. The specfc RBE algorhm we presen o complee hs ask s he Sgnal Dffuson Mappng (SDM) algorhm. The raonale for he name s because we wll map he dffuson graden of nformaon flowng beween he wo seres over he sae space of possble lags hs resuls n us beng able o plo he me evoluon of hs relaonshp on a heamap.e. he map of he flow of he sgnal beween he seres. In secon 2 we wll furher defne he me-sequencng problem n Bayesan erms and nroduce he noaonal convenons we wll adhere o hroughou hs paper. 2. A Bayesan Vew of he Tme-Sequencng Problem In hs secon we are gong o nerpre he Granger causal model n erms of he common General Dynamcal Model (GDM), hs a more general form of he Normal Lnear Dynamcal Model whch forms he bass of he Kalman Fler (Lopes and Tsay 2011). We wll hen oulne he general mehod of solvng he GDM equaons recursvely usng Bayes heorem. Fnally we wll descrbe he crera by whch an esmaor can be seen as opmal whch wll leave a clear pahway o nroducng SDM as he opmal soluon o hese equaons. The Bayesan approach o sascal decson makng s based around he defnon of relave belefs abou a se of dfferen evens. These belefs are represened as a sae space of probables assocaed wh each possble even - gven (1) our belefs are abou a seres of dscree causal relaonshps beween x and y. We are gong o hold hs se of belefs n a probably vecor conanng an enry for each of he lagged values under consderaon - we denoe hese opons as an Ns lengh probably vecor w where Ns s he number of consdered lags (read 'number of saes') a,.e.; w :[w 1...w Ns ] and = Ns 1= w. =1 The convenon we adhere o hroughou hs paper s ha subscrps wll denoe a vecors poson n me and superscrps denoe he relave poson of a value n he vecor, so w, would be read he 'h poson on vecor w a me. In erms of (1), hese probably weghs are concepually smlar o he parameer values β we dfferenae hem n he noaon o save confuson as hey are probably weghs raher han regresson coeffcens and wll furher defne hs dfference n secon 3.2. The second aspec of he Bayesan approach s o hen defne a measure of how well hese belefs map on o some measuremens of he processes under sudy. Le d be an Ns lengh measuremen vecor d :[d 1...d Ns ] conanng some measure of he x, y relaonshp correspondng o each of he values of w. For now he reader smply has o undersand hs as a comparave measure of he relaonshp, we wll make hs defnon concree n he followng secon. We noe ha s ypcal n he RBE leraure o use x for he sae vecor and y for he measuremen vecor. We have dffered from hs noaon because n he economercs leraure x and y are ypcally he meseres under sudy. As we beleve SDM s manly amed a he fnancal forecasng communy we have sded wh he economerc noaonal convenon. The GDM s hen gven by wo equaons he frs, usually referred o as he sysem model, governs he way n whch our belefs abou he sysem propagae forward hrough me, hs s defned n probablsc erms as; 4
w p(w w 1 ) (2) where he operaor should be read as 'vares n proporon o' - so he equaon saes ha he probably denses of w vary proporonally o a probably mass funcon appled o he denses a -1. The second, usually called he measuremen model, governs he way we are gong o nerpre he measuremen vecor n erms of he sae probables; d p(d w ) (3) whch should be read ha he measuremens vary proporonally o he lkelhood of he measuremen gven he sae vecor probables. If he sysem model can be characersed as a Markov chan.e. he values of w a +1 depend only on he values a, hen we can defne a Bayesan predcon model for he updaed values of w as; w 1 j= N = j=1 p(w j j w 1 )w 1 (4) Where w 1 denoes he esmae of he probably densy of he 'h poson on he probably vecor w gven he pror denses and he sysem model. All ha (4) really saes s ha, f we have a sysem model holdng he ranson probably of he 'h lag holdng useful nformaon gven he precedng vecor of lags, hen we can sum over hese probables o esmae he nex 's value for he lag. I s common n he leraure o see (4) wren as an negral, he reason for he summaon n our case s ha he sae space over he lags s dscree, raher han connuous. Gven an esmaon of he weghngs of w based on he known denses a -1 and he sysem model. We hen receve a se of measuremens d ha we use o updae hs forward projecon of he denses based on he observed evdence. Gven he measuremen model (3) we can wre hs due o Bayes' heorem as; w = j= Ns j=1 p(d w )w 1 p(d j j w )w 1 (5) Whch gves he lkelhood of he measuremen gven he daa, mulpled by he pror probably f he 'h lag normalsed over all Ns possble lags. Ths yelds he basc predcon and updae srucure of a Grd Based Fler, whch s a ype of RBE defned on a dscree se of possble saes n hs case lags, and forms he bass of a number of RBE algorhms, wo of he bes known beng he Boosrap Fler (Gordon e al 1993) and he Auxlary Parcle Fler (P and Shephard 1999). The aracon of hs formulaon s ha, provdng we can vald funconal forms for (2) and (3), he fler wll evolve o he opmal calculaon of he probably denses of w over repeaed eraons of (4) and (5) (Arulampalam e al 2002, Lopes and Tsay 2011). Opmaly n hs sense means he probably denses whch mnmse he measuremen error. So we need a measuremen model whch s conssen wh reasonable assumpons abou he x, y relaonshp. We also need o specfy a probably densy funcon for he values of hese measuremens, snce (5) requres ha we calculae he lkelhoods for hese observaons gven our belefs abou lag probables. Fnally, we requre a probably mass funcon for (3) whch capures he me-evoluon of he lag srucure n a heorecally jusfable way. If hese crera are me, we can clam he esmaor s opmal under he specfed assumpons. The rade-off n defnng he esmaor s hen how o pos relaxed enough assumpons abou he forms of (2) and (3) o make he esmaor generally applcable o a range of forecasng asks. Wha we are gong o show n nroducng SDM s ha we can specfy he form of (2) usng very weak assumpons. These assumpons are well suppored by oher areas of he leraure. Ths effecvely removes me-varyng lags as an ssue and allows us o plug n any relevan measuremen model avalable n he economercs leraure. Ths makes he SDM algorhm useful n a large number of praccal applcaons, snce he researcher can sll ulse all of he curren modellng approaches for bvarae seres and plug n 5
he SDM esmaor as he me-sequencng es supplemenng he specfed model. 3. Sgnal Dffuson Mappng In he followng subsecons we wll beng by descrbng he sysem model we are gong o use for (2). Ths s he key operaon n he SDM algorhm and remans nvaran despe he specfcaon of he measuremen model. We wll hen defne a measuremen model o f he smple Granger causal model (1). Afer llusrang hs smple case we wll hen go on o descrbe more exoc varans of he SDM algorhm.e. b-dreconal causaly srucures and posve-negave swchng causaly srucures. Fnally, we wll gve an algorhmc mplemenaon of he SDM algorhm and dscuss mplemenaon ssues and compuaonal complexy. 3.1. Sysem Model From a forecasng perspecve, clearly we would lke o be able o defne a model where he denses over he lags do no change over me, n hs case we would be able o esmae when he relaonshp was lkely o occur perfecly. Anoher way of sang hs would be ha he error n he forward projecon of our belefs n he sysems sae s 0 and w =w 1. If here s varaon n our belefs over me hen we need o pos an equaon descrbng hs varaon. Here, here are wo mporan quanes. Frsly; he srucure hs varaon akes.e. how hs varaon deforms he denses of w. Secondly; he magnude of he varaon.e. how much deformaon n he densy vecor we expec. Afer defnng hs funcon we hen need o fnd he paramersaon of he funcon whch mnmses he oal error for he sysem model over me. In erms of he srucure of he varaon, here s already a large body of leraure dealng wh varaons n hsorcal lag srucures. We are gong o follow hese leraures n defnng he srucure of he emporal varaon smlarly jus as a probablsc forward projecon of he sae vecor probables, raher han a hsorcal represenaon. Boh he Dynamc Tme Warpng (DTW) (see for example Keogh and Pazzan 1999, Senn 2008, Warren Lao 2005, Sakura e al 2005) and Opmal Thermal Causal Pah (OTCP) (Sorene and Zhou 2005, Zhou and Sornee 2007) leraures use sae space mehods o sudy hsorcal lead-lag relaonshp beween me-seres. The basc premse of eher algorhm s o defne a marx of all of he parwse relaonshps beween he varables on a marx conanng a measure of he relaonshp n each square. Then raverse he marx from a fxed sar o fxed end pon n such a way as o reveal he opmal or lowes cos relaonshp beween he varables based on he measure. In order o do hs, boh algorhms make he assumpon ha he srucure of he lagged relaonshp can vary only slowly n me. Slow varyng n hs sense means ha a lag pah can vary by only 1 un me-perod for every un ncrease n. Takng DTW as an example (alhough boh algorhms are derved from he same premse), gven wo me-seres c :[c 1...c k ] and q:[q 1...q k ], we can consruc a k*k marx of he parwse dsances,.e. c q j beween he wo seres for each lag lengh. The DTW algorhm hen seeks he lowes dsance, connuous 1 o 1 mappng beween he wo seres from a fxed sar pon o a fxed end pon. Ths ask s compleed by undersandng ha he lowes dsance 6 Fgure 1: Slow Varyng Lag Pahs; (A) descrbes he hree squares n he marx va whch we can reach square (9, 3) due o he recursve relaon used n he DTW and OTCP algorhms. (B) hen shows an example of a pah defned as a connuous mappng beween pons (0,0) and (4,11).
pah o a pon on he marx mus be he summaon over he lowes dsance pahways up o hs pon whch jusfes he recursve relaon; ε, j = c q j +mn [ε 1, j,ε 1, j 1,ε, j 1 ] (6) where ε, j s nerpreed as he relave cos of he pahway so ha he mnmum dsance pahway s he lowes cos pahway up o a gven pon on he marx. Defned n hs way he lag pah has some desrable properes we would expec n a realsc lagged relaonshp; 1) A relaonshp s defned for each here are no large jumps where no relaonshp s defned for a se of me-perods. 2) The relaonshp s a connuous 1 o 1 mappng f hs was no he case here could be overhangs or clffs n he lag pah mplyng he causal relaonshp ran from he fuure no he pas. (adaped from Sornee and Zhou 2005) The way we are gong o mplemen hese deas n he SDM esmaor s o make he same assumpon ha he me evoluon of he lag srucure s relavely slow. Gven ha he deal forecasng scenaro s ha our belefs abou he lag probables are accurae hs means characersng any devaon, or error, n hese belefs as a relavely slow varyng funcon of he sae vecor. In our noaon, (6) mples allowng a gven value of w o be nfluenced by one of he followng 3 values; 1 +, w 1,w 1 ) (7) (w 1 Assume ha we knew he amoun of varaon n he lag srucure wh cerany, so we could fx a parameer θ, whch capured he oal magnude of he varaon. Gven ha we are expecng w =w 1 n he 0 error case, (7) ells us he srucure of he varaon so we could subsue no (2) and wre; w w 1 +error so w w 1 + θ 3 (w 1 1 +w 1 +w + 1 ) (8) Dvdng he magnude of he error by 3 s due o he fac ha we are dsrbung hs quany evenly over he lags avalable due o he slow varyng consran. Clearly n mos neresng cases we are no gong o be able o assume a consan amoun of varaon, so a beer characersaon would be as a random sequence of perurbaons n he lag srucure v :[v 1... v ]. We could defne a mass or densy funcon for v bu as we wll show hs s no necessary and v can be any arbrary sequence. We would hen lke, a each, o choose a parameer value for θ whch mnmses he global error over repeaed eraons.e. mnmses he amoun of varaon n our belefs abou he lag srucure. To do hs we can defne some properes of v due o he properes of he probably vecors. Frsly, we noe ha 0 w w 1 2, so assumng no error we can also sae w w 1 =0. I follows hen ha he magnude of he error process a s he dsance beween he vecors v = w w 1. Furher, we can also nfer ha v always has a expeced value, snce he expecaon of a posve random varable s always defned, and ha hs expecaon mus le on he bounded nerval [0, 2]. Under hese crcumsances, he opmal choce of value for θ s he medan of he dsrbuon of he errors up o hs pon hs s due o he fac ha he medan s he mnmser of he dsance funcon where he expeced value of he funcon s defned - so we can defne he opmal choce of value for θ as; θ = v 1: 1 (9) Where he lde denoes he medan of he vecor of observed errors. Subsung no (4) hen yelds he sysem model for he forward projecon of he sae vecor as; w 1 =w 1 + θ 3 (w 1 1 +w 1 +w +1 ) (10) 7
An mporan noe s ha a he frs and las posons,.e. he boundares, on w no all of he 3 posons wll be defned snce we have no daa for eher w 1 1 1 or w N+1, here we smply se he probably of hese posons o 0. A furher pon of noe s ha (8) mples w 1 akes values of > 1 where θ > 0, so s no longer a probably vecor. A hs sage a beer nerpreaon of hese esmaes s as a se of weghs defned by her relave lkelhoods. We could normalse hs vecor o 1 by dvdng over he sum of he weghs bu hs s unnecessary as we wll compue he acual probables usng he updae sep (5). In erms of he opmaly condons we sae a he end of secon 2, (10) s always defned for any par of seres rrespecve of her spaal relaonshp. Ths follows because he nformaon abou he spaal relaonshp beween he seres s held as probables so for he reasons already specfed we can always calculae he medan of he dsrbuon and hs wll always be he opmal esmae for θ. The assumpon we make s ha he me-evoluon of he causaly srucure s relavely slow, hs assumpon s jusfed by a he large leraure on DTW algorhms and he nascen OTCP leraure. The aracveness of hs approach s allows us o concenrae on modellng he complexy of he dsance relaonshp beween he seres ndependenly of he emporal relaonshp o show hs we wll descrbe how o f (1) usng hs sysem model n he nex secon, hen expand o a number of oher cases. 3.2. Measuremen Models To complee he SDM esmaor we need o defne a measure of he x, y relaonshp and pos a funconal form for mappng hese measures no he same probably space as he sysem model. Ths means makng some assumpons abou he form of he dsances beween he wo seres hen usng hese assumpons o calculae he lkelhood of he measuremen. We han subsue hese lkelhoods no (5) and he esmaor s complee. We wll begn n secon 3.2.1 by descrbng how o f (1) where x and y are assumed o be posvely correlaed. We wll hen show, n subsequen secons, how we can relax hese assumpons so ha we can f b-dreconally causaly srucures and models where here are posve-negave regme shfs n he causaly srucure. 3.2.1. Smple Lnear Causal Models If we assume ha x and y are measured n he same uns, for example hey may have been rescaled usng her respecve means and sandard devaons, hen f here was 1 lagged value of x causally relaed o y we could wre hs as; y =x τ +u (11) If here s more han 1 lagged value of x causally relaed o y he OLS approach o model fng s o nerpre furher lags as ndependen random varables. So he model expands by addng more random varables no he regresson. The dfference n approach usng SDM s ha we are gong o manan he assumpon ha here s only 1 causal relaonshp beween he varables bu ha hs relaonshp s dsrbued over a number of lags. In hs way we can hnk of he probables as a weghed average over a paroned nerpreaon of x.e. = Ns y =u + w x (12) =1 We are gong o use he squared dsance beween x and y as he measure of he relaonshp, so ha n effec he SDM esmaor becomes he leas squares esmaor for a relaonshp wh me-varyng lags. Squarng he erms n (12) hen yelds. = Ns y 2 =(u + w x ) 2 (13) =1 Whch we can rearrange o; 8
= Ns u 2 = w (x y ) 2 (14) =1 Then seng he squared dsance as he measure of he relaonshp so ha d =( y x ) 2 we can wre (14) n he more compac form; = Ns u 2 = w d =1 or u 2 =w d, (15) where he do ndcaes he scalar produc of he wo vecors. The assumpon of he model (1) s ha u s a mean 0 random varable wh unknown varance. As a resul we would expec he squared values of u o be drawn from he Gamma dsrbuon wh a shape parameer of 1. The densy funcon for hs dsrbuon s gven as. f U 2(u 2 λ)=λ e λu2 (16) Where λ s he rae parameer wh he maxmum lkelhood esmaor gven as he mean of he observed values of u 2.e. s= λ = 1 u where u 2 2 = 1 s=1 w s d s. (17) We have ncluded he me-varyng subscrp for λ as we are gong o calculae he parameer of he dsrbuon hrough repeaed samples over me. Gven an esmae for λ we can hen calculae he lkelhood of a gven dsance causally nfluencng he values of y usng he lkelhood funcon condonal on he value of λ. p(d λ )=λ e λ d (18) Subsung hese lkelhoods no he sysem model equaon (5) hen yelds. w = p(d λ ) w 1 (19) p(d λ ) w 1 Noe ha dfferen o (5) we have used he scalar produc noaon for he denomnaor of (19). Combnng (5) and (19) wll hen gve he he opmal recursve esmaor of (1), under he specfed assumpons. 3.2.2. B-dreconal Causaly Srucures In he nroducon we dscussed ceran occasons where he me-orderng of nformaon-prce relaonshps may be reversed.e. when nsder radng occurs n he run up o an nformaon even. The esmaor we have presened so far only consders causaly runnng from nformaon o prce. In hs secon we wll show how s easy o generalse he SDM esmaor o cases of b-dreconal causaly. A smple b-dreconal analogue o (1) s he sysem of equaons. y =β 1 x 1 +β 2 x 2...+β τ x τ +u, u=n (0,σ u ) x =α 1 y 1 +α 2 y 2...+α τ y τ +μ, μ=n (0, σ μ ) (20) As here are now probably weghngs assocaed wh he lagged values of eher varable, o smplfy he noaon we nroduce he probably marx W ndexed a, bu conanng a column of values for eher seres. The frs column wll conan he lagged values of x and he second he lagged values of y so ha he enry w,1 would ndcae he 'h lag of x a me and w,2 ndcaes he 'h lag of y a me so ha; 9
={ w W 1,1 2,1 w 1,2 w 2,2 w...... w Ns,1 w Ns, 2} where snce W s a probably marx W =1. (21) We are gong o hen defne he dsance measures assocaed wh each of he lagged values n a marx wh correspondng enres o W.e. ={ 1,1 1,2 d d D...... where d Ns, 1 d d Ns,2},1 =( y x ) 2 and d,2 =(x y ) 2 Due o (15) we can hen wre he squared dsances for sysem of equaons descrbed n (20) as. = Ns u 2 = W,1,1 D =1 whch mples; =Ns (u +μ ) 2 =( =1 =Ns and μ 2 = W,2, D 2, =1 = Ns, W 1 D,1 )+( W,2 D,2 ) or (u +μ ) 2 =W D, (22) =1 usng he scalar produc noaon. Snce he summaon of wo Gaussan dsrbuons s anoher Gaussan dsrbuon, he maxmum lkelhood esmaor for he sum of he squared error erms u and μ s he mean of he globally observed errors - due o (17) we can hen wre hs as. s= λ =1/( 1 W s D s ) (23) s=1 The sysem model s hen appled ndependenly o eher column of he marx and he measuremen model s appled o he D and W marces usng., W j = p(d, j, j λ )W 1 (24) p(d λ ) W 1 The esmaor s agan opmal under he same assumpons as he un-dreconal case. 3.2.3. Posve-Negave Regme Shfs The formulaon of he esmaor we nroduce n secon 3.2.2. shows how we can allow dfferen funconal represenaons of he x, y relaonshp o compee agans each oher for a share of he global probably densy. Anoher case we mgh consder s where here s un-dreconal causaly runnng from x o y, bu here are regme shfs from posve o negave correlaon n he naure of he relaonshp. To capure hs srucure we can defne a dsance marx conanng he posve and negave squared dsances so ha; ={ 1,1 1,2 d d D...... where d Ns, 1 d d Ns,2},1 =( y x ) 2 and d,2 =( y +x ) 2 The res of he esmaor s consruced n he same way as he b-dreconal case descrbed n secon 3.2.2. and remans opmal under he same assumpons. 3.3 Algorhmc Implemenaon 10
Up o hs pon we have assumed ha we would nclude every lag of y n he forecas of x. In mos forecasng scenaros hs would lead o a large al of exremely low probably lags as more me-perods are consdered. I also means ha he compuaonal complexy of he SDM algorhm would scale exponenally wh. Clearly n mos forecasng asks we only wsh o consder a fne number of lagged values of he eher seres and so we can bound Ns o some reasonably small number, n hs case he algorhm scales lnearly wh. Consder he followng pseudo-code mplemenaon of he algorhm for he smple un-dreconal lnear causaly model descrbed n secon 3.2.1. Pseudo Code 1 npus : x[ x 1... x ], y[ y 1... y ], w =1 [w =1...w N =1 ],where w=1/n whle s : θ s = v 1 : s 1, 2 λ s =u 1: s 1 G=0 for [ Ns: 1]: w s s 1 =w s 1 + θ s 3 (w + s 1+w s 1 +w 1 s ) for [1: Ns ]: d s = y s x s, w s = p(d s λ s )w s s 1, G=G+ŵ s v s =0, u s =0 for [1: Ns ]: w s =ŵs /G, v s =v s + w s w s 1, u s =u s +w s 1 s=s+1 The mplemenaon of he algorhm as descrbed s hen no more complex han runnng 3 Ns lengh loops. Noe he necessy for he frs for loop n he code o be reversed runnng from Ns o 1, hs s due o he boundary condon ha where, w Ns+1 = 0, so whou reversng he order of he loop he probably of he w +1 would always be 0. A second mplemenaon consderaon s ha over some lags wll reurn nfnesmal probables n mos programmng languages hs wll eher cause an error as he floang pon numbers overflow he memory lm of he language, or n some cases hs wll resuls n he probably beng se o 0. A smple fx for hs ssue s o se a very small lower lm for he probably of each lag. No dong hs resuls n sub-opmal forecass as he lag srucure becomes ncreasngly pah dependen as more lags are se o 0 over me. d s 4. Expermens on Smulaed Daa In hs secon we are gong o show he resuls of esng he SDM algorhm on a number of seres consruced based on he assumpons of he Granger Causaly model, as specfed n (1), wh he ype of causaly srucures descrbed by Cheung e al (2004) and Soros (2008). The purpose of hs smulaons s no o defne exac mahemacal models represenave of hese heorecal nsghs, raher o provde a number of cases ha could arse n emprcal work amed a evdencng hs area of fnancal heory. In secon 4.1. we wll dscuss he consrucon of hese examples and presen he equaons used o generae he es seres. We wll hen presen resuls n secon 4.2. of he roo mean squared error forecass we acheve usng he SDM procedure o predc he values of he smulaed prce seres, we show forecass for a range of dfferen nose levels. 4.1. Consrucon of Smulaed Seres In he work of Cheung e al (2004) and Soros (2008) he noon of complex causaly srucures s only dscussed n qualave erms, hs leaves no clear gudance as o wha he bes mahemacal model o descrbe such seres would be. To make sure we are coverng a large range of possbles we are gong o es he SDM algorhm on 5 dfferen seres consrucons. For each we are gong o make he same assumpons as he Granger causaly model descrbed n (1), ha he laggng seres, n each of our example cases hs wll be he nformaon seres, s an auoregressve seres consruced usng; x =0.9 x 1 +N (0,1) (25) 11
where he auoregressve erm s 0.9, so ha he seres exhbs sgnfcan bu no nfne memory. Ths follows he esng framework used by Sornee and Zhou (2005). As we don' know he funconal form we would expec me-varyng lags o ake, we are gong o use wo smple funcons. The frs s a sep funcon so ha he lag shfs beween fxed regmes we specfy n advance. The second s a random walk so ha he lag srucure moves abou arbrarly whn a bounded nerval over me. We are only gong o focus on un-dreconal forecass, hs grealy smplfes our resuls as oherwse we have o show wo ses of sascs for each es, one for eher seres dependng on whch s leadng n me. We noe, however, ha we would expec he same qualy forecass n b-dreconal or posve-negave regme shfng cases as n he smple un-dreconal case. 4.1.1. Sep Funcon Models We are gong o consruc wo sep funcon models. The frs s a smple sep funcon where here s a sngle lag a each causally nfluencng he value of he prce seres. The second s a model where here are mulple lags nfluencng he prce seres. For he frs model he prce seres f 1 (x) s consruced as; f 1 (x) = y =x τ +u (26) where he lag lengh τ vares as. τ ={ 5 f 0< <200 20 f 201<< 400 (27) 10 f 401< <600 The error erm u s a mean 0 Gaussan nose, so ha σ u s equal o he roo mean square error (RMSE) we would expec from fng a model of he x, y relaonshp f we knew he complee lag srucure n advance. For he second model, f 2 (x), we wll also nclude he lagged values local o τ n he seres consrucon.e.; f 2 (x) = y = 1 =3 x 7 τ+ +u (28) = 3 So he prce seres s dependen on he 3 values of x mmedaely before and afer τ n me. 4.1.2. Random Walk Models We are also gong o consder wo models where he lag srucure vares due o a random walk model. Here, we consruc he seres smlarly o he sep-funcon models bu replace he lag-lengh funcon (27) wh a rnomal random walk of he form; 1/2, z [ 1, 0] f τ 25 τ =τ 1 +z where f Z 1 /3, z [ 1, 0,1 ] f 5<τ<25, (29) (z)={ 1/ 2, z [0, 1] f τ 5 ndcang ha he random walk s bounded so ha he lag-lengh can only vary beween 5 and 25 meperods. The seres f 3 (x) s hen consruced smlarly o (26).e. a random walk model where only 1 lag s causally nfluencng he prce seres. A second seres f 4 (x) s consruced where he averagng funcon (28) s also appled o he seres. 4.1.3. Fxed-Ineger-Lag Model A fnal model we consder s a fxed-neger-lag model where; f 5 ( x) = y =x 5 +u 12
we nclude hs case as a baselne o show how well he algorhm forecass n cases where here s no obfuscang lag varaon. 4.1.4. Descrpve Sascs and Model Fng For each of he models we use he SDM algorhm exacly as specfed n he pseudo-code example gven n secon 3.3. where we se Ns o 30 me-perods. To generae he forecas we ake he probably weghngs gven o each of he laggng values,.e. each of he values of w -1 and mulplyng hem by he correspondng values of x usng. ŷ =w 1 x Ns : 1 (30) We hen calculae he RMSE for hs forecas agans he observed values of y as; = RMSE( ŷ, y)= 1 ( ŷ y ) 2 (31) =1 Snce we know ha he expeced error beween he seres s equal o σ u we also calculae a sasc for he devaon from expeced forecas error as. FE( ŷ, y)=rmse( ŷ, y) σ u (32) 4.2. Expermenal Resuls Fgure 2 presens he resuls, each pon on eher plo represens he average over 500 rals for each model and nose level, σ μ. The lef hand plo shows he RMSE n 's raw form calculaed usng (31). The rgh hand plo shows he FE sasc calculaed usng (32). We can see from he lef hand plo ha he RMSE sasc racks he value of σ μ closely n all cases. For seres where here s sgnfcan emporal varaon, he random walk model f 3 (x) for example, we can see a perod where he nose level s low σ μ <1, here s a sgnfcan dfference beween he SDM model and he perfec f. Ths s explcable by he fac ha for hese models, he expeced f does no ake no accoun he exra nose from he emporal varaon n he model srucure. Fgure 2: Expermenal Resuls; he lef hand plo shows he RMSE for he SDM model f. The key ndcaes each of he dfferen model ypes so f 1(x) = sep funcon, f 2(x) = sep funcon averagng, f 3(x) = random walk, f 4(x) = random walk averagng and f 5(x) = fxed lag. The rgh hand plo shows he FE sasc for each model. The rgh hand plo also suppors hs heory, we see ha for he fxed-negerlag model SDM generaed model fs very close o he maxmum achevable < 0.1 sandard devaons dfference for all nose levels. As he emporal varance of he models ncreases, we see he nal (low nose) fs generaed by he SDM algorhm geng ncreasngly poorer - alhough he model fs are clearly sll very good. 5. Conclusons The work of Cheung e al (2004) and Soros (2008) hghlghs a key ssue wh he way mos fnancal 13
forecasng research deals wh me-sequencng. Parcpans n fnancal markes do no see he relaonshp beween nformaon and prce as emporally fxed, bu descrbe a suaon where flucuaons n he mng of he nformaon-prce relaonshp are key drvers of he varaon seen n asse prces. In he suaon he auhors descrbe undersandng when nformaon s lkely o effec prce s key o successful forecasng. In hs paper we have nroduced a mehodology ha can capure he ype of flucuaon he auhors descrbe. We have shown ha, under he same assumpons as he sandard Granger causaly approach o mesequencng, he SDM algorhm produces he opmal Bayesan esmae of he forecasng dsrbuon over he values of he laggng seres. Imporanly, he SDM algorhm we presen wll provde opmal Bayesan forecass of he leadng seres n suaons where he lag srucure s no me-varyng, bu also n suaons where s. We noe ha gven he consrucon of he sysem equaons we provde n hs paper would be possble o subsue he Gaussan measuremen model we use for a range of oher more complex models of b-varae relaonshps presen n he leraure. The resul s an esmaon framework for me-varyng lags, whch s opmal under he condons we descrbe, bu also hghly exensble o oher cases. References Arulampalam, M. S., Maskell, S., Gordon, N., & Clapp, T. (2002). A uoral on parcle flers for onlne nonlnear/non-gaussan Bayesan rackng. IEEE Transacons on Sgnal Processng, 50(2), 174 188. Carvalho, C., Lopes, H., & Polson, N. (2009). Parcle learnng for generalzed dynamc condonally lnear models, 1 19. Carvalho, C. M., & Lopes, H. F. (2007). Smulaon-based sequenal analyss of Markov swchng sochasc volaly models. Compuaonal Sascs & Daa Analyss, 51(9), 4526 4542. Cheung, Y-K., Chnn, M.D. and Marsh, I.W., 2004, How do UK-based foregn exchange dealers hnk her marke operaes?, Inernaonal Journal of Fnance and Economcs, 9: 289-306 Gordon N, Salmond D, Smh AFM. 1993. Novel approach o nonlnear/non-gaussan Bayesan sae esmaon. IEE Proceedngs F. Radar Sgnal Process 140: 107 113. Granger, C. W. J. (1969). Invesgang Causal Relaons by Economerc Models and Cross-specral Mehods. Economerca, 37(3), pp. 424 438. Halpn-Healy, T., & Zhang, Y. (1995). Knec roughenng phenomena, sochasc growh, dreced polymers and all ha. Aspecs of muldscplnary sascal mechancs. Physcs repors. Hemsra, C., & Jones, J. (1994). Tesng for Lnear and Nonlnear Granger Causaly n he Sock Prce Volume Relaon. The Journal of Fnance. Hong, Y., Lu, Y., & Wang, S. (2009). Granger causaly n rsk and deecon of exreme rsk spllover beween fnancal markes. Journal of Economercs. Keogh, E. J., and Pazzan, M. J. 1999. Scalng up dynamc me warpng o massve daases. In Prncples of Daa Mnng and Knowledge Dscovery (pp. 1-11). Sprnger Berln Hedelberg. Kalman, R. (1960). A new approach o lnear flerng and predcon problems. Journal of basc Engneerng, 82(Seres D), 35 45. Lopes, H., & Tsay, R. (2011). Parcle flers and Bayesan nference n fnancal economercs. Journal of Forecasng, 209(July 2010), 168 209. Manner, H., & Reznkova, O. (2012). A Survey on Tme-Varyng Copulas: Specfcaon, Smulaons, and Applcaon. Economerc Revews, 31(6), 654 687. P MK, Shephard N. 1999. Flerng va smulaon: auxlary parcle flers. Journal of he Amercan Sascal Assocaon 94: 590 599. Sakura, Y., Yoshkawa, M., & Falousos, C. (2005). FTW: fas smlary search under he me warpng dsance. Proceedngs of he weny-fourh, 1, 326 337. 14
Senn, P. (2008). Dynamc me warpng algorhm revew. Unversy of Hawa a Manoa Honolulu, (December), 1 23. Sharkas, A., Ruskn, H., & Crane, M. (2005). Inerrelaonshps among nernaonal sock marke ndces: Europe, Asa and he Amercas. Inernaonal Journal of, (Ocober 1987), 1 22. Sornee, D., and Zhou, W. X. 2005. Non-paramerc deermnaon of real-me lag srucure beween wo me seres: he opmal hermal causal pah mehod. Quanave Fnance, 5(6): 577-591. Soros, G., 2008, The new paradgm for fnancal markes: The cred crss of 2008 and wha means, PublcAffars, New York. Warren Lao, T. (2005). Cluserng of me seres daa a survey. Paern Recognon, 38(11), 1857 1874. Zhou, W. X., and Sornee, D. 2007. Lead-lag cross-seconal srucure and deecon of correlaed ancorrelaed regme shfs: Applcaon o he volales of nflaon and economc growh raes. Physca A: Sascal Mechancs and s Applcaons, 380: 287-296. 15