arxiv: v1 [stat.ap] 14 Jun 2018

Transcription

1 Sequenal Bayesan nference for spao-emporal models of emperaure and humdy daa Yngyng La Andrew Golghly Rchard Boys School of Mahemacs, Sascs and Physcs, Newcasle Unversy, Newcasle upon Tyne, NE1 7RU, UK arxv: v1 [sa.ap] 14 Jun 018 Summary We develop a spao-emporal model o forecas sensor oupu a fve locaons n Norh Eas England. The sgnal s descrbed usng coupled dynamc lnear models, wh spaal effecs specfed by a Gaussan process. Daa sreams are analysed usng a sochasc algorhm whch sequenally approxmaes he parameer poseror hrough a seres of reweghng and resamplng seps. An eraed bach mporance samplng scheme s used o crcumven parcle degeneracy hrough a resample-move sep. The algorhm s modfed o make more effcen and parallsable. The model s shown o gve a good descrpon of he underlyng process and provde reasonable forecas accuracy. Keywords: Dynamc lnear models (DLMs); sequenal Mone Carlo (SMC); eraed bach mporance samplng (IBIS); parallel compung. 1 Inroducon Clmae s one of he mos mporan envronmenal facors whch plays a crcal role on he global msson of urban susanably. Consequenly, has araced remendous aenon from academc scenss and ndusral expers n recen decades. In hs paper we focus on undersandng he relaonshp beween emperaure and humdy, as hese are wo of he mos mporan facors n drvng oher clmae processes. Our prmary objecve s he developmen of dynamc models whch can be used o undersand he sochasc naure of emperaure and humdy, as well as quanfy her spaal dependences. Moreover, n order o faclae accurae forecass n real me, we focus on developng algorhms whch allow nferences o made sequenally. The leraure conans several emporal models for emperaure a a sngle locaon. For example, [1] proposed an auoregressve (AR) model wh Fourer componens o accoun for seasonaly, a polynomal deermnsc rend and a generalsed auoregressve condonal heeroscedascy (GARCH) error process. Furher AR modellng approaches have been proposed by [], [3] and [4], wh he laer adopng a connuous-me approach. Alhough generc approaches for spaal daa ses are wdely avalable (see e.g. [5], [6], [7], [8], [9], [10] and [11]), relavely few papers have addressed he jon modellng of emperaure and humdy a mulple locaons. [1, 13] use a sochasc paral dfferenal equaon (SPDE) o model yearly emperaure and humdy daa a 10 locaons and perform fully Bayesan nference va an negraed nesed Laplace approxmaon [14]. The modellng approach developed here s movaed by he fne scale emporal naure of he avalable daa. Dynamc lnear models (DLMs) are wdely used for sysem evoluon learnng and shor erm forecasng due o her smple and praccal srucures; see, for example, [15] for an emal: andrew.golghly@ncl.ac.uk 1

2 nroducon. We explo hese properes here by specfyng a margnal DLM for emperaure and a condonal DLM for humdy gven emperaure. We accoun for spaal dependence a nearby locaons by addng a spaal Gaussan process o he sysem equaons, hereby smoohng spaal devaons from he underlyng emporal model. A smlar approach was used by [16] for polluan daa. We perform fully Bayesan nference for he model parameers as each observaon becomes avalable. Snce he poseror dsrbuon s nracable, we use sequenal Mone Carlo (SMC) mehods ha approxmae he poseror dsrbuon a each me pon hrough a se of weghed samples; see [17] for a recen revew of SMC mehods. Alhough he poseror s nracable, he observed daa lkelhood s avalable n closed form, allowng he mplemenaon of he eraed bach mporance samplng (IBIS) scheme, frs nroduced by [18]; see also [19] for a relaed approach. Essenally, parameer samples (known n hs conex as parcles) are ncremenally weghed by he observed daa lkelhood conrbuon of he currenly avalable observaon. Parcle degeneracy s mgaed va a resample-move sep [0] whch moves each parameer parcle hrough a Meropols-Hasngs kernel ha leaves he arge nvaran. Ths sep can be execued subjec o he fulflmen of some degeneracy creron e.g. small effecve sample sze. However, he compuaonal cos of he resample-move sep ncreases as he algorhm ncludes more daa, as requres calculaon of he observed daa lkelhood of all avalable nformaon. To oban an onlne IBIS algorhm, where he compuaonal cos of assmlang a sngle observaon s bounded, we modfy he resample-move sep by basng he observed daa lkelhood on an observaon wndow whose lengh s a unng parameer, chosen o balance accuracy and compuaonal effcency. We use a smulaon sudy o formulae praccal advce on how o choose he sze of hs wndow. Furher compuaonal savngs can be made by employng a hgh performance compung sysem. Whls he weghng and move seps can be performed ndependenly for each parcle, a basc mplemenaon of he resamplng sep requres collecve operaons, such as addng up he parcle weghs. Our approach s o use a smple sraegy whch performs he resamplng sep ndependenly for baches of parameer samples, hus allowng a fully parallel (per parameer bach) mplemenaon of he algorhm o be performed. We quanfy he effec of he approxmaon nduced by hs approach usng synhec daa. Fnally, we apply he onlne IBIS scheme (wh parallel mplemenaon) o he observed daase and examne he model relably and forecas accuracy hrough comparson of observed measuremens wh her poseror predcve dsrbuon. The remander of he paper s organsed as follows. A bref descrpon of he daa s gven n Secon 1.1. The srucures of he spaal DLMs for emperaure and humdy are dscussed n Secon. In Secon 3, we nroduce he IBIS scheme and develop a faser onlne verson and hen compare he performance of boh schemes n Secon 4 va a smulaon sudy. In Secon 5, we repor he full analyss on our Norh Eas daase on emperaure and humdy and draw conclusons n Secon Daa collecon Recen advances n sensor echnology and daa managemen mean ha s now possble o relably and affordably collec daa on many aspecs of cy lfe. The emperaure and relave humdy daa analysed n hs paper were colleced from he Urban Observaory [1], a bg daa hub provdng smar-cy daa va a grd of sensors n Norh Eas England. The daa are receved n real me, and hs requres effcen nework ransmsson and daa sorage soluons. Temperaure s measured n degree Celsus, and relave humdy s measured as he rao of he amoun of waer vapour held n he ar agans he he maxmum amoun of waer vapour he ar can hold a a specfc emperaure. The daa are capured and processed hrough a mcroprocessor nsde a sensor and ransmed va a hgh speed nework o he daabase []. We consder daa sreams a fve locaons: Newcasle upon Tyne, Seaham, Peerlee, Whley Bay and Conse. The observaon perod s from 8h July 017 o 31s December 017. Due o he dfferen recordng frequences of some of he sensors, we

3 Varable Locaon Mssng Prop. Mean Mn. 5% Medan 75% Max. Temperaure Newcasle % ( C) Seaham % Peerlee % Whley Bay % Conse % Humdy Newcasle % (%) Seaham % Peerlee % Whley Bay % Conse % Table 1: A summary of hourly average emperaure and humdy daa over he perod 8h July 017 o 31s December 017 a fve locaons n Norh Eas England. ake he average values of emperaure and relave humdy over every consecuve hour, gvng a oal of 439 me pons a whch a leas one locaon has a measuremen. Fgure 1 shows he mulple daa sreams over me a dfferen locaons. Boh emperaure and relave humdy exhb a clear snusodal paern over each 4 hour perod. Scaer plos of humdy agans emperaure for each locaon are shown n Fgure and reveal a srong negave lnear correlaon. Unforunaely, mssng daa are nevable due o nework dsconnecon or sensor falure. Table 1.1 and Fgure 1 summarse and dsplay he proporon of mssng daa a each locaon durng he observaon perod. Spaal dynamc lnear model (DLM) We develop a jon model for hourly average emperaure and humdy, recorded a each of L locaons. The model s specfed hrough a margnal model for emperaure and a condonal model for humdy gven emperaure. Le X = (X 1,..., X L ) T denoe hourly average emperaure aken over nervals (, +1 ], wh n hours ( = 1,..., n) and Y = (Y 1,..., Y L ) T denoe he correspondng humdy values. In wha follows we scale me so ha 1 = 0..1 Spaal emperaure DLM In Secon 1.1 we noed ha he daa show clear seasonaly n boh emperaure and humdy measuremens. Ths suggess ha margnally each varable should be modelled by a snusodal form wh a 4 hour perod. For smplcy, consder frs a sngle locaon j. We propose a DLM for emperaure wh observaon equaon X j = F x,j θ x,j + v x,j, v x,j ndep N(0, V x,j ), (1) where he observaon marx F x,j = (cos(π /1), sn(π /1), 1) and θ x,j = (θ x,j,1, θx,j,, θx,j,3 )T. Noe ha, afer droppng he superscrps for smplcy, he observaon equaon can be wren as ( ) X = θ π, cos 1 θ,1 + θ,3 + v () 3

4 0 Temperaure ( C) * * * 3/07/17 4/07/17 5/07/17 6/07/17 7/07/17 8/07/17 9/07/17 30/07/17 31/07/17 01/08/ Relave humdy (%) * * * 3/07/17 4/07/17 5/07/17 6/07/17 7/07/17 8/07/17 9/07/17 30/07/17 31/07/17 01/08/17 Fgure 1: Temperaure and relave humdy daa sreams over me a each locaon. Perods of mssngness are ndcaed jus above he x-axs. where he parameers n (1) and () are relaed usng ( ) θ,1 = θ,1 + θ,, θ, = an 1 θ,. (3) θ,1 We allow amplude, phase shf and basal emperaure o be me-varyng, and ake a sysem equaon of he form θ x,j = G x,j θ x,j 1 + k w x,j + p x,j, w x,j ndep N { 0, dag(w x,j ) } (4) where he sysem marx G x,j = I 3, he 3 3 deny marx, and W x,j = (W x,j 1, W x,j, W x,j 3 ) T. Noe ha ncludng k, where k = 1, allows for measuremens o be on an rregularly spaced emporal grd. Furher he erms p x,j = (p x,j,1, px,j,, px,j,3 )T allow for spaal varably beween amplude, phase shf and basal emperaure values a nearby locaons. We model he componens of he spaally smooh error process p x,j usng ndependen zero mean Gaussan process (GP) prors wh covarance funcons fm( ), x m = 1,, 3, ha s, p x,j,m GP {0, f x m( )}, m = 1,, 3. We ake hese covarance funcons o have a smple exponenal form f x m(d jj ) = Cov(θ x,j,m, θx,j,m ) = σ x,m exp( ψ x,m d jj ), m = 1,, 3 and depend on parameers σ x = (σ x,1, σ x,, σ x,3 ) and ψ x = (ψ x,1, ψ x,, ψ x,3 ), wh he laer deermnng he decay rao of he correlaon as he dsance beween wo locaons d jj ncreases [11]. 4

5 Se 1 Se Relave humdy Relave humdy Temperaure Temperaure Se 3 Se 4 Relave humdy Relave humdy Temperaure Temperaure Se 5 Relave humdy Temperaure Fgure : Scaer plos of emperaure agans relave humdy a each locaon. The full spaal DLM model (over all locaons) can be wren as X = F x θ x + v x, v x θ x = θ x 1 + k w x + p x, ndep N{0, dag(v x,1,..., V x, L )}, w x ndep N{0, dag(w x,1,..., W x, L )}, (5) where F x = dag(f x,1,..., F x, L ), θ x = ((θ x,1 ) T,..., (θ x, L ) T ) T and he 3 L-vecor of spaal effecs p x = ((px,1 ) T,..., (p x, L ) T ) T s normally dsrbued wh zero mean and covarance marx f x (d 11 )I 3... f x (d K x 1 L )I 3 = f x (d L1 )I 3... f x (d L L )I Addonal harmoncs Addonal harmoncs can be ncorporaed by usng a Fourer form srucure [see e.g. 15, 3]. For ease of exposon, we assume regularly spaced daa a mes = 1, = 1,..., n. The observaon marx n (1) s defned o be he 1 (q + 1) marx paroned as F x,j = (1, 0 1, ) so ha 5

6 he sae vecor θ x,j s of lengh q + 1 and sasfes a sysem equaon of he form (4) wh sysem marx G x,j = dag(h 1,..., H q, 1), where he H r are harmonc marces ( ) cos (πr/1) sn (πr/1) H r =, r = 1,..., q. sn (πr/1) cos (πr/1) The number of harmoncs q mus be specfed by he praconer. Noe ha for he full spaal emperaure DLM, specfyng q harmoncs wll gve L(q + 1) + 6 sac parameers o be nferred. Consequenly, Fourer models wh q = 1 or are ypcally used n pracce [3]. For he q = 1 harmonc and he rval case of W x,j = 0, he observaon equaon of he Fourer form DLM concdes wh ha he snusodal form n (1) gven by X j = θ x,j 0,1 cos (π /1) + θ x,j 0, sn (π /1) + θ x,j 0,3 + vx,j. However, when W x,j 0, he error srucures dffer due o he use of he harmonc n he sysem equaon of he Fourer form DLM, and n he observaon equaon for he snusodal form DLM. The ask of choosng beween compeng models s consdered n Appendx A.3.. Spaal humdy DLM Due o he srong lnear relaonshp beween emperaure and humdy, we specfy a condonal DLM for humdy by regressng on emperaure n he observaon equaon. For a parcular locaon j, he DLM akes he form Y j θ y,j = F y,j θ y,j + v y,j, v y,j ndep N ( 0, V y,j) = θ y,j 1 + k w y,j + p y,j, w y,j ndep N{0, dag(w y,j )} = (X j, 1), θ y,j = (θ y,j,1, θy,j, )T and W y,j = (W y,j 1, W y,j where F y,j he componens of he spaal error process p y,j process prors wh covarance funcons ) T. As n Secon.1, we assgn = (p y,j,1, py,j, )T ndependen zero mean Gaussan f y m(d jj ) = Cov(θ y,j,m, θy,j,m ) = σ y,m exp( ψ y,m d jj ), m = 1,. The spaal humdy DLM hen akes he form Y = F y θ y + v y, vy θ y = θ y 1 + k w x + p y, ndep N{0, dag(v y,1,..., V y, L )} wy ndep N{0, dag(w y,1,..., W y, L )} (6) where F y = dag(f y,1,..., F y, L ), θ y = ((θ y,1 ) T,..., (θ y, L ) T ) T and he L-vecor of spaal effecs p y s dsrbued analogously o p x. Noe ha he jon model gven by (5) and (6) nduces a margnal model for hourly average humdy wh he snusodal paern observed n he daa. Afer negrang ou X j n he observaon equaon for Y j, we oban Y j = F x,j θ x,j θ y,j,1 + θy,j, + vy,j + θ y,j,1 vx,j whch exhbs he same snusodal srucure of (1), albe wh a dfferen amplude, phase and basal level. I s clear ha he jon model for (X j, Y j ) T s no a DLM, as he margnal humdy n a nonlnear way. Neverheless, he facorsaon of he jon model as margnal and condonal DLMs can be exploed when performng nference for he model parameers, and hs s he subjec of he nex secon. model depends on θ x,j and θ y,j 6

7 3 Sequenal Bayesan nference 3.1 Seup Fng he model for emperaure and humdy descrbed n Secon o daa s complcaed by he fac ha n pracce, sensor daa s somemes mssng a one or more locaons. To deal wh hs scenaro, we le X o and Y o denoe he observed emperaure and humdy processes a me. We assume ha f emperaure s mssng a locaon j a me, hen so s humdy (and vce-versa), as s he case for our applcaon. The observaon model can hen be wren as X o = P X, Y o = P Y (7) where he n L ncdence marx P deermnes whch componens are observed a me. For example, f we have daa sreams from 5 dfferen locaons and daa are mssng a he second and hrd locaon a me, hen he ncdence marx s P = Le φ x denoe he flaened vecor of V x,1,..., V x, L, W x,1,..., W x, L, σ x and ψ x. Defne φ y smlarly. Gven observaons x o 0: and y o 0: a mes 0 = 1 < <... <, our prmarly goal s sequenal exploraon of he margnal poseror densy π(φ x, φ y x o 0:, y o 0: ). We assume ha φ x and φ y are ndependen a pror wh pror densy π(φ x, φ y ) = π(φ x )π(φ y ). Bayes heorem gves he poseror densy of neres as π(φ x, φ y x o 0:, y o 0: ) π(φ x )π(φ y )π(x o 0:, y o 0: φ x, φ y ) = π(φ x )π(φ y )π(x o 0: φ x )π(y o 0: x o 0: φ y ) π(φ x x o 0: )π(φ y x o 0:, y o 0: ) (8) and so he parameer ses φ x and φ y are ndependen a poseror. Moreover, we have ha π(φ x x o 0: ) π(φ x x o 0: 1 )π(x o x o 0: 1, φ x ) π(φ y x o 0:, y o 0: ) π(φ y x 0: 1, y o 0: 1 )π(y o x 0:, y o 0: 1, φ y ) (9) where he observed daa lkelhood conrbuons π(x o x o 0: 1, φ x ) and π(y o x o 0:, y o 0: 1, φ y ) can be calculaed usng a forward fler [15]. Deals of hs calculaon can be found n Appendx A Ieraed bach mporance samplng Alhough he parameer poseror s nracable, he form of (9) suggess a sequenal mporance samplng scheme ha repeaedly reweghs a se of N parameer samples (known as parcles n hs conex) by he observed daa lkelhood conrbuons. Ths approach s used n he eraed bach mporance samplng (IBIS) algorhm of [18], ogeher wh MCMC seps for rejuvenang parameer samples n order o crcumven parcle degeneracy. Gven he facorsaon of he poseror n (8), n wha follows we focus on recursve samplng from π(φ x x o 0: ) and noe ha he seps for samplng from π(φ y x o 0:, y0: o ) are smlar. Suppose ha a weghed sample {φ x (k), ω (k) } N k=1 from π(φ x x o 0: ) s avalable. The IBIS algorhm nvolves wo seps: an ncremenal weghng sep and a rejuvenaon (resample-move) sep. In he ncremenal wegh sep, he wegh s updaed for each parcle hrough he observed daa lkelhood conrbuon of he curren observaon,.e. ω (k) ω (k) 1 π(x o x o 0: 1, φ (k) x ). Noe 7

8 ha he calculaon of he observed daa lkelhood ncremen (as gven by he forward fler n Appendx A.1) requres he poseror summares m 1 (φ (k) x ) = m (k) 1 and C 1 (φ (k) x ) = C (k) 1 of π(θ x x o 0: 1, φ x ). Smply updang he ncremenal weghs over he me wll lead o parcle degeneracy. To bypass hs problem, he IBIS scheme uses a resample-move sep [see e.g. 0] ha frsly resamples parameer parcles (e.g. by drawng ndces from a mulnomal M(ω 1:N ) dsrbuon) and hen moves each parameer sample hrough a Meropols-Hasngs kernel whch leaves he arge poseror nvaran. The resample-move sep s only used f some degeneracy creron s fulflled. Typcally, a each me, he effecve sample sze (ESS) s compued as ESS = 1 / N (ω (k) ) k=1 and he resample-move sep s rggered f ESS < δn for δ (0, 1) and a sandard choce s δ = 0.5. As he parameers mus be srcly posve, we ake a proposal densy q(φ x φ x ) = log N { φ x; log φ x, γv ar(log φ x x o 0: ) } where log N( ; m, V ) denoes he densy assocaed wh he exponenal of a N(m, V ) random varable. We use he sandard rule of humb of [4] and [5] by akng he scalng parameer γ =.38 /n par, where n par s he number of parameers. The full IBIS scheme s gven by Algorhm 1. Fnally, we noe ha s sraghforward o esmae he evdence π(x o 0: n ) = n π(x o x o 0: 1 ) =1 usng he oupu of he IBIS scheme, a vrually no addonal compuaonal cos. Each facor L = π(x o x o 0: 1 ) n he produc above s esmaed by L 1 = N k=1 3.3 Onlne IBIS 1 N π(xo 0 φ (k) x ), L = N k=1 ω (k) 1 π(x o x o 0: 1, φ (k) x ), =,..., n. (10) The man compuaonal boleneck of IBIS s he resample-move sep. If hs sep s rggered a me, hen he observed daa lkelhood π(x o 0: φ x) mus be calculaed for each proposed parcle φ x. Consequenly, he compuaonal cos grows wh, precludng he use of IBIS as an onlne scheme. To bound he compuaonal cos of assmlang a sngle observaon, we modfy he resample-move sep by basng he observed daa lkelhood on an observaon wndow whose me lengh s chosen o balance accuracy and compuaonal effcency. We follow a smlar approach nroduced by [6] and defne a sequence of wndows wh equal wdhs, say T, over he observaon perod. Frs he observaon perod s dvded no b wndows and denoe by x o s he h observaon n wndow s {1,..., b}, for = 1,..., n s. The observaon mes sasfy s ((s 1)T, st ] when s = 1,..., b 1 and s ((b 1)T, b n b ] when s = b. The sandard IBIS scheme s run over he frs wndow. For wndows s =,..., b, he resample-move sep arges π(φ x x o 0: s) π(φ x x 0:(s 1)T )π(x o s 1 :s xo 0:(s 1)T, φ x) (11) where π(φ x x 0:(s 1)T ) = 1 N N k=1 8 log N(φ x ; log φ (k) x, h s)

9 Algorhm 1 IBIS scheme 1. Inalsaon. For k = 1,..., N sample φ (k) x = 1 of he forward fler. Sore m (k) 1 For =,..., n: and C (k) 1.. Sequenal mporance samplng. For k = 1,..., N: π( ) and se ω (k) 0 = π(x o 0 φ(k) x ) usng eraon (a) Perform eraon of he forward fler o oban π(x o x o 0: 1, φ (k) x ), m (k) Noe he convenon ha π(x o 0 φ(k) x ) = π(x o 0 xo 0: 1, φ (k) x ). (b) Updae and normalse he mporance weghs usng ω (k) = ω (k) 1 π(x o x o 0: 1, φ (k) x ), ω (k) = (c) Updae he observed daa lkelhood usng π(x o 0: φ (k) x ω (k) N j=1 ω(j) ) = π(x o 0: 1 φ (k) x )π(x o x o 0: 1, φ (k) x ). 3. If ESS < δn resample and move as follows. For k = 1,..., N: (a) Sample ndces a k M ( ω 1:N) and se {φ x (k), ω (k) π(x o 0: φ (a k) x ), m (k) := m (a k) and C (k) := C (a k). } := {φ (a k) x and C (k)., 1}, π(x o 0: φ (k) x ) := (b) Propose φ x q( φ (k) x ). Perform eraons 1,..., of he forward fler o oban π(x o 0: φ x). Wh probably pu φ (k) x mn { 1, π(φ x)π(x o 0: φ x) π(φ (k) x )π(x o 0: φ (k) x } x φ x) q(φ x φ (k) ) q(φ (k) x ) := φ x, π(x o 0: φ (k) x ) := π(x o 0: φ x), m (k) := m and C (k) := C. s a kernel densy esmae (KDE) of π(φ x x 0:(s 1)T ) and he bandwdh h s can be calculaed usng, for example, Slverman s rule of humb [7] as h s = 1.06 N /5 V ar(φ (1:N) x x o 0:(s 1)T ). Thus n order o evaluae (11), we need only evaluae he observed daa lkelhood conrbuon from he begnnng of he curren wndow unl he curren me. Furhermore, by akng he proposal densy o be q(φ x φ x ) = π(φ x x 0:(s 1)T ), he kernel densy esmae need no be evaluaed n he MH accepance rao. The choce of he wndow wdh has a drec nfluence on compuaonal effcency and poseror accuracy. A smulaon sudy comparng IBIS and onlne IBIS for dfferen wndow lenghs s gven n Secon 4.. The onlne IBIS scheme s summarsed by Algorhm. 3.4 Parallelsng he algorhm The ncremenal weghng seps are readly parallelsed n an SMC scheme. Addonally, for IBIS he move sep can be performed ndependenly for each parcle. However, commonly used resamplng schemes, such as he mulnomal approach consdered here, nvolve a collecve operaon 9

10 Algorhm Onlne IBIS scheme 1. Inalsaon. Dvde he observed perod no b wndows, s {1,..., b}. Denoe by s he h observaon me n wndow s, = 1,..., n s. For s = 1, mplemen he IBIS scheme (Algorhm 1). For s =,..., b and = 1,..., n s :. Sequenal mporance samplng. For k = 1,..., N: (a) Perform eraon (correspondng o me s ) of he forward fler o oban π(x o s xo 0: s, φ(k) x ), m (k) 1 s and C (k) s. (b) Updae and normalse he mporance weghs usng ω (k) s = ω (k) s π(xo s xo 1 0: s, φ(k) 1 x ), ω (k) s = ω (k) s N z=1 ω(z) s (c) Updae he observed daa lkelhood conrbuon n he curren wndow usng π(x o s 1 :s xo 0:(s 1)T, φ(k) x ) = π(x o s 1 :s xo 1 0:(s 1)T, φ(k) x )π(x o s xo 0: s, φ(k) 1 x ), wh he convenon ha π(x o s 1 :s xo 0:(s 1)T, φ(k) x ) = π(x s 1 x o 0:(s 1)T, φ(k) x ) for = If ESS < δn resample and move. For k = 1,..., N: (a) Sample ndces a k M ( ω 1:N) and se {φ (k) x C (a k) s, ω (k) s } := {φ (a k) x, 1}, m (k) s and π(x o s 1 :s xo 0:(s 1)T, φ(k) x ) := π(x o s 1 :s xo 0:(s 1)T, φ(a k) x ). := m (a k) s, C (k) s := (b) Propose φ x log N(log φ (k) x, h s). Usng m (s 1)T = m(k) (s 1)T and C (s 1)T = C(k) (s 1)T, perform eraons 1,..., (correspondng o mes s 1,..., s ) of he forward fler o oban π(x o s 1 :s xo 0:(s 1)T, φ x). Wh probably mn 1, π(x o s 1 :s x o 0:(s 1)T, φ x) π(x o s 1 :s xo 0:(s 1)T, φ(k) x ) pu φ (k) x C (k) s := C s. := φ x, π(x o s 1 :s xo 0:(s 1)T, φ(k) x ) := π(x o s 1 :s xo 0:(s 1)T, φ x), m (k) s := m s and (summng he weghs) precludng obvous parallelsaon of he full IBIS scheme. [8] and [9] descrbe a forward adder ree mehod whch parallelses he calculaon of he cumulave wegh. [30] sugges parallel Meropols resamplng and rejecon resamplng schemes o mgae numercal nsables of summng he weghs for a large number of parcles. However, hese mehods sll requre nformaon exchange and global operaons and hey are desgned manly for use on GPU shared memory sysems. Dsrbued memory sysems are naurally amenable o heavy parallelsed jobs, where runks of jobs are allocaed and processed over mulple cores n dfferen processors. In hs conex, a number of parallel resamplng mehods have been dscussed n he leraure; see, for example, [31] and [3, 33]. We follow he local resamplng mehod [31] by paronng parcles no dsjon subses, whn whch resamplng s performed. The algorhm proceeds by frs calculang a local ESS for each subse of parcles. If a local ESS s less han a hreshold, hen he rejuvenaon 10

11 sep s rggered locally. The nnovaon varance for he MH proposal n he move sep s also calculaed locally based on he ndvdual parcle subse. To mgae load-balance problems ha can occur when he resample-move sep s execued for some subses bu no ohers, we also carry ou a rejuvenaon sep a regular me pons, e.g. every 0 me pons. Ths approach naurally fs whn he dsrbued memory archecure and allows full parallelsaon of he IBIS scheme. In prncple, hs approach should sgnfcanly mprove compuaonal effcency of he nference scheme, as here s no need for ask communcaon. However, n pracce he number of nformave parcles may reduce sgnfcanly n some subses as he algorhm runs. Ths n urn resuls n he rejuvenaon sep beng execued more frequenly. Therefore, a rade-off has o be consdered carefully beween he number of parcle subses and he number of parcles n each subse. Secon 4.1 descrbes a smulaon sudy comparng a sandard seral mplemenaon wh a fully parallelsed verson (wh local resamplng). 4 Smulaon sudy In order o assess he performance of he proposed onlne IBIS scheme and he effec of local resamplng, we looked a resuls from synhec daa generaed from he margnal model n (5). We consder spaal locaons (gvng 14 parameers n oal) and smulaed n = 1300 observaons a each locaon. The rue parameer values used o produce he synhec daa are W j k = 0.01, V j = σ k = 1 and ψ k = 0.01 for j = 1, and k = 1,, 3, and hese values are shown n Fgure 3. As hs s a daa-rch scenaro, we assumed very weak ndependen nverse Gamma IG(1, 0.01) pror dsrbuons for all hese parameer componens, bu runcaed hem above a 10 as values n excess of 10 are far from plausble. We also ook he pror dsrbuon for he nal sysem sae as θ 0 N(m, C), where m = (0, 0, 17, 0, 0, 17) T and C = I 6. We used 10 7 parcles and an ESS hreshold of δ = 0.5 for rggerng he resample-move sep. All compuer code was wren n C and execued on a hgh performance cluser wh Inel Xeon E5-699 v4 processors (. GHz, 55 MB cache), where each processor has cores (.9 GB CPU memory per core). 4.1 Comparson of full IBIS wh seral resamplng and parallelsed local resamplng We consder frs wo parallelsed mplemenaons of he full IBIS scheme: () weghng and move seps are performed n parallel over cores hrough a shared memory sysem (whn one processor) wh he resamplng sep performed n seral; () parcles are dvded over 00 cores and local resamplng s used. Fgure 3 shows he parameer margnal poseror denses obaned by usng mehod 1 (IBIS wh seral resamplng) and mehod (IBIS wh parallelsed local resamplng). I s clear ha boh approaches gve poseror oupu conssen wh he rue values (used o smulae he daa). Moreover, he poseror denses from he fully parallelsed mehod mach up well wh hose from he exac (smulaon based) mehod 1. However he run me for mehod 1 (IBIS wh seral resamplng) s around 3 hours whereas ha for mehod (IBIS wh parallelsed local resamplng) s around 4 hours, a speed-up of around a facor of Comparson of full IBIS and onlne IBIS We now compare he full IBIS scheme wh onlne IBIS and n boh schemes we use he parallelsed local resamplng mehod. For onlne IBIS, we consder hree wdhs for he fxed wndow: T = 100, 300 and 500. Fgure 4 shows he oupu of he margnal poseror denses from he onlne IBIS scheme for each wndow sze, ogeher wh he denses from he full IBIS scheme. As expeced, as he larger wndow ncreases, so does poseror accuracy. The margnal poserors from onlne IBIS usng T = 300 and T = 500 almos overlay hose from full IBIS. However, here are noceable dfferences when usng T = 100. In erms of compuaonal effcency, onlne IBIS wh boh 11

12 T = 300 and T = 500 ake roughly CPU hours, ha wh T = 100 akes approxmaely 1 CPU hour. Consequenly, for hs example, onlne IBIS wh T = 300 and local parallel resamplng gves an overall reducon n compuaonal cos of around a facor of 1 compared o full IBIS wh seral resamplng. 5 Applcaon In hs secon we analyse he daa on hourly average emperaure and humdy values nroduced n Secon 1.1. Recall ha hese daa are measuremens recorded durng he perod 8h July 017 o 31s December 017 and ha he observaons are rregularly spaced due o nework and sensor falures. We ake ndependen nverse Gamma IG(1, 0.01) pror dsrbuons, runcaed above a 10, for all he sac parameers n boh emperaure and humdy DLMs. To ncorporae our pror belef ha he underlyng sysem should be smooher han he observaon process, we also mpose he consran ha a each locaon j = 1,..., 5, W x,j < V x,j ( = 1,, 3) and W y,j k < V y,j (k = 1, ). We ran he onlne IBIS scheme wh N = 10 7 parcles, fully parallelsed (wh local resamplng) over 00 cores usng an ESS hreshold of δ = 0.5. Regular parcle rejuvenaon seps were se up for he process a every 0 me pons, and he resample-move sep was execued n any bach whose ESS fell below half he number of parcles (n he bach). Fnally, o balance accuracy and compuaonal effcency, we used a wndow wdh of T = 1500, and hs gave a run me of approxmaely 9.5 days. 5.1 Inference resuls Table shows he margnal poseror medans and quanle-based 95% credble nervals for he sac parameers n he jon emperaure and humdy model. These summares were obaned from oupu of he onlne IBIS scheme. Inspecon of he poseror medans for he sysem varances (governng boh emperaure and humdy models) reveals ha hese componens are larger a locaon 1 (Newcasle) han a he oher locaons. Ths s perhaps no surprsng gven ha locaon 1 has he larges fracon of mssng daa (see Table 1.1). Also sampled poseror values of he observaon varance componens V x,j and V y,j are generally very much larger a locaon (Seaham), and hs oo s conssen wh he smple daa summares n Table 1.1 Seaham s he leas spaally conssen locaon n erms of medan emperaure and humdy. Varaon across ses s accouned for by he elemens of σ. The relavely large values of σx,3 and σ y, sugges ha here s some spaal nconssency n he dynamcally varyng mean level componens θ x,j,3 and θ y,j,. Spaal conssency of hese mean level componens can be assessed furher by nong ha Cor(θ x,j,3, θx,j,3 ) = exp( ψ x,3d jj ), Cor(θ y,j,, θy,j, ) = exp( ψ y,d jj ). Hence, fxng ψ x,3 and ψ y, a her poseror medans gves a smple lnear relaonshp beween dsance and log correlaon. For example, whn a 10km radus from each locaon, here s a spaal correlaon of a leas 0.76 for emperaure and 0.64 for humdy. These areas are dsplayed n Fgure 5. We noe ha s no surprsng ha spaal correlaon for humdy s lower han ha for emperaure, as he humdy records are also easly nfluenced by oher facors, such as urban srucure and dsance from he sea, n addon o emperaure. 5. Predcve performance We assess he valdy of he proposed model by comparng observed daa wh her model-based whn-sample poseror predcve dsrbuons and wh model-based ou-of-sample forecas dsrbuons. Smulaon mehods can be used o consruc hese dsrbuons and deals on how o generae draws from hem s provded n Appendx A.. Fgure 6 shows dscrepances beween 1

13 Densy Densy Densy W W W 3 1 Densy Densy Densy W W W 3 Densy Densy Densy V V σ 1 Densy Densy Densy σ σ ψ 1 Densy Densy ψ ψ 3 Fgure 3: Margnal parameer poseror denses obaned from he oupu of he full IBIS scheme wh a sandard seral resamplng sep (hsograms) and a parallelsed local resamplng sep ( ). The rue parameer values are shown as sold crcles. 13

14 Densy Densy Densy W W W 3 1 Densy Densy Densy W W W 3 Densy Densy Densy V V σ 1 Densy Densy Densy σ σ ψ 1 Densy Densy ψ ψ 3 Fgure 4: Margnal parameer poseror denses obaned from he oupu of he full IBIS scheme (hsograms) and he onlne IBIS scheme wh wndow wdhs T = 100 ( ), T = 300 ( ) and T = 500 ( ). The rue parameer values are are shown as sold crcles. 14

15 Temperaure Humdy φ x Medan.5% 97.5% φ y Medan.5% 97.5% W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, W y, W x, V y, W x, V y, W x, V y, W x, V y, W x, V y, V x, σy, V x, σ y, V x, ψ y, V x, ψ y, V x, σ x, σ x, σ x, ψ x, ψ x, ψ x, Table : Margnal parameer poseror medans and quanle-based 95% credble nervals obaned from he oupu of he onlne IBIS scheme. observaons and her whn-sample predcve dsrbuon over he frs 500 hours a each of he 5 locaons. These dsrbuons are characersed by her mean and 95% credble nerval. I s clear ha he mean dfference a each me-locaon combnaon s small and ha a mean dfference of zero s plausble (he 95% credble nervals nclude zero). Smlar resuls were obaned for he full daa se (no shown). Fgure 7 shows he mean and 95% credble nerval a each locaon for he one-sep ahead forecas. The mes dsplayed were chosen a random over a wo day perod and, for comparson purposes, he observaons a hese mes are also shown. Unsurprsngly forecas uncerany grows durng perods of prolonged mssngness. The fgure shows ha observaons ypcally le whn he forecas nerval and ha he model-based one-sep forecas dsrbuon s conssen wh he observed daa. Fgure 8 shows he mean and 95% credble nerval a each locaon for he wo-sep ahead forecas. Smlar o he one-sep forecass, hs fgure shows ha hese forecas dsrbuons are conssen wh he daa bu, of course, have larger uncerany. 15

16 Fgure 5: Map showng se locaons and a 10 km radus from each se, whn whch he spaal correlaon for emperaure s a leas 0.76, and for humdy, s a leas Dscusson We have developed and fed a spao-emporal model o around sx monhs of daa on hourly emperaure and humdy values a fve locaons n he Norh Eas of England. The daa were obaned from a sensor nework provdng sreamng daa on envronmenal varables such as clmae, polluon and raffc flow, held a he Newcasle Urban Observaory. The model we use for observed seasonaly n emperaure s a dynamc lnear model (DLM) whose observaon equaon akes he form of a snusod, wh me varyng amplude and phase descrbed by he sysem equaon. We capure he observed lnear relaonshp beween humdy and emperaure va a condonal DLM n whch humdy s regressed on emperaure. Also spaal conssency a nearby ses s accouned for by addng a Gaussan process n he sysem equaons. Our prmary goal s real me forecasng of emperaure and humdy. To hs end, we have developed a sequenal Mone Carlo (SMC) algorhm whch updaes he parameer poseror as each measuremen becomes avalable. The racably of he observed daa lkelhood allows us o consruc he SMC algorhm usng an eraed bach mporance samplng (IBIS) scheme, frs nroduced by [18]. The IBIS scheme res o deal wh parcle degeneracy by employng a resamplemove sep whch allows he parcle se o be rejuvenaed by movng each parcle hrough a Meropols-Hasngs kernel ha leaves he arge poseror nvaran. The compuaonal cos of hs sep ncreases as he algorhm runs, due o he me aken o calculae he observed daa 16

17 Temperaure (1) 0 1 Humdy (1) Temperaure () 0 1 Humdy () Temperaure (3) 0 1 Humdy (3) Temperaure (4) 0 1 Humdy (4) Temperaure (5) 0 1 Humdy (5) Fgure 6: Mean ( ) and 95% credble nervals for he dfference beween he whn-sample predcve and he observaons, a each locaon (1 5) over me. The observaon perod s from 8h July :00:00 o 9h July :00:00. 17

18 Temperaure (1) 10 0 Humdy (1) Temperaure () 10 0 Humdy () Temperaure (3) 10 0 Humdy (3) Temperaure (4) 10 0 Humdy (4) Temperaure (5) 10 0 Humdy (5) Fgure 7: One-sep ahead forecas means ( ) and 95% credble nervals, a each locaon (1 5) over me. The observaons are ndcaed ( ). The observaon perod s from 1h July :00:00 o 14h July :00:00. 18

19 Temperaure (1) 10 0 Humdy (1) Temperaure () 10 0 Humdy () Temperaure (3) 10 0 Humdy (3) Temperaure (4) 10 0 Humdy (4) Temperaure (5) 10 0 Humdy (5) Fgure 8: Two-sep ahead forecas means ( ) and 95% credble nervals, a each locaon (1 5) over me. The observaons are ndcaed ( ). The observaon perod s from 1h July :00:00 o 14h July :00:00. 19

20 lkelhood a each parcle, as more daa s ncluded. Ths problem s made much more accue by he long lengh of he observed me seres and he hgh dmenson of he parameer space and hs makes he algorhm unusable as an on-lne algorhm. To crcumven hs ssue, we have modfed he resample-move sep n wo ways. Frs, we use a sequence of observaon wndows and calculae he observed daa lkelhood for he daa whn he wndow. As he daa n each wndow are ncluded, he parameer poseror (a he sar of he wndow) s approxmaed usng a kernel densy esmae and hen updaed usng he observed daa lkelhood for he wndow. Ths places an upper bound on he compuaonal cos. We looked he effec of he choce of wndow lengh on compuaonal effcency and poseror accuracy and found ha reasonable poseror accuracy can be acheved for modes wndow lengh. Fnally, we speed up he algorhm by usng a fully parallel mplemenaon whch dvdes he parcles no baches and performs he resamplng sep locally, for each bach. We erm he resulng scheme onlne IBIS and fnd ha for our daa se, an observaon (conssng of boh emperaure and humdy hourly averages a each of fve locaons) can be assmlaed n around 3 mnues on average, wh hs average me domnaed by he rejuvenaon seps. One-sep and wo-sep forecas dsrbuons can hen be deermned very quckly. Gven ha observaons arrve every hour, hs makes he scheme enrely feasble for use n real me. Ths work can be exended n a number of ways. For example, covarae nformaon such as alude, dsance from he coas and wnd drecon/speed could be ncluded n he model. Unforunaely hs nformaon s no currenly avalable. Developng a jon model for all sensor sreams, whch would also nclude polluon daa and raffc daa, s also of neres. However, fng models of mulple heerogeneous sensors s lkely o requre furher mehodologcal developmen of he nference scheme consdered here. References [1] S.D. Campbell and F.X. Debold. Weaher forecasng for weaher dervaves. Journal of he Amercan Sascal Assocaon, 100:6 16, 005. [] W.K. Härdle and B.L. Cabrera. The mpled marke prce of weaher rsk. Appled Mahemacal Fnance, 19:59 95, 01. [3] F.E. Benh, J.S. Benh, and S. Koekebakker. Pung a prce on emperaure. Scadnavan Journal of Sascs, 34: , 007. [4] J.S. Benh and F.E. Benh. A crcal vew on emperaure modellng for applcaon n weaher dervaves markes. Energy Economcs, 34:59 60, 01. [5] N. Cresse. Sascs for Spaal Daa. Wley-Inerscence, [6] M.L. Sen. Inerpolaon of Spaal Daa: Some Theory for Krgng. Sprnger, [7] B.D. Rpley. Spaal Sascs. Wley-Inerscence, 004. [8] P. Dggle and P.J. Rbero. Model-based Geosascs. Sprnger, 004. [9] A.E. Gelfand, P. Dggle, M. Fuenes, and P. Guorp. Handbook of Spaal Sascs. CRC Press, 010. [10] N. Cresse and C.K. Wkle. Sascs for Spao-Temporal Daa. Wley-Inerscence, 011. [11] S. Banerjee, B.P. Carln, and A.E. Gelfand. Herarchcal Modelng and Analyss for Spaal Daa. CRC Press, nd edon,

21 [1] X. Hu, F. Lndgren, D. Smpson, and H. Rue. Mulvarae Gaussan random felds wh oscllang covarance funcons usng sysems of sochasc paral dfferenal equaons. Avalable from hps://arxv.org/abs/ , 013. [13] X. Hu, I. Sensland, D. Smpson, S. Marno, and H. Rue. Spaal modellng of emperaure and humdy usng sysems of sochasc paral dfferenal equaons. Avalable from hps://arxv.org/abs/ , 015. [14] H. Rue, S. Marno, and N. Chopn. Approxmae Bayesan nference for laen Gaussan models by usng negraed nesed Laplace approxmaons. Journal of he Royal Sascal Socey: Seres B (Sascal Mehodology), 71:319 39, 009. [15] M. Wes and J. Harrson. Bayesan Forecasng and Dynamc Models. Sprnger, nd edon, [16] G. Shaddck and J. Wakefeld. Modellng daly mulvarae polluan daa a mulple ses. J. R. Sas. Soc. C., 51:351 37, 00. [17] P. Fearnhead and H. Künsch. Parcle flers and daa assmlaon. Avalable from hps://arxv.org/abs/ , 018. [18] N. Chopn. A sequenal parcle fler mehod for sac models. Bomerka, 89: , 00. [19] N. Chopn, P.E. Jacob, and O. Papasplopoulos. SMC : an effcen algorhm for sequenal analyss of sae space models. J. R. Sas. Soc. B., 75:397 46, 013. [0] W. R. Glks and C. Berzun. Followng a movng arge Mone Carlo nference for dynamc Bayesan models. J. R. Sas. Soc. B., 63:17 146, 001. [1] P.M. James, R.J. Dawson, N. Harrs, and J. Joncyzk. Urban Observaory Envronmen. Newcasle Unversy. hp://dx.do.org/ / [] F. Galaoo, M.C. Bell, and G. Hll. Undersandng he characerscs of he mcroenvronmens n urban sree canyons hrough analyss of polluon measured usng a novel pervasve sensor array. Envronmenal Monorng and Assessmen, 186: , 014. [3] G. Pers, S. Perone, and P. Campagnol. Dynamc Lnear Models wh R. Sprnger, 009. [4] G.O. Robers, A. Gelman, and W.R. Glks. Weak convergence and opmal scalng of random walk Meropols algorhms. The Annals of Appled Probably, 7:110 10, [5] G.O. Robers and J.S. Rosenhal. Opmal scalng for varous Meropols-Hasngs algorhms. Sascal Scence, 16: , 001. [6] P. Del Moral, A. Jasra, and Y. Zhou. Based onlne parameer nference for sae-space models. Mehodology and Compung n Appled Probably, 19:77 749, 017. [7] B.W. Slverman. Densy Esmaon for Sascs and Daa Analyss. Chapman & Hall/CRC, [8] G. Hendeby, R. Karlsson, and F. Gusafsson. Parcle flerng: The need for speed. EURASIP Journal on Advances n Sgnal Processng, pages 1 9, 010. [9] P. Gong, Y. Bascfc, and F. Ozguner. A parallel resamplng algorhm for parcle flerng on shared-memory archecures. 01 IEEE 6h Inernaonal Parallel and Dsrbued Processng Symposum Workshops and PhD Forum, pages , 01. 1

22 [30] L.M. Murray, A. Lee, and P.E. Jacob. Parallel resamplng n he parcle fler. Journal of Compuaonal and Graphcal Sascs, 5: , 016. [31] O. Brun, V. Teulere, and J.M. Garca. Parallel parcle flerng. Journal of Parallel and Dsrbued Compung, 6: , 00. [3] M. Bolć, P.M. Djurć, and S. Hong. Resamplng algorhms for parcle flers: A compuaonal complexy perspecve. EURASIP Journal on Advances n Sgnal Processng, 15:67 77, 004. [33] M. Bolć, P.M. Djurć, and S. Hong. Resamplng algorhms and archecures for dsrbued parcle flers. IEEE Transacons on Sgnal Processng, 53:44 450, 005. [34] R. E. Kass and A. E. Rafery. Bayes facors. Journal of he Amercan Sascal Assocaon, 90: , [35] S. Frühwrh-Schnaer. Bayesan model dscrmnaon and Bayes facors for lnear Gaussan sae space models. Journal of he Royal Sascal Socey, Seres B, 57:37 46, A Appendx A.1 Forward fler To smplfy noaon we consder he spaal emperaure model and drop x. Gven he form of he observaon model n (7), we have ha X o = F θ + ṽ, θ = θ 1 + w, ṽ ndep N(0, Ṽ ), w ndep N(0, W ), (1) where F = P F, Ṽ = P dag(v 1,..., V L )P T and W = k dag(w 1,..., W L ) + K. Snce he parameers φ reman fxed hroughou hs secon, we drop hem from he noaon where possble. Now suppose ha θ 1 N(m, C) a pror and recall ha 1 = 0. The observed daa lkelhood ncremens π(x o x o 0: 1 ), and hence he full observed daa lkelhood π(x o 0: n φ), can be obaned from he forward fler descrbed n Algorhm 3. A. Whn-sample predcons and ou-of-sample forecass In order o compue whn-sample predcons, he smoohng densy π(θ 0:n x o 0: n, φ x ) s requred. Draws from hs densy can be readly obaned by usng a backward sampler ha recursvely draws from π(θ θ +1, x o 0:, φ x ) = N{θ ; m + B (θ +1 m ), C B R +1 B T }, (13) where B = C R 1 +1 and R +1 = C + W ; see, for example, [15]. Hence, gven an equally weghed sample {φ 1:N x } from he margnal poseror π(φ x x o 0: n ), we can negrae over parameer uncerany o generae draws from he whn-sample sysem poseror predcve densy π(θ 0:n x o 0: n ) by recursvely drawng from (13) for each parcle φ (k) x (and he assocaed quanes m (k), C (k) generaed by he forward fler). Subsequenly, he whn-sample observaon poseror predcve densy π(x 0:n x o 0: n ) can be sampled by drawng X (k) θ (k), φ (k) x N(F θ (k), V (k) ), = 1,..., n, k = 1,..., N.

23 Algorhm 3 Forward fler 1. Inalsaon ( = 1). Compue π(x o 1 ) = N(x 1 ; F1 m, F1 C F T 1 me 1 = 0 s herefore θ 1 x o 1 N(m 1, C 1 ), where Sore he values of m 1, C 1 and π(x o 1 ).. For =,..., n, m 1 = m + C F T 1 ( F 1 C F T 1 + Ṽ ) 1 (x o 1 F 1 m) C 1 = C C F T 1 ( F 1 C F T 1 + Ṽ ) 1 F1 C. + Ṽ ). The poseror a (a) Pror a. Usng he sysem equaon, we have ha θ x o 0: 1 N(m 1, C 1 + W ). (b) One sep forecas. Usng he observaon equaon, we have ha X o x o 0: 1 N{ F m 1, F (C 1 + W ) F T + Ṽ }. Compue he observed daa lkelhood ncremen π(x o x o 0: 1 ) = N{x o ; F m 1, F (C 1 + W ) F T + Ṽ }. (c) Poseror a. Combnng he dsrbuons n (a) and (b) gves he jon dsrbuon of θ and X o (condonal on x 0: 1 ) as ( θ X o ) {( ) ( m 1 C 1 + N, W F m 1 F (C 1 + W ) and herefore θ x o 0: N(m, C ), where (C 1 + W ) F T F (C 1 + W ) F T + Ṽ m = m 1 + (C 1 + W ) F T { F (C 1 + W ) F T + Ṽ } 1 (x o F m 1 ) C = C 1 + W (C 1 + W ) F T { F (C 1 + W ) F T + Ṽ } 1 F (C 1 + W ). Sore he values of m, C and π(x o x o 0: 1 ). )} Ou-of-sample sysem and observaon forecas dsrbuons can be obaned by agan explong he lnear Gaussan srucure of he DLM. Gven an equally weghed sample {φ 1:N x } from he margnal poseror π(φ x x o 0: n ), samples from π(θ n+1 x o 0: n ) and π(x n+1 x o 0: n ) can be obaned by recursvely drawng θ (k) n+1 φ (k) x x (k) n+1 φ (k) x A.3 Model selecon N(m (k) n, C (k) n + W (k) ), k = 1,..., n N{F n+1 m (k) n, F n+1 (C (k) n + W (k) )F T n+1 + V (k) }, k = 1,..., n. As noed n Secon.1, seasonaly n he margnal DLM can be accouned for n wo ways. A snusod can be specfed n he observaon equaon, wh a sysem equaon descrbng he evoluon of he parameers governng he amplude and phase. Alernavely, a Fourer form srucure can be used n he sysem equaon where he approprae number of harmoncs mus be specfed by he praconer. Our jon model consss of a margnal DLM for emperaure and a condonal DLM for humdy gven emperaure. Ths nduces a margnal DLM for humdy 3

24 wh he same form as ha for emperaure. We herefore consder hree canddae spaal DLMs for modellng emperaure and humdy daa margnally: 1. snusodal form DLM (sdlm);. Fourer form DLM wh 1 harmonc (FDLM1); 3. Fourer form DLM wh harmoncs (FDLM). Choosng beween hese compeng models s possble va compuaon of he Bayes facor [34, 35], whch, under he assumpon of equal pror probably for wo compeng models, say M1 and M, s defned as he rao of he evdence gven M1, and ha gven M. The Bayes facor based on emperaure daa s herefore BF = p(xo 0: n M1) p(x o 0: n M) wh a smlar form for he humdy daa Bayes facor. Noe ha BF < 1 suggess he daa suppor M. Equaon (10) gves an esmae of he evdence as a by-produc of he IBIS scheme. Unforunaely, he sze of he observed daase precludes calculaon of he Bayes facor usng all measuremens a all ses. Therefore, o gude our modellng approach we chose hree of he fve locaons a random and hen 400 consecuve observaons (sarng a a random observed me) a hese locaons. The evdence for each model was deermne usng he full IBIS scheme on hese daa wh a seral mulnomal resamplng sep for each model, usng N = 10 7 parcles. To accoun for Mone Carlo error, we repea hs process 30 mes. Takng FDLM as a baselne for comparson, we compue Bayes facors for sdlm vs FDLM and FDLM1 vs FDLM. Fgure 9 shows he mean log BF value (and 95% credble nerval) based on daa x o 0: and yo 0: agans. For he margnal emperaure DLM s clear ha FDLM s he leas favoured model. Furhermore, for > 80, he log Bayes facors correspondng o he snusodal form DLM agans FDLM are always srcly greaer han hose correspondng o FDLM1 agans FDLM. For he margnal humdy DLM, here s lle dfference n overall f beween he snusodal form DLM and FDLM1. Gven ha compuaonal cos scales as 1 : 1.1 : 1.3 for DLM : F DLM1 : F DLM, we conclude ha he snusodal form DLM offers he bes compromse beween model f and compuaonal effcency. 4

25 log Bayes facor (emperaure) snusodal DLM vs. FDLM FDLM1 vs. FDLM snusodal DLM vs. FDLM FDLM1 vs. FDLM log Bayes facor (humdy) Fgure 9: Mean and 95% credble nerval of he log Bayes facor comparng sdlm agans FDLM and FDLM1 agans FDLM, over me. (Top: emperaure models; boom: humdy models.) 5