Estimation for State Space Models: an Approximate Likelihood Approach

Transcription

1 Esimaio for Sae Sace Models: a Aroximae Likelihood Aroach Richard A. Davis ad Gabriel Rodriguez-Yam Colorado Sae Uiversiy h:// Joi work wih: William Dusmuir Uiversiy of New Souh Wales Yig Wag De of Public Healh W. Virgiia

2 Examle: Daily Ashma Preseaios 99: Ja Feb Mar Ar May Ju Jul Aug Se Oc Nov Dec Year Ja Feb Mar Ar May Ju Jul Aug Se Oc Nov Dec Year Ja Feb Mar Ar May Ju Jul Aug Se Oc Nov Dec Year Ja Feb Mar Ar May Ju Jul Aug Se Oc Nov Dec Year 993

3 Examle: Poud-Dollar Exchage Raes Oc 98 - Jue8 985; Kooma websie day lag 3 log reurs exchage raes - 4 ACF lag lag ACF of squares ACF of abs values

4 Moivaig Examles Time series of cous Sochasic volailiy Geeralized sae-sace models Observaio drive Parameer drive Model seu ad esimaio Geeralized liear models GLM Esimaig equaios Zeger MCEM Cha ad Ledoler Imorace samlig - Durbi ad Kooma Aroximaio o he likelihood Davis Dusmuir ad Wag Alicaio Time series of cous Sochasic volailiy 4

5 Geeralized Sae-Sace Models Observaios: y y... y Saes:... Observaio equaio: y : y - y - Sae equaio: -observaio drive + y : + - y -arameer drive + : + - y 5

6 Examles of observaio drive models Poisso model for ime series of cous Observaio equaio: e y e y ee y y y! y y!... Sae equaio: + µ + θy ex{ }/ex{ /}... where he equaio is defied recursively as a fucio of he as of he y s. Remarks: where This is a examle of a GLARMA see Davis Dusmuir Sree 3 for more deails. Esimaio is relaively sraighforward ca calculae he likelihood i closed form. Sabiliy behavior such as saioariy ad ergodicy is difficul o derive. 6

7 Examles of observaio drive models-co A observaio drive model for fiacial daa: Model GARCHq: Y σ Z {Z }~IID N σ + Y + L + Secial case ARCHGARCH: The resulig observaio ad sae rasiio desiy/equaios are y σ y ; σ σ + Y Y + β σ + L + β q σ q Proeries: Marigale differece sequece. Saioary for [e E E-Euler s cosa. Srogly mixig a a geomeric rae. For geeral ARCH GARCH roeries are difficul o esablish. 7

8 Examles of arameer drive models Poisso model for ime series of cous Observaio equaio: -e y e e y y y!... Sae equaio: Sae variables follow a regressio model wih Gaussia AR oise AR oise: β Τ x + W W φw - + Z {Z }~WNσ The resulig rasiio desiy of he sae variables is is + + ; β Τ x + + φ - β Τ x σ Remark: The case σ corresods o a log-liear model wih Poisso oise. 8

9 Examles of arameer drive models-co A sochasic volailiy model for fiacial daa Taylor `86: Model: Y σ Z {Z }~IID N φ - + ε {ε }~IID Nσ where log σ. The resulig observaio ad sae rasiio desiies are y y ; ex + + ; φ σ Proeries: Marigale differece sequece. Saioary. Srogly mixig a a geomeric rae. 9

10 Exoeial Family Seu for Parameer-Drive Model Time series daa: Y... Y Regressio exlaaory variable: x Observaio equaio: y ex{ + β Τ x y b + β Τ x + cy }. Sae equaio: { } follows a auoregressive rocess saisfyig he recursios γ+φ - +φ φ - + ε where {ε } ~ IID Nσ. Noe: corresods o sadard geeralized liear model. Origial rimary objecive: Iferece abou β.

11 Esimaio Mehods for Parameer Drive Models GLM igores he resece of he lae rocess i.e.. Esimaig equaios Zeger `88: Le βˆ be he soluio o he equaio µ β Γ y µ where µ exx β ad Γ vary. Moe Carlo EM Cha ad Ledoler `95 Imorace samlig Durbi & Kooma ` Kuk `99 Kuk & Che `97: Aroximae likelihood Davis Dusmuir & Wag 98

12 Model: Y x Poisexx T β +. GLM log-likelihood: This likelihood igores resece of he lae rocess. β + β β y y e l T x! log x T Esimaio Mehods Secialized o Poisso Examle GLM esimaio Assumios o regressors: x x x x T s T β Ω γ µ µ Ω β Ω µ Ω ε II s s II I I s x x x x T s T β Ω γ µ µ Ω β Ω µ Ω ε II s s II I I s

13 Theory of GLM Esimaio i Presece of Lae Process Theorem Davis Dusmuir Wag `. Le be he GLM esimae of β obaied by maximizig lβ for he Poisso regressio model wih a saioary logormal lae rocess. The βˆ d / ˆ β β N ΩI + ΩI ΩII ΩI. Noes:. - Ω I - is he asymoic cov marix from a sd GLM aalysis.. - Ω I - Ω II Ω I - is he addiioal coribuio due o he resece of he lae rocess. 3. Resul also valid for more geeral lae rocesses mixig ec 4. The x ca deed o he samle size. 3

14 Assume he { } follows a log-ormal AR where +σ / φ - + σ / +η {η }~IID N σ φ wih φ.8 σ.57. ^ β Z s.e. Ierce Tred cosπ/ -..6 siπ/ cosπ/6..4 siπ/ Alicaio o Model for Polio Daa Zeger GLM Fi Asym Simulaio ^ β GLM s.e. s.e ^ β GLM s.d

15 5 Esimaio Mehods Imorace Samlig Durbi ad Kooma Model: Y x Poisexx T β + φ - + ε {ε }~IID N σ Relaive Likelihood: Le β φ σ ad suose gy ; is a aroximaig joi desiy for Y Y... Y ' ad... '. d L y g d g g L L d g g g d g g ; y ; y y ; y ; y ; y y ; y ; y y g d g g L L d g g g d g g ; y ; y y ; y ; y ; y y ; y ; y y

16 Imorace Samlig co L L g y gy ; g y ; d E g y gy ; y ; where Noes: ~ N N j y gy j ; j j { ; j... N} ~ iid g y;. Aroximaio is oly good i a eighborhood of. Geyer suggess maximizig raio wr ad ierae relacig wih ˆ. This is a oe-samle aroximaio o he relaive likelihood. Tha is for oe realizaio of he s we have i ricile a aroximaio o he whole likelihood fucio. j 6

17 Imorace Samlig examle Simulaio examle: Y Poisex ε {ε }~IID N.3 N log likelihood hi 7

18 Imorace Samlig examle Simulaio examle: Y Poisex ε {ε }~IID N.3 N log likelihood hi 8

19 Imorace Samlig examle Simulaio examle: Y Poisex ε {ε }~IID N.3 N likelihood 5 3 likelihood hi_ hi_-.367 likelihood 8 6 likelihood hi_ hi_.3 9

20 Choice of imorace desiy g: Imorace Samlig co Durbi ad Kooma sugges a liear sae-sace aroximaig model wih Y µ + x T β+ +Z Z ~NH µ y ˆ x' y e ˆ + x' β + ˆ + x' β H e where he ˆ Eg y are calculaed recursively uder he aroximaig model uil covergece. Wih his choice of aroximaig model i urs ou ha g y ; ~ N Γ y~ Γ where y~ Γ y e diag e Xβ+ ˆ Xβ+ ˆ + e Xβ+ ˆ ˆ + E '.

21 Comoes required i he calculaio. gy y~ ' ~ Γ y de Γ simulae from comue simulae from N N Γ Γ y~ Imorace Samlig co y~ Γ Γ Remark: These quaiies ca be comued quickly usig a versio of he iovaios algorihm or he Kalma smoohig recursios. 3

22 Imorace Samlig examle Simulaio examle: β.7 φ.5 σ.3 N 5 realizaios loed likelihood likelihood likelihood hi_-.5 hi_-.5 hi_ likelihood 5 5 likelihood likelihood hi_ hi_ hi_.75 4

23 Esimaio Mehods Aroximaio o he likelihood Geeral seu: where y y de G / ex{ µ T G µ / } T G E µ µ Likelihood: L y d Cosider a Gaussia aroximaio a y φ ; µ Σ o he oserior y y Seig equal he resecive oserior modes a* ad * of a y ad y we have µ * where * is he soluio of he equaio log y G µ 6

24 7 Noes:. This aroximaig oserior is ideical o he imorace samlig desiy used by Durbi ad Kooma. Esimaio Mehods Aroximaio o he likelihood co Machig Fisher iformaio marices: Aroximaig oserior:. I radiioal Bayesia seig oserior is aroximaely a for large see Berardo ad Smih 994. * y log + Σ T G y log ; y * * + φ T a G

25 8 / * * * / * * * * y log de } / ex{ y y / y ;y + µ µ T T a a G G G L Esimaio Mehods Aroximaio o he likelihood co Aroximae likelihood: Noe ha which by solvig for L i he exressio a * y * y ;y y y L we obai ;y y y L / * * * / * * * * y log de } / ex{ y y / y ;y + µ µ T T a a G G G L

26 Esimaio Mehods Aroximaio o he likelihood co Case of exoeial family: L a / G T * T * * T * ;y ex{y { b cy} µ G µ / } K + G / where K diag{ ad * is he soluio o he equaio b } * y b G µ. Usig a Taylor exasio he laer equaio ca be solved ieraively. 9

27 Esimaio Mehods Aroximaio o he likelihood Imlemeaio:. Le be he coverged value of j where j+ b && j + G ad j j j j y~ y b& + & b + Gµ. - y~ j. Maximize y ; wih resec o. a 3

28 Simulaio Resuls Model: Y Poisex ε {ε }~IID N.3 Esimaio mehods: Imorace samlig N udaed a maximum of imes bea hi sigma mea sd Aroximaio o likelihood bea hi sigma mea sd

29 Model: Y Poisex ε {ε }~IID N.3 Arox likelihood desiy 3 4 desiy desiy bea hi Imorace Samlig sigma^ desiy 3 4 desiy desiy bea hi sigma^ 3

30 Alicaio o Model Fiig for he Polio Daa Model for { }: φ - +ε {ε }~IID N σ. Imorace samlig udaed 5 imes for each N 5 Simulaio based o relicaios ad he fied AL model. Imor Samlig Simulaio ˆβ IS Mea SD Ierce Tred cosπ/..3.4 siπ/ cosπ/ siπ/ φ σ Arox Like GLM Simulaio ˆβ AL Mea SD ˆβ GLM SD

31 Alicaio o Model Fiig for he Polio Daa co desiy Arox Likelihood desiy 3 4 desiy bea hi sigma^ desiy Imorace Samlig desiy 3 4 desiy bea hi simga^ 34

32 Sochasic volailiy model: Y σ Z {Z }~IID N Simulaio Resuls γ + φ - + ε {ε }~IID Nσ where log σ ; NR5 CV CV True AL γ.4.49 φ σ True AL γ φ σ.6.7 RMSE IS RMSE RMSE IS RMSE

33 Alicaio o Sydey Ashma Cou Daa Daa: Y... Y 46 daily ashma reseaios i a Cambellow hosial. Prelimiary aalysis ideified. o uward or dowward red aual cycle modeled by cosπ/365 siπ/365 seasoal effec modeled by.5 Tij Tij.55 Pij B where B.55 is he bea fucio ad T ij is he sar of he j h school erm i year i. day of he week effec modeled by searae idicaor variables for Suday ad Moday icrease i admiace o hese days comared o Tues-Sa. Of he meeorological variables max/mi em humidiy ad olluio variables ozoe NO NO oly humidiy a lags of - days ad NO max aear o have a associaio. 5 36

34 Resuls for Ashma Daa IS & AL Term IS Ierce.59 Suday effec.38 Moday effec.9 cosπ/ siπ/365. Term Term Term 99.8 Term Term 99.3 Term Term Term Humidiy H /.9 NO max -.5 AR φ.385 σ.53 AL Mea SD

35 Ashma Daa: observed ad codiioal mea cod mea observed Cous 4 6 Cous Day of Year 99 3 Day of Year 993 Cous Cous Day of Year 3 Day of Year 38

36 Is he oserior disribuio close o ormal? Comare oserior mea wih oserior mode: Ca comue he oserior mea usig SIR samlig imorace-resamlig Poserior mode: The mode of y is * foud a he las ieraio. Poserior mea: The mea of y ca be foud usig SIR. Le... N be ideede draws from he mulivariae disr a y. For N large a aroximae iid samle from y ca be obaied by drawig a radom samle from... N wih robabiliies i N w i i w i y y L ;y i wi i i a a y i i K N. 39

37 Polio daa: blue mea red mode Poserior mea vs oserior mode? smoohed sae vecor

38 Summary Remarks. Imorace samlig offers a ice clea mehod for esimaio i arameer drive models.. The iovaios algorihm allows for quick imlemeaio of imorace samlig. Exeds easily o higher-order AR srucure. 3. Relaive likelihood aroach is a oe-samle based rocedure. 4. Aroximaio o he likelihood is a o-simulaio based rocedure which may have grea oeial esecially wih large samle sizes ad/or large umber of exlaaory variables. 5. Aroximaio likelihood aroach is ameable o boosraig rocedures for bias correcio. 44