Minimally Conditioned Likelihood for a Nonstationary State Space Model. José Casals. Sonia Sotoca. Miguel Jerez. Universidad Complutense de Madrid

Transcription

1 Minimally Condiioned Likelihood for a Nonsaionary Sae Space Model José Casals Sonia Sooca Miguel Jerez Universidad Compluense de Madrid Absrac: Compuing he gaussian likelihood for a nonsaionary sae-space model is a difficul problem which has been ackled by he lieraure using wo main sraegies: daa ransformaion and diffuse likelihood. he daa ransformaion approach is cumbersome, as i requires nonsandard filering. On he oher hand, in some nonrivial cases he diffuse likelihood value depends on he scale of he diffuse saes, so one can obain differen likelihood values corresponding o differen observaionally equivalen models. In his paper we discuss he properies of he minimally-condiioned likelihood funcion, as well as wo efficien mehods o compue is erms wih compuaional advanages for specific models. hree convenien feaures of he minimally-condiioned likelihood are: (a) i can be compued wih sandard Kalman filers, (b) i is scale-free, and (c) is values are coheren wih hose resuling from differencing, being his he mos popular approach o deal wih nonsaionary daa. Keywords: Sae-space models; Condiional likelihood; iffuse likelihood; iffuse iniial condiions; Kalman filer; Nonsaionariy eparameno de Fundamenos del Análisis Económico II. Faculad de Ciencias Económicas. Campus de Somosaguas Madrid (SPAIN). jcasalsc@cajamadrid.es eparameno de Fundamenos del Análisis Económico II. Faculad de Ciencias Económicas. Campus de Somosaguas Madrid (SPAIN). sooca@ccee.ucm.es Corresponding auhor. eparameno de Fundamenos del Análisis Económico II. Faculad de Ciencias Económicas. Campus de Somosaguas Madrid (SPAIN). mjerez@ccee.ucm.es, el: (+34) , fax: (+34)

2 . Inroducion he mos popular approach o deal wih nonsaionary daa consiss of differencing he daa o induce saionariy, being his ransformaion useful boh, o specify a model and o compue is gaussian likelihood. his approach is simple and suiable in many cases. No so much in many ohers such as, e.g., when one wans o esimae non-muliplicaive models, such as ime-varying parameer regressions or srucural ime series models (Harvey, 989). Also, i resuls in unnecessary daa losses when he sample includes missing values or if he model has coinegraion consrains (Mauricio, 2006). Finally, for many pracical purposes such as, e.g., forecasing or signal exracion, i is more convenien working wih original insead of differenced daa. In all hese cases, i would be ineresing o esimae he nonsaionary model. Compuing he likelihood for a model wih uni roos is a difficul problem which has been ackled by he sae-space lieraure using wo main sraegies: daa ransformaion and diffuse iniializaion. he mos represenaive work in he daa ransformaion approach is Ansley and Kohn (985), hereafer AK, who proposed a sophisicaed daa ransformaion ha cancels he nonsaionary componens of he model. As AK recognize, heir approach has wo shorcomings: i needs a complex and nonsandard filering and requires he daa ransformaion o be independen of he parameer values. his requiremen is no fulfilled, for example, when one wans o esimae srucural ime series models (Harvey, 989). he AK approach has been furher developed in many relevan works (Kohn and Ansley, 986; Ansley and Kohn, 990, in he univariae case; Bell and Hillmer, 99; Gomez and Maravall 994), bu none of hem addressed he wo preciously menioned issues. he diffuse likelihood approach considers an iniial sae where some componens could have an arbirarily large covariance. Building on his idea, e Jong 2

3 (99) defined he diffuse likelihood funcion and proved ha i is a proper likelihood, as i is based in he daa ransformaion ha makes he daa invarian o he iniial diffuse sae. In comparison wih he AK algorihm, he main advanage of e Jong (99) proposal was ha i used a sandard filer, augmened wih he propagaion of a vecor and a marix, having each as many rows as he diffuse sae vecor. Following also he diffuse iniializaion sraegy, Koopman (997) proposed decomposing he iniial sae, x, as: x = Ah + Bd (.) where he erm Ah corresponds o he saionary srucure, where A is a fixedcoefficiens marix and h N ( 0, I). On he oher hand, B d corresponds o he diffuse saes, wih d N ( 0, ki) and k. Finally B is a coefficien marix ha mus be deermined heurisically in each case. Koopman (997) compues hen he likelihood by running wo differen filers, which propagae he covariances resuling from he diffuse and saionary subsysems respecively. Boh filers collapse o a unique sandard Kalman Filer (hereafer, KF) when he number of recursions is sufficien o eliminae he dependence on k. his algorihm has wo weak poins. Firs, he size of he sample required o eliminae his dependence is known only when he model is univariae and here are no missing values; in oher cases i mus be deermined heurisically. Second, he double filering procedure requires using generalized inverses, being hese inverses complex, unsable and compuaionally expensive. In his work we presen he compuaion and heoreical advanages of he minimally condiioned likelihood for a sae-space model. his approach has hree clear benefis. Firs, in comparison wih he daa ransformaion alernaives, i only requires sandard filering. Among oher advanages, his means ha our procedures can cope wih missing daa and coinegraion consrains. Second i is scale-invarian, while in 3

4 some cases he diffuse likelihood depends on he scale of he diffuse saes. his is illusraed by he examples in sub-secions 2. and 5.2, which show ha here could be differen diffuse likelihood values corresponding o observaionally equivalen models. hird, our mehod provides likelihood values idenical o hose resuling from differencing when boh approaches can be compared. he minimally condiioned likelihood funcion can be efficienly compued by wo differen bu equivalen mehods ha we call: Sae ecomposiion (S) and Column eleion (C), respecively. Secion 3 describes he S mehod, which is based on some ideas due o e Jong (988). I builds on a decomposiion of he condiional likelihood which separaes he effecs of boh, he diffuse and non-diffuse saes. Under hese condiions, one can compue he likelihood by applying a KF wih null iniial condiions o he sample and hen correcing he effec of he arbirary iniializaion. When he model marices are ime-invarian and here are no missing values in he sample, one can apply he filer simplificaion proposed by Casals e al. (999) o improve he sabiliy and compuaional efficiency of he algorihm. he C algorihm, described in Secion 4, is srucurally similar o ha of Koopman (997), as i uses an augmened filer o evaluae recursively he likelihood. Is main advanage in comparison wih Koopman s mehod is ha he columns corresponding o he augmened variables are auomaically eliminaed as he sample is processed. herefore, he recursion collapses o a sandard KF in he minimum number of ieraions and here is no need o se his number heurisically. Second, he augmened equaions are efficienly compued using he QR algorihm, hus avoiding he use of generalized inverses. Secion 5 presens wo examples illusraing he properies of our mehods and Secion 6 discusses in deail he relaive advanages of boh algorihms, provides some 4

5 concluding remarks and indicaes how o obain a free MALAB oolbox which implemens he mehods described in his paper. All he proofs for he formal resuls are given in he Appendices. 2. ifferen forms of he likelihood funcion 2.. iffuse likelihood model: Consider he m random vecor z, which is he oupu of he sae-space x + = Φ x + E w (2.) z = Hx + C v (2.2) where Φ, E, H, and C are fixed coefficien marices, x is a n vecor of sae variables and dimensions of w, v are zero-mean uncorrelaed vecors of errors, such ha he E w and v ( v ) R, and ( w v ) cov = cov, = S C are n and m respecively, wih ( ) cov w = Q, Noe ha model (2.)-(2.2) assumes wihou loss of generaliy ha: (a) he parameer marices are ime-invarian and (b) here are no exogenous inpus. Assuming he immemorial ime hypohesis (e Jong, 99) he iniial sae of a nonsaionary sysem includes a diffuse componen wih infinie uncerainy. I is hen easy o isolae his componen by applying a similar ransformaion o he iniial sae, which yields: é ù = x Mx ê ú (2.3) N ëx where M is he marix characerizing he ransformaion, x is a d vecor ha includes he diffuse saes, such ha cov( x ), and x N is a n d saionary componens. enoing - = [ ] M G we can wrie (2.3) as: vecor of 5

6 x = x N + G x (2.4) which is equivalen o he decomposiions of e Jong (99) and Koopman (997). On his basis, boh works discuss he evaluaion of he diffuse log-likelihood defined as: ( ) L( ) ( ) log L Z = log Z - log cov x (2.5) 2 where log L ( Z) denoes he diffuse log-likelihood of model (2.)-(2.2), log L ( Z ) is he corresponding gaussian log-likelihood and Z is he sample. AK (985, heorem 5.) and e Jong (99, heorem 4.2) proved ha (2.5) is a proper log-likelihood, as i is based on he componens of Z which are invarian o x. ha is, i coincides wih he log-likelihood of he sample afer ransforming i o avoid dependence on he diffuse componens of he iniial sae vecor. On his basis, e Jong (99) proposes an evaluaion algorihm based on he so-called diffuse KF, while Koopman (997) suggess using wo specialized filers for he diffuse and non-diffuse componens, respecively. he previous approach has a clear shorcoming, as he value of he diffuse likelihood may depend on he scale of he sae vecor. o see his, consider e.g., he observaionally equivalen models: x = + + x w z = a x + v (2.6) x = +a + x w z x v = + where a is an arbirary consan, var( w ) =, ( ) (2.7) cov w, v = 0 and x = a x. According o (2.5), he diffuse likelihood of (2.6) and (2.7) are, respecively: ( a) () ( ) log L z = log L z - log cov x (2.8) 2 ( a) () ( a ) log L = log L - z z log cov x (2.9) 2 6

7 and hese values do no coincide because log L ( a) - log L ( a) = log ( a) z z. Noe ha his problem also affecs he firs-order derivaives because: ( z a) log L ( z a) log L = + a a a (2.0) herefore, he values of he diffuse likelihood corresponding o equivalen represenaions, such as (2.6) and (2.7), can be differen. x In general, any linear ransformaion of he iniial diffuse vecor such ha = L x would yield he iniial sae decomposiion (2.4), wih ( x ) x L x G x, see - N = + cov. Under hese condiions, he diffuse likelihood would be: ( Z ) L( Z) ( x ) log L = log - log cov 2 (2.) which obviously depends on he ransformaion marix - L Condiional likelihood An alernaive o he diffuse likelihood would consis of compuing a gaussian likelihood, condiional o he minimum subse of he sample required o eliminae he effec of he diffuse saes. As we will see, his sraegy is closely relaed o he diffuse likelihood approach, bu is unaffeced by he scale of he diffuse saes. I is well known ha equaion (2.2) can be wrien in marix form as: where Z = O x + Z (2.2) Z is he par of he sample ha does no depend on x and O is he exended observabiliy marix, defined as: é H ù HΦ O = ê n- ú ëhφ (2.3) Applying he decomposiion (2.4) o (2.2) we obain: 7

8 N Z = O x + Z (2.4) where = + N N Z ON x Z, O = O is he exended observabiliy marix corresponding o he diffuse iniial saes and O = OG is he analogous marix N affecing he non-diffuse iniial saes. herefore, Z N is he par of he sample ha is no affeced by he diffuse iniial saes. Under hese condiions, here always exiss a marix A such ha AO = I wih rank( A) = dimension( x ). hen, premuliplying boh sides of (2.4) by A we obain: A Z = x N + A Z (2.5) enoing A Z º U and aking condiional expecaions of boh sides of (2.4) we obain: ( ) = ( ) E ZU O E x U (2.6) N Noe ha ( ) E Z U = 0 as his condiional expecaion depends on he inverse of cov( U ), which is null. Hence, he condiional covariance of he sample is: ( ZU) ( O A ) V( O A ) cov = I- I - (2.7) N where V ºcov( Z U) and he covariance marix given by (2.7) is finie and compuable, as i only depends on he saionary par of Z. Choosing = ( ) - A O O O yields he ransformaion proposed by AK (985). However, here are oher valid and more convenien choices for A. In his paper, we æ ö - will use O = A ç ( O O ), where O includes he firs columns of O so ha çè 0 ø - æ ö rank ( O) = rank ( O ) and ( ) ( ) O AO = O O O 0 = I, where O ç has been çèo æ ö 2ø pariioned as O O = ç. Noe ha hese expressions are paricularized for he firs çèo 2ø rank O rank O would observaions in he sample, bu any oher subsample wih ( ) ( ) have been a valid choice. = 8

9 he likelihood of Z condiional o U is given by wo main erms: (a) he deerminan of cov( Z U ), given in (2.7) and (b) a weighed sum of squares of he observaions. Abou he firs, expression (2.7) immediaely implies: ( Z U) ( O A ) V( O A ) cov = I- I - (2.8) and ( -O A ) I can be wrien as: æ ö = - ç - çè O ( ) 2 O O O I ø ( I O A ) (2.9) aking ino accoun he srucure of (2.9) and applying some well-known algebraic resuls, (2.8) can be wrien as: cov ( ZU) = - V O V O O O (2.20) As for he quadraic erm, is expression is: ( ) - Z cov Z U Z é ù ê = ë ú é - ù - = Z V Z -Z V O êë O V Oú O V Z (2.2) he mos efficien way o compue (2.20)-(2.2) consiss of applying a sandard KF o he observaions Z. If we denoe by F he en-bloc linear KF reducing he observaions o uncorrelaed innovaions, Z = FZ, hen B= FVF, where B is a block-diagonal marix of innovaion variances. herefore: - - V = F B F (2.22) and he quadraic erm in (2.2) would be: ( ) - Z écov ù ë Z U Z = é ù - Z B Z - Z B O êë O B O ú O B Z (2.23) where O is defined as O = F O, which is he resul of applying a KF o he columns of O. he firs addend in he righ-hand-side of (2.23) corresponds o he sum of squared innovaions associaed o a KF wih he iniial condiions ( G N, ) x P. 9

10 he second addend is a correcion ha compensaes he effec of condiioning o a minimal subsample over he likelihood. On he oher hand, using he resul (2.22) he deerminan in (2.20) reduces o: cov ( ZU) ( ) O O O O F B F O F B F O = = B O B O Finally he condiional log likelihood, ignoring consan erms, would be: é - ZU = log B+ log O B O - log O O + 2 êë Z B Z Z B O O B O O B Z ù ( ) ( ) ú (2.24) (2.25) Comparing (2.25) wih he diffuse likelihood of e Jong (99, heorem 4.2) i can be seen ha: ( ) log ( ) ZU Z O O (2.26) = L -log where log L ( Z ) denoes he diffuse log-likelihood. Expression (2.26) implies ha he condiional and diffuse log-likelihood funcions coincide bu for he addend -log O in sub-secion 2.. O which is very imporan, as i avoids he undesirable scale effec described Resul (2.26) can be very useful o implemen a likelihood compuaion procedure. Specifically, if one has he code required o calculae he diffuse log-likelihood, hen i would be enough o add he correcion -log O O o obain a condiional likelihood algorihm. On he oher hand, expression (2.26) is concepually imporan because i characerizes he condiioning se employed. For example, (2.26) is condiional o he beginning of he sample because he correcion is compued using O. I would be easy o compue correcions relying on oher sample sub-ses or even o he whole sample, which would require using -log O O. Noe ha he condiional likelihood in his case would coincide wih he marginal likelihood of Francke, Koopman and de Vos (200). 0

11 Finally, he condiional approach provides likelihood values ha are coheren wih hose obained by differencing he daa; for a formal proof, see Appendix. 3. Sae decomposiion (S) algorihm he efficiency of he algorihm oulined in Secion 2 can be improved by segregaing he erms affeced by he iniial condiions. o his end, consider again he expression (2.4): [ ] é ê x ù ú N Z = Ox + Z = O ON + Z ê Nú ëx (3.) where, Z is he par of he sample ha does no depend on N ( ) N x N Gx, P, wih P > 0. cov, and x ; ( x ) heorem: Expression (3.) implies ha he deerminan (2.20) and he quadraic erm (2.2) of he likelihood funcion can be wrien, respecively, as: cov ( ZU) = ( ) - V P + O V O P O O - ( ) ú - é ù ( ) - ( ) êp + ( ) ú ( ) é ù ê = ë Z cov Z U Z Z V Z Z V O O V O O V Z ë (3.2) (3.3) é0 0 ù where = ê - ú ë0 P V Z U is finie and O= [ O O ] P ; º cov( ) Proof. See Appendix 2. N he main advanage of using (3.2)-(3.3) insead of (2.20)-(2.2) is ha hese expressions separae he effecs of boh, he diffuse and non-diffuse iniial saes. his allows us o apply an idea due o e Jong (988), consising of compuing efficienly

12 he likelihood by propagaing a KF wih iniial condiions ( x N,0) G and aferwards correcing he effec of his ad-hoc iniializaion. his approach yields he simplified KF: z = z - x - H (3.4) B HP H CRC = + - (3.5) ( Φ ) K = P H + ESC B (3.6) - - x ˆ = Φxˆ + K z (3.7) + - ( Φ ) ( ) ( ) P = Φ P + EQE + K CRC K KESC -CS E K (3.8) where z are he innovaions, B is covariance marix, K is he KF gain, ˆ + x is an esimae of he sae vecor a ime + condiional o he informaion available up o ime, P+ is is covariance and Φ = - Φ KH. his filer is augmened wih he addiional equaions: ( ) ( ) - = w w F H B z wih w 0 = 0 (3.9) ( ) ( ) - = W W F H B HF wih W 0 = 0 (3.0) Φ = Φ Φ - wih Φ 0 = I (3.) - where he erms O ( V ) Z and ( ) by w N and - O V O in (3.3) and (3.2), respecively, are given W N, defined in (3.9) and (3.0). he condiional log-likelihood would hen be: ìæ - ( ZU) ïç é - = íç z B z + log Bù - wn M ( + MWNM) MwN + çå ê ë ú P 2 ïè ïî = + log P + log P+ M WN M - log O O + ë log p - ) é ( 2 )( ù N d)} (3.2) where is he sample size, N = m, being m he dimension of z, and d is he number of diffuse saes. If he sample includes some missing values he value of mus be adjused accordingly. 2

13 Iniializing he filer (3.4)-(3.) wih ( x N,0) G simplifies he propagaion equaions. Specifically, in a ime-invarian innovaions model (e.g., VARMAX) wih no missing values, he soluion o he Riccai algebraic equaion associaed o (3.8) is null and, herefore, he iniial condiion P = 0 implies ha P = 0 " - (Casals e al. 999). his propery simplifies he likelihood compuaion because if P = 0 ", i is - no necessary o propagae equaions (3.5), (3.6) and (3.8). Addiional efficiency can be obained by compuing he erm O O and he number of diffuse iniial saes, d, by applying he QR decomposiion o he marix H, where denoes he marix in (2.4), see Appendix 3. he simplificaion described above can be exended o any general ime invarian model, see Casals e al. (999), so i provides a very efficien way o evaluae he minimally-condiioned likelihood for many common represenaions such as, e.g., VARMAX or srucural ime series models. 4. Column deleion (C) algorihm he condiional log-likelihood ( ZU ) given in (2.26) can also be compued by an alernaive column deleion algorihm, which is srucurally similar o Koopman (997) mehod. d nd d efining x = x + x, wih x = x, see (2.4), we can wrie he sae d nd d nd equaion (2.) as: ( ) x + + x + = Φ x + x + E w, and hen, break i ino he corresponding diffuse (superindex d ) and non-diffuse (superindex nd ) equaions: x = x (4.) d d + Φ nd nd x + = Φx + E w (4.2) d nd Accordingly, (2.2) can be wrien as H( ) z = x + x + Cv or, equivalenly, as: 3

14 z = x + z (4.3) H d nd nd nd where z = H x + Cv is he non-diffuse componen of he endogenous variables. In =, he observer (4.3) would be: z = H x d + z nd, or, in compare noaion: nd z = H x + z (4.4) where H = H. his expression shows ha z depends on he diffuse vecor x, which uncerainy is infinie. Accordingly, i is no possible o deermine he likelihood of z or o apply a sandard KF o compue he likelihood of he sample. his problem can be ackled by: ) ecomposing x ino wo componens, one formed by he linear combinaions of x which affec z, and anoher one which componens affec he res of he sample ( z, z,, z 2 3 N ). 2) Esimaing he par of x which depends on z, condiional o his value. 3) Repeaing sep 2) by successively including he values z2, z3,, z N unil he dimensions of he erm affeced by he diffuse condiions collapse o zero. he following Subsecions describe in deail hese seps. 4. Firs Sep: ecomposiion Consider he marix Q, which spans a d-dimensional space, such ha i can be pariioned as Q = é ^ ù ê ë Q Q H H ú, where Q is he d d H marix ha generaes he rowsubspace of H ^ and he d d2 marix Q generaes he subspace orhogonal o ha of H H, so ha H Q ^ = 0. he dimensions of hese marices are d + d 2 = d, wih d m H. Under hese condiions x can be decomposed as follows: x = Q a + Q a ^ ^ H H (4.5) where a and a are ( d ) and ( ) ^ Subsiuing (4.5) in (4.4) yields: d vecors of diffuse iniial saes respecively. 2 4

15 where z = H a + z nd (4.6) = Q is a full rank marix, wih ( H ) H H H rank = d m. Noe ha he number of diffuse saes ha affec z reduces in his firs sep from d o d. In his siuaion we can esimae of a condiional o he informaion in z : - é - ù - ˆ B B z ( H ) H ( H ) a = ê ú (4.7) ë wih he condiional covariance: - é - ù H cov( ˆ ) ( H ) nd being = cov( ) a ˆ z = ê B ú (4.8) ë B z a compuable and finie value marix which can always be invered since H is full rank, see e Jong (988). Since here is a par of x ha can be esimaed wih he informaion in z, i would be convenien o derive a specialized filer for (4.5)-(4.6) such ha he propagaion of he diffuse saes disinguishes he par corresponding o a, which uncerainy condiional o z is finie and, accordingly, should be aken ino accoun. herefore, we can re-organize (4.)-(4.3) a =, aking ino accoun (4.5) as: d d d ^ x = Φ x wih x = Q a 2 H ^ nd nd aking ino accoun ha x = Φ x + E w, we obain: 2 nd nd x = ΦQ a + x (4.9) 2 H 2 and: nd z = HQ a + z (4.0) H Building on (4.9)-(4.0) and Casals, Jerez and Sooca (2000, pp. 6) he esimaes for he mean and variance of nd x 2, condiional on z, are: ( ) nd nd x ˆ = xˆ + Φ -K H Q a ˆ (4.) 2 2 H 5

16 ( Φ ) cov ˆ ( ˆ ) ( ) Φ nd nd P = P + -K H Q a z Q -K H (4.2) 2 2 H H and xˆ nd nd 2, P 2 and K (he KF gain) can be compued by applying a sandard KF o he saionary subsysem. On he oher hand, he esimae, â, and is covariance, ( ) cov ˆ a ˆ z, are given by (4.7) and (4.8). 4.2 Second Sep: Esimaion In =2, z = HF x + z d nd 2 2, so: z HFQ a z (4.3) ^ ^ nd 2 = + H 2 where he saes ^ a affec z bu do no affec z 2. hen, he second innovaion can be d nd wrien as z = z - zˆ = x H 2 z and we are in he same siuaion as in 2 2 = because cov( z nd 2 ) is finie and has he expression: ( z nd ) HP nd H CRC cov 2 = + (4.4) 2 bu comparing (4.3) wih (4.6) i is immediae o see ha he number of diffuse iniial saes is now he dimension of a, ha is d 2= d - d d. ^ We will herefore do he same as in he firs sep: (a) use he resuls in e Jong ^ (988) o esimae a and is variance, condiional o z 2, (b) apply he smooher due o Casals, Jerez and Sooca (2000) o condiion i o z 2, and (c) obain an esimae of he saionary sub-sysem sae vecor and is covariance, so ha he diffuse iniial condiions will no affec he filering resuls, see (4.)-(4.2). 4.3 hird Sep: Filering By inducion, i is easy o see ha, a any, he procedure given by he wo previous seps reduces o an augmened filer, including he following sandard KF equaions: = - ˆ - z z H x (4.5) 6

17 B HP H CRC = + - (4.6) ( Φ ) K = P H + ESC B (4.7) - - x ˆ = Φxˆ + K z (4.8) + - ( Φ ) ( ) ( ) P = Φ P + EQE + K CRC K KESC -CS E K (4.9) wih Φ = Φ- KH. On he oher hand, he augmened filer equaions are: - é - ù = ê ú ( ) A H B H (4.20) ë ( ) a A H B z (4.2) - = + H H ( ) ( ) ( ) P = P + Φ Q A Q Φ (4.22) + x ˆ xˆ Q a (4.23) = + Φ + + H = Φ Q ^ (4.24) + H where H H Q H = Obviously, hese equaions are required only if case hey would simplify o: x ˆ = xˆ and P = P has no null dimension. In his ^ he marices Q H, Q H and H can be obained efficienly by applying a column pivoing QR decomposiion o H =H, see Appendix 2, and he iniial condiions are hose corresponding o he saionary subsysem, ha is, ( N, ) G x P. he basic idea behind his procedure is ha he number of columns of + in (4.24) is he number of columns of minus he rank of he marix (4.20). herefore, he dimension of H, defined in decreases wih he number of observaions processed and, in a finie number of ieraions, is dimension will collapse o zero. Once 7

18 his criical size has been achieved, he augmened equaions are no longer needed and he filer collapses o a sandard KF for saionary sysems. 4.4 Likelihood evaluaion Given he resuls in he previous sub-secions, we will now discuss he analyical expression of he likelihood funcion, condiional on he minimum number of observaions required o deermine he diffuse iniial condiions. Consider a sysem such as (2.)-(2.2), wih diffuse iniial saes and, herefore, wih a finie condiional covariance and an infinie uncondiional uncerainy. Is innovaions, z, can be decomposed as: nd z = H a + z (4.25) where z nd is he saionary componen and a is a vecor of d < n diffuse saes. Under hese condiions, if we define U, he subse of Z corresponding o he firs d linearly independen rows of H according o (2.26) would be: where, he Gaussian log-likelihood of Z condiional o U - - ( ZU) = å{ z B z - a A a + log B -log A - 2 = - log H + é 0 H0 log - ë ( 2p)( ù N d)} (4.26) H 0 includes he firs d linearly independen rows of H and he values of, N and d are defined as in (3.2). Noe ha: ) All he erms in (4.26) can be evaluaed by propagaing he filer (4.5)-(4.24). In paricular, he compuaion of he erm log 0 0 H H and he number of iniial diffuse saes, d, are obained as by-producs of he QR decomposiion, see Appendix 3. 2) his procedure is efficien, as is only compuaional overhead in comparison wih evaluaing he likelihood of a saionary sysem, resuls from he 8

19 calculaion of a and A, given in (4.20) and (4.2), unil hese erms collapse o zero. Furhermore, i does no require generalized inverses, see Koopman (997), eqs. ()-(2). 3) he erm log H H in (4.26) is he difference beween he diffuse and he 0 0 minimally-condiioned likelihood and, herefore, is equivalen o he addend in (2.26). 4) Las, when he model does no include coinegraion resricions and he sample does no have missing values, he log-likelihood values given by (3.2) and (4.26) coincide wih hose obaining by differencing he daa, see Appendix. In he coinegraion or missing value cases our mehod is more efficien because differencing leads o a loss of sample informaion. 5. Examples 5.. Airline model his example illusraes he consisency of he condiional likelihood approach and is abiliy o work wih missing values. o his end, we will use he famous series G of inernaional airline passengers, from January 949 o ecember 960, see Box, Jenkins and Reinsel (994). able compares he esimaion resuls obained wih he full sample for boh, for he saionary MA() MA() ( q )( ) airline model: 2 z = - B -QB 2 e (5.) and he nonsaionary version of (5.): 2 2 ( - )( - ) = ( - )( -Q ) B B log P qb B e (5.2) 9

20 where P is he number of airline passengers a ime, B is he backshif operaor, ( )( )log z = -B -B 2 P and he error variance is var( e ) [Inser able ] = s 2 e. Noe ha: ) he esimaes displayed in he able are idenical up o he hird decimal place; in fac, he differences beween acual values are in he order of 0-4, 2) in he case of he saionary model (5.), he diffuse and condiional likelihood values on convergence are idenical, bu 3) when one esimaes he nonsaionary model (5.2) he condiional likelihood value on convergence is idenical o hose obained wih model (5.), while he diffuse likelihood value is subsanially smaller. Consisency of he likelihood values over various difference orders may be imporan when one wans o apply common economeric ools such as LR ess or Informaion crieria. A residual analysis of he previous model shows ha observaions # 62 and 35 may be impulse-ype ouliers. An efficien way o deal wih hese values consiss of agging hem as missing values, see Gomez, Maravall and Peña (999). able 2 compares he esimaes of models (5.) and (5.2) obained for he sample wih hese missing values. Noe ha he esimaes corresponding o he saionary and nonsaionary models are remarkably differen. his happens because differencing propagaes he missing values, hus desroying poenially valuable sample informaion. In his case working wih model (5.2) is cerainly more adequae. [Inser able 2] 20

21 5.2. ynamic facor model Consider wo observable ime series generaed by a common dynamic facor: éy ù éaù é ù a = f + ê ú êbú ê ú ëy2 ë ëa2 where a and b are unknown parameers and he common facor, process: (5.3) f, is given by he ( f )( ) - B - B f = e (5.4) f wih a saionary auoregressive parameer f f. Finally, he error erms e, a and a 2 are muually independen gaussian whie noise sequences wih consan variances s 2 e, s 2 and s2 2. As i is well known, facor models such as (5.3)-(5.4) are no idenified. o esimae hem one mus herefore impose a normalizing consrain. ables 3 and 4 summarize he resuls of an exercise consising of: simulaing 00 values of y and y 2, wih he rue parameer values indicaed in he firs column, and hen esimaing he model parameers, considering differen normalizing consrains and likelihood funcions. [Inser able 3] [Inser able 4] he resuls in able 3 display a remarkable sabiliy. Parameer esimaes and likelihood values on convergence are pracically idenical, clearly showing ha differen normalizing consrains yield observaionally equivalen models. On he oher hand, he diffuse likelihood resuls in able 4 are pracically idenical and good enough for he normalizing consrains a = and b =, bu change subsanially when he consrain is s e =.. his sensiiviy o he normalizing consrain is due o he las addend in (2.26) which, for his model, depends on he parameers o be esimaed. As his 2

22 example shows, is omission in he diffuse likelihood can be he source of subsanial changes in he parameer esimaes. 6. Concluding remarks In his paper we discussed he minimally condiioned likelihood for a sae-space model, allowing for uni roos. his approach is relaively simple, as i is based on a sandard KF, and has specific advanages in comparison wih daa ransformaion, diffuse likelihood and differencing. Abou he former, our mehod avoids using nonsandard filers and is general, meaning ha i allows for missing daa and can be applied o any dynamic model including, for example, coinegraed srucures. Our approach also has advanages in comparison wih diffuse likelihood because, as we showed in sub-secion 2., he diffuse likelihood value depends in some cases on he scale of he sae vecor. Our log-likelihood includes a normalizing addend, see Exp. (2.26), which avoids his problem by making i insensiive o scale facors. In some nonrivial cases such as, e.g., dynamic facor models o models wih coinegraion consrains, his addend depends on he parameers o be esimaed. As he example in sub-secion 5.2 shows, ignoring his erm makes he esimaes sensiive o idenifying consrains ha should be neural. Las, we have proved ha he minimally condiioned likelihood is consisen wih he resuls provided by differencing (see Appendix ) so, when boh mehods are comparable, heir resuls are idenical. On he oher hand our mehod is more complex han differencing, bu has many advanages as i is: (a) more flexible, as i can be applied o esimae non-muliplicaive models, such as ime-varying parameer regressions or srucural ime series models; (b) more efficien when here are 22

23 coinegraion consrains or missing in-sample, because i avoids unnecessary daa losses; and (c) more convenien when one wans o compue forecass or apply a signal exracion procedure. he erms of he minimally condiioned likelihood can be compued using eiher he S or C procedures described in Secions 3 and 4, respecively. Boh mehods are muually consisen in he sense ha, when applied o a given sample and model, hey reurn he same likelihood value, allowing for insignifican numerical differences. espie his equivalence, each one has specific advanages in differen siuaions. Specifically, he filers required by boh algorihms have some compuaional overhead in comparison wih a sandard KF. In paricular, he S procedure requires propagaing equaions (3.9)-(3.) for all, while he C mehod only requires addiional calculaions unil he augmened filer collapses o a sandard KF. Furhermore, he C algorihm includes an efficien and sable mehod, based on he QR decomposiion, o include he diffuse iniial condiions in he filer. As a consequence, he C procedure is more efficien in he general case. However, he S mehod is compuaionally cheaper when here are no missing values in he sample and he model parameers are ime-invarian. his happens because, under hese condiions, one can ake advanage of he simplificaions derived from he null soluion o he Riccai equaion, which more han compensae is inrinsic overhead. Compuaional efficiency in boh cases is furher enhanced by he QR algorihm employed o compue he deerminans log O O or log H H 0 0 which, when compued using sandard approaches, can add a subsanial compuaional overhead. he procedures described in his aricle are implemened in he E 4 funcions lfsd (S mehod) and lfcd (C mehod). E 4 is a MALAB oolbox for ime series modeling, which can be downloaded a: he source code for all he funcions in he oolbox is freely provided under he erms of he GNU General 23

24 Public License. his sie also includes a complee user manual and oher reference maerials. References Ansley, C.F. and R. Kohn (985). Esimaion, Filering and Smoohing in Sae Space Models wih Incompleely Specified Iniial Condiions, Annals of Saisics 3, Ansley, C.F. and Kohn, R. (990). Filering and Smoohing in Sae Space Models wih Parially iffuse Iniial Condiions, Journal ime Series Analysis, Bell, W., and Hillmer, S. C. (99). Iniializing he Kalman Filer for Nonsaionary ime Series Models, Journal of ime Series Analysis, 2, 4, Box, G.E.P.; G. M. Jenkins y G. C. Reinsel (2008). ime Series Analysis: Forecasing and Conrol, Wiley, New York. Casals, J., S. Sooca and M. Jerez (999) A Fas and Sable Mehod o Compue he Likelihood of ime Invarian Sae-Space Models, Economics Leers, 65, Casals, J., M. Jerez and S. Sooca (2000). Exac Smoohing for Saionary and Nonsaionary ime Series, Inernaional Journal of Forecasing 6,, e Jong, P. (988). he Likelihood for a Sae Space Model, Biomerika 75,, e Jong, P. (99). he iffuse Kalman Filer, Annals of Saisics 9, e Jong, P and S. Chu-Chun-Lin (994). Fas Likelihood Evaluaion and Predicion for Nonsaionary Sae Space Models, Biomerika, 8, Francke, M.K., S. J. Koopman and A. de Vos (200). Likelihood Funcions for Sae Space Models wih iffuse Iniial Condiions, Journal of ime Series Analysis, 3, 6, Gomez, V. and Maravall, A. (994). Esimaion, Predicion and Inerpolaion for Nonsaionary Series wih he Kalman Filer, Journal of he American Saisical Associaion, 89,

25 Gomez, V., Maravall, A., and. Peña, (999). Missing Observaions in ARIMA Models: Skipping Approach versus Addiive Oulier Approach, Journal of Economerics, 88, 2, Harvey, A.C. (989). Forecasing, Srucural ime Series Models and he Kalman Filer, Cambridge Universiy Press, Cambridge (UK). Kohn, R. and Ansley, C.F. (986). Esimaion, Predicion, and Inerpolaion for ARIMA Models wih Missing aa, Journal of he American Saisical Associaion, 8, Koopman, S.J. (997). Exac Iniial Kalman Filering and Smoohing for Non- Saionary ime Series Models, Journal of he American Saisical Associaion, 92, 440, Mauricio, A. (2006). Exac Maximum Likelihood Esimaion of Parially Nonsaionary Vecor ARMA Models, Compuaional Saisics and aa Analysis 50, 2, Acknowledgemens: Financial suppor from Miniserio de Ciencia e Innovación, Ref: ECO , is graefully acknowledged. 25

26 APPENIX. Equivalence beween he likelihood of differenced daa and he minimally-condiioned likelihood. Consider he mulivariae sochasic process z N and assume, wihou loss of generaliy, ha i has m firs-order inegraed componens, such ha is firs-order difference, w B z, is a saionary and inverible process. his saionary ransformaion can be wrien in compac form as: w F z, where z and w are N vecors and F is a m N m N block-marix, composed of m m null and ideniy marices, wih he following srucure: I I I 0 0 F 0 0 I I Under hese condiions, he Gaussian densiy of z is: z Fz z, w,, w f f f 2 N where z conains he firs observaion in he sample, w2 z2 z and so on. Under diffuse iniial condiions i holds ha cov z coincides wih he number of uni roos. Accordingly z z z, w2,, wn z w2,, w N f f f, because he dimension of cov z 0 and hen: So he densiy of he saionary ransformaion f w 2,, w N coincides wih he condiional densiy f z z, being z is he minimal sub-sample required o deermine he diffuse iniial condiions in his case. z 26

27 APPENIX 2. Proof of he heorem. N Par (): enoing V º cov( Z U) and º cov( ) (3.2), we firs wan o prove ha: V Z U, see expression P (A.) - - V O V O = V + O V O P é0 0 ù P = ê P - ú ë0 Firs, we know ha: wih O = [ O O N ] and P+ = P+ - O V O O W W O (A.2) where W is a marix such ha WV W = Ι, so - V = W W. efining O º WO ˆ expression (A.2) can be reformulaed as: P P P ˆ ˆ Oˆ Oˆ Oˆ Oˆ = = Oˆ V Oˆ V P - Oˆ Oˆ Oˆ Oˆ P - + O V O = + O W W O = + O O = N N N N (A.3) where V = O PO + V [see equaion (3.)] and N N WVW = WO PO W + I N N herefore, i is easy o see ha: V=Oˆ POˆ + I N N (A.4) wih WVW = V. Applying he marix inversion lemma o expression (A.4): - - I ˆ é ˆ ˆ ù ˆ N N N N V = O êo O P ë ú O (A.5) Las, (A.3) is: Oˆ V Oˆ V P = O V O W V W P (A.6) and subsiuing (A.6) in (A.) we obain: V O V O = V P + O V O V V P P 27

28 which is he resul ha we waned o prove. Par (2). Now, we wan o prove ha: V V O é( O ) V O ù( O ) V = V - V Oé O V Où êë P + ú O V ê = ë ú see expression (3.3).o prove (3.3) i is enough o pre and pos-muliply (A.7) by (A.7) Z and Z, respecively. Hereafer, we will denoe he erms in (A.7) as AA - BB = CC - where: - AA = V BB = V O é( ) ù ê O V O ú( O ) V ë CC = V = V O é + ù êë P O V Oú O V Under hese condiions, we will prove ha = BB + CC - AA - = W WOé ù êë P + O W W Oú O W W = - ˆ é ˆ ˆù ˆ = W O + = ˆ êp O O ë ú O W W PW (A.8) wih - ˆ ˆ é ˆ ˆù ˆ P= OêP+ O O ë ú O or, - N - N N N N é ˆ ˆ ˆ ˆ ù é ˆ ù ˆ = é ˆ ˆ ùê O O O O O P êo úê ú ê ú = ë ON ˆ ˆ ˆ ˆ ëo O O O + P ˆ ëo - - ˆ é ˆ - ˆ ù ˆ - - I = V O êo V O ú O V + -V ë (A.9) where we applied he pariioned marix inversion lemma and expression (A.5). Subsiuing (A.9) in (A.8) we obain: 28

29 = W ˆ W = W V Oˆ éoˆ V Oˆ ù Oˆ V + - V W = P { ê I ë ú } - - { ˆ é ˆ - ˆ ù ˆ - - } + - W V O êo ë V Oú O V W W W W V W and, aking ino accoun ha, V - = W W and WVW = V é - ù = V O êë O V O ú O V + V - V = BB + CC -AA which is he resul ha we waned o prove. 29

30 APPENIX 3. QR algorihm. I is well known ha he QR decomposiion of any m n real-valued marix A is: A RQ Where Q is an orhonormal n n marix and R is a m n lower riangular marix. he column-pivoing QR decomposiion is a reordering such ha he elemens in he main diagonal of R are sored in decreasing order according o heir absolue value. In his case, he decomposiion resuls in: A E RQ where E is a m m permuaion marix. Afer he previously defined re-ordering, R can be wrien as: R R 0 Where he dimensions of R are m d, being d he rank of A, which can be deermined as he number of nonzero elemens in he main diagonal of R. If we pariion Q as: Q Q Q 2 being Q a d n marix, hen A can be wrien as A ER Q, where Q spawns he row-space of A and Q 2 is orhogonal o his row-space. In he Column eleion algorihm we apply his decomposiion direcly o H H Q H 2, obaining Q Q, H Q, H ER and he erm of he likelihood given in (4.26) can be compued as log H H log R ii, 0 0 i d he Sae ecomposiion mehods requires including he decomposiion of H in he filer, where is given by (2.4). he marix Μ G can be easily compued by he iniializaion procedure proposed by e Jong and Chu-Chun-Lin (994). 30

31 able. Esimaion resuls obained for he full airline daase. Model (5.), diffuse and condiional likelihood Model (5.2), diffuse likelihood Model (5.2), condiional likelihood Minus loglikelihood qˆ.402 Qˆ.557 s ˆe.037 3

32 able 2. Esimaion resuls obained for he airline daase, reaing observaions # 62 and 35 as missing values. Model (5.), diffuse and condiional likelihood Model (5.2), diffuse likelihood Model (5.2), condiional likelihood Minus loglikelihood qˆ Qˆ s ˆe

33 able 3. Resuls obained wih he Sae ecomposiion algorihm (idenical o hose of he Column eleion mehod). he rue parameer values are given in he firs column and have been employed in all he cases as iniial condiions for he ieraive opimizaion. Consrained parameers are denoed by an aserisk. Normalizing consrain a = b = s a =. Likelihood value on convergence a = b = s = s = f f = s = e 33

34 able 4. Resuls obained wih he diffuse likelihood. he rue parameer values are given in he firs column and have been employed in all he cases as iniial condiions for he ieraive opimizaion. Consrained parameers are denoed by an aserisk. Normalizing consrain a = b = s a =. Likelihood value on convergence a = b = s = s = f f = s = e 34