Non-Gaussian Test Models for Prediction and State Estimation with Model Errors

Transcription

1 Non-Gassian Tes Models for Predicion and Sae Esimaion wih Model Errors M. Branicki, N. Chen, and A.J. Majda Deparmen of Mahemaics and Cener for Amosphere Ocean Science, Coran Insie of Mahemaical Sciences New York Universiy, New York, USA Absrac Trblen dynamical sysems involve dynamics wih boh a large dimensional phase space and a large nmber of posiive Lyapnov exponens. Sch sysems are biqios in applicaions in conemporary science and engineering where saisical ensemble predicion and real-ime filering/sae esimaion are needed despie he nderlying complexiy of he sysem. Saisically exacly solvable es models have a crcial role o provide firm mahemaical nderpinning or new algorihms for vasly more complex scienific phenomena. Here a class of saisically exacly solvable non-gassian es models are inrodced where a generalized Feynman-Kac formlaion redces he exac behavior of condiional saisical momens o he solion of inhomogeneos Fokker-Planck eqaions modified by linear lower order copling and sorce erms. This procedre is applied o a es model wih hidden insabiliies and combined wih informaion heory o address wo imporan isses in conemporary saisical predicion of rblen dynamical sysems: coarse-grained ensemble predicion in a perfec model and improving long range forecasing in imperfec models. The models discssed here shold be sefl for many oher applicaions and algorihms for real ime predicion and sae esimaion. Inrodcion Trblen dynamical sysems involve dynamics wih boh a large dimensional phase space and a large nmber of posiive Lyapnov exponens. Sch exremely complex sysems are biqios in many disciplines of conemporary science and engineering sch as climae-amosphere-ocean science, neral science, maerial science, and engineering rblence. Topics of wide conemporary ineres involve saisical ensemble predicion [3] and real ime sae esimaion/filering [34] for he exremely complex sysems while coping wih he fndamenal limiaions of model error and he crse of small ensemble size []. An imporan role of mahemaics in applied sciences is o develop simpler exacly or easily solvable es models wih nambigos mahemaical feares which neverheless capre crcial feares of vasly more complex sysems in science and engineering. Sch models provide firm nderpinning for boh advancing scienific ndersanding and developing new nmerical or saisical ndersanding. One of he ahors has been developing his approach wih varios collaboraors over he pas few years for paradigm problems for rblen dynamical sysems. For example, simple saisically exacly solvable es models have been developed for slow-fas sysems [, 3], rblen racers [, 4,, 33] and as sochasic parameerizaion algorihms for real-ime filering of rblen dynamical sysems wih jdicios model error [9, 8, 5, 35, 34]. Sch models have been ilized as nambigos es models for improving predicion wih imperfec models in climae science hrogh empirical informaion heory [, 8, 9, 3, 5] and for esing algorihms for ncerainy qanificaion [5, 4, 5].

2 Here, we sdy non-gassian saisics in a class of es models which are saisically exacly solvable hrogh a generalized Feynman-Kac formla [6, ] which redces he exac behavior of condiional saisical momens o he solion of inhomogeneos Fokker-Planck eqaions modified by linear lowerorder copling erms and sorce erms. This exac procedre is developed in secion below and involves only marginal averaging and inegraion by pars. In secion 3 elemenary es models are inrodced where he general procedre from secion can be evalaed hrogh elemenary nmerical solions of he copled generalized Fokker-Planck eqaions CGFPE. Secion 4 conains a brief inrodcion o he se of informaion heory o qanify model error in a framework adaped o he presen conex. Secion 5 conains wo applicaions of he maerial in secion 3 and 4 o saisical ensemble forecasing: he firs applicaion involves coarse-grained ensemble predicion in a perfec model wih hidden insabiliies; he second applicaion involves he se of imperfec models for long range forecasing. Tes Models wih Exacly Solvable Condiional Momens Here, we consider a special class of es models and illsrae he evolion of he exac condiional saisical momens can be calclaed hrogh he solion of copled generalized Fokker-Planck eqaions CGFPE. Or elemenary derivaion follows he philosophy of generalized Feynman-Kac framework [6, ] alhogh we do no know any specific reference for he general principle developed below. Consider a vecor IR M pariioned ino componens = i, ii wih i IR Mi, ii IR Mii, and M = M i + M ii. We focs on he special class of es models given by he sysem of Iô SDE s, d i = F i, d + σ i i, dw i, d ii = F ii i, + Γ i, ii d + σ ii i, dw i + σ ii,a i, dw ii,a + σ ii, + σ ii,m i, ii dw ii,m, where W i is an M i -dimensional Wiener process and W ii,a, W ii,, W ii,m are independen M ii -dimensional Wiener processes. Noe ha he dynamics of i is arbirary while he dynamics of ii is qasilinear, i.e., linear in ii in boh he drif and noise b wih general nonlinear coefficiens depending on i. Also, noe ha he noise for i and ii can be correlaed hrogh W i appearing in boh he eqaion for i and ii. All of he nonlinear es models for slow-fas sysems [, 3], rblen racers [, 4,, 33] and exacly solvable sochasic parameerized filers [9, 8, 5, 35, 34] have he srcral form as in. Sch sysems are known o have exacly solvable non-gassian saisics for filers where i is observed condiionally over a ime inerval [, ]. Below, we derive explici closed eqaions for he evolion of condiional momens of hrogh CGFPE. The Fokker-Planck eqaion for he probabiliy densiy p i, ii, associaed wih is given by [7, 36] p = i F i p ii F ii + Γ ii p + Q p + i i Q a p + ii ii Q m p, where = i, ii, and Q = σ i, σ ii σ T i, σ T ii, Q a = σ ii,a σ T ii,a, Q m = σ ii, + σ ii,m ii σ T ii, + T ii σ T ii,m. 3 We are ineresed in developing exac saisical approximaions for p i, ii, which, by Bayes heorem, can be wrien as p i, ii, = p ii i, π i, 4 where π i, is he marginal disribion π i, p i, ii, d ii. 5

3 We firs inegrae wih respec o ii and se he divergence heorem o calclae ha he marginal densiy, π i,, saisfies he Fokker-Planck eqaion wih π = L fp,i π, 6 L fp,i π = i F i π + i i Q i π, Q i = σ i σ T i. 7 Nex, we derive he closed sysem of Copled Generalized Fokker-Planck Eqaions CGFPE for he condiional momens M α i, α ii p i, ii, d ii = π i, α ii p ii i, d ii. 8 Noe ha M i, = π i, is js he marginal densiy of in i. sandard mli-index noaion α = α, α,..., α Mii IR Mii wih Here and below, we se he α ii ii α ii α... ii α Mii M ii. 9 We have he following general principles for comping he vecor, M α i, M α i,, α = N, of condiional momens of order N: Proposiion. Generalized Feynman-Kac formla The vecor M N i, of condiional momens of order N associaed wih he probabiliy densiy of saisfies he CGFPE M N i, = L fp M N i, + L N i, M N i, + F N i, M N i,, i M N i,, M N i,, wih he convenion M = M = where F N is an explici linear fncion wih coefficiens depending on i of he lower order momens; L N is an N N Feynman-Kac marix poenial which is an explici linear fncion wih coefficiens depending on i of he qaniies which vanishes when boh Γ =, Q ii,m =. Γ i,, Q ii,m = σ ii,m σ T ii,m, The proof below immediaely yields explici formlas for L N and F N in any concree applicaion see secion 3 below b a general noaion for hese coefficiens wold be edios and nnecessary o develop here. The advanage of CGFPE in is ha high resolion nmerical inegraors can be developed for o find hese saisics provided M i is low-dimensional or has special algebraic srcre see secion 3. The skech of he proof below emphasizes he main conribions o he operaor L N in. As in he derivaion of 6, we firs mliply he Fokker-Planck eqaion by α ii and inegrae wih respec o ii o obain M N i, = L fp M N i, + α ii ii Γ i, ii p d ii α ii ii ii σ ii,m ii T ii σii,m T p d ii +..., 3

4 where +... denoes all he remaining erms which define he recrsive sorce erm F N. We simplify sing inegraion by pars of he wo las erms on he righ hand side; namely α ii ii Γ i, ii p d ii = ii α ii Γ i, ii p d ii = L, N M N i,, 3 and α ii ii ii σ ii,m ii T ii σii,m T p d ii = ii ii α ii σ ii,m ii T ii σii,m T p d ii = L, N M N i, 4 so ha L N = L, N + L, N in. The remaining erms in +... are comped explicily by similar inegraion by pars o define F N. The correlaed noise erms in involving W i which defines he noise Q in deermine he dependence on M N i, in F N since hey have he ypical form α ii i ii σ i σii T p d ii = i ii ii α σ i σii T p d ii = F N i, i M N i,. 5 I is worh poining o ha F N depends only on he poin-wise vales of M N i,, M N i, if here are non-correlaed noise ineracions and σ ii =. 3 Applicaion of he Condiional Momen PDE s o a Non-Gassian Tes Model Here, we develop he simples non-gassian es model where we can explicily evalae non-rivial saisical feares ilizing he copled sysem of PDE s in from for he condiional momens M α i,. We hen derive and validae a nmerical procedre for accrae nmerical solion of he closed sysem of eqaions in for he condiional momens in several sringen es problems. This explici solion procedre is applied in 5 o ndersand he role of coarse-graining and non-gassian saisics wih model error in ensemble predicions. Clearly, he simples models o consider wih he srcre as in have M i = M ii = so ha he recrsion formlas in involve scalar fields and he CGFPE are inegraed in a single spaial dimension. For i we choose he general nonlinear scalar Iô SDE d i = F i i, d + σ i i, dw i, 6 while for ii we ilize he qasi-linear eqaion d ii = i ii + f d + σ ii dw ii, 7 where f does no depend on i, and he noise σ ii is consan. Noe ha i eners in 7 as a mliplicaive coefficien and flcaions in i can inrodce growh and inermien insabiliies wih highly non-gassian behavior even when i in 6 has a posiive mean [3, 5, 5]. The sochasic models for i in 6 will vary from linear sochasic models a special case of he SPEKF models for filering [9, 34, 35, 5] o cbic nonlinear models wih addiive and mliplicaive noise [5]. For sysems wih dynamics as in 6-7, he closed eqaions for he condiional momens M α in 8 become M N i, = L fp M N i, N i M N i, + Nf M N i, + NN σ ii M N i,, 8 where N =,,..., N max and M = M =. Sch models illsrae a wide range of inermien non- Gassian behavior mimicking ha in vasly more complex sysems []. These simple ye revealing models will be sed in 5 o sdy varios new aspecs of model error in ensemble predicions for non-gassian rblen sysems. 4

5 3. Validaion of a Nmerical Mehod for Solving he CGFPE Deerminaion of he ime evolion of he condiional momens M α in 8 reqires an accrae nmerical procedre for solving he inhomogeneos sysem of copled Fokker-Planck eqaions CGFPE in. The algorihms discssed below apply o he case when i IR i.e., M i = which is sfficien for or prposes and leads o many new insighs on model error in imperfec ensemble predicions of rblen sysems wih posiive Lyapnov exponens, as discssed in 5. Similar o he case of he homogeneos Fokker-Planck eqaion, solving he inhomogeneos CGFPE sysem for M i 3 poses a formidable challenge which, convenienly, is nnecessary here. Here, he copled sysem in is solved sing he hird-order emporal discreizaion hrogh he backward differeniaion formlas e.g., [7] and he second-order spaial discreizaion via he finie volme mehod [9] see Appendix A for deails. The performance of he nmerical procedre for solving CGFPE in one spaial dimension i.e., i IR in is esed in he following widely varying dynamical configraions: i Dynamics wih ime-invarian saisics on he aracor/eqilibrim wih Nearly Gassian marginal eqilibrim PDFs in ii and linear Gassian dynamics for i in 6, Fa-ailed marginal eqilibrim PDfs in ii and linear Gassian dynamics for i in 6, Highly non-gassian marginal eqilibrim PDFs in ii and cbic dynamics for i in 6 wih highly skewed eqilibrim PDFs. ii Dynamics wih ime-periodic saisics on he aracor wih ime-periodic regime swiching beween nearly Gassian and highly skewed regimes wih cbic dynamics for i in 6 and highly non- Gassian dynamics of ii in 7. Below, we inrodce he relevan es models in 3.. and provide evidence for good accracy of he developed echniqe in 3.., as well as is advanages over direc Mone Carlo sampling. 3.. Non-Gassian es models for validaing CGFPE Here, we consider wo non-gassian models wih inermien insabiliies and wih he srcre as in 6-7 where we adop he following noaion i =, ii =. The firs model is a simplified version of he SPEKF model developed originally for filering rblen sysems wih sochasically parameerized nresolved variables [9, 8, 5, 35, 34] and given by a d = d ˆ + f d + σ dw 9 b d = + f d + σ dw. Noe ha despie he Gassian dynamics of he damping flcaions, he dynamics of in 9 can be highly non-gassian wih inermienly posiive Lyapnov exponens even when he eqilibrim mean, ˆ, is posiive [3, 5, 4, 5]. The sysem 9 possesses a wide range of rblen dynamical regimes ranging from highly non-gassian dynamics wih inermiency and fa-ailed marginal PDFs for o laminar regimes wih nearly Gassian saisics; a deailed discssion of properies of his sysem can be fond in [3, 5]. In he nmerical ess discssed in he nex secion we examine he accracy of he nmerical algorihm for solving CGFPE in he dynamical regime characerized by a highly inermien marginal dynamics in associaed wih fa-ailed marginal eqilibrim PDFs for see figre for examples of sch dynamics. 5

6 The second model we examine has a cbic nonlineariy in he dynamics of he damping flcaions,, and is given by [ ] a d = a + b c 3 + f d + A B dw C + σ dw, b d = + f d + σ dw, The above nonlinear model for wih correlaed addiive and mliplicaive noise W C and exacly solvable eqilibrim saisics was firs derived in [6] as a normal form for a single low-freqency variable in climae models where he noise correlaions arise hrogh advecion of he large scales by he small scales and simlaneosly srong cbic damping. The nonlinear dynamics of has many ineresing feares which were sdied in deail elsewhere [5]. Here, we consider a more complex problem where he dynamics of in a is copled wih hrogh he qadraic nonlineariy. In he nmerical ess below we focs on he pariclarly ineresing regime where he damping flcaions exhibi regime swiching despie nimodaliy of he associaed eqilibrim saisics see figre for an example. This configraion represens he simples possible es model for he analogos behavior occrring in comprehensive climae models [7, 3]. Anoher imporan configraion of esed below wih relevance o amospheric/climae dynamics corresponds o ime-periodic ransiions in beween a highly skewed and a nearly Gassian phases in wih he dynamics in remaining highly non-gassian hrogho he evolion see figre for an illsraion of sch dynamics. The above wo non-gassian models are ilized below o validae he accracy of or nmerical mehod for solving he CGFPE sysem ; his framework is hen sed o analyze model error in imperfec predicions of rblen non-gassian sysems in Nmerical ess Here, we se he es models inrodced in he previos secion o analyze he performance of he nmerical scheme for solving he CGFPE sysem in one-spaial dimension. In order o assess he accracy of he algorihm, we consider following wo ypes of relaive error in he condiional momens: he poin-wise relaive error in he N-h condiional momen ɛ N, = and he L relaive error for each fixed ime M cgfpe N, M ref N,,, M ref N ɛ N = Mcgfpe N, M ref N, L M ref N,. L The reference vales for he condiional momens, M ref N, in he above formlas are obained from eiher he analyical solions in he case of sysem 9 hrogh he formlas derived in [9], or via he Mone Carlo esimaes. The condiional momens are normalized in he sandard fashion, wih he condiional 6

7 3.5 π Gassian fi Tre 3.5 π p π Tre Gassian fi.5 logp π p.5 logp Figre : Top Marginal saisics, p eq, and a pah-wise solion on he aracor of he sysem 9 in he non-gassian regime wih invarian measre characerized by inermien ransien insabiliies and fa-ailed marginal PDFs dynamics of is Gassian in his model. Boom Marginal saisics, p eq, p eq, and a pah-wise solions,,, on he aracor of he sysem in he non-gassian regime wih regime swiching in he pah-wise dynamics despie a nimodal, skewed marginal PDF in. mean, variance, skewness and krosis given by M, = M,, M, = M,, 3 M, = M, p,, d = M, M,, 4 M 3, = M 4, = M 3/, M, 3 p,, d = M 3, 3M, M, + M 3,, 5 M 3/, M, 4 p,, d M, = M 4, 4M, M 3, + 6M, M, 3M 4, M,. 6 The L errors for he wo es models discssed in he previos secion and parameers as specified below are lised in Tables -3; noe ha he errors in he condiional momens do no exceed 6% for he wide range of dynamical regimes considered. Moreover, comparison of he resls in Tables - shows ha he nmerical algorihm developed here is more efficien and accrae han he Mone Carlo esimaes, 7

8 Skewness of π B A.5.5 Marginal disribion π a A Tre Gassian fi.5.5 Marginal disribion π a B Tre Gassian fi 5 Skewness of π.6.4. B A Marginal disribion π a A.4 Tre Gassian fi Marginal disribion π a B.4 Tre Gassian fi Figre : Top Time-periodic evolion of he skewness of he marginal dynamics of in he non-gassian sysem wih cbic nonlineariy in in he configraion where cycles beween a highly skewed op middle and a nearly Gassian op righ phases. The phases of high/low skewness in he marginal saisics of are correlaed wih hose in he marginal saisics of ; noe, however, ha he dynamics of remains highly non-gassian hrogho he evolion. The snapshos of he marginal PDFs in on he boom are shown for he imes indicaed on he op panel. even when a relaively large sample size 7 is sed in he MC simlaions. In figre 3 we illsrae he performance of he algorihm for comping he condiional momens in associaed wih he condiional eqilibrim densiy p eq for he sysem 9. The sysem parameers, d, σ, ˆ, σ, in 9 are chosen o represen he non-gassian dynamics in he regime wih inermien insabiliies and a fa-ailed marginal eqilibrim PDF in ; in pariclar, we choose σ =, d =, ˆ = 3, f = f = ; see figre for an example of he corresponding dynamics. In figres 5-6 we illsrae he performance of or algorihm for comping he condiional momens, M α,, of in he sysem wih he cbic nonlineariy in which is copled mliplicaively o he dynamics in. Here, we consider wo disinc configraions. For consan forcing we choose he parameers in in sch a way ha displays regime swiching wih he nimodal, highly skewed marginal eqilibrim PDF for, while he marginal dynamics of is highly non-gassian and secondorder sable; his dynamical configraion can be achieved by seing, for example, a =, b =, c =, A =.5, B =, σ =, f =, f = 3; see figre for an illsraion of sch a dynamics. For ime-periodic forcing, when he dynamics in cycles beween highly skewed and a nearly Gassian phases while remains highly non-gassian, we se a =, b =, c =, A =.5, B =.5, σ =.5, f =.5, 8

9 M M M M 3 M 4 Sysem 9: Nearly Gassian reg Sysem 9: Fa algebraic ail reg Sysem : High skewness reg Table : Relaive errors, ɛ N, in he condiional momens M M a eqilibrim for he wo es models 9 and wih he reference inp obained from Mone Carlo esimaes from 7 rns. M M M M 3 M 4 Sysem 9: Nearly Gassian reg..5 5 Sysem 9: Fa algebraic ail reg Sysem : High skewness reg..8 Table : Relaive errors in he condiional momens M, M, and M 3 a eqilibrim for he wo es models 9 and wih he reference inp obained from analyical solions. M M M M 3 M 4 = = = = Table 3: Relaive error in ime-periodic condiional momens M M for he es model in he regime wih ransiions see figre beween highly skewed and nearly Gassian marginal densiies π a; he reference inp obained from Mone Carlo esimaes from 7 rns. wih he ime-periodic forcing in given by f = 6.5 sinπ π/ +.5. Based on he resls smmarized in figres 3-6 and Tables -3, we make he following poins: The nmerical algorihm for solving he copled sysem in he CGFPE framework wih i IR provides robs and accrae esimaes for he condiional momens 8. The discrepancies beween he esimaes obained from and direc Mone Carlo esimaes wih large sample size 7 are below 6% for boh ime-periodic and ime-invarian aracor saisics. The larges discrepancies in he normalized condiional momens obained from CGFPE and Mone Carlo esimaes he in he normalized momens occr in ail regions where he corresponding probabiliy densiies are very small. The developed algorihm for solving he CGFPE sysem is more efficien and more accrae han he Mone Carlo esimaes wih relaively large sample sizes 7. 9

10 . M x 4 M.4 M.5.3 M. M M x 4 M M 4 Momen Eqaion Mone Carlo M 3 M Figre 3: Eqilibrim condiional saisics of he sysem 9 wih Gassian damping flcaions, inermien insabiliies and fa-ailed marginal PDF in. Unnormalized condiional momens, M M 4, 8 of a eqilibrim of he wo-dimensional non-gassian rblen sysem 9 wih inermien insabiliies de o Gassian damping flcaions; he resls of CGFPE and Mone Carlo esimaes from 7 rns are compared. In he dynamical regime shown he marginal eqilibrim PDF, p eq, is symmeric and fa-ailed de o hese inermien insabiliies see figre. Noe he errors in he Mone Carlo esimaes in he odd momens. π = M Rel Error 5 x M Rel Error 5 x 3 M M Rel Error.5..5 M Momen Eqaion Mone Carlo Figre 4: Same as in figre 3 above b for cenered, normalized condiional momens, M M 4, which correspond o marginal densiy π, he condiional mean, variance, skewness and krosis given by 3-6.

11 π = M 5 M 4 M 3 Rel Error Rel Error M Rel Error Rel Error M Rel Error Momen Eqaion Mone Carlo Figre 5: Snapsho of ime-periodic condiional saisics on he aracor of he sysem ; cbic nonlineariy in damping flcaions, highly skewed PDF phase. Normalized condiional momens, M, M 4,, 3-6 of on he ime-periodic aracor of he wo-dimensional non- Gassian rblen sysem wih cbic dynamics of damping flcaions ; he resls obained via CGFPE and Mone Carlo simlaions wih 7 sample rns are compared a ime = 8.4 which corresponds o he highly non-gassian phase wih highly skewed marginal PDFs, π a,, π a, see figre. The normalized condiional momens are he condiional mean, variance skewness and krosis. π = M 4 M 3 M 3 3 Rel Error Rel Error M Rel Error Rel Error M Rel Error Momen Eqaion Mone Carlo Figre 6: Same as figre 5 above b showing he normalized condiional momens, M, M 4, i.e., he condiional mean, variance, skewness and krosis, a = 7 which corresponds o he nearly Gassian phase in b a highly skewed marginal π a, see also figre.

12 4 Qanifying Model Error Throgh Empirical Informaion Theory As discssed exensively recenly [8, 9, 3,, 5, 5], a very naral way o qanify model error in saisical solions of complex sysems is hrogh he relaive enropy, Pp, q for wo probabiliy measres, p, q, given by Pp, q = p ln p q = S p p ln q, 7 where S p = p ln p 8 is he Shannon enropy of he probabiliy measre p. The relaive enropy,pp, q, measres he lack of informaion in q abo he probabiliy measre p. If p is he perfec densiy and p m, m M is a class of probabiliy densiies, hen m is a beer model han m provided ha and he bes model m M saisfies Pp, p m < Pp, p m, 9 Pp, p m = min m M Pp, p m. 3 There are exensive applicaions of informaion heory o improve imperfec models in climae science developed recenly [8, 9, 3,, 5, 5]; he ineresed reader can hese references. The goal here is o develop and illsrae his informaion heory perspecive on model error for direc applicaion o esimae model error for he sep developed above in secions, 3; hese formlas are ilized in 5 below. as We consider a probabiliy densiy for he perfec model p i, ii which can be wrien by Bayes heorem p i, ii = p ii i π i, 3 where, here and below, π i is he marginal π i = p i, ii d ii. 3 From he CGFPE procedre developed in secions, 3, we have exac expressions for he condiional momens p o some order L for p ii i evolving in ime already, his is a sorce of informaion loss hrogh coarse graining of p i, ii. To qanify his informaion loss by measring only he condiional momens p o order L, le p L i, ii = p L ii i π i 33 where for each vale i he condiional densiy p L ii i saisfies he maximm enropy leas biased crierion [3, 4, 3] S p L ii i = max S π L ii, 34 π L L where L is a class of marginal densiies π L wih idenical momens p o order L, i.e., α ii π L ii d ii = α ii p L i, ii d ii = α ii p i, ii d ii, α L. 35 Below and in secion 5, we will always apply he variaional problem in 34 for L = which garanees ha p L ii i is a Gassian densiy wih specified condiional mean and variance. In general, for L even and L >, i is a sble isse as o wheher he solion of he variaional problem 34 exiss [34] b here

13 we acily assme his. We remark here ha highly non-gassian densiies can have Gassian condiional densiies like p L ii i as discssed in 5. Naral imperfec densiies wih model error have he form p m L i, ii = p m L ii i π m i. 36 The simples model wih model error is a Gassian densiy, p G i, ii, which is defined by is mean and variance; he sandard regression formla for Gassian densiies [7] aomaically garanees ha he form in 36 applies wih L = in his imporan case. Anoher imporan way o generae an imperfec model wih he form 36 is o have a differen model [, 5] for he sochasic dynamics of i han ha in and compe he condiional momens p o order L in he approximae model hrogh CGFPE so ha he model approximaions aomaically have he form 36 see secion 5 below. Here is a precise way o qanify he model error in an imperfec model in he presen sep: Proposiion 4. Given he perfec model disribion, p i, ii, is condiional approximaion, p L i, ii in 33 and he imperfec model densiy, p m L i, ii defined in 36, we have P p i, ii, p m L i, ii = P p i, ii, p L i, ii + P p L i, ii, p m L i, ii 37 where P p i, ii, p L i, ii = = [ ] π i Sp L ii i Sp ii i d i π i Pp ii i, p L ii i d i, 38 and P p L i, ii, p m L i, ii = P π i, π m i + = P π i, π m i + [ ] π i Sp m L ii i Sp m L ii i d i π i P p ii i, p m L ii i d i. 39 In pariclar, P p i, ii, p L i, ii qanifies an inrinsic informaion barrier [9, 3,, 5, 5] for all imperfec model densiies wih he form as in 36. The proof of Proposiion 4. is by direc calclaion ilizing he general ideniy [6] P p L i, ii, p m L i, ii = P π i, π m i + π i P p ii i, p m L ii i d i, 4 which is easily verified by he reader. Nex, for each i, se he general ideniy for leas biased densiies which follows from he max-enropy principle in 34 see [4, Chaper ] P p i, ii, p m L i, ii = P p i, ii, p m L i, ii + P p i, ii, p m L i, ii, 4 and inser his in 4. Nex, comping P p i, ii, p L i, ii and P p L i, ii, p m L i, ii by he formla in 4 once again, wih simple algebra we arrive a he reqired formlas in

14 5 Non-Gassian Tes Models for Saisical Predicion wih Model Error Here, we apply he maerial developed in secions 3 and 4 wih L = o gain new insigh ino saisical predicions wih he effecs of coarse graining and model error in non-gassian seing. In he firs par of his secion, we consider he effec of model error hrogh coarse graining he saisics in a perfec model seing [8] for shor, medim, and long range forecasing. In he second par of his secion, we consider he effec of model error in he dynamics of i [5] on he long range forecasing skill. The error in boh he fll probabiliy densiy as well as he marginal densiies in ii is considered. 5. Choice of iniial saisical condiions As already menioned in 4, we are pariclarly ineresed in assessing he model error de o varios coarse-grainings of he perfec saisics; hese model errors arise narally when eiher deriving he approximae leas-biased condiional densiies hrogh esimaing he condiional momens in he CGFPE framework of, or when deriving he Gassian esimaors of non-gassian densiies. The effecs of iniial condiions are clearly imporan in he shor and medim range predicion, for boh he perfec and he coarse-grained saisics, and he choice of a represenaive se of saisical iniial condiions reqires some care. In he following secions we consider he leas-biased condiionally Gassian esimaors i.e., L = in 4 of he re saisics p,,, leading o he non-gassian densiies p,,, as well as flly Gassian approximaions p G,, of he re non-gassian saisics p,,. Therefore, in order o compare he effecs of coarse-graining he srcre of he PDFs in a sandardized seing, we consider he iniial join densiies wih idenical second-order momens, i.e., any wo iniial densiies, p i, p j, saisfy α β p i, dd = α β p j, dd, α + β. 4 Here, for simpliciy we choose he iniial densiies wih ncorrelaed variables, p i, = π i π i, where he marginal densiies π i, π i are given by he mixres of simple densiies see Appendix B for more deails. This procedre is sfficien for he presen prposes and redces he complexiy of exposiion. Analogos procedre can be sed o generae PDFs wih correlaed variables by, for example, changing he coordinae frame; sch a sep migh be necessary when sdying he model error in filering problems. The following se of non-gassian iniial condiions, shown in figre 7 and consrced in he way described above, is sed in he sie of ess discssed nex see also Appendix B: p, : Nearly Gassian PDF wih he Gassian marginal in being and a weakly sb-gassian marginal in. p, : PDF wih a bimodal marginal in and a weakly skewed marginal in. 3 p 3, : Mlimodal PDF wih a bimodal marginal in and a ri-modal marginal in. 4 p 4, : PDF wih a highly skewed marginal in and a bimodal marginal in. 5 p 5, : PDF wih a weakly skewed marginal in and a highly skewed marginal in. 6 p 6, : Mlimodal PDF wih a Gassian marginal in and a ri-modal marginal in. 7 p 7, : Mlimodal PDF wih a bimodal marginal in and a Gassian marginal. 4

15 A. Join Disribion.5 B. Marginal disribion in π.6 C. Marginal disribion in g π D. Logarihm scale of B op and C boom π π π π π π π π π π π π π π π π π π π π π π π π π π Figre 7: Se of seven non-gassian iniial condiions wih idenical second-order saisics sed in he ess in figres 9-; see 5. and Appendix B for more deails. 5

16 5. Ensemble predicion wih model error de o coarse-graining he perfec dynamics Here, we consider he dynamics of he same non-gassian sysem 9 wih inermien insabiliies as in 3.. which has he general srcre as in 6-7. The wide range of ineresing rblen dynamical regimes [3, 5, 4, 5] makes his saisically exacly solvable sysem an nambigos esbed for sdying he effecs of model error inrodced hrogh varios coarse-grainings of he perfec densiy p,, as discssed in 4. In his secion, following he mehodology inrodced in 4, we focs on he model error arising from wo pariclar coarse-grainings of he perfec model densiy p,, : p,, : non-gassian densiy obained hrogh he leas-biased condiionally Gassian approximaion of he re condiional densiies sch ha he re densiy, p,,, and he coarse-grained densiy, p,,, have he same firs wo condiional momens, i.e., for each fixed and, we se S p, = max M N, =M N S q, where M N = N p, d, M N, = N qd, n. Noe ha, despie he Gassian approximaions for he condiional densiies p,, he coarsegrained join and marginal densiies p,, = p, π,, π, = p,, d can be highly non-gassian. p G,, : Gassian approximaion of he join densiy p,,. The error in he Gassian esimaors, p G,, and π G, = p G,, d, arises from he leas-biased approximaion of he re non-gassian densiy p,,, which for each fixed maximizes he enropy S p G,, = max M ij,g =M ij S q,, sbjec o he following momens consrains M i,j = i j p,, dd, M ij,g = i j q, dd, i + j. In he above se-p he condiional approximaions, p and π, represen he bes possible leas-biased esimaes for he re join and marginal densiies given he firs wo condiional momens. Ths, he errors Pp, p and Pπ, π represen he inrinsic informaion barriers which canno be overcome by models based ilizing wo-momen approximaions of he re densiies see Proposiion 4. in 4. In figres 9- we show he evolion of model error 37 de o differen coarse-grainings in p and p G in he following hree dynamical regimes of he sysem 9 wih Gassian damping flcaions see also figre 8: Regime I figre : Regime wih plenifl, shor-lasing ransien insabiliies in he resolved componen wih fa-ailed marginal eqilibrim densiies π; here, he parameers sed in 9 are ˆ =, σ = d =, σ =, f =. Regime II figre : Regime wih inermien large-amplide brss of insabiliy in wih fa-ailed marginal eqilibrim densiies π; here, he parameers sed in 9 are ˆ =, σ = d =, σ =, f =. 6

17 Regime I Regime II Regime III p p p Figre 8: Three dynamical regimes of he non-gassian sysem 9 characerized by differen eqilibrim marginal densiies π eq sed for sdying he model error in coarse-grained densiies in 5. see figres 9-. Regimes I and II of 9 are characerized by inermien dynamics of de o ransien insabiliies indced by damping flcaions. Regime III figre 9: Regime wih nearly Gassian marginal eqilibrim densiy π; here, he parameers sed in 9 are ˆ = 7, σ = d =, σ =, f = In each regime he model error in he ensemble predicions is examined for he se of seven differen iniial densiies inrodced in 5. and figre 7 wih idenical second order-saisics. The evolion of he re densiy, p,,, is esimaed via Mone Carlo simlaions wih 7 samples, while he coarse-grained join densiies p, p G, and heir marginals, π, π G, are comped according o he momen-consrained maximm enropy principle in 34 sing he condiional momens comped from he CGFPE procedre. The op row in figres 9- shows he evolion of model error in he Gassian esimaors, p G,,, π G,, of he re densiy. The inrinsic informaion barrier in he Gassian approximaion see Proposiion 4., represened by he lack of informaion in he leas-biased densiy, p, based on wo condiional momen consrains is shown for each regime in he middle row. I can be seen in figres 9- ha he common feare of he model error evolion in all he examined regimes of 9 is he presence of a large error a he inermediae lead imes. The sorce of his phenomenon is illsraed in figre in regime III of 9 wih a nearly Gassian aracor saisics; he large error arises from he presence of a robs ransien phase of fa-ailed dynamics in he sysem 9 which is poorly capred by he coarse grained saisics. Below, we smmarize he resls illsraed in figres 9- wih he focs on he model error in he Gassian approximaions p G,, and π G, : For boh he Gassian esimaors p G,, π G and he condiionally Gassian esimaors p,, π, here exiss a phase of large model error a inermediae lead imes. This phase exiss in all he examined regimes of 9 irrespecive of he iniial condiions and i is arises de o a ransien highly non-gassian fa-ailed dynamical phase in 9 which he Gassian esimaors fail o capre. The rends in he model error evolion for he join and he marginal densiies are similar; his is o be expeced based on Proposiion 4.. 7

18 The conribions o he model error in he Gassian esimaors p G,, π G from he inrinsic informaion barrier, Pp, p see Proposiion 4., and from he error Pp, p G de o he flly Gassian vs condiionally Gassian approximaions depends on he dynamical regime. The effecs of he inrinsic informaion barrier are he mos prononced in he non-gassian regime I of 9 wih abndan ransien insabiliies in see figre and 8; in his regime he informaion barrier dominaes he oal model error. In he nearly Gassian regime he inrinsic informaion barrier is negligible excep a shor imes de o he errors in coarse-graining he highly-non-gassian iniial condiions; see figres 9 and 7. In he highly non-gassian regime I wih abndan insabiliies and fa-ailed eqilibrim PDF figre 8 he differences in model error beween differen iniial condiions qickly become irrelevan; he inrinsic informaion barrier dominaes he model error and here is a significan error for long range predicions in boh he join and he marginal coarse-grained densiies. In he non-gassian regime II of 9 wih large amplide inermien insabiliies he inrinsic informaion barrier dominaes he error in he Gassian esimaors a shor ranges. A inermediae lead imes he error de o he flly Gassian vs condiionally Gassian approximaions exceeds he inrinsic barrier. The error a long lead imes is significanly smaller han in regime I wih comparable conribions from Pp, p and Pp, p G. In he nearly Gassian regime III of 9 he inrinsic informaion barrier in he Gassian esimaors error is small and dominaed by he errors in coarse-graining he non-gassian iniial condiions. The inrinsic informaion barriers in he join densiy, Pp, p, and in he marginal densiy, Pπ, π, are comparable hrogho he evolion and almos idenical a shor lead imes. 5.3 Ensemble predicion wih model error de o imperfec dynamics Here, we focs on he model error which arises hrogh common approximaions associaed wih ensemble predicion: i errors de o imperfec/simplified dynamics and ii errors de o coarse-graining he saisics of he perfec sysem which is sed for ning he imperfec models. While he above wo approximaions are ofen simlaneosly presen in applicaions and are generally difficl o disenangle, i is imporan o ndersand he effecs of hese wo conribions in a conrolled environmen which is developed below. Similar o he framework sed in he previos secions, we consider he dynamics wih he srcre as in he es model 6-7 where he non-gassian perfec sysem, as in, is given by [ ] a d = a + b c 3 + f d + A B dw C + σ dw, 43 b d = + f d + σ dw, wih cbic nonlineariy in he damping flcaions. The imperfec non-gassian model inrodces errors by assming Gassian dynamics in he damping flcaions, as in 9, a d m = d m m ˆ m + f m d + σ m dw m, 44 b d m = m m + f m d + σ m dw m. The imperfec model 44 is opimized by ning is marginal aracor saisics, in eiher m or m depending on he conex, o reprodce he respecive re marginal saisics. This is a prooype problem for a nmber of imporan isses; wo opical examples are: 8

19 Redced models wih a sbse of nresolved variables here m whose saisics is ned for saisical fideliy in he resolved variables here m, Simplificaion of pars of dynamics in complex mli-componen models sch as he copled amosphereocean-land models in climae science; in he presen oy-model seing can be regarded as he amospheric forcing of he ocean dynamics. In order o illsrae he framework developed in -4, we compare he model error arising in he opimized imperfec saisics, p m,, or π m,, associaed wih 44 wih he model error in p,, or π, de o he Gassian coarse-graining of he condiional densiies p, of he perfec sysem 43 sing he CGFPE framework of. In pariclar, we show ha small model error can be achieved a medim and long lead imes for imperfec predicions of he marginal dynamics π m sing models wih ned nresolved dynamics models despie a large model error in he join densiy p m, Ensemble predicions wih imperfec dynamics and ime-independen saisics on he aracor Here, we consider he perfec sysem 44 and is model 44 wih invarian measres a heir respecive eqilibria; his configraion is achieved by assming consan forcing f =.8, f =.5, f m =, f m =.5 in boh 43 and 44. We firs examine he effecs of model error associaed wih wo disinc ways of opimizing he imperfec model 44: I Tning he marginal eqilibrim saisics of he damping flcaions m in 44 for fideliy o he re saisics of in 43. In order o ne he mean and variance of m o coincide wih he re momens, we simply se ˆ m = eq, σ m/ dm = V ar eq, 45 which leads o a one-parameer family of models in 44 wih correc marginal eqilibrim densiy in m. Below, we choose he damping, d m, in 44 as he free parameer and sdy he dependence of model error in he class of models saisfying 45 and parameerized by he damping/decorrelaion dime in m see figre 3. Noe ha only one model in his family can mach boh he eqilibrim densiy, π, and he decorrelaion ime, τ = Corr τdτ, of he re damping flcaions in 43; for sch a model we have, in addiion o 45, τ m = /d m = τ. 46 Examples of predicion error in models 44 opimized for eqilibrim fideliy in m b differen dampings d m are shown in figre 3 for he wo-sae nimodal regime of 43 see figre. We highligh wo imporan observaions here: Underdamped models 9 opimized for eqilibrim fideliy in he damping flcaions m have he smalles error for medim range forecass all models are comparable for long range forecass. These resls are similar o hose repored recenly in [5] where he shor and medim range predicive skill of linear models wih opimized marginal saisics of he nresolved dynamics was shown o ofen exceed he skill of models wih correc marginal saisics and decorrelaion ime. Despie he sriking redcion in model error inermediae lead imes achieved hrogh nderdamping he nresolved dynamics in 9, caion is needed when ning imperfec models for shor range forecass or forced response predicion where he damping, in boh he resolved and nresolved dynamics, is relevan for correc sysem response e.g., [5]. 9

20 II Tning he marginal eqilibrim saisics of he damping flcaions m in 44 for fideliy o he re saisics of in 43. This case corresponds o he siaion in which we consrc a simplified model of a sysem wih nresolved degrees of freedom here ; hese sochasically sperparameerized nresolved dynamics are hen ned o correcly reprodce he saisical feares of he resolved dynamics here. We consider his opimizaion in he Gassian framework and opimize he imperfec model 44 by ning he dynamics of he damping flcaions m in order o minimize he lack of informaion in he imperfec marginal densiy for he resolved variable, i.e., he opimal imperfec model saisfies P π G, πg m = min P π G, πg m d m,σm,ˆm, 47 where π G and πg m are he Gassian esimaors of he respecive marginal densiies associaed wih 43 and 44. Wih he condiional momens of in he perfec sysem 43, M and M, obained by solving in he CGFPE framework in, he mean and variance of p G are given by ū = M d, R = M d ū. 48 Analogos expressions hold for he mean and variance of p m G which are sed in he opimizaion 47. The wo ypes of model opimizaion are compared in figre 4 for he wo-sae nimodal regime of 43 see figre ; boh procedres yield comparably good resls a long lead imes when he model error in he marginal densiies in π m, is considered. Unsrprisingly, opimizing he marginal dynamics of m by ning he dynamics of m generally leads o a smaller model error for shor and medim range predicions b he ype of he opimizaion largely depends on he applicaions. In figres 5-8 we illsrae he evolion of model error in he imperfec saisical predicion of 43 which is opimized according o procedre I above. Two non-gassian regimes of he re sysem 43 and illsraed in figre are sed o analyze he error in imperfec predicions wih opimized models in Ensemble predicions wih imperfec dynamics and ime-periodic saisics on he aracor We finish he analysis by considering he dynamics of he perfec sysem 44 and of is model 44 wih ime-periodic saisics on he aracor. We focs on he highly non-gassian regime of he perfec sysem 43 wih he cbic nonlineariy in he damping flcaions periodic ransiions beween he nearly Gassian and highly skewed marginal densiy in he damping flcaions which are indced by he simple ime-periodic forcing f = f, + f, sinω + φ; his regime was previosly sed in 3.. o validae he CGFPE framework see figre. Similarly o he configraions sdied wih ime-independen eqilibrim saisics in he previos secion, we are ineresed in he differences beween he model error arising in he opimized imperfec dynamics, p m,,, π m, and he error de o he coarse-graining he perfec saisics in he densiies p,,, π obained hrogh he Gassian approximaions of he condiionals p,. The isse of ning he marginal aracor saisics of he damping flcaions m in he imperfec model 44 reqires more care han in he case wih ime-independen eqilibrim saisics; his is de o he presence of an inrinsic informaion barrier see 4 or [5, 5] when ning he saisics of he Gassian damping flcaions m in 44 o he re saisics of 43 in. Similar o he ime-independen case, we aim a ning he marginal aracor saisics in m for bes fideliy o he re marginal saisics in

21 . Here, however, here exiss an informaion barrier associaed wih he fac ha he aracor variance of he Gassian flcaions m is always consan regardless of he forcing f m. One way o opimize he imperfec saisics of is o ne is decorrelaion ime, and ime-averaged mean and variance on aracor o reprodce he re ime-averaged qaniies. However, sch an approach is clearly insensiive o phase variaions of he respecive saisical momens. Here, insead, we opimize he imperfec model by firs ning he decorrelaion imes of m and and hen minimizing he period-averaged relaive enropy beween he marginal densiies for he damping flcaions, i.e., he opimized model 44 saisfies P π G,, πg m, = min P π G,, π m σ m,{f m} G,, 49 where he overbar denoes he emporal average over one period and {f m } denoes a se of parameers in he forcing f m in 44; in he examples below we assme he form of he forcing f m wih he same ime dependence as he re one, i.e., f m = f m, + f m, sinω m + φ m, wih ω m = ω, φ m = φ, so ha he opimizaion in 49 is carried o over a hree parameer space {σ m, f,, f, } opimizaion in he phase and freqency are ofen crcial and ineresing b we skip he discssion for he sake of breviy. In figres 9- we show he model error for he coarse-grained join and marginal densiies p, π and compare hem wih he model error in he join and marginal densiies associaed wih he opimized imperfec model 44. Here, he parameers sed in 43 are a =, b =, c =, A =.5, B =.5, σ =.5, σ =, f =.5, f, =.5, f, = 6.5, ω = π, φ = π/. 5 In figre 9 he decorrelaion ime, τ m = /d m, of he damping flcaions m is he same as he one in he re dynamics while he resls shown in figre illsrae he dependence of model error in he opimized imperfec model on he decorrelaion ime see also figre 3 for he configraion wih ime-independen eqilibrim saisics. The following poins smmarize he resls of 5.3., and 5.3.: Small model error can be achieved a medim and long lead imes for imperfec predicions of he marginal dynamics π m sing models wih ned nresolved dynamics models despie a large model error in he join densiy p m, ; figres 3, 4, 5, 7, 9,. The error in he coarse-grained densiies p,,, π, is mch smaller han ha in he opimized models wih imperfec dynamics wih p m,,, π m, ; figres 5-8. The larges error in he opimized models 9 is associaed wih he presence of ransien mlimodal phases which canno be capred by he imperfec models in he class 9; figres 5-9. A long lead imes he model error in he join densiies, P p,,, p m,,, is largely insensiive o he variaion of he damping d m figre. The model error in he marginal densiies π m, of he opimized models has non-rivial dependence on he decorrelaion ime /d m of he damping flcaions; he overall rend is ha nderdamped imperfec models have smaller error in he marginals π m, for consan or slow forcing, while he overdamped imperfec models are beer for srongly varying forcing see figres 3, for wo exreme cases.

22 Pπ re,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,,.5..5 Pπ re, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re,, π, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re, π p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Figre 9: de o coarse-graining he perfec dynamics of he sysem 9 in he nearly Gassian regime Regime III in figre 8. Top wo rows Evolion of model error 37 de o differen coarsegrainings of he perfec dynamics in he sysem 9 wih Gassian damping flcaions; he non-gassian join and marginal densiies, p, π, are obained hrogh he Gassian coarse-graining of he condiional saisics p see 3,4, while p G, π G are he join and he marginal densiy of he Gassian esimaors see 4. The informaion barrier boom row eqals Pp, p G Pp, p G see 37. The respecive saisical iniial condiions, all wih he same second-order momens, are described in 5. and shown in figre 7.

23 Pπ re,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,,.5..5 Pπ re, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re,, π, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re, π p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Figre : de o coarse-graining perfec dynamics; sysem 9 in regime wih inermien large amplide insabiliies.top wo rows Evolion of model error 37 de o differen coarsegrainings of he perfec dynamics in he sysem 9 wih Gassian damping flcaions; he non-gassian join and marginal densiies, p, π, are obained hrogh he Gassian coarse-graining of he condiional saisics p see 3,4, while p G, π G are he join and he marginal densiy of he Gassian esimaors see 4. The informaion barrier boom row eqals Pp, p G Pp, p G see 37. The respecive saisical iniial condiions, all wih he same second-order momens, are described in 5. and shown in figre 7. 3

24 Pπ re,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,,.5..5 Pπ re, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re,, π, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ re, π p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Pπ,, π G, p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, x Pπ, π G p,, p,, p 3,, p 4,, p 5,, p 6,, p 7,, Figre : de o coarse-graining perfec dynamics; sysem 9 in regime wih abndan ransien insabiliies.top wo rows Evolion of model error 37 de o differen coarse-grainings of he perfec dynamics in he sysem 9 wih Gassian damping flcaions; he non-gassian join and marginal densiies, p, π, are obained hrogh he Gassian coarse-graining of he condiional saisics p see 3,4, while p G, π G are he join and he marginal densiy of he Gassian esimaors see 4. The informaion barrier boom row eqals Pp, p G Pp, p G see 37. The respecive saisical iniial condiions, all wih he same second-order momens, are described in 5. and shown in figre 7. 4

25 * =.4.3. π, π re π π G.4.3. π re π π G π,.. * =.4 * =" " π re π π G π re π π G π re π π G π re π π G Figre : Three disinc sages in he saisical evolion of he sysem 9 illsraed for he regime wih nearly Gassian dynamics and highly non-gassian mlimodal iniial saisical condiions p 3, figre 7; hese hree sages exiss regardless of he dynamical regime of 9 and he form of he iniial condiions no shown. Top The iniial configraion projeced on he marginal densiies a =, Middle The fa-ailed phase in he marginal π, corresponding o he large error phase in he coarse grained models see figres 9-, Boom Eqilibrim marginal saisics on he aracor in he regime wih nearly Gassian saisics see Regime III in figre 8. 5