Implementation of two statistical methods for Ensemble Prediction Systems in the management of electrical systems

Transcription

1 CS-BIGS 5(2) : CS-BIGS hp:// Implemeaio of wo saisical mehods for Esemble Predicio Sysems i he maageme of elecrical sysems Adriaa Gogoel Elecricié de Frace ad Uiversiy of Paris Descares, Frace Jérôme Colle Elecricié de Frace, Frace Aver Bar-He Uiversiy of Paris Descares (MAP5), Frace This paper preses a sudy of wo saisical pos-processig mehods implemeed o forecass of Meeo-Frace emperaures provided by he esemble predicio sysem (EPS) The resuls are useful i he maageme of elecriciy cosumpio a EDF Frace. Those mehods are he Bes-Member Mehod (BMM) proposed by Fori (2006), ad he Bayesia Model Averagig mehod (BMA) proposed by Rafery (2004). The idea behid he BMM is o desig for each lead ime i he daa se he bes forecas amog all forecass provided by he emperaure predicio sysem, o cosruc a error paer usig oly he errors made by hose "bes members" ad o he "dress" all he members of he iiial predicio sysem wih his error paer. The BMA mehod is a saisical mehod which combies predicive disribuios from differe sources. The BMA predicive probabiliy desiy fucio (PDF) of he quaiy of ieres is a weighed average of PDFs ceered o he bias-correced forecass, where he weighs are equal o poserior probabiliies of he models geeraig he forecass ad reflec he accuracy (sill) of he models over he raiig period. The resulig forecass implemeed o our daa se are compared wih oe aoher ad compared o he iiial forecass, usig scores which measure he accuracy ad/or he spread of he EPS: he Mea Absolue Error (MAE), he Roo Mea Square Error (RMSE), he Igorace Score, he Coiuous Ra Probabiliy Score (CRPS), he Talagrad Diagram, he Bias ad he Mea. The purpose is o improve he probabiliy desiy fucio of he forecass, preservig a he same ime he qualiy of he mea forecass. The preseaio is accessible o readers wih a iermediae level of saisics. Keywords: forecasig; esemble predicio sysems; eergy; Bayesia aalysis 1. Iroducio 1.1 Coex The eergy secor is highly weaher-depede, hece i eeds accurae forecass o guaray ad opimize is aciviies. Predicios of producio are eeded o opimize he sellig ad disribuio of elecriciy. Of

2 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. course eeds ielecriciy deped o meeorological codiios. Numerous specialiss i physics (for example meeorologiss) build sophisicaed deermiisic umerical models, wih uceraiy o he ipu daa. To ae io accou his uceraiy, hey ru he same model several imes, wih sligh bu o radom perurbaio of he daa. I seems obvious ha he resuls of physical models coai irreplaceable iformaio. Neverheless we oice ha he probabiliy disribuio obaied from he model is ypically o a perfec represeaio of he ris facor, hus we eed o submi i o a saisical processig before usig i. Correlaio bewee emperaure ad elecriciy cosumpio Temperaure is he mai ris facor for EDF as a elecriciy producer i Frace, a coury where elecric heaig is well developed. If we ae io accou he variabiliy i emperaure, he power cosumed for heaig o a wier give day ca vary by abou 20GW, which represes 40 % of he average cosumpio. Regardig eergy, he climaic ris facor is quaiaively less impora, because he differece i eergy cosumed bewee he warmes ad he coldes wiers represes approximaely 5 % of he eergy over he year. To explai he correlaio bewee emperaure ad elecriciy cosumpio we oe ha he Frech elecrical load is very sesiive o emperaure because of he developme of elecric heaig sice he 70 s. The ifluece of emperaure o he Frech load is mosly ow, excep for he impac of air codiioig whose red remais difficul o esimae. Elecric heaig is used o maiai a emperaure close o 20 C iside buildigs. Taig io accou he "free" coribuios o hea (su, huma hea), i is cosidered ha elecric heaig urs o a approximaely below 1 6 C. Beyod ha emperaure, he hea loss beig proporioal o he emperaure differece bewee iside ad ouside, he cosumpio icreases approximaely liearly. Moreover, sice buildigs ae a cerai amou of ime o warm up or o cool dow, he reacio o ouside emperaure variaios is delayed.to ae io accou his delay, oe uses a smoohed emperaure (based o a "average" emperaure) as a predicor of he cosumpio (Bruhs e al., 2005). The siuaio is similar for air codiioig. 1.2 Purpose of he wor This sudy has for objecive o improve he probabilisic disribuio of forecass provided by he Esemble Predicio Sysems (EPS) of Meeo-Frace, while preservig he accuracy (sill) of he mea forecass. The iiial EPS coais = 51 members - scearios of he same model - oe sarig wih uperurbed iiial weaher codiios (he corol forecass) ad 50 from perurbed iiial codiios defied by addig small dyamically acive perurbaios o he operaioal aalysis for he day. Each oe of he 51 members of he sudied EPS provides rajecories of emperaures for 14 imehorizos (1 horizo correspods o 1 day). We impleme saisical pos-processig mehods o improve is use for he maageme of he elecric sysem a EDF Frace. The use of he EPS mehod allows o he oe had o exed he horizo where we have good forecass ad o he oher had o give a measure of forecas uceraiy. Ulie deermiisic soluios he probabiliy forecas is beer adaped o he aalysis of ris ad decisio-maig. Firs, we sudy he Meeo-Frace emperaure forecass ad he emperaure realizaios i rerospecive mode i order o esablish he saisical li bewee hese wo variables. We he examie wo saisical processig mehods of he paer s oupus. From sae of he ar exisig mehods ad from he resuls obaied by he examiaio of he probabiliy forecass, a pos-processig module will be developed ad esed. The goal is o achieve a robus mehod for calibraig saisical forecass. This mehod should hus ae io accou he uceraiies of he ipus (represeed by he 51 differe iiial codiios added o he paer). The firs mehod is he Bes-Member Mehod (BMM) ad i has bee proposed by Fori e al. (2006). The idea is o desig for each lead ime i he daa se he bes forecas amog all forecass provided by he emperaure predicio sysem, o cosruc a error paer usig oly he errors made by hose "bes members" ad he o "dress" all he members of he iiial predicio sysem wih his error paer. This approach fails i cases where he iiial predicio sysems are already over dispersed. This is he reaso why a secod mehod was iroduced which allows dressig ad weighig each member differely by classes of is saisical order. We prese i his paper he secod mehod, ha we call W-BMM.

3 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. The secod mehod we impleme is he Bayesia Model Averagig (BMA) mehod proposed by Rafery e al. (2004). I is a saisical mehod for pos processig model oupus which allows o provide calibraed ad sharp predicive Probabiliy Disribuio Fucios (PDFs) eve if he oupu iself is o calibraed (forecasig are well calibraed if for a forecas wih probabiliy p, he prediced eve is observed p imes). The mehod allows for usig a slidig-widow raiig period o esimae ew models parameers, isead of usig he w h o l e daabase of pas forecass ad observaios. Resuls will be compared usig scores which measure he sill ad/or he spread of he EPS: Mea Absolue Error (MAE), Roo Mea Square Error (RMSE), Igorace Score, Coiuous Ra Probabiliy Score (CRPS), Talagrad Diagram, Reliabiliy diagram, Bias, Mea. 2. Esemble predicio sysems (EPS) Esemble predicio sysems are a raher ew ool i operaioal forecasig which allows for faser ad scieifically jusified comparisos of several forecas models. The EPS is coceived i order o yield he probabiliy of meeorological eves ad he zoe of ihere uceraiy i every plaed siuaio. I is a echique o predic he probabiliy disribuio of forecas saes, give a probabiliy disribuio of radom aalysis error ad model error. The priciple of he EPS is o ru several scearios of he same model wih slighly differe ipu daa i order o simulae he uceraiy. I he curre sysem Meeo-Frace is usig, each EPS perurbaio is a liear combiaio of sigular vecors wih maximum growh compued usig a oal eergy orm. The assumpio uderlyig he liear combiaio is ha iiial error is ormally disribued i he space spaed by h e sigular vecors. A Gaussia samplig echique is used o sample realizaios from his disribuio (IFS Documeaio 2006). The EPS is based o he oio ha forecas uceraiy is domiaed by error or uceraiy i iiial codiios. This is cosise wih sudies ha show ha, whe wo operaioal forecass differ, he differeces ed o origiae i he aalyses raher ha i model formulaio see (IFS Documeaio 2002). We oe some of he primary objecives of EPS (Malle 2008): 1. Allows for esimaig he uceraiy, obaiig a represeaive spread of he ypical uceraiy, esurig a empirical sadard deviaio of he forecass comparable wih he sadard deviaio of he observaios; 2. Provides a good esimaio of he probabiliy of a eve; 3. Allows for a coveie use of liear combiaios of models i forecas. 3. Verificaio mehods for EPS Meeorologiss have bee usig EPS for several years ow ad a he same ime may mehods for evaluaig heir performaces have bee developed (Sasi e al. 1989). A proper scorig rule maximizes he expeced reward (or miimizes he expeced pealy) for forecasig oe s rue beliefs, hereby discouragig hedgig or cheaig (Jolliffe ad Sepheso 2007). Oe ca disiguish wo ypes of mehods: he oes permiig o evaluae he qualiy of he spread ad he oes givig a score (a umerical resul) permiig o evaluae he performace of he forecass (Pei 2008). Therefore, we eed o examie he sill or accuracy (how close he forecass are o he observaios) ad spread or variabiliy (how well he forecass represe he uceraiy). If model errors played o role, ad if iiial uceraiies were fully icluded i he EPS iiial perurbaios, a small spread amog he EPS members would be a idicaio of a very predicable siuaio i.e. whaever small errors here migh be i he iiial codiios, hey would o seriously affec he deermiisic forecas. By coras, a large spread idicaes a large uceraiy of he deermiisic forecas (Persso 2003). As for he sill, i idicaes he correspodece bewee a give probabiliy, ad he observed frequecy of a eve. Saisical cosideraios sugges ha eve for a perfec esemble (oe i which all sources of forecas error are sampled correcly) here may o be a high correlaio bewee spread ad sill (Whiaer ad Loughe 1998). 3.1 Sadard Saisical Measures Le y be he vecor of model oupus ad le o be he vecor of he correspodig observaios. These vecors boh have compoes. Their meas are respecively y ad o.

4 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. The Bias is give by: 1 yi i1 Biasm 1 oi i1 The Correlaio Coefficie is give by: r 1 i1 ( y y)( o o) i 2 2 ( yi y) ( oi o) i1 i1 i The Mea Absolue Error (MAE) measures overall accuracy ad is defied as: 1 i i i 1 (1) (2) MAE y o (3) The roo mea square error (RMSE) has he advaage of beig recorded i he same ui as he observaios ad i is he roo square of he MSE where 1 2 MSE is give by MSE ( yi oi). 3.2 Reliabiliy i 1 The reliabiliy (or spread) measures how well he prediced probabiliy of a eve correspods o is observed probabiliy of occurrece. For a p probabiliy forecas, he prediced eve should be observed roud ( p ) imes. Talagrad Diagram. I is a ype of bar char i which caegories are represeed by bars of varyig ras raher ha specific values - a hisogram of ras. The Talagrad diagram has is origis i he Probabiliy Iegral Trasform diagram (PIT, see Dordoa ad Colle 2010). I measures how well he spread of he esemble forecas represes he rue variabiliy (uceraiy) of he observaios. For each period (day) we cosider he esemble of he forecass values (icludig he observaio value). The values wihi his esemble are ordered ad he posiio of he observaio is oed (he ra). For example he ra will be 0 if he observaio is below all he forecass ad N if he observaio is above all he forecass. Repeaig he procedure for all he forecass we obai a hisogram of observaio ras. By examiig he shape of he Talagrad diagram, we ca draw coclusios o he bias of he overall sysem ad he adequacy of is dispersio: A fla hisogram: he esemble spread correcly represes forecas uceraiy. I does o ecessarily idicae a silled forecas bu oly measures wheher he observed probabiliy disribuio is well represeed by he esemble. A U-shaped hisogram: he esemble spread oo small, so ha may observaios fall ouside he exremes of he esemble A Dome-shaped hisogram: he esemble spread is oo large, so ha oo may observaios fall ear he ceer of he esemble Asymmeric hisogram: he esemble coais bias. 3.3 Resoluio (sharpess) The resoluio (or accuracy, or sill) is he measure of he accuracy of he forecass. Coiuous Ra Probabiliy Score (CRPS). The CRPS measures he differece bewee he forecas ad observed cumulaive disribuio fucios (CDFs). The CRPS compares he full disribuio wih he observaio, where boh are represeed as CDFs. If F is he CDF of he forecas disribuio ad x is he observaio, he CRPS is defied as: CRPS( F, x) F ( y) 1 y x dy (4) deoes a sep fucio alog he real lie ha aais he value 1 if y x ad he value 0 oherwise. I he case of probabilisic forecass he CRPS is a probabiliy-weighed average of all possible absolue differeces bewee forecass ad observaios. The CRPS eds o icrease wih forecas bias ad be reduced by he effecs of he correlaio bewee forecass ad observaios (Schaae e al. 2007). where 1 y x Oe of is advaages is ha i has he same uis as he prediced variable (so is comparable o he MAE) ad does o deped o predefied classes. I is he geeralizaio of he Brier score for he case of he coiuous variables. The CRPS provides a diagosic of he global sill of a EPS, he perfec CRPS is 0, a 2

5 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. higher value of he CRPS idicaes a lower sill of he EPS. 4. Pos-processig mehods 4.1 The Bes-Member mehod The Bes-Member Mehod was proposed by V.Fori e al. (2006) ad improves o sudies iously led by Roulso ad Smih (2002) he by Wag ad Bishop (2005). The idea is o desig for each lead ime i he daa se he bes forecas amog all 51 forecass (i our case), o cosruc a error paer usig oly errors made by hose "bes members" ad o he "dress" all members wih his error paer. This approach does o wor i cases where he udressed esemble members are already over- or uder-dispersed; he soluio is he o weigh ad dress each member differely, ha is usig a differe error disribuio for each order saisic of he esemble. So we ca disiguish wo specialized mehods: he oe wih cosa dressig, or he u-weighed members mehod ad he oe wih variable dressig, or he weighed members mehod. We impleme ad prese i his paper he Weighed Members Mehod (W-BMM) The Weighed Members Mehod Fori applied his mehod o a syheic EPS (where he EPS was buil uder cerai codiios - esemble members are idepede ad ideically disribued - ad where we ca vary he parameers we wa i order o es differe hypoheses or mehods). He observed ha his mehod failed i he case of over-dispersed or uder-dispersed EPS. The explaaio is ha whe a EPS is uder-dispersed, he oucome ofe lies ouside he spread of he esemble. Hece, a exreme forecas has a much more chace of givig he bes predicio ha a forecas close o he esemble mea. Coversely, whe a esemble is over-dispersed members close o he esemble mea have a much higher chace o be bes members ha exreme forecass. Hece, he probabiliy ha a esemble member gives he bes forecas as well as he error disribuio of he bes member depeds o he disace o he esemble mea. For uivariae forecass we ca sor esemble members from he smalles o he bigges, oe heirs ras ad cosider he ra of a member a he dressig sequece. x Le,, j be he emperaure predicios provided by a give EPS, where is he sceario s umber, is he ime for which he forecas is made ad j is he imehorizo. The mehod is preseed i he uivariae case so j is fixed, hece x,, j becomes x,. Le y be he uow variable which is forecased a ime, ad le X { x, 1,2,..., K} be he se of, all esemble members of he forecasig sysem. Give X he purpose is o obai a probabilisic forecas i.e. p( y X) i order o provide may more predicive simulaios sampled from p( y X ) where X { x,m,m 1,2,..., M} wih M K. The cocep of codiioal probabiliy allows for icorporaig addiioal iformaio io he forecass (i his case i will be he forecass give by Meeo Frace). The basic idea of he mehod is o "dress" each esemble member x, wih a probabiliy disribuio equal o ha of he error made by his member whe i happeed o give he bes forecas. The bes sceario deoed by x * is defied as he oe miimizig y x. for a give orm.. As we are worig i a uivariae space, he orm is he absolue value so ha: Le:,( ) x arg mi y x. * x, x be he h member of he esemble X { x, 1,2,..., K} ordered by ra., ( ), 1,2,..,,( ) T y x x x be he errors of he bes esemble members for every ime i a daabase of pas forecass, where he bes forecas has ra. p be he probabiliy ha x,() is he bes member, i.e p Pr[ x x ].,() To dress each esemble member differely, isead of resamplig from he archive of all bes-member errors, oe resamples from ( ) o obai dressed esemble members. Hece, he simulaed forecass are obaied as:,

6 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. y x (5),,,,, where: he ε,(), are draw a radom from ( ), is he umber of simulaios per ime sep, 2 s ad s 2 s 1 is he T 2 2 (,( ) ) T 1 1 esimaed variace of he bes-member error. This compuaio fails i he case of EPSs where he uceraiy is already over esimaed ( 2 s egaive). 4.2 Bayesia model averagig The Bayesia approach is based o he fac ha he probabiliy of realizaio of a eve does o deped oly o is frequecy of appearace bu also o he owledge ad experiece of he researcher. We base our sudy o he Bayesia Model Averagig mehod (BMA) (Rafery e al. 2004). The BMA predicive probabiliy desiy fucio (PDF) of he quaiy of ieres is a weighed average of PDFs ceered o he bias-correced forecass, where he weighs are equal o he poserior probabiliies of he models geeraig he forecass, reflecig he models sill over he raiig period. A origial idea i his approach is o use a movig raiig period (slidig-widow) o esimae ew models parameers, isead of usig he whole daabase of pas forecass ad observaios. This implies a choice of legh for his slidig-widow raiig period; he priciple guidig his choice is ha probabilisic forecasig mehods should be desiged o maximize sharpess subjec o calibraio. I is a advaage o use a shor raiig period i order o be able o adap rapidly o chages (sice weaher paers ad model specificaio chage over ime) bu he loger he raiig period, he beer he BMA parameers are esimaed (Rafery e al. 2004). Afer comparig measuremes such as he RMSE, he MAE, he CRPS for various raiig period leghs (from 10 o 60) Rafery e al. coclude ha here are subsaial gais i icreasig he raiig period o up o 30 days, ad ha beyod ha here is lile gai. The mai differece bewee heir case ad ours is ha hey have 5 models ad we have 51 (scearios). T Le y be he quaiy o be forecased ad M 1,..., M K be K saisical models providig forecass. Accordig o he law of oal probabiliy, he PDF of he forecass p(y) is give by: K T p y = p(y M ) p(m y ) (6) 1 where p(y M ) is he forecas PDF based o M T ad p(m y ) is he poserior probabiliy of model M beig correc give he raiig daa which a measure of wheher he model fis he raiig daa. The sum of all poserior probabiliies correspodig o K T he models is 1: p(m y )=1. 1 This allows us o use hem as weighs, so o defie he BMA PDF as a weighed average of he codiioal PDFs. This approach uses he idea ha here is a bes "model" for each predicio esemble bu i is uow. Le f he bias-correced forecas provided by he model M which yields he bes predicio, correspodig o a PDF g (y f ). The BMA predicive model he is give by: K p(y f,.., f ) = w g (y f ) (7) 1 1 where w is he poserior probabiliy of forecas beig he bes oe ad is based o forecas s performace i he raiig period ad where K w = 1 =1. For emperaure ad sea- level pressure, he codiioal PDF ca be fied reasoably well usig a ormal disribuio ceered a a bias-correced forecas a +b f as show by Rafery e al. (2004): 2 y f N(a bf, ) The parameers a,b as well as he esimaed o he basis of he raiig daa se: b by simple liear regressio of y o w are o be a ad f for he

7 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. raiig daa ad w, = 1,.., K, ad σ by maximum lielihood (Aldrich 1997) from he raiig daa. For algebraic simpliciy ad umerical sabiliy reasos i is more coveie o maximize he logarihm of he lielihood fucio raher ha he lielihood fucio iself; he expecaio-maximizaio (EM) algorihm (Dempser e al. 1977) is used. Fially he BMA PDF is a weighed sum of ormal PDFs ad he weighs w reflec he overall performace of he esemble members over he raiig period, relaive o oher members. 5. Applicaio 5.1 Daa descripio We are worig o emperaure forecass provided by Meeo-Frace as a esemble of weaher predicio sysems which coais 51 members, or 51 equiprobable scearios obaied by ruig he same forecasig model wih slighly differe iiial codiios. The daa se correspods o he period bewee March ad 20 of April ad coais forecass up o 14 ime-horizos correspodig o 14 days (1 horizo correspods o 24 hours). Currely he value used o predic he cosumpio is he mea of he 51 forecass. I Figure 1, o op we represe for hree fixed ime-seps he curves of he predicio errors for he 51 scearios as a fucio of ime-horizo (from 1 o 14-days ahead). The errors ypically icrease wih he ime-horizo; hey are paricularly small up o he 4h imehorizo. Hece, we cosider ha up o he 4h imehorizo he deermiisic forecass give high qualiy forecass ad we impleme our improveme mehods sarig wih he 5h horizo. I he same figure, o he boom, we ca see he predicio errors for all periods bu for hree differe imehorizos; we oice he same ypical correlaio bewee he errors ad he ime-horizo. As meioed above every sceario, amog he 51, gives forecass up o 14-days ahead. The differece bewee he scearios comes from he small dyamically acive perurbaio added o heir iiial codiios. Hece his perurbaio is o relaed o he ame of he sceario (umbers bewee 0 ad 50) ad is o he same from oe day of forecasig sar o aoher 1. The emperaure measuremes are performed by 26 differe Frech saios, of which we ae a weighed average o obai a sigle emperaure for Frace. The weighs are defied so as o bes explai elecriciy cosumpio for differe Frech regios. We sar by seig he ime-horizo. Oce he horizo is fixed, we sudy he forecass, sarig wih 5-days ahead horizos sice up o 4-days ahead he deermiis forecass are very good (he Meeo-Frace paer is purposely buil o be uder- dispersed up o 3 days). I his paper we prese he 5-days ahead resuls. We ca see i Figure 2 superposed o he same graph he curve of he realizaios ad he curve of he average prediced emperaures. 5.2 Implemeaio of he weighed Bes- Member mehod Le y be he emperaure variable we are forecasig a ime ad le X ={x,, = 1,2,..., K} be he se of all esemble members of he Meeo-Frace forecasig sysem. We would lie o obai a probabilisic forecas i.e. p(y X ). The codiioal probabiliy allows for aig io accou addiioal iformaio i a forecas, i our case he forecass give by Meeo-Frace. The bes sceario x is he oe miimizig y x,. To compue he W-BMM mehod we use he SAS sofware. We use a cross-validaio mehod o build ad es our models: we divide he four years i our daa se io wo equal pars: he firs par serves as a model buildig period for he model we will validae o he secod par ad vice versa. As meioed i he preseaio of he mehod, he saisical ra of he esemble members is ae io accou. We deoe he h forecas by x, ad recall ha { y x x x,, 1,2,,..., T} as defied above (see 4.1.1). 1 For example: forecass give by sceario 15 compued o July 1s for he period July 1s-July 7h ae io accou from he begiig a cerai perurbaio. Tha perurbaio will o be he same as he oe ae io accou by sceario 15 whe o July 2d i provides forecass for he period July 2d- July 8 h.

8 Figure 1. Figures correspodig o iiial predicios. O op, predicio error curves for he 51 scearios for hree fixed days, for all 14 ime-horizos (i gray are sceario errors from 1 o 51, i blac sceario 0 - he oe wih o perurbed iiial codiios - ad i red he mea of he 51 scearios). A he boom are he forecasig errors for hree differe ime-horizos; we ca see he errors become larger wih he ime-horizo.

9 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. Figure 2. Graphs correspodig o iiial predicios for 5-days ahead. I blac is he observed emperaures curve, i blue he iiial predicio meas. We oice a good precisio of he mea forecass excep for exreme emperaures. Give he archive of pas forecass ad a orm we creae a probabiliy disribuio from he realizaios of * = y - x : (),() () = () + exp( ())N(0, 1) (8) () where: is he ime-sep, he µ are he values of he errors, prediced by a liear regressio model M 1 as described below, he are he logarihms of he absolue values of he residuals from he M 2 paer, prediced by a liear regressio model M 2 as described below, he saisical ra does o ierfere direcly a his sage of he sudy bu ierferes idirecly i he creaio of he M 1 ad M 2 paers. The M 1 paer explais he predicio error wih he iiial forecas, he day-posiio wihi he year ad he saisical ra τ : 3 µ x [ a i b i ]. (9) 1 2, i 3, i 4 i1 The M 2 paer explais he logarihm of he absolue value of he residuals from he M 1 paer i.e., wih he emperaure, day-posiio wihi he year ad he saisical ra τ : 3 i i x [.a b ] 1 2, i 3, i 4 i1 (10) We herefore also have wo parameers µ (prediced by he M 1 paer) ad (prediced by he M 2 paer). Boh are geeraed by 7 parameers:,, µ (i = 1, 2, 3) 1 2,i 3,i for ad 1, 2,i, 3,i for (i = 1, 2, 3). Boh have he same legh as he sudied period 1,459. As meioed above we use hem as parameers of he ormal disribuio o simulae our ew forecass. We wa o obai M=10 K=10 x 51=510 simulaios so we will draw N = p M dressed esemble

10 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. members from each x, ( ). This way classes wih poserior probabiliies of givig beer forecass will be simulaed more ha classes wih small such probabiliies: y,, x, (11),, where p Pr[ x x ] is he probabiliy ha,() x, ( ) gives he bes forecass amog he K=51 scearios. Figure 3. Probabiliy iervals (10%-90%) for he 510 daily simulaios (i red), heir media (i gree) ad he curve wih realizaios (i blue) for a oe-year period. We oe o his graph ha he media of he simulaios lies i he [10%, 90%] ierval. Figure 4. BMA mehod. CRPS ad MAE for 5-days ahead for differe leghs of he raiig period, from 10 o 50, by 5- day seps.

11 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. I Figure 3 we ca observe he media of simulaed forecass, he real emperaures curve ad he 10% - 90% probabiliy ierval for he simulaed forecass. The curve of he forecass we simulaed is sill o perfecly close o he curve of observaios. I is ieresig o observe o ha graph wheher he real emperaures curve always lies wihi he 10% - 90% ierval. Oher resuls of he ess verifyig sill ad spread are preseed i Secio Implemeaio of he Bayesia Model Averagig mehod Applyig he Bayesia Model Averagig mehod cosiss i cosrucig he BMA PDF as a weighed sum of ormal PDFs, where he weighs reflec he overall performace of he esemble members over he raiig period. To impleme his mehod we use a R pacage for probabilisic forecasig, esemble BMA creaed by Fraley e al. (2009) usig esemble pos processig via Bayesia Model Averagig o provide fucios for modelig ad forecasig daa. Whe we cosruc he Bayesia model we cosider ha forecass esemble members are ierchageable (because of he idepedece of he forecas scearios, see secio 2) ha is, heir forecass ca be assumed o come from he same disribuio. The firs ad a impora sep i his mehod is o choose he legh of he raiig period. We are looig for a good compromise. The advaage of a shor raiig period is ha i is able o adap rapidly o chages (sice weaher paers ad model specificaio chage over ime). The advaage of a loger raiig period is ha he BMA parameers are beer esimaed. We compare measuremes such as he Mea Absolue Error (MAE) ad he Coiuous Raed Probabiliy Score (CRPS) for differe raiig period leghs (from 10 o 60 days, by 5 or 10 day seps). The values of CRPS ad he values of he MAE correspodig o differe leghs of he raiig period are give i Figure 4. We oice ha he wo curves are alie; he CRPS decreases from 1.0 (10 days) o 0.73 (60 days) ad he MAE decreases from 4.2 (10 days) o 1.5 (50 days) ad he icreases agai o 3.1 a 60 days. A 50-days raiig period is chose. Oce we have decided o he legh period we cosruc he paer ha fi hose daa, so ha we ca obai he ew forecass sysem ad he correspodig probabiliies. Scores are calculaed i he ex secio o decide o he spread ad sill of he BMA forecass. 6. Compariso of he mehods by meas of he crieria To compare he qualiy of he forecass provided by he saisical pos processig mehods cosidered here, we use some of he crieria preseed earlier i his paper. We compare hree ypes of scores: sadard measures, reliabiliy scores ad resoluio scores for he iiial forecass, u-weighed forecass from he bes member mehod, weighed forecass from he bes member mehod ad he forecass obaied wih he Bayesia Model Averagig mehod. 6.1 Sadard Measures Bias: We compare he bias for he iiial forecass ad he bias for he forecass obaied by he hree mehods (see Table 1). A perfec score is 1. Scores obaied for all hree mehods are 1, showig good forecass bu we oe ha i is possible o ge a perfec score for a bad forecas if here are compesaig errors. Correlaio Coefficie: The R 2 we obai for wo of he mehods has values very close o 1: 0.96 for he Bayesia forecass ad 0.97 for he W-BMM forecass (see Table 1). Sice a perfec correlaio coefficie is 1 our scores show a good correlaio bewee observaios ad forecass. The correlaio coefficie for he iiial predicios is 0.99 so he degree of correlaio is o los afer pos processig he forecass. Roo mea square error (RMSE): The RMSE s values for wo of he mehods show small model errors. Neverheless he RMSE for he iiial forecass is smaller ha he RMSE for he forecass we simulaed by W-BMM (see Table 1), ad he BMA RMSE is eve larger. Oe possible explaaio would be ha he RMSE is iflueced more by large errors raher ha small errors. From he poi of view of sadard measures, he forecass we creaed have predicive qualiies almos as good as he iiial predicios. Me a absolue error (MAE): The smaller he MAE, he beer. Whe we compare he MAE for he iiial forecass ad he MAE for he forecass obaied by he oher wo mehods, we fid a larger value for he W-

12 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. BMM: 1.30 ad eve larger for he BMA 1.53 (see Table 1). Hece, pos processig he forecass by hese wo mehods slowly icreases he MAE, as well as he RMSE. Neverheless hose are good values of he MAE. Table 1. Values of he sadard measures for he hree mehods. Forecass Bias 2 R RMSE( C ) MAE CRPS Iiial W-BMM Bayesia Reliabiliy crieria Talagrad diagram: For he iiial sysem of forecass for 5-days ahead, he ra hisogram is give i Figure 5a. We oice a asymmeric U-shaped hisogram meaig ha he esemble spread is oo small (uder-dispersive) wih may observaios fallig ouside he exremes of he esemble. The EPS is uder-dispersive, so he uceraiy is uder esimaed. The ra hisogram of he esemble obaied by he Bes Member Weighed Mehod has a raher fla shape he esemble spread correcly represes forecas uceraiy (see Figure 5b) bu we oice ha he exremes ras are o so well represeed. The ra hisogram of he esemble obaied by he BMA Mehod is give i Figure 5c. We sill oice a U-diagram, bu more symmerical ha he BMM oe. a. Iiial Predicios b. W-BMM weighed c. BMA Figure 5. Compariso of he Talagrad Ra Diagrams of he wo mehods ad wih he iiial predicios 6.3 Resoluio Crieria Coiuous ra probabiliy score (CRPS): The CRPS measures he differece bewee he forecas ad observed CDFs. The values of CRPS for he wo mehods calculaed for he eire sudied period are give i Table 1. Those are good values, owig ha he perfec CRPS is 0, provig a high sill of he ew creaed EPS. The CRPS of he W-BMM forecass is as good as he oe of he iiial predicios. 7. Coclusio The objecive of his paper was o exed he umber of simulaed forecass of emperaure (he 51 per day) provided by Meeo-Frace ad sill have a forecasig sysem wih a good qualiy (spread ad sill) ha will be useful for he maageme of he elecric sysem a EDF Frace. Up o he 4h ime horizo (1 horizo correspods o 1 day) he deermiisic forecass give high qualiy forecass so we ried o improve forecass beyod his ime-horizo. Therefore we examied wo mehods of saisical processig of he paer s oupus which ae io accou he uceraiies of he ipus (represeed by he 51 differe iiial codiios added o he paer). We sudied heir implemeaio o he daa-se provided by Meeo-Frace. I coais forecass for he March April period. There are 51 values of forecass for 14 ime-horizos. We sudied separaely several horizos-ime, sarig wih he 5-days ahead horizo (he sudy of he 5h horizo preseed i he curre aricle). We wa o improve he probabiliy desiy fucio of he forecass, preservig a he same ime he qualiy of he mea forecass. The firs mehod is he Bes-Member mehod proposed by Fori e al. (2006). The idea is o desig for each lead ime i he daa se he bes forecas amog all forecass provided by he emperaure predicio sysem, o cosruc a error paer usig oly he errors made by hose "bes members" ad o he "dress" all he members of he iiial predicio sysem wih his error paer. This mehod allows us o exed he umber of simulaed emperaures. We preseed here he case where he

13 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. esemble members are dressed ad weighed differely by classes of is saisical order. The secod mehod we have implemeed is he Bayesia Model Averagig mehod proposed by Rafery e al. (2004). I is a saisical mehod for pos processig model oupus which allows for providig calibraed ad sharp predicive PDFs eve if he oupu iself is o calibraed. The mehod uses a slidig-widow raiig period o esimae ew models parameers, isead of usig he whole daabase of pas forecass ad observaios. Reliabiliy ad resoluio are he aribues ha deermie he qualiy of a probabilisic predicio sysem. Therefore, comparig EPS (hree i our case, icludig he iiial sysem) ivolves comparig scores which measure he sill ad he spread. From he spread poi of view here is sigifica improveme of he disribuio whe usig he mehod W-BMM. The Ras Hisogram of he iiial EPS shows uder-dispersio ad a cold ad ho bias (see Figure 5) ad he Ras Hisogram of he BMA maiais he same shape, bu he Ras Diagram for he W-BMM is raher fla, alhough here is a small effec o he exreme ras ha migh be correced by usig a differe modelig approach for he exreme values of he forecass. From he sill poi of view, he Bayesia Mehod gives less good resuls ha he iiial predicios (see he CRPS, RMSE, MAE values i Table 1). The W-BMM mehod has a beer (smaller) CRPS bu he RMSE ad MAE are larger. So from he spread poi of view we ca say ha he qualiy of he iiial predicio sysem is preserved bu o improved. The resuls we obaied are coveie, cosiderig he objecive: icreasig he umber of forecass for improvig he disribuio of he Esemble Predicio Sysem, wihou losig he precisio of is mea forecass. The ex sep is o build a mixure model usig he W- BMM for he ceer of he disribuio ad Geeralized Exreme Value (GEV) models for he ails of he disribuio. Correspodece: adriaa.gogoel@gmail.com Refereces Aldrich, J R. a. fisher ad he maig of maximum lielihood , Saisical Sciece 12 (1997), Bruhs, A. Deurveilher, G. ad Roy, J-S A o-liear regressio model for mid-erm load forecasig ad improvemes i seasoaliy, 15h PSCC, Liege Dordoa, V. ad Colle, J Méhodes de prévisio e loi : éa de l ar, Tech. repor, EDF R&D, Dempser, A. P., Laird, N. M., ad Rubi, D. B., Maximum lielihood from icomplee daa via he EM algorihm, Joural of he Royal Saisical Sociey 39 (1977), IFS Documeaio, The esemble predicio sysem, Tech. repor, ECMWF, IFS Documeaio, 2006 Par v: The esemble predicio sysems, Tech. repor, ECMWF, V. FORTIN, A.C. FAVRE, ad M. SAID, Probabilisic forecasig from esemble pre-dicio sysems: Improvig upo he bes-member mehod by usig a differe weigh ad dressig erel for each member, Q. J. R. Meeorol. Soc. 132 (2006), Farley, C., Rafery, A. E. Geiig, T., ad J. M. SLOUGHTER, EsembleBMA: A r pacage for probabilisic forecasig usig esembles ad bayesia model averagig, Tech. repor, Deparme of Saisics Uiversiy of Washigo, Jolliffe, I. T. ad Sepheso, D. B., Proper scores for probabiliy forecass ca ever be equiable, America Meeorological Sociey (2007). Malle, V., Prévisio d esemble, Tech. repor, INRIA Roquecour, Persso, A., User guide o ecmwf forecas producs, Tech. repor, ECMWF, Pei, T., Evaluaio de la performace de prévisios hydrologiques logiques d esemble issues de prévisios mééorologiques d esemble, Ph.D. hesis, Faculé Des Scieces E De Géie Uiversié Laval Québec, Rafery, A.E., Geiig, T., Balabdaoui, F., ad Polaowsi, M., Usig Bayesia model averagig o calibrae forecas esembles, Physical Review (2004), 20. Roulso ad Smih, Combiig dyamical ad saisical esembles, Tellus 55A(2002), Schaae, J., Demarge, J., Harma, R., Mullusy, M., Welles, E., Wu, L., Herr, H., Fa, X., ad Seo, D. J., Precipiaio ad emperaure esemble forecass from sigle-value forecass, Hydrology ad Earh Sysem Scieces Discussios 4 (2007),

14 Implemeaio of wo Saisical Mehods for Esemble Predicio Sysems / Gogoel, e al. Sasi, H.R., Wilso, L.H., ad Burrows, W. R., Survey of commo verificaio mehods i meeorology, Tech. repor, Eviroeme Caada, Service de l eviroeme amosphérique., Wag, X. ad Bishop, C.H., Improveme of esemble reliabiliy wih a ew dressig erel., Q. J. R. Meeorol. Soc. 131 (2005), Whiaer, J.S. ad Loughe A. F., The relaioship bewee esemble spread ad esemble mea sil l, Mohly Weaher Review 126 (1998),