Artificial Immune System for Forecasting Time Series with Multiple Seasonal Cycles

Transcription

1 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles Grzegorz Dudek Deparmen of Elecrcal Engneerng, Czesochowa Unversy of Technology, Al. Arm Krajowej 17, Czesochowa, Poland Absrac. Many me seres exhb seasonal varaons relaed o he daly, weekly or annual acvy. In hs paper a new mmune nspred unvarae mehod for forecasng me seres wh mulple seasonal perods s proposed. Ths mehod s based on he paerns of me seres seasonal sequences: npu ones represenng sequences precedng he forecas and forecas ones represenng he forecased sequences. The mmune sysem ncludes wo populaons of mmune memory cells anbodes, whch recognze boh ypes of paerns represened by angens. The emprcal probables ha he forecas paern s deeced by he kh anbody from he second populaon whle he correspondng npu paern s deeced by he jh anbody from he frs populaon, are compued and appled o he forecas consrucon. The emprcal sudy of he model ncludng sensvy analyss o changes n parameer values and he robusness o nosy and mssng daa s performed. The suably of he proposed approach s llusraed hrough applcaons o elecrcal load forecasng and compared wh ARIMA and exponenal smoohng approaches. Keywords: arfcal mmune sysem, seasonal me seres forecasng, smlary-based mehods. 1 Inroducon In general, a me seres can be hough of as conssng of four dfferen componens: rend, seasonal varaons, cyclcal varaons, and rregular componen. The specfc funconal relaonshp beween hese componens can assume dfferen forms. Usually hey combne n an addve or a mulplcave fashon. Seasonaly s defned o be he endency of me seres daa o exhb behavor ha repeas self every n perods. The dfference beween a cyclcal and a seasonal componen s ha he laer occurs a regular (seasonal) nervals, whle cyclcal facors have usually a longer duraon ha vares from cycle o cycle. The presence of he cyclcal and mulple seasonal cycles hampers he consrucon of forecasng models. In hs arcle we concenrae on he seasonal cycles. Seasonal paerns of me seres can be examned va correlograms or perodograms based on a Fourer decomposon. Many economcal, busness and ndusral me seres exhb seasonal behavor. Examples of daa wh recurren paerns are: real sales, ndusral producon, raffc, N.T. Nguyen (Ed.): Transacons on CCI XI, LNCS 8065, pp , Sprnger-Verlag Berln Hedelberg 2013

2 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 177 weaher phenomena, elecrcy load, calls o call cener, gas and waer consumpon. The recurren paerns n hese cases can be observed whn daly, weekly and/or annual perods. Seasonaly s ofen conneced wh he rhyhm of lfe of he populaon and s relaonshp o he varably of he seasons, professonal acvy, radons and habs. A varey of mehods have been proposed for seasonal me seres forecasng. These nclude [1]: seasonal ARIMA, exponenal smoohng, arfcal neural neworks, dynamc harmonc regresson, vecor auoregresson, random effec models, and many ohers. The frs hree approaches are he mos commonly employed o modelng seasonal paerns. Accordng o Box e al. [2] we can exend he base ARIMA model wh jus one seasonal paern for he case of mulple seasonales. Such an exenson we can fnd n [3]. The nconvenence n he me seres modelng usng mulple seasonal ARIMA s a combnaoral problem of selecng approprae model orders. The basc Hol-Wners exponenal smoohng was adaped by Taylor so ha can accommodae wo seasonales [3]. Emprcal comparson showed ha he resulng forecass for he new double seasonal Hol-Wners mehod ouperformed hose from sandard Hol-Wners and also hose from a double seasonal ARIMA model. An advanage of he exponenal smoohng models s, besdes her relave smplcy, ha hey can be nonlnear. On he oher hand can be vewed as beng of hgh dmenson, as nvolves nalzaon and updang of a large number of erms (level, perods of he nraday and nraweek cycles). More parsmonous formulaon s proposed n [1]. Recenly fve new exponenally weghed mehods for forecasng me seres ha conss of boh nraweek and nraday seasonal cycles were proposed n [4]. Gould e al. descrbed a sae space model developed for he seres usng he nnovaon approach whch enables o develop explc models for boh addve and mulplcave seasonaly [5]. The nnovaon sae space approach provdes a heorecal foundaon for exponenal smoohng mehods. Ths procedure mproves on he curren approaches by provdng a common sense srucure o he models, flexbly n modelng seasonal paerns, a poenal reducon n he number of parameers o be esmaed, and model based predcon nervals. Arfcal neural neworks (ANNs), beng nonlnear and daa-drven n naure, may be well sued o he seasonal me seres modelng. One of he major advanage of ANNs, ha makes hey are so ofen used n pracce, s her grea capacy o exrac unknown and general nformaon from a gven daa se even n hgh-dmensonal ask. The auomaed ANN learnng releases a desgner from he cumbersome procedures of a pror model selecon. Alhough here s anoher problem: he selecon of nework archecure as well as he learnng algorhm. The mos popular ANN ype used n forecasng ask s mullayer percepron, whch has a propery of unversal approxmaon. ANNs are able o drecly model seasonaly, whou he pror seasonal adjusmen. An example we can fnd n [6] where auhors conduc a comparave sudy beween ANN and ARIMA models. However Nelson e al. [7] conclude

3 178 G. Dudek ha ANNs raned on deseasonalzed daa perform sgnfcanly beer han hose wh raw daa. Zhang and Q [8] noce ha no only deseasonaly s mporan bu also derendng. They sudy he effecveness of daa preprocessng on ANN modelng and forecasng performance. In a large-scale emprcal sudy [9] Zhang and Klne f a lnear rend for derendng and hen subrac he esmaed rend componen from he raw daa. For deseasonalzng hey employ he mehod of seasonal ndex based on cenered movng averages. Decomposon of he me seres o cope wh seasonales and rend, s a procedure used no only n ANNs, bu also n oher models, e.g. ARIMA and exponenal smoohng. The componens showng less complexy han he orgnal me seres can be predced ndependenly and more accurae. A frequenly used approach s o decompose he me seres on seasonal, rend and sochasc componens (e.g. usng STL flerng procedure based on LOESS smooher [10]). Oher mehods of decomposons apply he Fourer ransform [11] or he wavele ransform [12]. The smple way o remove seasonaly s o defne he separae me seres for each observaon n a cycle,.e. n he case of cycle of lengh n, n me seres s defned ncludng observaons n he same posons n a cycle. In hs paper we propose an approach based on he paerns of he me seres seasonal sequences. Usng paerns we do no need o decompose a me seres. A rend and many seasonal cycles as well as he nonsaonary and heeroscedascy s no a problem here when usng proper paern defnons. The proposed approach belongs o he class of smlary-based mehods [13] and s dedcaed o forecasng me seres wh mulple seasonal perods. The forecas here s consruced usng analoges beween sequences of he me seres wh perodces. An arfcal mmune sysem (AIS) s used for deecon of smlar paerns of sequences. The clusers of paerns are represened by anbodes (AB). Two populaon of ABs are creaed whch recognze wo populaons of paerns (angens) npu ones and forecas ones. The emprcal probables ha he paern of forecased sequence s deeced by he kh AB from he second populaon whle he correspondng paern of npu sequence s deeced by he jh AB from he frs populaon are compued and appled o he forecas consrucon. Ths dea s aken from [14], where he Kohonen ne was used as a cluserng mehod. The mers of AIS le n s paern recognon and memorzaon capables. The applcaon areas for AIS can be summarzed as [15]: learnng (cluserng, classfcaon, recognon, roboc and conrol applcaons), anomaly deecon (faul deecon, compuer and nework secury applcaons), and opmzaon (connuous and combnaoral). Angen recognon, self-organzng memory, mmune response shapng, learnng from examples, and generalzaon capably are valuable properes of mmune sysems whch can be brough o poenal forecasng models. The frs AIS model dedcaed o he me seres forecasng was proposed by Dudek [16]. The novely of he AIS proposed n hs paper n he comparson wh [16] s ha he mmune memory s composed of wo AB populaons and he cross-reacvy hresholds are adaped o learnng daa durng he mmune memory creaon process.

4 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles Smlary-Based Forecasng Mehods The smlary-based (SB) mehods use analoges beween sequences of he me seres wh seasonal cycles. A course of a me seres can be deduced from he behavor of hs me seres n smlar condons n he pas or from he behavor of oher me seres wh smlar changes n me. In he frs sage of hs approach he me seres s dvded no sequences of lengh n, whch usually conan one seasonal cycle. Fg. 1 shows a perodcal me seres, where we can observe annual, weekly and daly varaons. Ths seres represens hourly elecrcal loads of he Polsh power sysem. Our ask s o forecas he me seres elemens n he daly perod, so he sequences nclude 24 successve elemens of he daly perods. (a) 2.4 x 104 Load, MW Year (b) 2.2 x Load, MW Hour Fg. 1. The load me seres of he Polsh power sysem n hree-year (a) and hree-week (b) nervals In order o elmnae rend and seasonal varaons of perods longer han n (weekly and annual varaons n our example), he sequence elemens are preprocessed o oban her paerns. The paern s a vecor wh componens ha are funcons of acual me seres elemens. The npu and oupu (forecas) paerns are defned: x = [x 1 x 2 x n ] T and y = [y 1 y 2 y n ] T, respecvely. The paerns are pared (x, y ), where y s a paern of he me seres sequence succeedng he sequence represened by x and he nerval beween hese sequences (forecas horzon τ) s consan. The SB

5 180 G. Dudek mehods are based on he followng assumpon: f he process paern x a n a perod precedng he forecas momen s smlar o he paern x b from he hsory of hs process, hen he forecas paern y a s smlar o he forecas paern y b. Paerns x a, x b and y b are deermned from he hsory of he process. Pars x a x b and y a y b are defned n he same way and are shfed n me by he same number of seres elemens. The way of how he x and y paerns are defned depends on he me seres naure (seasonal varaons, rend), he forecas perod and he forecas horzon. Funcons ransformng seres elemens no paerns should be defned so ha paerns carry mos nformaon abou he process. Moreover, funcons ransformng forecas sequences no paerns y should ensure he oppose ransformaon: from he forecased paern y o he forecased me seres sequence. The forecas paern y = [y,1 y,2 y,n ] encodes he followng acual me seres elemens z n he forecas perod +τ: z +τ = [z +τ,1 z +τ,2 z +τ,n ], and he npu paern x = [x,1 x,2 x,n ] maps he me seres elemens n he perod precedng he forecas perod: z = [z,1 z,2 z,n ]. In general, he npu paern can be defned on he bass of a sequence longer han one perod, and he me seres elemens conaned n hs sequence can be seleced n order o ensure he bes qualy of he model. Vecors y are encoded usng acual process parameers Ψ (from he neares pas), whch allows o ake no consderaon curren varably of he process and ensures possbly of decodng. For seres wh daly, weekly and annual seasons we defne some funcons mappng he orgnal feaure space Z no he paern spaces X and Y,.e. f x : Z X and f y : Z Y: f ( z x,, Ψ ) = n z l = 1, ( z z, l z ) 2, f ( z y,, Ψ ) = n z l = 1 + τ, ( z + τ, l z z ) 2, (1) f f ( z x, z, z +τ,, Ψ ) =, f y ( z,, Ψ ) =, (2) z' z" x( z,, y,, z, Ψ ) = z z', f ( z, Ψ ) = z +τ ", (3) where: = 1, 2,, N he perod number, = 1, 2,, n he me seres elemen number n he perod, τ he forecas horzon, z, he h me seres elemen n he perod, z he mean value of elemens n perod, z' { z, z -1,, z -7,, z,-1 }, z" { z, z,, z +τ-7, }, Ψ he se of codng parameers such as z, z' and z". The funcon f x defned usng (1) expresses normalzaon of he vecors z. Afer normalzaon hese vecors have he uny lengh, zero mean and he same varance. When we use he sandard devaon of he vecor z componens n he denomnaor of equaon (1), we receve vecor x wh he uny varance and zero mean. Noe ha he nonsaonary and heeroscedasc me seres s represened by paerns havng he same mean and varance.

6 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 181 The componens of he x-paerns defned usng equaons (2) and (3) express, respecvely, ndces and dfferences of z, and z' or z". Forecas paerns are defned usng analogous funcons o npu paern funcons f x, bu hey are encoded usng he me seres elemens or characerscs deermned from he process hsory, wha enables decodng of he forecased vecor z +τ afer he forecas of paern y s deermned. To calculae he me seres elemen values on he 1 1 bass of her paerns we use he nverse funcons: fx ( x,, Ψ ) or f y ( y,, Ψ ). For example he nverse funcons for (1) are: 1 x,, n 2 2 f ( x, Ψ ) = x ( z z ) + z, f ( y, Ψ ) = y ( z τ z ) + z. (4) l= 1, l 1 y As a smlary measure beween wo paerns (real-valued vecors) we can use [17]: he nner produc (when he vecors are normalzed) or closely relaed o cosne smlary measure, Pearson s correlaon coeffcen or Tanmoo measure. I s useful o defne he smlary measures on he base of he dsance measures, e.g usng lnear mappng: s(x a,x b ) = c d(x a,x b ) or nonlnear mappng: s(x a,x b ) = 1/(1+ d(x a,x b )), where: s(.,.) s a smlary, d(.,.) s a dsance and c s a consan greaer han he hghes value of he dsance. The popular dsance measures are: Eucldean, Manhaan or Canberra dsances. If he componens of he vecors are expressed n dfferen uns or change n dfferen ranges, n order o offse her mpac on he dsance, her weghng s recommended. If for a gven me seres he sascal analyss confrms he hypohess ha he dependence beween smlares of npu paerns and smlares beween forecas paerns pared wh hem are no caused by random characer of he sample, jusfes he sense of buldng and usng models based on he smlares of paerns of hs me seres. The sascal analyss of paern smlares s descrbed n [13]. The forecasng procedure n he case of SB mehods can be summarzed as follows: 1. Elmnaon of he rend and seasonal varaons of perods longer han n usng paern funcons f x and f y. 2. Forecasng he paern y usng smlares beween paerns. 3. Reconsrucon he me seres elemens from he forecased paern y usng he nverse funcon f 1 y.,, n l= 1 +, l 3 Immune Inspred Forecasng Model The proposed AIS conans mmune memory conssng of wo populaons of ABs. The populaon of x-anbodes (ABx) deecs angens represenng paerns x = [x 1, x 2,..., x n ] T AGx, whle he populaon of y-anbodes (ABy) deecs angens represenng paerns y = [y 1, y 2,..., y n ] T AGy. The vecors x and y form he epopes of AGs and paraopes of ABs. ABx has he cross-reacvy hreshold r defnng he AB recognon regon. Ths recognon regon s represened by he n-dmensonal

7 182 G. Dudek hypersphere of radus r wh cener a he pon x. Smlarly ABy has he recognon regon of radus s wh cener a he pon y. The cross-reacvy hresholds are adjused ndvdually durng ranng. The recognon regons conan AGs wh smlar epopes. AG can be bound o many dfferen ABs of he same ype (x or y). The srengh of bndng (affny) s dependen on he dsance beween an epope and a paraope. AB represens a cluser of smlar AGs n he paern space X or Y. The clusers are overlapped and her szes depend on he smlary beween AGs belongng o hem, measured n he boh paern spaces X and Y. The kh ABx can be wren as a par {p k, r k }, where p k = x k, and he kh ABy as {q k, s k }, where q k = y k. Afer he wo populaon of mmune memory have been creaed, he emprcal condonal probables P(ABy k ABx j ), j, k = 1, 2,, N, ha he h AGy smulaes (s recognzed by) he kh ABy, when he correspondng h AGx smulaes he jh ABx, are deermned. These probables are calculaed for each par of ABx and ABy on he bass of recognon of he ranng populaon of AGs. In he forecasng phase he new AGx, represenng paern x*, s presened o he raned mmune memory. The forecased paern y pared wh x* s calculaed as he mean of ABy paraopes weghed by he condonal probables and affnes. The dealed algorhm of he mmune sysem o forecasng seasonal me seres s descrbed below. Tranng (mmune memory creaon) 1. Loadng of he ranng populaons of angens. 2. Generaon of he anbody populaons. 3. Calculaon of he cross-reacvy hresholds of x-anbodes. 4. Calculaon of he cross-reacvy hresholds of y-anbodes. 5. Calculaon of he emprcal condonal probables P(ABy k ABx j ). Tes 6. Forecas deermnaon usng y-anbodes, probables P(ABy k ABx j ) and affnes. Fg. 2. Pseudocode of he AIS for he seasonal me seres forecasng Sep 1. Loadng of he ranng populaons of angens. An AGx represens a sngle x paern, and AGy represens a sngle y paern. Boh populaons of AGx and AGy are dvded no ranng and es pars n he same way. Immune memory s raned usng he ranng populaons, and afer learnng he model s esed usng he es populaons. Sep 2. Generaon of he anbody populaons. The AB populaons are creaed by copyng he ranng populaons of AGs (ABs and AGs have he same srucure). Thus he paraopes ake he form: p k = x k, q k = y k, k = 1, 2,, N. The number of AGs and ABs of boh ypes s he same as he number of learnng paerns.

8 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 183 Sep 3. Calculaon of he cross-reacvy hresholds of x-anbodes. The recognon regon of he kh ABx should be as large as possble and cover only he AGx ha sasfy wo condons: () her epops x are smlar o he paraope p k, and () he AGy pared wh hem have epopes y smlar o he kh ABy paraope q k. The measure of smlary of he h AGx o he kh ABx s an affny: 0, f d( pk, x ) > rk or rk = 0 a( p, ) = ( k, ) k x d p x 1, oherwse, (5) rk where d(p k, x ) s he dsance beween vecors p k and x, a(p k, x ) [0, 1]. Affny a(p k, x ) nforms abou he degree of membershp of he h AGx o he cluser represened by he kh ABx. The smlary of he h AGy o he kh ABy menoned n () s measured usng he forecas error of he me seres elemens encoded n he paraope of he kh ABy. These elemens are forecased usng he epope of he h AGy. The forecas error s: 100 z f y ) δ =, (6) k, n 1 k + τ, y (,, Ψk n = 1 zk + τ, where: z k+τ, he h me seres elemen of he perod k+τ whch s encoded n he 1 paraope of he kh ABy: qk, = f y ( z k +τ,, Ψk ), f y ( y,, Ψk ) he nverse funcon of he paern y reurnng he forecas of me seres elemen z k+τ, usng he epope of he h AGy. If he condon δ k, δ y s sasfed, whereδ y s he error hreshold value, s assumed ha he h AGy s smlar o he kh ABy, and h AGx, pared wh hs AGy, s classfed o class 1. When he above condon s no me he h AGx s classfed o class 2. Thus class 1 ndcaes he hgh smlary beween ABy and AGy. The classfcaon procedure s performed for each ABx. The cross-reacvy hreshold of kh ABx s defned as follows: r k = d p, x ) + c[ d( p, x ) d( p, x )], (7) ( k A k B k A where B denoes he neares AGx of class 2 o he kh ABx, and A denoes he furhes AGx of class 1 sasfyng he condon d(p k, x A ) < d(p k, x B ). The parameer c [0, 1) allows o adjus he cross-reacvy hreshold value from r kmn = d(p k, x A ) o r kmax = d(p k, x B ). Sep 4. Calculaon of he cross-reacvy hresholds of y-anbodes. The crossreacvy hreshold of kh ABy s calculaed smlarly o he above: s = d q, y ) + b[ d( q, y ) d( q, y )], (8) k ( k A k B k A

9 184 G. Dudek where B denoes he neares AGy of class 2 o he kh ABy, and A denoes he furhes AGy of class 1 sasfyng he condon d(q k, y A ) < d(q k, y B ). The parameer b [0, 1) plays he same role as he parameer c. The h AGy s classfed o class 1, f for he h AGx pared wh, here s ε k, ε x, where ε x s he hreshold value and ε k, s he forecas error of he me seres elemens encoded n he paraope of he kh ABx. These elemens are forecased usng he epope of he h AGx. The forecas error s: 100 z f ( x,, Ψk ) ε =, (9) k, n 1 k, x n = 1 zk, where: z k, he h me seres elemen of he perod k whch s encoded n he paraope of he kh ABx: pk, 1 = f x ( zk,, Ψk ), f x ( x,, Ψk ) he nverse funcon of paern x reurnng he forecas of me seres elemen z k, usng he epope of he h AGx. The h AGy s recognzed by kh ABy f affny a(q k, y ) > 0, where: 0, f d( qk, y ) > sk or sk = 0 a( q, ) = ( k, ) k y d q y 1, oherwse, (10) sk a(q k, y ) [0, 1] expresses he degree of membershp of paern y o he cluser represened by he kh ABy. Procedure for deermnng he hreshold s k s hus analogous o he procedure for deermnng he hreshold r k. The recognon regon of kh ABy s as large as possble and covers AGy ha sasfy wo condons: () her epops y are smlar o he paraope q k, and () he AGx pared wh hem have epopes x smlar o he kh ABx paraope p k. Ths way of formng clusers n paern space X (Y) makes ha her szes are dependen on he dsperson of y-paerns (x-paerns) pared wh paerns belongng o hese clusers. Anoher paern x (y ) s appended o he cluser ABx k (ABy k ) (hs s acheved by ncreasng he cross-reacvy hreshold of AB represenng hs cluser), f he paern pared wh x (y ) s suffcenly smlar o he paraope of he kh ABy k (ABx k ). The paern s consdered suffcenly smlar o he paraope, f allows o forecas he me seres encoded n he paraope wh an error no greaer han he hreshold value. Ths ensures ha he forecas error for he paern x (y) has a value no greaer han ε x (δ y ). Lower error hresholds mply smaller clusers, lower bas and greaer varance of he model. Mean absolue percenage error (MAPE) here s used as an error measure ((6) and (9)) bu oher error measures can be appled. Sep 5. Calculaon of he emprcal condonal probables P(ABy k ABx j ). Afer he cluserng of boh spaces s ready, he successve pars of angens (AGx, AGy ), = 1, 2,, N, are presened o he raned mmune memory. The smulaed ABx and

10 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 185 ABy are couned and he emprcal frequences of ABy k gven ABx j, esmang condonal probables P(ABy k ABx j ), are deermned. Sep 6. Forecas procedure. In he forecas procedure new AGx, represenng he paern x*, s presened o he mmune memory. Le Ω be a se of ABx smulaed by hs AGx. The forecased paern y correspondng o x* s esmaed as follows: N y = w q, (11) k = 1 k k where w k j Ω = N l= 1 j Ω P( ABy k ABx ) a( p, x*) P( ABy ABx ) a( p, x*) l j j j j (12) N and w = 1. k = 1 k The forecas s calculaed as he weghed mean of paraopes q k. The weghs w express he sum of producs of he affnes of smulaed memory cells ABx o he AGx and probables P(ABy k ABx j ). The clusers represened by ABs have sphercal shapes, hey overlap and her szes are lmed by cross-reacvy hresholds. The number of clusers s here equal o he number of learnng paerns, and he means of clusers n he paern spaces X and Y (paraopes ABx and ABy) are fxed hey le on he learnng paerns. The cross-reacvy hresholds, deermnng he cluser szes, are uned o he ranng daa n he mmune memory learnng process. In resuls he clusers n he space X correspond o compac clusers n he space Y, and vce versa. I leads o more accurae mappng X Y. The model has four parameers: he error hresholds (δ y and ε x ) and he parameers unng he cross-reacvy hresholds (b and c). Increasng he value of hese parameers mply an ncrease n sze of clusers, an ncrease n he model bas and reducon n s varance. The ranng roune s deermnsc, whch means he fas learnng process. The mmune memory learnng needs only one pass of he ranng daa. The runme complexy of he ranng roune s O(N 2 n). The mos cosly operaon s he dsance calculaon beween each ABs and AGs. The runme complexy of he forecasng procedure s also O(N 2 n). 4 Emprcal Sudy The descrbed above AIS was appled o he nex day elecrcal load curve forecasng (τ = 1). Shor-erm load forecasng plays a key role n conrol and schedulng of power sysems and s exremely mporan for energy supplers, sysem operaors, fnancal nsuons, and oher parcpans n elecrc energy generaon,

11 186 G. Dudek ransmsson, dsrbuon, and markes. Precse load forecass are necessary for elecrc companes o make mporan decsons conneced wh elecrc power producon and ransmsson plannng, such as un commmen, generaon dspach, hydro schedulng, hydro-hermal coordnaon, spnnng reserve allocaon and nerchange evaluaon. The seres suded n hs secon represens he hourly elecrcal load of he Polsh power sysem from he perod Ths seres s shown n Fg. 1. The me seres were dvded no ranng and es pars. The es se conaned 30 pars of paerns from January 2004 (from January 2 o 31) and 31 pars of paerns from July The ranng se conaned paerns from he perod from January 1, 2002 o he day precedng he day of forecas. For each day from he es par he separae mmune memory was creaed usng he ranng subse conanng AGy represenng days of he same ype (Monday,, Sunday) as he day of forecas and pared wh hem AGx represenng he precedng days (e.g. for forecasng he Sunday load curve, model learns from AGx represenng he Saurday paerns and AGy represenng he Sunday paerns). Ths roune of model learnng provdes fne-unng s parameers o he changes observed n he curren behavor of he me seres. The dsance beween ABs and AGs was calculaed usng Eucldean merc. The paerns were defned usng (1). MAPE, whch s radonally used n shor-erm load forecasng, was a forecas error measure. The model parameers were deermned usng he grd search mehod on he ranng subses n he local verson of he leave-one-ou procedure. In hs procedure no all paerns are successvely removed from he ranng se o esmae he generalzaon error bu only he k-neares neghbors of he es x-paern (k was arbrarly se o 5). As a resul, he model s opmzed locally n he neghborhood of he es paern. I leads o a reducon n learnng me. In he grd search procedure he parameers were changed as follows: () δ y = 1.00, 1.25,, 3.00, ε x = 1.00, 1.25,, δ y, a he consan values of b = c = 1, and () b = c = 0, 0.2,, 1.0, a he opmal values of δ y and ε x deermned n (). I was observed ha a lower values of δ y and c many x-paerns are unrecognzed. If δ y 2.25 and c = 1 approxmaely 99% of he x-paerns are deeced by ABx. Increasng δ y above 2.25 resuls n ncreasng he error. Mnmum error (MAPE) was observed for δ y = 2.25, ε x = 1.75 and b = c = 1. The forecas es MAPE for January was 1.37 and for July was Fg. 3 llusraes he emprcal condonal probables P(ABy k ABx j ) esmaed on he ranng se for July 1, You can observe a specfc paern ha ndcaes whch ABs n boh populaons are acvaed smulaneously (e.g. he acvaon of x-anbodes #19 31 corresponds he sronger acvaon of y-anbodes #19 32, and ). These are ABs represenng daly cycles lyng n he same perod of he year. Hgher probables mply a sronger relaonshp X Y and greaer confdence o he forecas. The weghs of acvaed ABs for hs forecasng ask are shown n Fg. 4. Here

12 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 187 we also can see ha he npu paern smulaes ABs represenng paerns from he same perod of he year as he npu paern. The reconsruced forecas paern on he background of paerns represened by acvaed ABs s shown n Fg x ABy number ABx number 0 Fg. 3. The emprcal condonal probables P(ABy k ABx j ) esmaed on he ranng se for July 1, 2004 w ABy number Fg. 4. The weghs of acvaed ABs esmaed on he ranng se for July 1, Model Sensvy o Changes n Parameer Values The am of hs analyss s o evaluae he nfluence of he parameer values on he forecass generaed by he model. Fg. 6 shows he es sample forecas errors dependng on he parameer values whch were changed ndvdually n he ranges: ε x [0.5ε x *, 1.5ε x *], δ y [0.5δ y *, 1.5δ y *] and b = [0.5b*, mn(1.5b*, 1)], where asersk denoes he opmal value of he parameer esmaed on he ranng se. I was assumed c = b.

13 188 G. Dudek y,q Hour Fg. 5. The acvaed ABs (gray lnes), he reconsruced forecas paern (dashed lne) and he acual forecas paern (black connuous lne) load forecasng for July 1, 2004 To avod he suaons when for he smaller parameer values many es AGx are unrecognzed, was assumed ha an unrecognzed AGx s ncluded o he group represened by he neares ABx, alhough he recognon regon of hs ABx does no nclude he AGx. The same assumpon s made for analyses descrbed n Secons 4.2, 4.3 and 5. Fg. 7 shows he relave percenage dfference beween he forecass generaed by he opmal model ( z+ τ, ( p*) ) and he model wh non-opmal parameer value ( z ( )): + τ, p 100 RPD( p) = Nn N n z ( p) z + τ, = 1 = 1 + τ, z + τ, ( p*) ( p*). (13) where p denoes he parameer. The change of he parameer value can cause a sep change n he model response whch makes he lnes n Fg. 6 and 7 are no smooh. From Fg. 6 we can see ha overesmaon of ε x and δ y resuls n more rapdly ncrease n he forecas error han underesmaon. The values of ε x and δ y for he mnmum ranng and es errors are no he same. Smaller values are observed for he es se. RPD acheves % for he border values of ε x and δ y and only 0.074% for b and c. The model sensvy o changes n parameers s defned by he sensvy ndex: MAPEs max MAPEs mn SI p = 100. (14) MAPE s mn Index (14) nforms wha s he relave percenage dfference beween he maxmum and mnmum errors MAPE s when he parameer vares n a gven range. Is values were: 10.10% for ε x, 7.42% for δ y and 2.27% for b and c. Thus he sensvy o he error hresholds are much greaer han for b and c.

14 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles MAPE s ε x/ε x δ y/δ y b/b* p/p* Fg. 6. The forecas error dependng on he parameer values ε x/ε x δ y/δ y b/b* RPD p/p* Fg. 7. Relave percenage dfference (13) dependng on he parameer values 4.2 Model Robusness o Nosy Daa The am of hs analyss s o deermne he model robusness o nosy daa resulng from he measuremen or esmaon errors of he me seres erms. The nosy paerns are locaed n he space dfferenly relave o each oher han he orgnal paerns. Ths affecs he dsances beween hem, and hus, he probables P(ABy k ABx j ), affnes and he forecased paern. I s assumed ha he acual me seres elemens are dsurbed by random errors: z ' = ξ, (15) l z l where ξ l ~ N(1,σ). The sandard devaon σ was changng n he range from 0 o 0.1. I corresponds o a share of nose n he daa (100 z' z / z) from 0 o 8%. For each σ value 30 ranng sessons were performed and hen mean forecas error was recorded (Fg. 8). The sensvy ndex o nosy daa s defned as follows: l

15 190 G. Dudek MAPEs ( σ ) MAPEs ( σ = 0) SI n ( σ ) = 100. (16) σ Ths ndex expresses he rao of he change n forecas error due o he nosy daa o he nensy of he nose. The mean value of SI n for all σ, whch corresponds o he slope of he sragh lne approxmang he characerscs presened n Fg. 8, was equal o 65.96% MAPE s σ Fg. 8. The forecas error dependng on he σ value 4.3 Model Robusness o Mssng Daa In hs sudy he robusness o mssng npu nformaon s analyzed. We assume ha some componens of he npu paern x* are mssng. I corresponds o he mssng erms of he me seres (mssng componens of vecor z*). In many models (e.g. ARIMA, exponenal smoohng, neural neworks) hs s a serous problem, and he mssng daa reconsrucon s needed. One sraegy for dealng wh he mssng componen value s o assgn he value ha s mos common among ranng examples. Anoher sraegy s o assgn he values of he correspondng componens of he mos smlar paerns. The proposed AIS copes well wh mssng componens of he npu paern. In such a case he epopes of AGx and paraopes of ABx are composed of non-mssng componens. The mmune memory creaon and es procedures reman unchanged. When paerns x and y are defned usng he mean value of elemens z, he values of non-mssng componens are dfferen from her orgnal values. Ths s caused by dfferen value of z whch s deermned now whou he mssng componens. As n he case of nosy daa, paerns wh mssng componens locae dfferenly n he space han he orgnal ones, whch cause he change of he forecas paern. To examne he robusness of he model o mssng daa we remove m componens of he vecors z* and hen we redefne paerns x and y. The mmune memory s consruced on he ranng se and he es s performed usng x* paern havng he same componens as z*. The m componens are chosen by random ndependenly for each

16 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 191 z*. The forecas errors dependng on he relave number of mssng componens are shown n Fg. 9. When he number of mssng componens s low he deeroraon n he model accuracy s no observed. Errors begn o grow rapdly when m exceeds 16. As a measure of he model sensvy o he mssng componens he ndex SI m s proposed: MAPEs ( m) MAPEs ( m = 0) SIm ( m) = 100. (17) m / n The SI m value for m = 6 was -3.82%, for m = 12 was 11.82% and for m = 18 was 41.81%. 2.5 MAPE s Relave number of mssng componens, % Fg. 9. The forecas error dependng on he relave number of mssng componens 5 Emprcal Comparson wh Oher Models In hs secon we compare he proposed AIS wh oher popular models of he seasonal me seres forecasng: ARIMA, exponenal smoohng (ES) and double seasonal Hol-Wners (DSHW) mehod. These models were esed n he nex day elecrcal load curve forecasng problem on hree me seres of elecrcal load: TS1: me seres of he hourly loads of he Polsh power sysem from he perod (hs me seres was used n analyses descrbed n Secon 4). The es sample ncludes daa from 2004 wh he excepon of 13 unypcal days (e.g. holdays). TS2: me seres of he hourly loads of he local power sysem from he perod July 2001-December The es sample ncludes daa from he perod July- December 2002 excep for 8 unypcal days. TS3: me seres of he hourly loads of he local power sysem from he perod The es sample ncludes daa from 2001 excep for 13 unypcal days. Tme seres TS1 s presened n Fg. 1, TS2 and TS3 n Fg. 10. Tme seres TS2 and TS3 are more rregular and harder o forecasng han TS1. The measure of he load me seres regulary could be he forecas error deermned by he naïve mehod.

17 192 G. Dudek The forecas rule n hs case s as follows: he forecased daly cycle s he same as seven days ago. The mean forecas errors, calculaed accordng o hs naïve rule, are presened n he las row of Table TS TS Load, MW 1000 Load, MW Hour TS Hour TS Load, MW Load, MW Hour x Hour Fg. 10. The TS2 and TS3 me seres (hree-week deals on he rgh) In ARIMA he me seres were decomposed no 24 seres,.e. for each hour of a day a separae seres s creaed. In hs way he daly seasonaly s removed. For he ndependen modelng of hese seres ARIMA(p, d, q) (P, D, Q) m model s used: Φ ( B Θ, (18) m m D d m ) φ ( B)(1 B ) (1 B) z = c + ( B ) θ ( B) ξ where {z } s he me seres, {ξ } s a whe nose process wh mean zero and varance σ 2, B s he backshf operaor, Φ(.), φ(.), Θ(.), and θ(.) are polynomals of order P, p, Q and q respecvely, m s he seasonal perod (for our daa m = 7), d and D are orders of nonseasonal and seasonal dfferencng, respecvelly, and c s a consan. To fnd he bes ARIMA model for each me seres we use a sep-wse procedure for raversng he model space whch s mplemened n he forecas package for he R sysem for sascal compung [18]. Ths auomac procedure reurns he model wh he lowes Akake's Informaon Creron (AIC) value. ARIMA model parameers, as well as he parameers of he ES and DSHW models descrbed below, were esmaed usng 12-week me seres fragmens mmedaely precedng he forecased daly perod. Unypcal days n hese fragmens were replaced wh he days from he prevous weeks.

18 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 193 The ES sae space models [19] are classfed no 30 ypes dependng on how he seasonal, rend and error componens are aken no accoun. These componens can be expressed addvely or mulplcavely, and he rend can be damped or no. For example, he ES model wh a dumped addve rend, mulplcave seasonaly and mulplcave errors s of he form: Level : Growh : Seasonal : Forecas : l = ( l s = s 1 b = φb 1 m μ = ( l 1 + φb 1 + β ( l + φb 1 (1 + γξ ) )(1 + αξ ) 1 ) s + φb m 1 ) ξ, (19) where l represens he level of he seres a me, b denoes he growh (or slope) a me, s s he seasonal componen of he seres a me, μ s he expeced value of he forecas a me, α, β, γ (0, 1) are he smoohng parameers, and φ (0, 1) denoes a dampng parameer. The rend componen s a combnaon of a level erm l and a growh erm b. In model (19) here s only one seasonal componen. For hs reason, as n he case of he ARIMA model, me seres s decomposed no 24 seres, each of whch represens he load a he same hour of a day. These seres were modeled ndependenly usng an auomaed procedure mplemened n he forecas package for he R sysem [18]. In hs procedure he nal saes of he level, growh and seasonal componens are esmaed as well as he smoohng and dampng parameers. AIC was used for selecng he bes model for a gven me seres. The DSHW model was proposed by Taylor [3]. Ths s an exponenal smoohng formulaon ha can accommodae more han one seasonal paern. Ths approach does no requre he decomposon of he problem. The rend componen s reaed addvely and he seasonal componens are reaed mulplcavely: Level : Growh : Seasonaly 1: Seasonaly 2 : Forecas : l b = β ( l d w y = αy /( d + h = ( l m1 l 1 = δy /( l w = ωy /( l d m2 m1 m2 + hb ) d ) + (1 α)( l ) + (1 β ) b w ) + (1 δ ) d ) + (1 ω) w m1 + h w 1 m2 + h m1 m2 1 h + λ + b 1 ) ( y ( l + b ) d w ) 1 1 m1 m2, (20) where d and w are he seasonal componens (daly and weekly n our examples) of he seres a me, y + h s he h sep-ahead forecas made from forecas orgn, α, β, δ, ω (0, 1) are he smoohng parameers, m 1 and m 2 are he seasonal perods (for our daa m 1 = 24 and m 2 = 168). The erm nvolvng he parameer λ n (20) s a smple adjusmen for frs-order auocorrelaon. All he parameers: α, β, δ, ω and λ are esmaed n a sngle procedure by mnmzng he sum of squared one sep-ahead n-sample errors. The nal smoohed values for he level, rend and seasonal componens are esmaed by averagng he early observaons. These calculaons were performed usng dshw funcon from he forecas package for he R sysem.

19 194 G. Dudek The model selecon and ranng procedures for AIS were he same as descrbed n Secon 4. The model was opmzed for each es sample, as well as oher models. In Table 1 resuls of forecass are presened: MAPE for he es samples and he nerquarle range (IQR) of MAPE. The acual and forecased fragmen of TS1 are shown n Fg. 11. Table 1. Resuls of forecasng Model TS1 TS2 TS3 MAPE s IQR MAPE s IQR MAPE s IQR AIS ARIMA ES DSHW Naïve x TS1 Acual AIS ARIMA ES DSHW f(pe s ) PE s Fg. 11. The acual and forecased fragmen of TS1 In order o ndcae he bes model we check f he dfference beween he accuraces of each par of models s sascally sgnfcan usng he Wlcoxon rank sum es for equaly of medans. The 5% sgnfcance level s appled n hs sudy. For only one case: AIS, ES and TS1 he es faled o rejec he null hypohess ha errors have he same medans. In all oher cases he es ndcaes he sascally sgnfcan dfference beween errors. Thus he bes mehod for TS1 are AIS and ES, for TS2 s ES, and for TS3 s AIS. The wors mehod for all cases was DSHW. Ths model produced several compleely ncorrec forecass for TS2 (no ncluded n mean error presened n Table 1). From Table 1 we can see ha he naïve mehod was subsanally

20 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles 195 ouperformed by all oher mehods. I s worh nong as well ha he ES model performed beer han ARIMA n all cases. I was observed ha he ARIMA and ES models produced wors forecass han AIS and DSHW for he frs hours of a day. The proposed AIS was he bes model for TS1 and TS3 bu no for TS2. Ths s probably because of he nsuffcen number of learnng pons for TS2 (TS2 me seres s wce shorer han TS1 and TS3). In hs case here are fewer ABs n he mmune memory and he local modelng s less accurae. In Fg. 12 he densy funcons of he percenage errors (PE) are shown. For ARIMA and ES he shf of he PE denses on he rgh s observed. I means ha hese models produced underesmaed forecass. 25 TS1 f(pe s ) AIS ARIMA ES DSHW PE s TS2 15 TS f(pe s ) 5 f(pe s ) PE s PE s Fg. 12. The emprcal PDF of PE s 6 Conclusons In hs arcle we deal wh forecasng of he seasonal me seres whch can be nonsaonary and heeroscedasc usng paerns of he me seres seasonal perods. The proposed forecasng mehod belongs o he class of smlary-based models. These models are based on he assumpon ha, f paerns of he me seres sequences are smlar o each oher, hen he paerns of sequences followng hem are smlar o each oher as well. I means ha paerns of neghborng sequences are sayng n a ceran relaon, whch does no change sgnfcanly n me. The more sable hs relaon s, he more accurae forecass are. Ths relaon can be shaped by proper paern defnons and srenghened by elmnaon of oulers.

21 196 G. Dudek The dea of usng AIS as a forecasng model s a very promsng one. The mmune sysem has some mechansms useful n he forecasng asks, such as an ably o recognze and o respond o dfferen paerns, an ably o learn, memorze, encode and decode nformaon. Unlke oher cluserng mehods used n forecasng models [13], [14], he proposed AIS forms clusers akng no accoun he forecas error. The cluser szes are uned o he daa n such a way o mnmze he forecas error. Due o he deermnsc naure of he model he resuls are sable and he learnng process s rapd. The AIS model also offers robusness o mssng daa. The dsadvanage of he proposed mmune sysem s lmed ably o exrapolaon. Regons whou he angens are no represened n he mmune memory. However, a lo of models, e.g. neural neworks, have problems wh exrapolaon. Acknowledgmens. The sudy was suppored by he Research Projec N N fnanced by he Polsh Mnsry of Scence and Hgher Educaon. References 1. Taylor, J.W., Snyder, R.D.: Forecasng Inraday Tme Seres wh Mulple Seasonal Cycles Usng Parsmonous Seasonal Exponenal Smoohng. Deparmen of Economercs and Busness Sascs Workng Paper 9/09, Monash Unversy (2009) 2. Box, G.E.P., Jenkns, G.M., Rensel, G.C.: Tme Seres Analyss: Forecasng and Conrol, 3rd edn. Englewod Clffs, Prence Hall, New Jersey (1994) 3. Taylor, J.W.: Shor-Term Elecrcy Demand Forecasng Usng Double Seasonal Exponenal Smoohng. Journal of he Operaonal Research Socey 54, (2003) 4. Taylor, J.W.: Exponenally Weghed Mehods for Forecasng Inraday Tme Seres wh Mulple Seasonal Cycles. Inernaonal Journal of Forecasng 26(4), (2010) 5. Gould, P.G., Koehler, A.B., Ord, J.K., Snyder, R.D., Hyndman, R.J., Vahd-Aragh, F.: Forecasng Tme-Seres wh Mulple Seasonal Paerns. European Journal of Operaonal Research 191, (2008) 6. Sharda, R., Pal, R.B.: Connecons Approach o Tme Seres Predcon: An Emprcal Tes. Journal of Inellgen Manufacurng 3, (1992) 7. Nelson, M., Hll, T., Remus, T., O Connor, M.: Tme Seres Forecasng Usng NNs: Should he Daa Be Deseasonalzed Frs? Journal of Forecasng 18, (1999) 8. Zhang, G.P., Q, M.: Neural Nework Forecasng for Seasonal and Trend Tme Seres. European Journal of Operaonal Research 160, (2005) 9. Zhang, G.P., Klne, D.M.: Quarerly Tme-Seres Forecasng wh Neural Neworks. IEEE Transacons on Neural Neworks 18(6), (2007) 10. Cleveland, R.B., Cleveland, W.S., McRae, J.E., Terpennng, I.: STL: A Seasonal-Trend Decomposon Procedure Based on Loess. Journal of Offcal Sascs 6, 3 73 (1990) 11. Aya, A., El-Shoura, S., Shaheen, S., El-Sherf, M.: A Comparson Beween Neural Neworks Forecasng Technques Case Sudy: Rver Flow Forecasng. IEEE Transacons on Neural Neworks 10(2), (1999) 12. Soares, L.J., Mederos, M.C.: Modelng and Forecasng Shor-Term Elecrcy Load: A Comparson of Mehods wh an Applcaon o Brazlan Daa. Inernaonal Journal of Forecasng 24, (2008)

22 Arfcal Immune Sysem for Forecasng Tme Seres wh Mulple Seasonal Cycles Dudek, G.: Smlary-based Approaches o Shor-Term Load Forecasng. In: Zhu, J.J., Fung, G.P.C. (eds.) Forecasng Models: Mehods and Applcaons, pp Concep Press (2010), hp:// php?paper_d= Lendasse, A., Verleysen, M., de Bod, E., Corell, M., Gregore, P.: Forecasng Tme- Seres by Kohonen Classfcaon. In: Proc. he European Symposum on Arfcal Neural Neworks, pp Bruges, Belgum (1998) 15. Har, E., Tmms, J.: Applcaon Areas of AIS: The Pas, he Presen and he Fuure. Appled Sof Compung 8(1), (2008) 16. Dudek, G.: Arfcal Immune Sysem for Shor-erm Elecrc Load Forecasng. In: Rukowsk, L., Tadeusewcz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC LNCS (LNAI), vol. 5097, pp Sprnger, Hedelberg (2008) 17. Theodords, S., Kouroumbas, K.: Paern Recognon, 4h edn. Elsever Academc Press (2009) 18. Hyndman, R.J., Khandakar, Y.: Auomac Tme Seres Forecasng: The Forecas Package for R. Journal of Sascal Sofware 27(3), 1 22 (2008) 19. Hyndman, R.J., Koehler, A.B., Ord, J.K., Snyder, R.D.: Forecasng wh Exponenal Smoohng: The Sae Space Approach. Sprnger (2008)