FORECASTING NATURAL GAS CONSUMPTION USING PSO OPTIMIZED LEAST SQUARES SUPPORT VECTOR MACHINES

Transcription

1 FORECASING NAURAL GAS CONSUMPION USING PSO OPIMIZED LEAS SQUARES SUPPOR VECOR MACHINES Hossen Iranmanesh 1, Majd Abdollahzade 2 and 3 Arash Mranan 1 Deparmen of Indusral Engneerng, Unversy of ehran & Insue for Inernaonal Energy Sudes, ehran, Iran hranmanesh@u.ac.r 2 Deparmen of Mechancal Engneerng, K.N.oos Unversy of echnology & Insue for Inernaonal Energy Sudes, ehran, Iran m.abdollahzade@gmal.com 3 School of Elecrcal and Compuer Engneerng, Unversy of ehran & Insue for Inernaonal Energy Sudes, ehran, Iran ar.mranan@gmal.com ABSRAC hs paper proposes an effecve model based on he leas squares suppor vecor machnes (LS- SVM) and he parcle swarm opmzaon (PSO), ermed PSO-LSSVM, for predcon of naural gas consumpon, as an mporan energy resource. he salen feaure of mappng nonlnear daa no hgh dmenson feaure space, dsngushes LS-SVM as a powerful approach for forecasng and esmaon. Opmzaon of he model s parameers by a fas and effcen PSO algorhm resuls n an opmzed model whch s employed for predcon of annual naural gas consumpon n Iran and Unes Saes. Promsng resuls were obaned for predcon of Iranan gas consumpon from 1998 o 2006 and U.S. gas consumpon from 2001 o Besdes, comparson o an opmzed mul-layer precepron (MLP) nework, usng error ndces of MAPE and NMSE demonsraed he superor performance of he proposed PSO-LSSVM approach. KEYWORDS Leas square suppor vecor machnes, parcle swarm opmzaon, naural gas consumpon, forecasng 1. INRODUCION Naural gas as a clean and effcen fossl fuel accouns for a consderable poron of world energy consumpon. In 2008, 20.0% of oal energy consumpon of he world was suppled by naural gas [1]. Naural gas producon and consumpon are experencng an ncreasng rend owng o he growh n world populaon and he economc developmen worldwde. Due o he favourable characerscs of naural gas, such as beng clean, envronmen frendly and hghly effcen as well as s sraegc saus, he accurae predcon of gas consumpon s crucal. Hence, varous mehods and approaches have been developed by he researchers for hs purpose, whch can be denfed as deermnsc or sochasc, dynamc or sac and lnear or nonlnear models [2]. As anoher classfcaon, naural gas predcon mehods can be dsngushed as radonal (me seres) and compuaonal nellgence (CI)-based approaches. Lu and Ln employed me seres models for forecasng consumpon of naural gas n awan whn he resdenal secor [3]. In her sudy, hey explored he relaonshps among resdenal gas consumpon and several DOI : /jaa

2 relevan me seres varables, such as emperaure of he servce area and gas prce, and hen developed he forecas model. hey provded boh monhly and quarerly forecas usng her developed model. A logsc curve nerpreaon approach was presened by Semek e al for esmaon of naural gas consumpon [4]. In hs approach he hypohecal naural-gas demand was descrbed based on average rend of he economy developmen durng recen decades. In anoher sudy, Akkur e al used dfferen me seres models for predcon of naural gas consumpon n urkey [5]. hey proposed dfferen models such as such as exponenal smoohng, wners forecasng and Box-Jenkns mehods, o forecas naural gas consumpons of urkey n dfferen me perods. A sysem dynamcs model has been developed by L e al for Forecasng he growh of Chnese naural gas consumpon [6]. hey appled hs model o provde an oulook for Chnese gas consumpon unl Sochasc Gomperz nnovaon dffuson model, whch s a sascal model, was used by Guérrez e al o forecas Span naural gas consumpon [2]. hs approach s based on obanng he probably densy funcon of he process and hen forecasng he fuure values of he process. here are many unceran facors nfluencng naural gas consumpon whch make gas consumpon seres hghly complex and nonlnear [7]. herefore, radonal lnear models and sascal approaches such as lnear regresson or he mehod one proposed n [2], are no suable for gas consumpon predcon. Compuaonal nellgence (CI) based models, ncludng fuzzy logc, neural neworks (NN) and suppor vecor machnes (SVM) are elaborae models whch are effecve n dealng wh hghly nonlnear and complex processes [8]. he CI-based models have been used for energy demand predcons o a grea exen [9, 10]. Predcon of daly naural gas consumpon by combnaon of arfcal neural-nework forecasers has been also carred ou [11]. In hs sudy, Khoanzad e al proposed a wo-sage sysem wh he frs sage conanng wo NN forecasers. he second sage conssed of a combnaon module o mx he wo ndvdual forecass produced n he frs sage. hey mplemened her approach on real daa from sx dfferen gas ules. Suppor vecor machnes, esablshed based on he sascal learnng heory, exhb dsncve advanages o solve complex problems [12, 13]. In hs paper we propose he dea of opmzng leas squares suppor vecor machnes (LS-SVM) parameers usng he fas and effcen algorhm of parcle swarm opmzaon. he developed PSO-LSSVM wll be used for predcon of annual naural gas consumpon n Iran and Uned Saes. 2. LEAS SQUARES SUPPOR VECOR MACHINES Suppor vecor machnes have been developed based on he sascal learnng heory by Vapnk [14]. he man heme of SVMs les n mappng he npu space no a hgher dmensonal feaure space, and hen performng he lnear regresson usng suppor vecor regresson (SVR). he less adjusable parameers of he SVMS compared o neural neworks, has made hem popular for predcon, conrol and sgnal processng applcaons [15], [16]. Furhermore, SVMs ranng nvolves opmzaon of a quadrac problem wh a unque soluon; herefore random nalzaon of he model's weghng facors s prevened. Varous applcaons have been repored for SVMs, ncludng paern recognon, classfcaon and regresson analyss [17]-[19]. In case of me seres predcon, SVR esmaes a funcon usng observed daa and he SVMs are raned. In he res of hs paper, we resrc our aenon o he mahemacal formulaons of he SVMs for he purpose of me seres forecasng. x defned a 0,1,..., N 1 y N + as he predced values n Consder me seres ( ) = and ( ) he fuure. he predcon funcon f ( x ), defnes he predced oupu based on he m prevous observaons, y ( N + ) = f ( x ( N a1 ), x ( N a2 ),..., x ( N am )) (1) 50

3 where a 1,, a m are me lags. By applyng regresson analyss he predcon funcon for nonlnear regresson applcaons s defned as below, ( ) ( φ ( )) f x = w x + b (2) where φ ( x ) s he kernel funcon, w s he vecor of weghs and b s he bas. he nonlnear regresson n (2) maps he npu space no a hgher dmenson feaure space by means of he kernel funcon and hen a lnear regresson s performed [14]. Nex, he opmal weghs w and he bas b mus be found hrough an opmzaon procedure, consderng he proper opmzaon 2 crera, namely he flaness of he weghs, measured by he Eucldean norm w and he esmaon error, defned by a loss funcon. wo commonly used loss funcons for SVMs are sensve and quadrac loss funcons. he laer s assocaed wh he leas squares suppor vecor machnes (LS-SVM), employed n hs paper. he mahemacal represenaon of he opmzaon problem for LS-SVM, gven N pars of ranng daa ( x, y ), = 1,..., N s as follows, Mnmze 1 N w w + λ ξ 2 = 1 (3) Subjec o y w φ ( x ) b + = 1 ξ (4) where λ s referred o as he regularzaon consan and deermnes he penales o he esmaon error and ξ are slack varables whch allow for some errors n he opmzaon problem. By usng he Lagrange mulplers and consderng he Karush-Kuhn-ucker (KK) condons, he followng s obaned, = ( ) α = γξ y w φ ( x ) b + 1+ ξ = 0 N w α 1 y φ x = N α 0 1 y = = (5) Le us also defne he (, j ) K x x as he nner produc of ( ) Kernel funcon) and consder a se of oher defnons saed below: Z = φ ( x 1) y 1, φ ( x 2 ) y 2,..., φ ( x ) y Y = [ y 1, y 2,..., y ] ξ = [ ξ1, ξ2,..., ξ ] α = [ α1, α2,..., α ] φ x and ( x j ) By elmnang w and γ and usng (5) and (6), he followng equaon s obaned: φ vecors (called (6) 51

4 0 Y b 0 1 = Y ZZ γ I α I + (7) = 1,, 1, Bas 2, Oupu Neuron N npus, Hdden Layer Fgure 1. Srucure of SVM where I = [ 1,1,...,1 ]. By applyng Mercer s condon [15] whn he n hs marx wll have he followng form: ZZ marx, each elemen ( ZZ ) y y jφ ( x ) φ ( x j ) j = (8) Fnally he resulng LS-SVM model can be represened as: N * f ( x ) = ( α α ) K ( x, x ) + b (9) = 1 where α and * α are Lagrange mulplers. I s noceable ha wh he aforemenoned φ. he Kernel funcon whch s nner produc of defnons, here s no need o compue ( x ) wo φ ( x ) funcons s nsead ncorporaed n compuaons. Some common Kernel funcons are nroduced n (10)-(13). he RBF Kernel s used n hs paper. Do Produc Kernel: (, ) (, ) K x x = x x (10) Polynomal Kernel: ( ) ( ) K x, x = x, x + 1 d MLP Kernel: (, ) anh (, ) K x x = x x + b (11) (12) 52

5 RBF Kernel: 2 x x K ( x, x ) = exp 2 (13) 2σ Based on he presened descrpon, γ and σ are he only parameers of he LS-SVM whch should be opmally uned. he PSO algorhm s presened n nex secon and wll be furher used for opmal selecon of LS-SVM's parameers. 3. PROPOSED FORECAS FRAMEWORK he LS-SVM model, descrbed n prevous secon, conans wo adjusable parameers whch have a key role n he accuracy of predcons of he produced of he model. Varous opmzaon echnques, such as genec algorhms (GA), smulaed annealng (SA) and parcle swarm opmzaon (PSO) can be ulzed for fne unng of he LS-SVM's parameers. In hs paper, PSO algorhms, due o he speed of convergence, smplcy of mplemenaon and less suscepbly of beng rapped n local opma, are preferred [20]. In PSO, parcles flow n a mul-dmensonal search space and he poson of each parcle s uned based on he experences ganed by hm and hs neghbours. In hs paper we adop a gbes PSO algorhm. In gbes algorhm he new poson of he parcle s found by addng he velocy componen, as followng: 53

6 ( + 1) = ( ) + ( + 1) ( + 1) = ( ) + ( ) ( ) ( ) + ( ) ( ) ( ) x x v j ) (14) v j v j c1r1 j y j x j c2r2 j y x j where, x ( ) s he poson of parcle a me, v j ( ) ) me, y ( ) s he bes poson found by parcle, y ( ) and c 2 are acceleraon consans and r1 ( ) and 2 ( ) s velocy of parcle a dmenson j a s he bes poson found by swarm, c 1 r are unformly dsrbued number n [0, 1]. For opmal selecon of he LS-SVM's parameers,.e. σ and λ, wo dmensonal parcles are randomly dsrbued n he search space. he overall procedure of he LS-SVM opmzaon by he PSO algorhm s llusraed n Fg. 2(a). he framework of he proposed forecas approach s shown n Fg 2(b). hs fgure llusraes how he LS-SVM model s opmzed by PSO algorhm hrough ranng daa and he opmzed model s employed for predcon of he es daa. In nex secon, he proposed PSO opmzed LS-SVM model wll be appled o predcon of gas consumpon n Iran and U.S. 4. GAS CONSUMPION PREDICION In hs secon he annual gas consumpon of Iran and he Uned Saes wll be forecased usng he proposed PSO-LSSVM model. he consumpon n prevous perod as well as he populaon o he las pon are he sandard npu varables for he predcon. he ranng and es daa se for each case are presened n able 1. hese daa are colleced from Insue for Inernaonal Energy Sudes (IIES) webpage, World Bank Developmen Indcaor daases and he U.S. energy nformaon admnsraon webse [20-22]. For evaluaon of he performance of he PSO- LSSVM model, he followng error measures are compued, Mean absolue percenage error (MAPE): ) 1 y y MAPE = 100 = 1 y Absolue percenage error (APE): ) y y APE = 100 y Normalzed mean squares error (NMSE): ) 2 ( y ) 1 y = NMSE = 2 y y = 1 ( ) (15) (15) (16) 54

7 y 1 = 1 = y Fgure. 3 ranng MAPE versus PSO eraons where, y and y ) are he acual and predced consumpons a perod, respecvely Predcon of Gas Consumpon n Iran In hs case sudy, he annual consumpon of naural gas n Iran form 1998 o 2006 wll be forecased. For hs purpose he followng npu-oupu ses wll be used for ranng he proposed model. Inpu vecor Oupu { y ( 1 ), x ( 4 ), x ( 3 ), x ( 2 ), x ( 1) } y ( ) (17) where, y ( ) and ( ) x are gas consumpon and populaon a me, respecvely. Furhermore, for he purpose of comparson, a mul-layer percepron (MLP) nework was raned and opmzed usng ranng daa. Frs, he opmzaon of he LS-SVM model was carred ou by PSO algorhm for 30 eraons. he number of parcles, dmenson of each parcle, c 1,c 2 for he PSO algorhm are se as 30, 2, 2, 2 respecvely. he MAPE for he ranng daa was seleced as he fness funcon n PSO. he fness value for PSO eraons s shown n Fg. 3. he acual and forecased values of Iranan gas consumpon for ranng and es daa are llusraed n Fg. 4, revealng he remarkable performance of he proposed model n esmang ol consumpon seres. he acual and forecased gas consumpon for es perod s shown n able 2. A comparson beween performance of he proposed mehod and he MLP nework s presened n able 3. Accordng o hs able he APE ranges from 0.28% o 27.5%. he maxmum APE% was occurred n 2000 when an abrup change happens n he gas consumpon seres, as shown n Fg. 4, and he PSO- 55

8 LSSVM faled o capure hs change n consumpon. he opmzed MLP nework has one hdden layer wh 4 neurons. he surpassng performance of he proposed mehod s evden. Mllon barrel of ol euqvalen Acual Predcon Error ran es Year Fg. 4 Acual and predced values for ran and es daa for Iran Gas consumpon 4.2. Predcon of Gas Consumpon n U.S. Due o unavalably of he daa for U.S. gas consumpon pror o 1980, ranng and es daa dfferen o he prevous case sudy are employed here. As presened n able 1, here are 20 ranng daa pons, whle he es daa conan 5 samples. For sngle sep ahead predcon of he U.S. gas consumpon, followng npu feaures are consdered, Inpu vecor Oupu (18) able 2. Acual and forecased gas consumpon for case sudy 1 Year Acual Forecas APE% able 3. Compsrson beween he PSO-LLNF and MLP models for case sudy 1 Mehod Daa MAPE% NMSE MLP ranng es PSO-LSSVM ranng es

9 { y ( 1 ), x ( 1 ), w ( 1) } y ( ) x 104 Acual Predcon Error Bllon cubc fee ran es Year Fg. 5 Acual and predced values for ran and es daa for U.S. Gas consumpon where, y ( ), x ( ) and ( ) w are gas consumpon and populaon and GDP per capa a me, respecvely. Agan, he LS-SVM model was opmzed usng PSO algorhm and MAPE as he fness funcon. Smlar o he prevous case, an MLP nework was also opmzed wh 3 neurons for makng a comparson o he resuls obaned by he proposed PSO-LSSVM. he predcons of he PSO-LSSVM as well as he acual gas consumpons and he forecas error are depced n Fg. 5. he remarkable forecas performance and accuracy of he proposed mehod s obvous n 57

10 hs fgure. he acual and forecased values of he es daa of he U.S. gas consumpon are gven n able 4. he mnmum and maxmum values of APE% are 3.43% and 0.01%, respecvely. Besdes, he performance of he PSO-LSSVM and he MLP nework n erms of error ndces MAPE and NMSE are presened n able 5. he resuls n hs able demonsrae he noeworhy performance of he proposed mehod as well as s superory over he opmzed MLP nework. 58

11 4.3. Comparson of he Resuls More dealed dscusson on he predcon resuls s presened n hs sub-secon. For hs purpose, he maxmum and mnmum values of APE, acheved hrough he proposed mehod and he MLP model, for boh case sudes s are shown n able 6. For boh case sudes, here s a consderable dfference beween he mnmum and maxmum APEs assocaed wh he PSO- LSSVM mehod and he MLP model. For nsance, he mnmum APE of he PSO-LSSVM n he frs case sudy s 0.28%, whle hs value for he MLP model s 1.96% (clearly 7 mes ha of PSO-LSSVM). o horoughly analyze he superory of he proposed mehod over MLP, he mprovemen n error ndces,.e. MAPE and NMSE, s compued and summarzed n able 7. Clearly, sgnfcan mprovemen has been acheved n boh case sudes by employng he proposed PSO-LSSVM mode. Furhermore, an overall comparson beween he proposed approach and he MLP nework for boh case sudes s provded by Fg. 6. Obvously, he proposed PSO-LSSVM model has ouperformed MLP nework n boh case sudes. As anoher fndng from Fg. 6, boh forecas models had beer accuracy for he second case sudy. he reason for hs can be undersood by comparng acual gas consumpon seres of Iran and he U.S. n Fgs. 4 and 5, respecvely. As shown n hese fgures. he Iranan gas consumpon seres exhbs more changes and flucuaons hrough he me. Hence s less predcable n comparson he U.S. gas consumpon seres, whch s a more smooh seres. 5. CONCLUSION Accurae forecasng of naural gas consumpon, due o s large conrbuon n provdng he world energy demand, needs specal aenon. hs paper proposed a PSO opmzed LS-SVM approach for predcon of naural gas consumpon n Iran and Uned Saes. Suppor vecor machnes show noceable forecas and esmaon capables owng o mappng nonlnear daa no hgh dmensonal feaure space and hen performng lnear regresson. Opmzaon of parameers of he LS-SVM model by a smple bus fas and effcen PSO algorhm resuled n a hybrd model, appled for gas consumpon predcon. Fnally, wo dfferen case sudes were consdered for evaluang he performance of he proposed PSO-LSSVM approach. Forecasng annual gas consumpon n Iran, as one of he world larges gas producers, and he U.S., as one he world larges gas consumers, revealed he promsng forecas ably of he mehod. Assessng performance of he proposed approach n erms of NMSE and MAPE and comparson o an opmzed MLP nework showed he superor performance of he PSO-LSSVM model. 59

12 APPENDIX: RAW DAA FOR PRESENED CASE SUDIES able A1: Raw daa for Iran Year Gas consumpon (mllon barrel of equvalen) Populaon (mllon people)

13 able A2: Raw daa for he U.S. Year Gas consumpon Populaon (mllon (bllon cubc fee) people) GDP per capa REFERENCES [1] Inernaonal Energy Agency (IEA) [onlne] Avalable: [2] Guérrez, A., Nafd, A., Guérrez Sánchez, R., Forecasng oal naural-gas consumpon n Span by usng he sochasc Gomperz nnovaon dffuson model. Appled Energy; 80(2), [3] Lu, L.M, Ln, M.W., Forecasng resdenal consumpon of naural gas usng monhly and quarerly me seres, Inernaonal Journal of Forecasng; 7(1), [4] Semek, J., Nagy, S., Rychlck, S., Esmaon of naural-gas consumpon n Poland based on he logsc-curve nerpreaon. Appled Energy; 75(1-2), 1-7. [5] Akkur, M., Demrel, O.F., Zam, F., Forecasng urkey s naural gas consumpon by usng me seres mehods. European Journal of Economc and Polcal Sudes; 3(2). [6] L, J., Dong, X., Shangguan, J., Höök, M., Forecasng he growh of Chnese naural gas consumpon. Energy; 36(3),

14 [7] Sánchez-Úbeda, E.F., Berzosa, A., Modelng and forecasng ndusral end-use naural gas consumpon. Energy Economcs; 29(4), [8] Jang, J.S., Sun, C.., Mzuan, E., Neuro-Fuzzy and Sof Compung: A Compuaonal Approach o Learnng and Machne Inellgence. Prence-Hall Inernaonal, Inc. [9] Hpper, H.S., Pedrera, C.E., Souza, R.C Neural neworks for shor-erm load forecasng: a revew and evaluaon. IEEE ransacons on Power Sysems; 16(1), [10] Laure, P., Fock, E., Randranarvony, R.N., Mancom-Ramasamy, J.F., Bayesan neural nework approach o shor me load forecasng. Energy Converson and Managemen;49(5), [11] Khoanzad, A., Elragal H., Lu,,L., Dallas,.X., Combnaon of arfcal neural-nework forecasers for predcon of naural gas consumpon. IEEE ransacons on Neural Neworks; 11(2), [12] Crsann, N., Shawe-aylor, J. An Inroducon o Suppor Vecor Machnes and Oher Kernel- Based Learnng Mehods. Cambrdge, U.K.: Cambrdge Unv. Press, [13] Suykens, J.A.K., Van Gesel,., De Brabaner, J., De Moor, B., Vandewalle, J. Leas Squares Suppor Vecor Machnes. Sngapore: World Scenfc, [14] Vapnk, V. N. Sascal Learnng heory. New York: Wley, [15] Sapankevych, N., Snakar, R me seres predcon usng suppor vecor machnes. IEEE compuaonal nellgence magazne, [16] Lee, M.C., o, C., Comparson of Suppor Vecor Machne and Back Propagaon Neural Nework n Evaluang he Enerprse Fnancal Dsress. IJAIA; 1(3), [17] Km, S.K., Park, Y.J., oh, K.A., Lee, S., SVM-based feaure exracon for face recognon. Paern Recognon; 43(8), [18] Meha, S.S., Lngaya, N.S., Developmen of SVM based classfcaon echnques for he delneaon of wave componens n 12-lead elecrocardogram. Bomedcal Sgnal Processng and Conrol; 3(4), Pages [19] Xu, S.Q., Luo, Q.Y., Xu, G.H., Z., Le Asymmercal nerval regresson usng exended ε- SVM wh robus algorhm. Fuzzy Ses and Sysems; 160(7), [20] Engelber, A. P. Compuaonal Inellgence- An Inroducon, Wley, [21] Insue for Inernaonal Energy Sudes. [onlne] Avalable: hp://es.ac.r [22] World Bank, World Developmen Indcaors World Bank, Washngon, DC. [23] U.S. Energy Informaon Admnsraon. [onlne] Avalable: hp:// 62