Energy Consumption Forecasting in Process Industry Using Support Vector Machines and Particle Swarm Optimization

Transcription

1 Energy Consumpton Forecastng n Process Industry Usng Support Vector Machnes and Partcle Swarm Optmzaton MILENA R. PEKOVIĆ MILAN R. RAPAIĆ BORIS B. JAKOVLJEVIĆ Computng and Control Department Unversty of Nov Sad rg Dosteja Obradovća 6 SERBIA mlena5@uns.ac.rs Abstract: - In ths paper, Support Vector Machnes (SVMs) are appled n predctng energy consumpton n the frst phase of ol refnng at a partcular ol refnery. Durng cross-valdaton process of the SVM tranng Partcle Swarm Optmzaton (PSO) algorthm was utlzed n selecton of free SVM parameters, wdths of radal bass functons to be exact. Incorporaton of PSO nto SVM tranng process has greatly enhanced the qualty of predcton. Key-Words: - Support Vector Machnes (SVM), Partcle Swarm Optmzaton (PSO), Energy Predcton 1 Introducton Hgh qualty predcton of consumpton of fuels, ncludng both electrcal energy and fossl fuels (heatng ol, natural and refned gas and steam) n the frst phase of ol refnng (atmospherc dstllaton) s vtal for control and optmzaton of the ol refnng process. Energy and fuel consumpton are quanttes of utmost mportance, snce they affect overall cost of the entre ol refnng process, and consequently, the defnte prces of all refned ol products. Fuel consumpton analyss and predctons are therefore vtal for producton and an nterestng feld of research. In terms of predcton methods, Support Vector Machnes (SVM) are attractve as relatvely new, yet effectve technque n modelng of complex functonal correlatons. herefore, utlzaton of SVMs n estmatng and predctng the consumpton of varous types of fuels for ndustral systems s a promsng approach, as stpulated by the present paper as well as by other authors [1]. he present paper advocates utlzaton of regresson SVM model [2] wth PSO algorthm ncorporated n cross-valdaton phase of tranng. Support Vector Machnes mplement the prncple of structural rsk mnmzaton n place of emprcal rsk mnmzaton, whch gves them excellent generalzaton ablty n the stuaton of small tranng sample [3]. In addton, SVMs can change a nonlnear learnng problem nto a lnear one, n order to reduce the algorthm complexty by usng the kernel functon dea (the kernel trck ). At present, SVMs have been utlzed n solvng nonlnear regresson estmaton problems n fnancal tme seres forecastng [4], relablty predcton [5], power load forecastng [6], etc. However, SVMs have rarely been appled to forecast fuel consumpton n ol refnng, though, t s the opnon of the authors, the technque has great potental n ths area. A short account of SVM regresson s gven n secton 2. he man dsadvantage of SVM s the necessty to set a number of parameters n advance. Standard procedure, known as cross-valdaton, s to make several consecutve trals wth dfferent parameter sets and then choose the set gvng the best performance. In the present paper, the crossvaldaton procedure s conducted by PSO algorthm. he dea to use global optmzaton procedure n cross-valdaton process s not entrely new. Successful applcatons of genetc algorthm (GA) have prevously been reported n lterature [7]. PSO s novel optmzaton procedure, known for ts effcency, well adopted for solvng non-convex, multmodal optmzaton problems. PSO s ntroduced n secton 3. Applcaton of PSO to crossvaldaton process s addressed n secton 4. Results obtaned n ths study clearly demonstrate effectveness of PSO-based cross-valdaton. In partcular, predcton offset presented n the prevous studes [8] was completely suppressed n the current one. ISSN: ISBN:

2 SVM models developed n the present paper were traned on a one year data-base consstng of 1) daly refnng of ol, 2) daly usage of ndustral unts (n percents), 3) type of ol beng refned, 4) the daly consumpton of fuels (both electrc energy and fossl fuels) and 5) clmate condtons (season). he data concerns partcular faclty for atmospherc ol dstllaton, that we named faclty A. hs faclty was selected based on analyss of the producton process of the refnery and the fact that ths faclty uses a consderable fracton of the overall fuel consumed n the ol refnng process. Results and conclusons are presented n secton 5 and 6, respectvely. 2 Support Vector Regresson he general regresson learnng problem s set as follows: the learnng machnes s gven n tranng data pars from whch t attempts to learn the nputoutput relatonshp y= f( x). A tranng data set X = {( x, y), = 1,..., n} consst of n tranng pars. he nputs x are m- dmensonal vectors, whle the target outputs y are real valued scalars. We ntroduce all the relevant and necessary concepts of SVM regresson n a gentle way startng wth a lnear ˆ = gven as regresson hyperplane y f( x, w, b) y= f( x, w, b) = w x+ b ˆ (1) where ŷ s predcted output, x s nput pattern, w s weght vector and b s bas [9,10]. Both weght and bas are set durng the tranng process. he most mportant dfference of SVM wth respect to classcal regresson technques s the use of a novel loss (error) functon [6] Vapnk s lnear loss functon wth ε-nsenstvty zone, defned as E( x, y, f ) = y = max f( x, ( x, ( 0, y f ε) ε = (2) hus, the loss s equal to zero f the dfference between the predcted f ( x, and the measured value y s less than ε. In contrast, f the dfference s larger than ε, ths dfference s used as the error. In other words, loss functon (2) defnes an ε tube as shown n Fg 1. If the predcted value s wthn the tube, the loss s zero; for all other predcted ponts outsde the tube, the loss equals the magntude of the dfference between the predcted value and the radus ε of the tube. It can be shown [11] that generalzaton ablty of the SVM depends on the magntude of the weght vector: the smaller the magntude w the greater the generalzaton ablty of the SVM becomes. herefore, lnear regresson hyperplane s constructed by mnmzng n 1 2 R= w + C y f( x, (3) ε 2 = 1 where C s a postve constant (regularzaton parameter) [9]. From (2) and Fg 1 t follows that for all tranng patterns one can defne postve quanttes known as slack varables ξ = max( y f( x, ε,0) (4) ξ = max( f( x, y ε,0) Notce that at least one of these quanttes s equal to zero for each tranng pattern. For patterns nsde the tube, both of them are zero. hus, the mnmzaton of the rsk R above equals the mnmzaton of n 1 2 * Rξ = w + C ( ξ + ξ ) (5) 2 = 1 under constrants y w x b ε + ξ (7) * y + w x + b ε + ξ (8) ξ 0, ξ 0 (9) Fg 1. he parameters used n (1-D) Support Vector regresson. Fled data are support vectors Support Vector Machnes used n the present paper were all traned and smulated by means of LIBSVM software [12], freely avalable lbrary of functons for creatng SVM models. 3 PSO Algorthm Partcle Swarm Optmzaton (PSO) algorthm has been ntroduced for the frst tme by Kennedy and Eberhart [13] as a new populaton based optmzaton technque nspred by anmal socal behavor. he algorthm nvestgates soluton space usng a set of vectors, usually referred to as partcles. Each partcle s a potental soluton, and the entre set s referred to as the populaton, or sometmes as the swarm. A partcle s descrbed ISSN: ISBN:

3 by ts poston (x) and speed (v), and s able to memorze the poston wth the hghest ftness value t has acheved so far (p). Intally, the swarm s randomly dspersed wthn the search space, and random velocty s assgned to each partcle. Partcles nteract by sharng nformaton. Although dfferent patterns of nteractons have been nvestgated n lterature, we focus our efforts to the so-called star topology, also known as the gbest PSO model [14]. In ths settng, the swarm as a whole memorzes the best poston acheved so far by any of ts partcles (g); at each step, a partcle cares over a porton of ts prevous speed, and s, n addton, smultaneously accelerated towards ts personal best poston and the best poston found by any other partcle n the swarm. Dynamcs of each partcle s, therefore, determned by the followng set of equatons v[ k] = w v[ k 1] + cp rp[ k] ( p[ k] x[ k] ) + (7) + cg rg[ k] ( g[ k] x[ k] ) x [ k+1 ] = x[ k] + v[ k] (8) Postve parameter w, the nerta weght, was ntroduced by Sh and Eberhart [15] n an attempt to control dversty of the swarm durng the optmzaton process. It s generally true, n populaton-based optmzaton methods, that hgh dversty s necessary n early stages of the search n order to fully nvestgate the search space and reduce possblty of beng trapped n local optmum. On the other hand, n later stages, algorthm should focus on fne-tunng good solutons already found, so reduced dversty s desrable. Sh and Eberhart found that consderable mprovements n performance of the orgnal PSO are acheved by lnear decreasng nerta over the generatons from 0.9 to 0.4. Postve coeffcents cp and cg are usually called the acceleraton factors [14]. Random values rp and rg are mutually ndependent and unformly dstrbuted n range [0, 1]. Factor cp s sometmes referred to as the cogntve parameter, whle cg s referred to as socal parameter [14]. Due to orgnal work of Kennedy and Eberhart, t s common choce to set both acceleraton factors equal to 2. However, t s known that relatvely hgh cogntve component enhances exploraton, whle relatvely hgh socal component forces partcles to cluster. Acknowledgng ths fact, Ratnaweera et al. suggested [16] that tme-varyng acceleraton coeffcents may further mprove performance of the optmzer. hey reported mprovements for most of the benchmarks when decreasng cp form 2.5 to 0.5, and smultaneously ncreasng cg from 0.5 to PSO - Based Cross Valdaton he selecton of SVM parameters, namely C, ε and parameters of kernel functon, s mportant for the forecastng accuracy. Selectng approprate values of these parameters s crucal n ganng excellent forecastng performance. In ths paper, we chose Radal Bass Functon for a kernel functon and t was necessary to select ts wdths. However, t s not known beforehand what values of the parameters are approprate. herefore, PSO s used to optmze parameters n the proposed SVM model. he schematc dagram of PSO-based crossvaldaton s presented n Fg 2. Encodng SVM Intalzaton ranng SVM model Evaluaton Is stop condton satsfed? Movng the swarm New parameters Optmal parameters obtaned Optmal SVM forecastng model obtaned Fg 2.Schematc dagram of PSO- based cross valdaton 5 Results 5.1 PSO vs. Classcal Cross- Valdaton In followng examples, output s the amount of electrcal energy consumed (n kw), whle the nputs are: the amount of refned ol (n tons), the type of ol, plant utlzaton (n %), consumpton of other fuels (heatng ol, natural and refned gas and steam), as well as crtcal events such are plant shutdown and restart. ISSN: ISBN:

4 Fg 6. Relatve test-set predcton errors (n %) wth PSO-based cross-valdaton as a functon of plant utlzaton (n %). Fg 3. est-set predcton results wth classcal cross-valdaton (LIBSVM mplementaton). he ordnate depcts kw of power consumed. Fg 3. and Fg 5. depct true and predcted power consumpton usng classcal and PSO-based crossvaldaton, respectvely. It s clear that, when applyng classcal cross-valdaton there s a problem wth offset, as well as wth peaks occurrng durng crtcal events. hese errors were also reported n prevous studes [8]. PSO-based SVM regresson proposed n the current paper s not prone to such errors. Relatve errors presented n Fg 4 and Fg 6 only confrm the effectveness of our method. However, conclusons presented n [8] concernng the mportance of suffcent plant utlzaton for hgh qualty predcton reman. Fg 4. Relatve test-set predcton errors (n %) wth classcal cross-valdaton (LIBSVM mplementaton) as a functon of plant utlzaton (n %). 5.2 Standard Refnery Fuel Commonly, the consumpton of fuels n ol refnng processes s expressed n tons of Standard Refnery Fuel [17,18]. Standard Refnery Fuel (SRF) s a reference fuel whose lower heatng value s 9673 kcal/kg. In ths example, output s the amount of electrcal energy consumed (n kw), whle the nputs are: the amount of refned ol (n tons), the type of ol, plant utlzaton (n %), sum of consumptons of all other fuels (heatng ol, natural and refned gas and steam) expressed n tons of SRF, as well as crtcal events such are plant shut-down and restart. Fg 5. est-set predcton results wth PSO-based cross-valdaton. he ordnate depcts kw of power consumed. Fg 7. est-set predcton results wth PSO-based cross-valdaton and fuel consumptons expressed n tons of SRF. he ordnate depcts kw of power consumed. ISSN: ISBN:

5 Fg 8. Relatve test-set predcton errors (n %) wth PSO-based cross-valdaton and fuel consumptons expressed n tons of SRF as a functon of plant utlzaton (n %). Fg. 7 and Fg. 8 show that the modelng of nput parameters of fuel consumpton over SRF addtonally enhances the qualty of predcton of energy consumpton n ol refnng processes. 6 Concluson hs paper s dedcated to the problem of predctng the energy consumpton n ol refnng process usng SVM regresson wth cross-valdaton based on PSO algorthm. We used the SVM method n whch the parameters were determned by the PSO algorthm, a new method for solvng ths type of problems. he results of ths paper clearly demonstrate the effectveness of the proposed procedure n comparson to the classcal one. References: [1] S. Žvkovć, Prmena Support vector machnes u predkcj potrošnje elektrčne energje, (n Serban) MSc hesys, Faculty of echncal Scences, Nov Sad, Serba, [2] A. Smola and B. Schölkopf, A utoral on Support Vector Regresson, Statstcs and Computng 14: , (2004), Kluwer Academc Publshers. Manufactured n he Netherlands, (2004) [3] H. Shn, S. Cho, Response modelng wth support vector machnes, Expert Syst. Appl. 30 (4) (2006) [4] K.-j. Km, Fnancal tme seres forecastng usng support vector machnes, Neurocomputng 55 (1 2) (2003) [5] P.-F. Pa, System relablty forecastng by support vector machnes wth genetc algorthms, Math. Computer Model. 4 (3-4) (2006) [6] P.-F. Pa,W.-C. Hong, Forecastng regonal electrcty load based on recurrent support vector machnes wth genetc algorthms, Electrc Power Syst. Res. 74 (3) (2005) [7] Sheng-We Fe, Yu Sun, Forecastng dssolved gases content n power transformer ol based on support vector machne wth genetc algorthm, Electrc Power Systems Research 78 (2008) [8] Mlena Petrujkć, Marjana Bobar, Olvera Papć Predkcja potrošnje energenata u prmarnoj prerad nafte prmenom Support Vector Machnes, (n Serban) ERAN, Herceg Nov [9] B. Schölkopf, A. Smola, Learnng wth Kernels, Supprt Vector Machnes, Regularzaton, Optmzaton and Beyond, MI Press, Cambrdge, MA, [10] Kecman V., Hgh Dmensonal Functon Approxmaton (Regresson, Hypersurface Fttng) by an Actve Set Least Squares Learnng Algorthm, School of Engneerng Report 643, he Unversty of Auckland, Auckland, NZ, (53 p.), 2006 [11] V. Kecman, Learnng and Soft Computng, Support vector machnes, Neural Networks and Fuzzy Logc, he MI Press, Cambrdge, MA, (2001) [12] C. C. Chang and C. J. Ln (2001) LIBSVM: A Lbrary for Support Vector Machnes [Onlne]. Avalable: [13] J. Kennedy, R.C. Eberhart, Partcle Swarm Optmzaton, Proc. of IEEE Int. Conf. on Neural Networks, Perth, Australa (1995) [14] F. van den Bergh, An analyss of partcle swarm optmzers, PhD hess, Unversty of Pretora, [15] Y. Sh, R.C. Eberhart, Emprcal study of partcle swarm optmzaton, Proc. IEEE Int. Congr. Evolutonary Computaton, vol 3, 1999, [16] A. Ratnaweera, et al, Self-Organzng Herarchcal Partcle Swarm Optmzer Wth me-varyng Acceleraton Coefcents, IEEE ransactons on Evolutonary Computaton, vol. 8, no. 3, June 2004, [17] G. G. Rayan, Practcal energy effcency optmzaton, ISBN: [18] ISSN: ISBN: