Research Article Crude Oil Price Prediction Based on a Dynamic Correcting Support Vector Regression Machine

Transcription

1 Abstract ad Applied Aalysis Volume 2013, Article ID , 7 pages Research Article Crude Oil Price Predictio Based o a Dyamic Correctig Support Vector Regressio Machie Li Shu-rog ad Ge Yu-lei College of Iformatio ad Cotrol Egieerig, Chia Uiversity of Petroleum, Qigdao, Shadog , Chia Correspodece should be addressed to Li Shu-rog; lishuro@upc.edu.c Received 10 December 2012; Accepted 28 Jauary 2013 Academic Editor: Fudig Xie Copyright 2013 L. Shu-rog ad G. Yu-lei. This is a ope access article distributed uder the Creative Commos Attributio Licese, which permits urestricted use, distributio, ad reproductio i ay medium, provided the origial work is properly cited. A ew accurate method o predictig crude oil price is preseted, which is based o ε-support vector regressio (ε-svr) machie with dyamic correctio factor correctig forecastig errors. We also propose the hybrid RNA geetic algorithm (HRGA) with the positio displacemet idea of bare boes particle swarm optimizatio (PSO) chagig the mutatio operator. The validity of the algorithm is tested by usig three bechmark fuctios. From the compariso of the results obtaied by usig HRGA ad stadard RNA geetic algorithm (RGA), respectively, the accuracy of HRGA is much better tha that of RGA. I the ed, to make the forecastig result more accurate, the HRGA is applied to the optimize parameters of ε-svr. The predictig result is very good. The method proposed i this paper ca be easily used to predict crude oil price i our life. 1. Itroductio I recet years, crude oil prices have experieced four jumps adtwoslumps.thefluctuatioofcrudeoilpriceaddsmore chages to the developmet of world ecoomy. Graspig thechageofoilpricecaprovideguidaceforecoomic developmet [1]. Therefore, it is very importat to predict the crude oil price accurately. The predictig methods ca be divided ito two aspects. Oe is from the qualitative agle [2]; the other is from quatitative agle, such as ecoometric model ad statistical model [3, 4]. Ad the latter method is adopted by most scholars. But it is a difficult job to predict crude oil price, sice the price is oliear ad ostatioary time series [5]. The traditioal predictig methods such as AR(p)model, MA(q)model,adARMA(p,q)model,baseoliearmodel. Theyareolysuitableforliearpredictioadcaotbe applied to model ad predict oliear time series [6]. Wag got the predictig model by usig time series ad artificial eural etwork i 2005 [7], Xie proposed a ew method for crudeoilpriceforecastigbasedosupportvectormachie (SVM) i 2006 [8], Mohammad proposed a hybrid artificial itelligece model for crude oil price forecastig by meas of feed-forward eural etworks ad geetic algorithm i 2007 [9],adGuoproposedahybridtimeseriesmodelo the base of GMTD model i 2010[10]. The experimetal resultstellusthatthepredictioaccuracyofthesemethods is better tha traditioal models. But the results is still existig biggish errors especially whe the crude oil price is fluctuatig violetly. Neural etwork techique provides a favorable tool for oliear time series forecastig. But the predictive ability of covetioal eural etwork is low, because of the problems such as the local miimum, over learig, ad the lackig of theoretical directio for selectig the hidde layer odes. The SVM was proposed i the 1990s [11]; it ca get the optimal results o the basis of the curret iformatio. The basic idea of SVM is that it fits the sample capacity of fuctios o the basis of regulatig the upper boud of the miimum VC dimesio, which also meas the umbers of support vector. Compared with eural etwork [12, 13], SVM has strog geeralizatio ability of learig small samples ad with the iferior depedece o quatity. But the predictio performace of SVM is very sesible to parameter selectio. O the other had, the research o parameter optimizatio of SVM is very few at the momet. The parameters are usually determied o experiece or trial method. I this way, if the parameters are ot suitably chose, the SVM will lead to poor

2 2 Abstract ad Applied Aalysis predictio performace. So, it is importat to fid oe good method to get the optimal parameters of SVM. I this paper, a ε-support vector regressio machie withdyamiccorrectiofactorisproposed.adaovel hybrid RNA geetic algorithm (HRGA) is proposed to obtai the optimal parameters for a SVM. The HRGA is from the developmet of biological sciece ad techology; the structure ad iformatio of RNA molecular are kow profoudly. To improve the optimal performace of geetic algorithm, oe geetic algorithm which bases o codig ad biological molecular operatio has bee widely cocered [14]. This method improves the search efficiecy ad optimizatio performace through codig the idividuals ito biological molecules by use of bases [15, 16]. The appropriate mutatio operator ca improve the populatio diversity ad prevet premature. While the mutatio operator of classical RNA geetic algorithm (RGA) is fixed, so we eed to fid a suitable method to determie the mutatio operator. I 2003, Keedy did some improvemet o particle swarm optimizatio (PSO) ad proposed the bare boes particle swarm algorithm [17]. I the proposed HRGA, the positio displacemet idea of bare boes PSO is applied to chage the mutatio operator. The ucleotide base ecodig, RNA recodig operatio, ad protei foldig operatio are reserved i the ew algorithm. Thus,thestrogglobalsearchcapabilityiskept.Atthesame time,tomakesureofthedirectivityoflocalsearchig,the optimal experiece of the whole populatio ad the historical experiece of idividuals are used. The covergece speed ad solutio precisio are improved. Furthermore, to test the validity of HRGA, three bechmark fuctios are adopted. The mea value of optimum of HRGA is smaller tha that of traditioal RNA geetic algorithm. Oce the support vector regressio machie is desiged optimally,itcabeusedtopredictcrudeoilprice.dyamic correctio factor is brought i to improve the predictive effect ad ca stregthe the robustess of systems. I order to test the performace of the proposed predictig method, we provided the predictig results by usig a back propagatio eural etwork ad a traditioal support vector regressio machie which are also improved with dyamic correctio factor [7, 8].Theresultsshowthatourpredictigmethod obtais greater accuracy tha that of the other two i this paper. The paper is orgaized as follows. Sectio 2 discusses the support vector regressio machie with dyamic correctio factor. Sectio 3 presets HRGA based o bare boes PSO, ad some testig examples are applied to verify the effectiveess of the algorithm. Sectio 4 applies the dyamic correctig ε -SVR to predict the crude oil price. Sectio 5 cocludes the paper. 2. Support Vector Regressio Machie with Dyamic Correctio Factor Cosider the traiig sample set (x i,y i ), i = 1,2,...,, x i R,astheiputvariableady i Ras the output variable. The basic idea of SVM is to fid a oliear mappig φ from iput space to output space [18 20]. Data x is mapped to a high-dimesioal characteristic space F o the basis of the oliear mappig. The estimatig fuctio of liear regressio i characteristic space F is as follows: f (x) =[ω φ(x)]+b, (1) φ:r F, ω F, where b deotes threshold value. Fuctio approximatio problem is equal to the followig fuctio: R reg (f) = R emp (f) + λ ω 2 = s C(e i )+λ ω 2, (2) where R reg (f) deotes the objective fuctio, R emp (f) deotes the empirical risk fuctio, s deotes the sample quatity, λ deotes adjustig costat, ad C deotes the error pealty factor. ω 2 reflects the complexity of f i the high-dimesioal characteristic space. Sice liear ε isesitive loss fuctio has better sparsity, we ca get the followig loss fuctio: y f(x) ε = max {0, y f(x) ε }. (3) Theempiricalriskfuctioisasfollows: R ε emp (f) = 1 y f(x) ε. (4) Accordigtothestatisticaltheory,webrigitwogroups of oegative slack variable {ξ i } ad {ξ i }.The,the questiocabecovertedtothefollowigoliearεsupport vector regressio machie (ε -SVR) problem: mi (ξ i,ξi ) {1 2 ω 2 +C(ξ i +ξ i )}, y i ω φ(x) b ε+ξ i, ω φ(x) +b y i ε+ξ i, ξ i,ξ i 0, where ε deotes the isesitive loss fuctio. C is used to balace the complex item ad the traiig error of the model. We brig ito Lagrage multipliers α i ad α i, the the covex quadratic programmig problem above ca be chaged ito the below dual problem: max [ 1 2 [ (α i,α i ) + i,j=1 (α i α i )(α j α j)k(x i,x j ) α i (y i ε) 0 α i,α i (α i α i )=0, α i (y i ε)], C,,2,...,, (5) (6)

3 Abstract ad Applied Aalysis 3 where K(X i,x j ) deotes the ier product kerel satisfyig Mercer theorem. We ca get the ε-svr fuctio through solvig the above dual problem: f (x) = (α i α i )K(X i,x)+b. (7) Whe ε-svr is used o predictio, it may have a certai error sice the data fluctuates violetly such as the crude oilprice.toreducetheerrorisomecertaiaspossibleas we ca, we brig i the dyamic correctio factor ε.the mai idea of the dyamic correctio factor is that we use the errorofbackstepwithmultiplyigε to revise the curret predictig results. Thus, we ca reduce the curret predictig error. The dyamic correctig SVR ca be defied as follows: Y d (i+1) =Y 1 (i+1) +ε [Y (i) Y d (i)], (8) where Y deotes the real results, Y d deotes the fial predictio results, Y 1 deotes the iitial predictig results, ε deotes the dyamic correctio factor, ad i deotes the predictio steps. I order to make the predictig results more accurate, theoptimalvalueofε ad the parameters of ε-svr ivolvig C, δ (the variable i gauss kerel fuctio) should be desiged (i (8)). To this ed, a HRGA is studied below to optimize the followig problem: mi [Y d (i) Y(i)] 2. (9) (ε,c,σ) i 3. HRGA Based o Bare Boes PSO Assumig that populatio size is N,the dimesio of particle is m. Thepositioofparticlei o geeratio t is X i (t) =(x i1 (t),...,x ij (t),...,x im (t)),i = 1,2,...,N. The speed of particle i o geeratio t is V i (t) = (V i1 (t),..., V ij (t),..., V im (t)). ThehistoricoptimalvalueofidividualsisPBest i (t) =(pbest i1 (t),...,pbest ij (t),...,pbest im (t)). Let the global optimal value be GBest(t) = (gbest 1 (t),..., gbest j (t),...,gbest m (t)). As to stadard particle swarm, the positio ad speed are updated as V ij (t+1) =wv ij (t) +c 1 r 1j (pbest ij (t) x ij (t)) +c 2 r 2j (gbest j x ij (t)), x ij (t+1) =x ij (t) + V ij (t+1), (10) where ω deotes the iertia weight [21], c 1 ad c 2 deote the acceleratig operators, ad r 1j ad r 2j are uiform distributed radom umbers i [0, 1]. I the bare boes particle swarm optimizatio (PSO), (10) is replaced by (11) as the evolutio equatio of particle swarm algorithm: x ij (t+1) = N (0.5 (pbest ij (t) +gbest j (t)) pbest ij (t) gbest j (t) ). (11) The positio of particle is some radom umbers which are gotte from the Gauss distributio. The distributio has the mea value of (pbest ij (t) + gbest j (t))/2 ad the stadard deviatio of pbest ij (t) gbest j (t). RNA geetic algorithm is o the basis of base codig ad biological molecules operatio. Sice i the biological molecule, every three bases compose oe amio acid. I other words, the bases legth of idividuals must be divided exactly by 3. Whe RNA recodig ad protei foldig [22], to reduce calculatio ad to cotrol populatio size, we assume that the protei foldig operatio oly occurs o the idividuals without RNA recodig. The the most importat work is to chage the mutatio probability [23, 24]. Agelie told us that the essece of particle swarm s positio updatig was oe mutatio operatio i 1998 [25]. Traditioal RNA geetic algorithm mutates as the fixed mutatio probability, ad the mutatio is radom with oe directio. However HRGA ca reflect the historic iformatio of idividuals ad the sharig iformatio of the populatio. HRGA ca make every idividual do directioal mutatio ad improve search efficiecy. Moreover, HRGA esures the strog global search capability, sice it does ot chage the selectio ad crossover operator. The procedure of HRGA based o bare boes particle swarm algorithm to optimize the ε-svr parameters ad the dyamiccorrectiofactorisasfollows. Step 1. Get oegroupofε-svr parameters, ad the dyamic correctio factor radomly, code every parameter, ad get the iitial RNA populatio with N idividuals, crossover probability P c,admutatioprobabilityp m.assigvalues for every PBest i (idividual s historic optimal solutio) ad GBest (populatio s global optimal solutio). Step 2. Compute its error fuctio ad get the fitess fuctio. Comparig it with correspodig fitess value of PBest i ad GBest,theupdatePBest i ad GBest. Step 3. Execute the selectio operatio. Get curret geeratio through copig N idividuals from the iitial or the last geeratio. Step 4. Decide whether the value meets the RNA recodig coditio or ot. If Y, recoderna,thegotostep 6. IfN, go to Step 5. Step 5. Decide meet the protei mutual foldig coditio or ot. If Y, execute the protei mutual foldig operatio. If N, execute the protei ow foldig operatio. Step 6. Execute the mutatio operatio as (11) forallthe crossover idividuals, o the basis of the PBest i ad GBest, which have bee gotte fromstep 2. Step 7. Repeat Step 2 to Step 6 util the traiig target meets the coditio. At last, we get the optimal parameters of ε-svr ad the dyamic correctio factor.

4 4 Abstract ad Applied Aalysis Table 1: Bechmark fuctios. Fuctio Formula Global miimum Sphere Rosebrock f 2 = =1 Griewak f 3 = f 1 (x) = x 2 i, x i [ 100, 100]. f 1 (x) =0, x = (0, 0,..., 0) (100 (x i+1 x 2 i )2 )+(x i 1) 2, x i [ 30, 30]. f 2 (x) = 0, x = (1, 1,..., 1) x 2 i cos ( x i i )+1, x i [ 600, 600]. f 3 (x) = 0, x = (0, 0,..., 0) Table 2: Testig results of HRGA ad stadard RGA. Fuctio Hutig zoe Dimesio Iterative times ( 100, 100) Sphere ( 100, 100) ( 100, 100) ( 30, 30) Rosebrock ( 30, 30) ( 30, 30) ( 600, 600) Griewak ( 600, 600) ( 600, 600) Parameter selectio P c = 0.9 P c = 0.9 P c = 0.9 P c = P c = P c = RGA Mea value of optimum e Parameter selectio P c = 0.6 P c = 0.6 P c = 0.6 P c = 0.5 P m = 0.4 P c = 0.5 P m = 0.4 P c = 0.5 P m = 0.4 HRGA Mea value of optimum e e e e 3 The flowchart of HRGA to optimize the ε-svr parameters ad the dyamic correctio factor is show i Figure HRGA Testig. Three classical bechmark fuctios show i Table 1 are used to test the property of HRGA. I additio, amog the three fuctios, Sphere is uimodal fuctio, ad the other two are multimodal fuctio. With the populatio size N=50, ad other parameters determied by multiple test for each fuctio. Each fuctio is tested by HRGA ad stadard RGA i differet dimesios. Each experiece is carried o 100 times. Record the mea value of target fuctio s optimum (show i (12)). The result is displayed i Table 2: MVO = 1 N N f i (x). (12) I this equatio, MVO deotes the mea value of target fuctio s optimum; f i (x) deotes the optimum of bechmark fuctios i every experimet. As to the experimetal results, with differet dimesios havigthesameiterativetimes,themeavalueofoptimum ofhrgaissmallerthathatofrgaforthethreebechmark fuctios. The average performace of HRGA is closer to the optimum. We ca icrease the mutatio probability appropriately ad ehace the covergece speed, sice the mutatio operator of HRGA has directioal local search. 4. Crude Oil Price Predictio Based o a Dyamic Correctig ε-svr I this paper, we get the crude oil price from the US Eergy Iformatio Admiistratio Web [26]. Sice the oil price fluctuates violetly, i order to facilitate the processig ad decreasetheerror,weadoptthecushig,okwtispotprice FOB (dollars per barrel)mothly from Jauary 1986 to ow. We take the oe hudred data from Jauary 1986 to April 1994asthetestsample.Adgivetheext20-mothdyamic predictig data from May 1994 to December The relative errorofforecastigisshowitable 2. Thepredictioeffect figure of HRGA ad ε-svr with dyamic correctio factor is show i Figure 2. WeuseGaussfuctioasthekerel fuctio of ε-svr, which is give as follows: k(x,y)=exp ( x y 2 2 σ 2 ). (13)

5 Abstract ad Applied Aalysis 5 Start Parameters of ε-svr ad dyamic correctio factor Populatio iitializatio Fitess fuctio computatio Selectig operatio F(x) x: parameters of ε-svr ad dyamic correctio factor ε-svr model F(x): fitess Satisfyig the coditio of RNA recodig? Y RNA recodig operatio N Satisfyig the protei mutual foldig coditio? Y Mutual foldig operatio N Ow foldig operatio Executig mutatio operatio as (11) N Satisfyig the termiatio coditio? Y Get the optimal parameters of ε-svr ad dyamic correctio factor Ed Figure 1: The flowchart of HRGA. WTI spot price FOB (dollars per barrel) Real data Predicted data Date (moths from 1986) Figure 2: The predictio curve of HRGA-ε-SVR. Relative error of forecastig (%) Date (moths from to ) BP SVM HRGA-SVM Figure 3: Errors aalysis with differet models. Parameter settig of HRGA-ε-SVR iswithpopulatio size beig 100, maximum evolutio geeratio beig 150, codig legth of C beig 9, codig legth of ε beig 8, codig legth of σ beig 13, codig legth of ε beig 8, P c beig 0.8, ad P m beig 0.1. The optimizatio iterval is set to be 1 C 10000, ε 0.1, 0.01 σ 500, 0.5 ε 2. (14)

6 6 Abstract ad Applied Aalysis Table3:Aalysisresultsofforecastigerror. Date BP/% ε-svr% HRGA-ε-SVR /% δ Whe aalyzig the results, we defie the evaluatio idex: E r = x i y i 100%, σ = 1 ( x 2 i y i ). (15) x i x i The forecastig error aalysis results are show i Figure 3. Ithisfigure,SVMreferstoε-SVR. TheBPeural etwork ad ε-svr are with dyamic correctiofactorwhich differs them to the traditioal method. From Figure 2, we ca kow that the predictio result is very close to the real value. The HRGA-ε-SVR ca be used to predictthe crude oil price. Table 3 tells us the WTI crude oil price predictig relative errors of twety moths. Amog the three methods i twety moths, the biggest absolute value of relative error of HRGA- ε-svr is the smallest, which is 7.35%, ad the smallest rootmea-square of relative error is 3.87%. As to Figure 3, the fluctuatio rage of HRGA-ε-SVR is smaller tha those of the other two methods obviously. This meas that HRGA-ε-SVR is the best oe amog the three methods. 5. Coclusios I this paper, we have preseted a ovel method o predictig crude oil price. This method bases o a ε-support vector regressio machie with dyamic correctio factor correctig predictig errors. We also proposed the HRGA, with the positio displacemet idea of bare boes PSO chagig the mutatio operator, to optimize the parameters i a ε-svr. The predictig result of crude oil price shows the validity of the proposed method. Thus, the ε-svr model ca also be applied to predict tedecy i other practical areas. Ackowledgmets The research was partially supported by Grat o from the Natioal Sciece Foudatio of Chia ad by Grat o. ZR2011FM002 from the Natural Sciece Foudatio of Shadog Provice. Refereces [1] B. Hut, P. Isard, ad D. Laxto, The macroecoomic effects of higher oil prices, IMF Workig Paper No.wp/01/14, [2]Y.Fa,K.Wag,Y.J.Zhagetal., Iteratioalcrudeoil market aalysis ad price forecast i 2009, Bulleti of Chiese Academy of Scieces,vol.4,o.1,pp.42 45,2009. [3] C. Moraa, A semiparametric approach to short-term oil price forecastig, Eergy Ecoomics,vol.23,o.3,pp , [4] S. Mirmirai ad H. Cheg Li, A compariso of VAR ad eural etworks with geetic algorithm i forecastig price of oil, Advaces i Ecoometrics,vol.19, pp ,2004. [5] Z. J. Dig, Q. Mi, ad Y. Li, Applicatio of ARIMA model i forecastig prude oil price, Logistics Techology,vol.27, o.10, pp ,2008. [6] J.P.Liu,S.Li,T.Guo,adH.Y.Che, Nolieartimeseries forecastig model ad its applicatio for oil price forecastig, Joural of Maagemet Sciece,vol.24,o.6,pp ,2011. [7] S. Y. Wag, L. Yu, ad K. K. Lai, Crude oil price forecastig with TEI@ I methodology, Joural of Systems Scieces ad Complexity,vol.18,o.2,pp ,2005. [8]W.Xie,L.Yu,S.Xu,adS.Wag, Aewmethodforcrude oil price forecastig based o support vector machies, Lecture Notes i Computer Sciece,vol.3994,pp ,2006. [9] R. A. N. Mohammad ad A. G. Ehsa, A hybrid artificial itelligece approach to mothly forecastig of crude oil price time series, i The Proceedigs of the 10th Iteratioal Coferece o Egieerig Applicatios of Neural Networks,pp , [10] S. Guo ad P. Lai, The time series mixed model ad its applicatio i price predictio of iteratioal crude oil, Joural of Najig Uiversity of Iformatio Sciece & Techology, vol.2, o.3,pp ,2010. [11] Y. B. Hou, J. Y. Du, ad M. Wag, Neural Networks, Xidia Uiversity Press, Xi a, Chia, [12] H. Zhu, L. Qu, ad H. Zhag, Face detectio based o wavelet trasform ad support vector machie, Joural of Xi a Jiaotog Uiversity,vol.36,o.9,pp ,2002. [13] R.Feg,C.L.Sog,Y.Z.Zhag,adH.H.Shao, Comparative study of soft sesor models based o support vector machies ad RBF eural etworks, Joural of Shaghai Jiaotog Uiversity,vol.37,pp ,2003. [14] J. Tao ad N. Wag, DNA computig based RNA geetic algorithm with applicatios i parameter estimatio of chemical egieerig processes, Computers & Chemical Egieerig, vol. 31,o.12,pp ,2007. [15] K. Wag ad N. Wag, A protei ispired RNA geetic algorithm for parameter estimatio i hydrocrackig of heavy oil, Chemical Egieerig Joural, vol. 167, o. 1, pp , 2011.

7 Abstract ad Applied Aalysis 7 [16] K. Wag ad N. Wag, A ovel RNA geetic algorithm for parameter estimatio of dyamic systems, Chemical Egieerig Research & Desig,vol.88,o.11,pp ,2010. [17] D. Bratto ad J. Keedy, Defiig a stadard for particle swarm optimizatio, i Proceedigs of the IEEE Swarm Itelligece Symposium (SIS 07), pp , April [18] N. Y. Deg ad Y. J. Tia, ANewMethodofDataMiig adgermay:supportvectormachies, Sciece Press, Beijig, Chia, [19] U. Thisse, R. Va Brakel, A. P. De Weijer, W. J. Melsse, ad L. M. C. Buydes, Usig support vector machies for time series predictio, Chemometrics ad Itelliget Laboratory Systems, vol.69,o.1-2,pp.35 49,2003. [20] K. J. Kim, Fiacial time series forecastig usig support vector machies, Neurocomputig,vol.55, o.1-2,pp , [21] Y. Shi ad R. Eberhart, Modified particle swarm optimizer, i Proceedigs of the IEEE Cogress o Evolutioary Computatio, pp ,1998. [22] D. P. Clark, Molecular Biology: Uderstadig the Geetic Revolutio, Academic Press, New York, NY, USA, [23] J. Lis, Geetic algorithm with the dyamic probability of mutatio i the classificatio problem, Patter Recogitio Letters,vol.16,o.12,pp ,1995. [24] M. Serpell ad J. E. Smith, Self-adaptatio of mutatio operator ad probability for permutatio represetatios i geetic algorithms, Evolutioary Computatio, vol.18,o.3,pp , [25] P. J. Agelie, Evolutioary optimizatio versus PSO: philosophy ad performace differeces, Evolutioary Programmig, vol. 7, pp , [26] &s=rwtc&f=m.

8 Advaces i Operatios Research Advaces i Decisio Scieces Joural of Applied Mathematics Algebra Joural of Probability ad Statistics The Scietific World Joural Iteratioal Joural of Differetial Equatios Submit your mauscripts at Iteratioal Joural of Advaces i Combiatorics Mathematical Physics Joural of Complex Aalysis Iteratioal Joural of Mathematics ad Mathematical Scieces Mathematical Problems i Egieerig Joural of Mathematics Discrete Mathematics Joural of Discrete Dyamics i Nature ad Society Joural of Fuctio Spaces Abstract ad Applied Aalysis Iteratioal Joural of Joural of Stochastic Aalysis Optimizatio