Vegetable Price Prediction Using Atypical Web-Search Data

Transcription

1 Vegeable Prce Predcon Usng Aypcal Web-Search Daa Do-l Yoo Deparmen of Agrculural Economcs Chungbuk Naonal Unversy Emal: Seleced Paper prepared for presenaon a he 2016 Agrculural & Appled Economcs Assocaon Annual Meeng, Boson, Massachuses, July 31-Augus 2 Copyrgh 2016 by Do-l Yoo. All rghs reserved. Readers may make verbam copes of hs documen for non-commercal purposes by any means, provded ha hs copyrgh noce appears on all such copes.

2 1. Inroducon In vegeable marke, relable prce predcon s expeced o preven loss of socal welfare caused by excess supply or excess demand. For example, by referrng o predced fuure prce, farmers may produce less vegeable beforehand n he excess supply marke, where prce s expeced o drop. And, farmers effcen quany adjusmen can save poenal socal coss of unshpped produc wase landflls, long-erm sorage, farm subsdes, and ec. Thus, s necessary o predc vegeable prce as accurae as possble. Tradonally, a consderable prevous leraure reles on me seres or neural nework models n predcng prce n he sense ha pas prces may mpac on curren and fuure prces. Thanks o nnovave nformaon echnology, recen Bg-Daa boom receves huge aenon as s possble o analyze large daase gahered from onlne webses lke Google and socal nework servces (SNS) such as blogs, Twer, Facebook, and ec. Assocaed leraure pays aenon o he mpac of aypcal web-search daa composed of specfc lexcon on relevan produc sales or prces assumng ha hose lexcons reflec consumers psychology n makng economc decsons. Represenavely, Google search engne query daa are used o predc economc ndcaors such as auomoble sales, unemploymen clams, consumer senmen, and gun sales (Cho and Varan, 2012; Sco and Varan, 2013). Bollen e al. (2011) predcs he sock marke by analyzng he nfluence of publc Twer mood on he value of he Dow Jones Indusral Average. Though Bg-Daa ssue s acvely spreadng n he feld of fnance, markeng, and economcs, sudes concernng agrculural economcs are relavely rare. Rare sudes resul from he fac ha agrculural producs marke s more unceran and unpredcable han oher ndusral producs marke; agrculural producs are easly pershable and frequenly affeced by clmae facors, leadng o flucuang prces. Therefore, would be mely o nroduce aypcal web-search daa analyss no he feld of agrculural economcs. We pay aenon o

3 he mpac of lexcons concernng vegeables on webses on vegeable prces. The objec of hs sudy s o develop vegeable prce predcon model wh hgher predcon power. Based on he ypcal me-seres models, we pay aenon o he role of aypcal web-search daa obaned from on-lne webses. Here, we beleve ha such aypcal daa could provde more robus prce predcon. To do so, we depend on he Bayesan srucural me seres (BSTS) model suggesed by Sco and Varan (2013). Whle ypcal me-seres models focus on he relaons beween curren prces and lagged prces, srucural me seres models could be more useful n he sense ha explanaory varables mpacng prces are nroduced n he srucural form (Harvey and Shephard, 1993). In addon, he Bayesan approach s wdely used o provde beer predcon concernng random walk by usng updaed poseror nformaon from pror nformaon of random walk (Koop, 2003). The paper s organzed as follows. Secon 2 presens boh concepual and emprcal BSTS models for vegeable prce predcon. Secon 3 presens an applcaon of he approach o hree vegeables of dred red pepper, garlc, and onon n Korean wholesale marke. Predced prce resuls are repored n secon 4. Fnally, secon 5 concludes. 2. Bayesan Srucural Tme Seres (BSTS) Model Based on he sae space form, where unobserved laen varables are consdered as sae varables, a ypcal concepual model usng BSTS s composed of wo equaons as follows: y Z 2, where ~ N 0, T T R 1 (1), where ~ 0, N Q (2)

4 Equaon (1) s called as an observaon equaon, lnkng observable me-seres daa y wh unobserved laen varables (sae varables). And, equaon (2) s called as a ranson equaon descrbng he law of moon beween he curren sae varables and he nex sae varables 1. In (1), Z s a vecor ncludng explanaory varables and parameers. In (2), T corresponds o a ranson marx accounng for relaon beween and 1, and R s a vecor ncludng parameers. Boh and are random noses followng he Gaussan dsrbuon wh zero mean and he varance 2 and Q, respecvely (Harvey and Peers, 1990; Sco and Varan, 2013a). For each vegeable garlc, onon, drp 1, equaons (1) and (2) can be specfed wh conceps of rend and seasonaly for prce me-seres y as follows: y x (3) T u (4) 1 1 (5) 1 v S 1 s w s1 (6), where x s a vecor ncludng explanaory varables mpacng y wh s assocaed parameer vecor. In equaon (4) and (5), s he slope of rend. In (6), S ndcaes he number of seasons consdered n he model for -vegeable. Excep for equaon (3), he oher equaons (4) ~ (6) accoun for ypcal me-seres models. Through equaon (3) 1 drp ndcaes dred red pepper.

5 ~ (6),,,, u v w are also assumed o be Gaussan random noses wh me-nvaran varances ,,,, respecvely. u v w Now, s necessary o dsnc equaon (3) by ypes of explanaory varables. Tha s o say, for our emprcal analyss, we need evaluae whch approaches can provde beer prce predcon wh and whou aypcal web-search daa concernng -vegeable. Under C, A x, le s consder C s a vecor composed of clmae facors for -vegeable. Also, le s consder A be a vecor ncludng aypcal ndexes obaned from aypcal websearch daa for -vegeable. Holdng equaons (4) ~ (6) same, equaon (3) s specfed no hree emprcal models by he ype of x as follows: y (7) y C (8) T y C A (9) T T, where and are parameer vecors assocaed wh clmae facors and aypcal websearch daa, respecvely for -vegeable. Also, equaons (7) ~ (9) are named as BSTS-I, BSTS-II, and BSTS-III. Then, BSTS-I s a benchmark model for comparng oher models BSTS-II and BSTS-III. As seen n equaon (7), BSTS-I has no explanaory varables n s form, mplyng a pure me-seres model consderng only rend and seasonaly. BSTS-II s a BSTS model wh only clmae facors, whose mpacs are assumed o mpac vegeable prce volaly hrough unsable demand and supply due o clmae volaly. Fnally, BSTS-III s a BSTS model consderng boh clmae facors and aypcal ndexes usng aypcal web-search daa. Furher deals concernng C and A are presened n secon 3.

6 Esmaon mehod for BSTS models depend on sochasc esmaon. Through equaons (7) ~ (9), parameers assocaed wh models are, and for each 2 2 vegeable. Ther pror probably dsrbuons p and p are assumed o follow he Gaussan and he nverse Gamma dsrbuons, respecvely as follows (Koop, 2003; Sco and Varan, 2013a): 2 2 ~ N o, (10) 1 v ss ~ G, (11), where 1 T T s a pror nformaon marx wh X X dag X X /2n and T,, X x1 x n when x, C A. Also, v and ss ndcae a pror sample sze and he pror sum of squares for -vegeable (Sco and Varan, 2013b). 2 Due o he properes of conjugacy n he Gaussan and he nverse Gamma dsrbuon, he poseror probably dsrbuons for parameers 2, 1,, 2 p y1,, y n p y y n and also follow same dsrbuons wh pror dsrbuons as follows (DeGroo, 2004): 1 T n , y 1,, y ~ N X T X X T y T 1,, y, X X n (12) 2 Furher deals are encouraged o refer o Sco and Varan (2013b).

7 v n, 2 T ss y1,, y y1,, y n n 1,, y1,, y ~ G n T T T X X X y1 y n T 1 X X 1 1 T T T X X X y1,, y n T (13) Followng Durbn and Koopman (2002), he poseror probably dsrbuons 2, 1,, 2 and 1,, p y y n Carlo (MCMC) smulaon usng Gbbs samplng. Denong p y y n are esmaed by he Markov chan Mone y as a prce predcon and,,,,,,,, u v w as a combned parameer vecor across all equaons for -vegeable, he poseror predcve dsrbuon s derved from he followng equaon:,, 1,, 1 p y y y p y p y y d n (14) n, mplyng Bayes heorem. Emprcally, equaon (14) s obaned by calculang E y y,, y n 1 based on randomly derved usng Mone Carlo esmaon. 3. Daa Concepual and emprcal models developed n secon 2 are appled o he Korean wholesale vegeable markes for garlc, onon, and dred red pepper a he monhly level. Remndng our

8 goal s o provde beer vegeable prce predcon across BSTS models, we specfy assocaed explanaory varables for each vegeable. Frs, clmae facors n C ncludes emperaure emp, mnmum emperaure mn emp, precpaon precp, sunshne amoun sun, and her square erms. C emp, emp,mn emp,mn emp, precp, precp, sun, sun (15) Here, square-erms are used for reflecng clmae volaly nsead of each clmae facor s varance erms, leadng o non-lnear models. All values are averaged values by monh as we predc monhly vegeable prces. Descrpve sascs for clmae facors, prces, and quanes for each vegeable are descrbed from <Table 1> o <Table 3>. Noe ha all averaged values for each clmae facor for -vegeable are calculaed from chef producng dsrcs for each vegeable as llusraed n <Fgure 1> ~ <Fgure 3>. <Fgure 1 ~ Fgure 3, here> <Table 1 ~ Table 3, here> Second, aypcal ndexes n A are derved from aypcal web-search daa obaned from varous on-lne webses ncludng SNS. We sugges fve aypcal ndexes accordng o recen ex-mnng approaches wdely used n he Bg-Daa research, reflecng consumers aenon on hree vegeables from SNS webses and major poral ses such as Google and Naver n Korea. Specfcally, usng ex mnng program Texom and UNICET 6, we gaher assocae web-search keywords. Then, we make smple query daa measurng

9 frequency on webses and Term Frequency Inverse Documen Frequency (TF-IDF) consderng weghs of core keywords on webses (Salon and McGll, 1983). So, fve aypcal ndexes are as follows: A nfo, search, unb, pec, lnk (16), where nfo s an ndex for nformaon exraced from web documens usng ex-mnng approach, mplyng a oal amoun of all web-documens ncludng a parcular lexcon (e.g., he name of a parcular vegeable) durng a pecular perod. search sands for search, whch s he oal number used for searchng a parcular lexcon durng a parcular perod. unb sands for unbalanced, mplyng TF-IDF suggesed by Salon and McGll (1983). pec sands for pecular, ndcang an ndex for pecular lexcon whch doesn appear a ordnary me. So, f a pecular lexcon appears durng a ceran perods, could be a lexcon people are suddenly neresed n (Sebasan, 2002). Fnally, lnk sands for lnk, and means an ndex for measurng he mporance of lnkages among lexcons (Freeman, 1979). 4. Resuls Based on me-seres daa from 2007/07 o 2016/03, we predc each vegeable prce for hree monhs from 2016/04 o 2016/06 across BSTS models (BSTS I ~ BSTS III). In order o measure how well each BSTS model predcs vegeable prce, we use he followng mean absolue percenage error (MAPE) as he crera of predcon performance. n 1 ACTUAL PREDICT MAPE (17) n ACTUAL 1

10 , where ACTUAL and PREDICT are acual prce and predced prce for -vegeable a me perod. Dvdng he whole perod for predcon perods no he n-sample performance perod and he ou-of-sample performance perod, we apply MAPE only o he n-sample performance perod. Whereas, fuure prces from 2016/04 o 2016/06 are predced only n he ou-of-sample performance perod. Those performance perods could be se up dfferenly accordng o he properes of vegeables. Resuls are shown n <Table 4> ~ <Table 6> across BSTS models wh calculaed MAPEs. <Table 4 ~ Table 6, here> For garlc, predcon power s hgher as aypcal ndexes are nroduced movng from BSTS-I o BSTS-III wh lower MAPEs. As for aypcal ndexes, search and unbalance ndexes are consdered n he model. For onon, he effecs of nroducon of aypcal ndexes n BSTS-III are he sronges among hree vegeables. As for aypcal ndexes, unbalance and lnk ndexes are used. Ths s neresng resul n our paper. There s a popular snger named as Onon n Korea, whch means he same lexcon could be yped va webses. So, among aypcal ndexes, some parcular ndexes are suable for parcular vegeables. For dred red pepper, he overall resuls are smlar o hose of garlc and onon. As for onon, even n BSTS-I and BSTS-II, MAPEs are low, mplyng ha BSTS models are mos approprae for predcng dred red pepper prces.

11 5. Conclusons By nroducng aypcal ndexes no he Bayesan srucural me seres models, we could see ha predcon power for vegeable prces are mproved. In oher words, can provde beer performances n predcng prces o combne recen Bg-Daa generaed aypcal web-search daa. Especally, would be valuable f we apply more aypcal daa no he feld of agrculural economcs such as food secor, yeld, and ec. oher han prce lke n our paper. Resuls show as follows: frs, he nroducon of aypcal ndex obaned from aypcal web-search daa can mprove prce predcon power. Second, he mprovemen across BSTS models could be dfferen by he knd of vegeables. Thrd, dfferen ypes of aypcal ndexes can be used by reflecng he properes of vegeables due o complcae meanng of lexcons lke he case of onon n Korea.

12 References Bollen, J., H. Mao, and X. Zeng, 2011, Twer Mood Predcs he Sock Marke, Journal of Compuaonal Scence, 2(1): 1-8. Cho, H. and H. Varan, 2012, Predcng he Presen wh Google Trends, Economc Record, 88(1): 2-9. DeGroo, M. H., 2004, Opmal Sascal Decsons, John Wley & Sons. Durbn, J. and S. J. Koopman, 2002, "A Smple and Effcen Smulaon Smooher for Sae Space Tme Seres Analyss," Bomerka, 89, Freeman, L. C., 1979, Cenraly n Socal Neworks Concepual Clarfcaon, Socal Neworks, 1: Salon, G. and M. J. McGll, 1983, Inroducon o Modern Informaon Rereval, McGraw Hll Book Co., New York. Harvey, A. C. and N. Shephard, 1993, "Srucural Tme Seres Models," Handbook of Sascs, Vol. 11, Elsever Scence Publshers. Harvey, A. C. and S. Peers, 1990, "Esmaon Procedure for Srucural Tme Seres Models," Journal of Forecasng, Vol. 9, Koop, G., 2003, Bayesan Economercs, Chaper 8. Inroducon o Tme Seres: Sae Space Models, Wley, U. K. Sco, S. and H. Varan, 2013a, Bayesan Varable Selecon for Nowcasng Economc Tme Seres, NBER Workng Paper Sco, S. and H. Varan, 2013b, Predcng he Presen wh Bayesan Srucural Tme Seres, Avalable a SSRN: hp://ssrn.com/absrac= Sebasan, F., 2002, Machne Learnng n Auomaed Tex Caegorzaon, ACM Compung Surveys, 34(1): Wen, I. H., E. Frank, and M. A. Hall, 2011, Daa Mnng: Praccal Machne Learnng Tools and Technques, 3 rd edon, Burlngon, MA: Morgan Kaufmann.

13 <Fgure 1: Chef Producng Dsrc of Garlc> <Fgure 2: Chef Producng Dsrc of Onon>

14 <Fgure 3: Chef Producng Dsrc of Dred Red Pepper>

15 <Table 1: Descrpve Sascs for Garlc> Obs. Mean s. d. Mn Max Prce (KRW/kg) Quany (kg) Temperaure ( ) Mnmum Temperaure ( ) Precpaon (mm) Wnd speed (m/s) Sunshne (Hr)

16 <Table 2: Descrpve Sascs for Onon> Obs. Mean s. d. Mn Max Prce (KRW/kg) Quany (kg) Temperaure ( ) Mnmum Temperaure ( ) Precpaon (mm) Wnd speed (m/s) Sunshne (Hr)

17 <Table 3: Descrpve Sascs for Dred Red Pepper> Obs. Mean s. d. Mn Max Prce (KRW/kg) Quany (kg) Temperaure ( ) Mnmum Temperaure ( ) Precpaon (mm) Wnd speed (m/s) Sunshne (Hr)

18 <Table 4: Prce Predcon for Garlc across BSTS Models, 2016/04 ~ 2016/06 > (Un: KRW/kg) BSTS Model BSTS I BSTS II BSTS III Monh pure me seres w/ clmae facors w/ clmae facors & aypcal ndexes 2015/9 4,584 4,584 4, /10 5,190 5,190 5, /11 5,570 5,570 5, /12 5,716 5,716 5, /1 5,862 5,862 5, /2 6,030 6,030 6, /3 5,781 5,781 5, /4 4,631 4,831 5, /5 4,709 4,966 6, /6 4,774 5,121 6,242 MAPE ( ~ )

19 <Table 5: Prce Predcon for Onon across BSTS Models, 2016/04 ~ 2016/06 > (Un: KRW/kg) BSTS Model BSTS I BSTS II BSTS III Monh pure me seres w/ clmae facors w/ clmae facors & aypcal ndexes 2015/9 1,400 1,400 1, /10 1,417 1,417 1, /11 1,594 1,594 1, /12 1,717 1,717 1, /1 1,673 1,673 1, /2 1,632 1,632 1, /3 1,608 1,608 1, / ,172 1, / ,261 1, / ,341 1,681 MAPE ( ~ )

20 <Table 6: Prce Predcon for Dred Red Pepper across BSTS Models, 2016/04 ~ 2016/06 > (Un: KRW/kg) BSTS Model BSTS I BSTS II BSTS III Monh pure me seres w/ clmae facors w/ clmae facors & aypcal ndexes 2015/4 13,667 13,667 13, /5 13,667 13,667 13, /6 13,667 13,667 13, /7 13,667 13,667 13, /8 13,670 13,670 13, /9 13,883 13,883 13, /10 13,687 13,687 13, /11 13,497 13,497 13, /12 13,332 13,332 13, /1 13,013 13,013 13, /2 13,000 13,000 13, /3 12,891 12,891 12, /4 11,478 15,601 12, /5 11,561 15,511 12, /6 11,508 15,677 12,569 MAPE ( ~ )