D big lsses in judgmental adjustments affect experts behaviur? Ftis Petrpuls, Rbert Fildes and Paul Gdwin
This material has been created and cpyrighted by Lancaster Centre fr Frecasting, Lancaster University Management Schl, all rights reserved. Yu may use this material fr yur private educatinal purpses, s lng as they are clearly identified as being created and cpyrighted by Lancaster Centre fr Frecasting, Lancaster University Management Schl. Yu are nt permitted t alter, change, r enhance them. Yu may nt use any f the cntent, text and images, in full r in part, in a tutrial, training, educatin, written papers, vides r ther recrdings. Yu are nt permitted t distribute r make available directly r indirectly, within r utside yur cmpany, nr explit them withut explicit prir written permissin frm Lancaster Centre fr Frecasting, Lancaster University Management Schl, email: training@frecastingcentre.cm
Mtivatin [Smith et al., 2009, MnSc] Pker players tend t change their behaviur after winning r lsing big pts: Big wins lead t a less aggressive playing behaviur. Big lsses are fllwed by playing less cautiusly. The empirical results supprt the break-even hypthesis and, secndarily, the gambler s fallacy. Dyle Bransn s rush pker strategy is nt fund t be applied in practice. At the same time, behaviural theries such as huse mney and revised assessment are nt supprted. Can we link these insights t judgmental adjustments fr frecasting?
Theries and Research Questins Break-even hypthesis: Balancing-ut the effects f judgmental adjustments t crrect the inventry signals. A judgmental adjustment that led t a big lss will be fllwed by anther equally large adjustment in the ppsite directin. Gambler s fallacy: It will happen this time (because it is verdue)! A judgmental adjustment that led t a big lss will be fllwed by anther equally large adjustment in the same directin. RQ1 Des experts behaviur change after big lsses? RQ2 If yes, what can we d t turn this t ur benefit?
Types f judgmental adjustments Wrng Directin Actuals (X). Mdel Frecast (MF) r Statistical Frecast r System Frecast. Expert Frecast (EF) r Judgmental Adjustment. This is usually used as the Final Frecast (FF).
Types f judgmental adjustments Undersht Actuals (X). Mdel Frecast (MF) r Statistical Frecast r System Frecast. Expert Frecast (EF) r Judgmental Adjustment. This is usually used as the Final Frecast (FF).
Types f judgmental adjustments Oversht Actuals (X). Mdel Frecast (MF) r Statistical Frecast r System Frecast. Expert Frecast (EF) r Judgmental Adjustment. This is usually used as the Final Frecast (FF).
Hw are big lsses defined? We define: Difference between frecasts Actual difference f statistical estimate and real utcme Prperties f : scale- and directin-free measure fr identifying the type and the magnitude f a judgmental adjustment. Type f adjustment Value f β XL Oversht β > 3 L Oversht 2 < β 3 Oversht 1 < β 2 Undersht 0 < β < 1 Wrng Directin -1 β < 0 L Wrng Directin β < -1 Big Lsses
Database and measuring accuracy Mnthly sales f SKUs (pharma prducts). [Franses & Legerstee, 2009, IJF] We cnsider the 774 series where the triplet X, MF, EF is available fr all bservatins. X MF EF Average Relative Mean Abslute Errr: [Davydenk & Fildes, 2013, IJF]
Analysis f all judgmental adjustments ARMAE 4.81 1.39 0.35 0.56 1.31 3.63 Sample: 18192 1.02 Only 49% f the adjustments lead t imprvements 25.4% f the adjustments lead t big lsses
Analysis f judgmental adjustments after big lss ARMAE 4.75 1.39 0.34 0.49 1.35 3.74 Sample: 4595 1.14 Less than 44% f the adjustments lead t imprvements 1/3 f the adjustments lead t big lsses
Analysis f judgmental adjustments after big lss After an XL versht After a L wrng directin After a big lss Adjustments after a big lss are mre prbable t be f the same type and/r the same directin.
Crrecting frecasters behaviur Guidance and restrictiveness thrugh FSSs: [Fildes et al., 2009, IJF] Prvide autmated advices that wuld prevent the frecaster frm making adjustments after big lsses. Apply a lck-ut, meaning nt allwing the frecaster t perfrm changes n the statistical frecast after big lsses. Adjusting the adjustments: [Franses & Legerstee, 2011, ESwA] Fr : Damping the judgmental adjustments: the Blattberg-Hch apprach (50% mdel + 50% manager) [1990, MnSc]
Imprving the frecasting perfrmance ARMAE After big lsses After L wrng directin After XL vershts Overall Current Practice 1.14 1.16 1.12 1.02 Guidance* 1.07 1.08 1.06 1.01 Restrictiveness 1.00 1.00 1.00 0.99 Blattberg-Hch 0.97 0.98 0.94 0.98 * assuming that in 50% f the cases the adjustment was prevented. The Blattberg-Hch apprach wrks fr almst 2/3 f the cases after big lsses, a percentage which is higher than the general case. Accuracy imprvements f up t 16% after big lsses.
Cnclusins We examined the behaviur f frecasters after big lsses : The prbability f perfrming an adjustment that leads t a big lss increases by 29%. At the same time the prbability f making an adjustment in the same directin is even higher, giving supprt t the gambler s fallacy thery. Simple crrectin strategies can be applied t imprve the frecasting perfrmance: imprvement f up t 16% fr the perids after big lsses. Finally, we defined a new measure t identify the type and magnitude f a judgmental interventin.
Next steps Further explre the differences between psitive and negative adjustments (rule-based crrectins n the adjustments?). Explre mre sphisticated strategies fr adjusting the adjustments: Use errr btstrap rules. [Fildes et al., 2009, IJF] Crrelate the applied weights with experts experience and/r behaviur ver multiple lags. [Franses & Legerstee, 2011, ESwA] Examine if autmatically adjusting the adjustments leads the frecasters t change their behaviur in perfrming judgmental interventins. Explre experts behaviur after big wins.
Thank yu fr yur attentin! f.petrpuls@lancaster.ac.uk
bringing researchers and frecasters tgether w w w.frsc.net