Learning to Forecast with Genetic Algorithms Mikhail Anufriev 1 Cars Hommes 2,3 Tomasz Makarewicz 2,3 1 EDG, University of Technology, Sydney 2 CeNDEF, University of Amsterdam 3 Tinbergen Institute Computation in Economics and Finance Taipei 21 June 215
Computational Markov model explaining experimental data: Agents use Genetic Algorithms to optimize a linear forecasting rule. Contribution: 1 Alternative to the RE model of expectation formation motivated by the experimental evidence 2 Generalization of the existing theoretical model, Heuristic Switching Model (Anufriev, Hommes, 212, AEJ-Micro); 3 Realism: heterogeneous agents using individual learning; 4 Model replicates both aggregate and individual characteristics of data from four Learning-to-Forecast experiments. Focus of the presentation: experimental data from Bao et al, 212, JEDC (but also a bit from Heemeijer et al, 29, JEDC).
Price Expectations Price/predictions feedback cornerstone of any dynamic model. Classical framework: as-if perfect rationality leads to rational expectations (RE), i.e., model-consistent predictions. Problems: (i) model is silent on how the agents get RE; (ii) data do not confirm RE hypothesis and its implications Surveys: housing market (Case, Schiller, Thompson, 212); inflation expectations of consumers (Malmendier and Nagel, 29) and professional forecasters (Nunes, 21). Financial market examples: bubbles and crashes (dot-com bubble of 1997 2; US bear market of 27 9). Experiments: macro-level bubbles in Smith et al. experiments; micro-level expectations in Learning-to-Forecast experiments (Hommes, 211).
Agenda A mechanism of bubbles: self-fulfilling trend-following type of price expectations (Brock, Hommes, 1997, 1998) LtF Experiments find individual evidence in favor of this mechanism in the market with positive feedback (Hommes et al, 25) Heuristic Switching Model: agents switch between simple forecasting heuristics like adaptive vs. trend following expectations. HSM fits well the experimental aggregate data for different types of markets (Anufriev, Hommes 212; Anufriev et al, 213). Problem: heuristics in HSM should be specified, but how and how many? Genetic Algorithms (Hommes, Lux, 213) This Paper: micro-foundations for HSM; model of endogenous learning.
Learning-to-Forecast General structure of a LtF experiment Focus on a market of a specific commodity, e.g., financial asset (demand-driven market with positive feedback) or agricultural good (supply-driven market with negative feedback). Subjects play a role of price forecasters to computer agents. The submit forecasts during 5 periods. Computer agents trade rationally given the submitted forecasts. Price is determined from the market clearing condition. Feedback between subject price predictions and prices through optimal demand/supply decisions. Subjects are rewarded for their forecasting accuracy.
Learning-to-Forecast Computer Screen earnings per period: ( ) e t,h = max 1 1 49 (p t pt,h e )2, 1 2 euro
Learning-to-Forecast GA is applied to four experiments 1 Heemeijer et al. (29, JEDC): simple linear framework. Negative feedback Positive feedback 12 1 8 6 4 2 12 1 8 6 4 2 p t = 6 2 ( 21 p e t 6 ) + ε t p t = 6 + 2 ( 21 p e t 6 ) + ε t 2 Bao et al. (212, JEDC): large breaks in fundamental price. 3 vd Velden (21) and Hommes et al. (27, MD): nonlinear cobweb producers economy. 4 Hommes et al. (25, RFS): two-period ahead nonlinear asset pricing economy.
Learning-to-Forecast Experimental outcome Producer market (negative feedback between forecasts and prices) quickly converges to RE. Asset pricing market (positive feedback) may converge slowly, but typically oscillates. With non-linear feedback more oscillations and instability is observed. Large between-treatment heterogeneity in forecasting and prediction rules; small within-treatment heterogeneity in forecasts (coordination).
Learning-to-Forecast Experimental outcome Negative feedback Positive feedback 8 8 6 6 Price 4 Price 4 2 1 2 3 4 5 Time 2 1 2 3 4 5 Time Prices in all groups, 6 for negative feedback and 6 for positive feedback (Heemeijer et al., 29).
Learning-to-Forecast Experimental outcome 1 Negative feedback 1 Positive feedback 8 8 6 6 4 4 2 2 1 2 3 4 5 6 1 2 3 4 5 6 Individual predictions (green) and prices (black) from the selected groups (Bao et al, 212).
GA model Individual Prediction Rules Market with I = 6 artificial agents. Agent i uses a simple forecasting heuristic (adaptive expectations + trend extrapolation) to predict p t : p e i,t = α i p t 1 + (1 α i )p e i,t 1 + β i (p t 1 p t 2 ). The rule requires specific parameters, e.g., rule may extrapolate trend stronger or weaker. General constraint: α i [, 1], β i [ 1.1, 1.1]. Active rule is picked from H = 2 specifications p e i,h,t = α i,hp t 1 + (1 α i,h )p e i,t 1 + β i,h (p t 1 p t 2 ).
GA model Individual learning In period t: Each agent has a pool of 2 heuristics, i.e., different (α i,h, β i,h ). Forecast is generated by the active rule taken with probability exp( MSE i,h ) 2 k=1 exp( MSE i,k), MSE i,h = (p e i,h,t 1 p t 1) 2 Between periods t and t + 1: individual learning agents independently update the set of heuristics {(α i,h, β i,h )} with Genetic Algorithms evolutionary operators (Haupt and Haupt, 24): each heuristic {(α i,h, β i,h )} is encoded as a binary chromosome. GA operators: procreation, crossover, mutation, election update the pool of heuristics We use GA, because it is efficient and simple. Heuristic parameters evolve with time as the price series unfolds!
GA model Individual Learning: agent s heuristics update Between periods t and t + 1 agent i updates the pool of heuristics H i,t = {(α i,h, β i,h )} 2 h=1 in order to get the new pool H i,t+1 Procreation Agent samples 2 heuristics from the pool with probabilities based on the logit function of MSE i,h = (p e i,h,t p t) 2 Mutation Each bit reverses its value with probability δ m =.1 Crossover Every pair of heuristics swap their α s entries with probability δ c =.9 Election So generated 2 new heuristics are compared pairwise with the 2 old heuristics composing the previous pool H i,t. New heuristic takes place in the new pool, if its MSE is strictly larger than of the old.
GA model Whole Model Fix the experimental environment (i.e., pricing equation) Initialization Take as many agents as subjects in the experiment. Initialize 2 heuristics per agent randomly Sample the first forecasts randomly from exogenous distribution. Next time draw randomly one heuristic to form forecast. Loop of one iteration 1 Agents observe the price and evaluate heuristics hypothetical MSE. 2 Every agent update own pool of heuristics with GA. 3 Every agent stochastically picks one heuristic from the new pool based on its hypothetical past performance and submit the corresponding price forecast. 4 New price is generated from the individual predictions.
Model fitness Sample experiment and simulation: Heemeijer et al. (29) 1 Negative feedback 1 Positive feedback 8 8 6 6 4 4 2 2 1 2 3 4 5 1 1 2 3 4 5 1 8 8 6 6 4 4 2 2 1 2 3 4 5 1 2 3 4 5
Model fitness Sample experiment and simulation: Bao et al. (212) 1 Negative feedback 1 Positive feedback 8 8 6 6 4 4 2 2 1 2 3 4 5 6 1 1 2 3 4 5 6 1 8 8 6 6 4 4 2 2 1 2 3 4 5 6 1 2 3 4 5 6
Model fitness Empirical analysis 1 Monte Carlo study run the model 1 times with different learning and initial predictions: Does the GA model match the stylized facts from the experiments? 2 5-period ahead simulations take experimental initial predictions and run the model 1 times with different learning: How far is the model from the data in the long-run? 3 One-period ahead simulations take data available to experimental subjects until period t and predict period t + 1. (Sequential Monte Carlo analysis.) How far is the model from the data in the short-run? Result: GA model beats simple homogeneous models, RE and HSM.
Model fitness Measuring prediction error: 5-period ahead MSE Negative feedback Positive feedback MSE for HSTV12 Prices Forecasts Prices Forecasts Trend extrapolation 2736 1289 11.3 113.3 Adaptive 3.629 1.75 55 62.14 Contrarian 6.984 14.45 58.46 65.95 Naive 94.44 11.9 46.62 52.9 RE 13.871 2.923 55.133 6.859 HSM 73.57 87.86 9.8 11.8 GA-S1: β [ 1.1, 1.1] 8.1 21.97 43.49 49.44 GA-S2: β [, 1.1] 6.333 17.39 43.49 49.64
Model fitness Measuring predicting error: 1-period ahead MSE Negative feedback Positive feedback MSE for HSTV12 Prices Forecasts Prices Forecasts Trend extrapolation 114.61 121.329 1.183 2.165 Adaptive 3.689 1.332 3.776 4.618 Contrarian 5.92 12.534 4.737 5.559 Naive 9.979 16.81 2.411 3.286 RE 13.871 2.923 55.133 6.859 HSM 38.39 45.679.9996 2.24 Genetic Algorithm model: Sequential Monte Carlo GA-S1: β [ 1.1, 1.1] 1.247 21.464.342 2.59 GA-S2: β [, 1.1] 4.28 15.267.341 2.36
Individual learning in the model What drives the difference between the treatments? In short: Reinforcement between price-predictions feedback and the realized learning. Agents learn to extrapolate the trend under the positive feedback.
1.5 -.5-1 1 2 3 4 5 6 1.5 -.5-1 1 2 3 4 5 6 Individual learning in the model Monte Carlo results: Bao et al. (212) 1 Negative feedback 1 Positive feedback 8 8 6 6 4 4 2 2 1 2 3 4 5 6 1 2 3 4 5 6 Prices...
Individual learning in the model Monte Carlo results: Bao et al. (212) 1 Negative feedback 1 Positive feedback 8 8 6 6 4 4 2 2 1 2 3 4 5 6 1 1 2 3 4 5 6 1.5.5 -.5 -.5-1 -1 1 2 3 4 5 6 1 2 3 4 5 6 Prices and learned trend extrapolation.
Individual learning in the model Positive feedback: complexity matters Last period: distribution of trend extrapolation learned by agents..3.3.3.2.2.2.1.1.1-1.2 -.9 -.6 -.3.3.6.9 1.2 Linear pos. feedback -1.2 -.9 -.6 -.3.3.6.9 1.2 Linear pos. feedback with shocks to the fundamental -1.2 -.9 -.6 -.3.3.6.9 1.2 2-period ahead non-linear feedback Result: the more difficult feedback, the more trend extrapolation.
Conclusions GA model of individual learning: agents fine-tune forecast heuristics the specific market. The model replicates well how individuals learn to forecast prices in experiments (both stylized facts and individual data). Producers economy is more likely to converge to RE. Asset pricing market: individuals learn to extrapolate price trend, thus reinforcing price oscillations. The more difficult feedback, the more trend chasing. Degree of the learned trend extrapolation positively depends on the complexity of the market itself.
Questions? Comments?
Questions? Comments? Thank you for your attention!
Literature experiments 1 Bao, T., Heemeijer, P., Hommes, C., Sonnemans, J. and Tunistra, J. (212): Individual expectations, limited rationality and aggregate outcomes, Journal of Economic Dynamics and Control, in press. 2 Heemeijer, P., Hommes C., Sonnemans J. and Tuinstra J. (29): Price stability and volatility in markets with positive and negative expectations feedback: An experimental investigation, Journal of Economic Dynamics and Control, 33(5), pp. 152-172, Complexity in Economics and Finance. 3 Hommes, C., Sonnemans, J., Tuinstra, J. and van de Velden, H. (27): Learning in Cobweb Experiments, Macroeconomic Dynamics, 11(Supplement S1), 8-33. 4, (25): Coordination of Expectations in Asset Pricing Experiments, The Review of Financial Studies, 18(3), pp. 955-98. 5 van de Velden, H. (21): An experimental approach to expectation formation in dynamic economic systems, Ph.D. thesis, Tinbergen Institute and Universiteit van Amsterdam.
Literature Genetic Algorithms 1 Arifovic, J. (1996): The Behavior of the Exchange Rate in the Genetic Algorithm and Experimental Economies, Journal of Political Economy 14(3), pp. 51-541 2 Dawid, H. and Kopel, M. (1998): On Economic Applications of the Genetic Algorithm: A Model of the Cobweb Type, Journal of Evolutionary Economics 8, pp. 297-315 3 Haupt, R. and Haupt S. (24): Practical Genetic Algorithms, John Wiley & Sons, Inc., New Jersey, 2nd edn. 4 Hommes, C.H. and Lux, T. (211): Individual expectations and aggregate behavior in learning to forecast experiments, Macroeconomic Dynamics, pp. 1-29 5 Vriend, N. (2): An illustration of the essential difference between individual and social learning, and its consequences for computational analyses, Journal of Economic Dynamics and Control 24(1), pp. 1-19
Appendix
Model Specification
GA parameters Parameter Notation Value Number of agents I 6 Number of heuristics per agent H 2 Allowed α, price weight [α L, α H ] [, 1] Allowed β, trend extrapolation coefficient Specification 1 [β L, β H ] [ 1.1, 1.1] Specification 2 [β L, β H ] [, 1.1] Number of bites per parameter {L 1, L 2 } {2, 2} Mutation rate δ m.1 Crossover rate δ c.9 Fitness measure V ( ) exp( MSE( ))
Fitness measure Consider agent i in period t with heuristic h, i.e., (α i,h,t, β i,h,t ). Hypothetical prediction of this heuristic for the price at period s is p e i,h,t(s) = α i,h,t p s 1 + (1 α i,h,t )p e i,s 1 + β i,h,t (p s 1 p s 2 ) The fitness measure of this heuristic at time t against its hypothetical performance in period s is exp( MSE i,h (t, s)) 2 k=1 exp( MSE i,k(t, s)), MSE i,h(t, s) = (p e i,h,t(s) p s ) 2. To pick an active rule in period t, agent uses MSE i,h (t, t 1), For individual learning between periods t and t + 1, agent uses MSE i,h (t, t), hypothetical performance against the realized price.
GA model Results for the GA model For details, see the paper (no time :( ). For the first three experiments (linear feedback, linear feedback with shocks to the fundamental price, cobweb economy) our GA model works better than any other model (RE, homogeneous expectations, HSM). But for the 25 two-period ahead asset pricing experiment, GA model improves very little over other models.
25 experiment Experiment setup Hommes, Sonnemans, Tuinstra, vd Velden (25): nonlinear two-period ahead asset pricing market. Subjects have to predict a price of an asset in the next period. Standard myopic mean-variance optimizing computer traders, but also robotic fundamental traders. Supply equals demand gives the law of motion: p t = 1 ] [(1 n t ) p t+1 e + n t p f + y + ε t 1 + r where n t = 1 exp ( 1 2 p t 1 p f ) is the share of robotic fundamentalists, y is the dividend, p f is the fundamental price, p e t+1 is the average prediction and p t is the realized price.
25 experiment Results: heterogeneity and learning
GA model and the experiment 5-period ahead simulations MSE Prices Forecasts Trend extr. 178.2 174.9 Adaptive 96.12 145.9 Contrarian 157 146.8 Naive 95.29 144.6 RE 96.328 145.998 GA: β [ 1.1, 1.1] 13.9 155.8 GA: β [, 1.1] 114.9 169.1
GA model and the experiment One-period ahead simulations MSE Prices Forecasts Trend extr. 17.4527 55.898 Adaptive 44.125 25.3157 Contrarian 59.395 3.8646 Naive 31.6864 2.8416 RE 96.328 145.998 GA: β [ 1.1, 1.1] 19.794 39.226 GA: β [, 1.1] 17.899 39.256
GA model and the experiment Subject 1, group 8, real data vs. APF (GA model)
Multiple equilibria 2 periods
Multiple equilibria 2 periods: trend coefficient
Multiple equilibria Coordination measure
Multiple equilibria Coordination measure Stylized facts: 1 Negative feedback: slow coordination of the subjects around the fundamental value; 2 Positive feedback: fast coordination of the subjects far from the fundamental value. To measure the degree of coordination between experimental subjects/ga agents at period t, take the standard deviation of their predictions from that period: [ 6 ].5 ( σ t p e i,t pt e ) 2 (1) i=1 where pt e is the average prediction at t 6 pt e = pi,t. e (2) i=1
Multiple equilibria Chosen heuristics
Multiple equilibria 29 experiment: price weight 1.8.6.4.2 Negative feedback 1 2 3 4 5 1.8.6.4.2 Positive feedback 1 2 3 4 5
Multiple equilibria 29 experiment: trend extrapolation 1.5 -.5-1 Negative feedback 1 2 3 4 5 1.5 -.5-1 Positive feedback 1 2 3 4 5
Multiple equilibria 212 experiment: price weight 1.8.6.4.2 Negative feedback 1 2 3 4 5 6 1.8.6.4.2 Positive feedback 1 2 3 4 5 6
Multiple equilibria 212 experiment: trend extrapolation 1.5 -.5-1 Negative feedback 1 2 3 4 5 6 1.5 -.5-1 Positive feedback 1 2 3 4 5 6
Multiple equilibria Robustness: trend extrapolation
Multiple equilibria 29 experiment: β [.5,.5] 1 8 Negative feedback 6 4 2 1 2 3 4 5 1 8 Positive feedback 6 4 2 1 2 3 4 5
Multiple equilibria 29 experiment: β [ 1.5, 1.5] 1 8 Negative feedback 6 4 2 1 2 3 4 5 1 8 Positive feedback 6 4 2 1 2 3 4 5
Multiple equilibria Initial predictions
Multiple equilibria Initial predictions first period No learning is possible in the first period. Following Diks, C. and Makarewicz, T. (212): Initial predictions in the Learning-to-Forecast experiment, Lecture Notes in Economics and Mathematical Systems, forthcoming, we sample initial predictions from calibrated distributions ε 1 i U(9.546, 5) with probability.45739, pi,1 e = 5 with probability.3379, (3) ε 2 i U(5, 62.793) with probability.23882 for the 29 experiment and for the 212 experiment: ε 1 i U(16.46, 5) with probability.32296, pi,1 e = 5 with probability.35159, ε 2 i U(5, 7.312) with probability.32296. (4)
Multiple equilibria Initial predictions second period In the second period, learning is already possible (first price is already observed), but no trend can be visible. We assume that the GA agents use the first order heuristic already in the second period. This requires hypothetical (counter-factual) price zero p. Calibration of the model: the best results are obtained under the assumption that the GA agents behave as-if p = p 1 p 1 p = (5) Interpretation: agents initially disregard the trend, since it cannot be observed in the first place.