Big Data and Macroeconomic Nowcasting

Transcription

1 Big Data and Macroeconomic Nowcasting George Kapetanios, Massimiliano Marcellino & Fotis Papailias King s College London, UK Bocconi University, IT Queen s Management School, UK Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

2 Outline Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

3 Outline Introduction Types of Big Data Pros and Cons of Big Data for Macroeconomic Nowcasting Data & Methodological Issues Part 1: Literature Review Existing literature on the use of Big Data Existing literature on Macroeconomic Nowcasting What about their combination? Part 2: Big Data Modelling Machine Learning Heuristic Optimisation Dimensionality Reduction Forecast Combination & Model Averaging Mixed Frequency Methods for Big Data Big Data Access, Cleaning and Preparation Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

4 Outline Part 3: Empirical Analysis Data Description Nowcasting & Short-Term Forecasting Evaluation Conclusions Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

5 Introduction Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

6 Introduction The recent crisis has emphasized the importance for policy-makers and economic agents of a real-time assessment of the current state of the economy and its expected developments, when a large but incomplete information set is available. The main obstacle is the delay with which key macroeconomic indicators such as GDP and its components, but also fiscal variables, regional/sectoral indicators and disaggregate data, are released. Example: GDP data are only available on a quarterly basis and the advance/flash estimate is only published with a 4-6 week delay or longer, depending on the country. Moreover, preliminary data are often revised afterwards, in particular around turning points of the business cycle. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

7 Introduction On the other hand, a large and growing number of more timely leading and coincident indicators is available, at a monthly, daily or even higher frequency, based in particular on financial, survey and internet data, though sometimes subject to short samples, missing observations and other data irregularities. This has stimulated a vast amount of statistical and econometric research on how to exploit the large, timely and higher frequency but irregular information to provide estimates for key low frequency economic indicators. A parallel, more empirical, literature has instead focused specifically on the use of big data for nowcasting economic indicators, often using rather simple econometric techniques and specific big data sources, mainly Google Trends. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

8 Introduction Finally, a more theoretical literature has developed new, or adapted old, statistical and econometric methods to handle very large sets of explanatory variables, such as those associated with big data. Broadly speaking, they are based either on summarizing the many variables, or on selecting them, or on combining many small models, after a proper data pre-treatment. Some techniques are borrowed from machine learning, where prediction is of key interest but data are typically assumed to be i.i.d. rather than serially correlated and possibly with changing variances over time, so that these techniques have to be properly extended prior to use on economic data. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

9 Introduction: Big Data Types The literature provides various definitions of big data. Part of the problem is that the meaning of big data can differ significantly across disciplines. One possibility to obtain a general classification is to adopt the 4 Vs classification, originated by the IBM, which relates to: (i) Volume (Scale of data), (ii) Velocity (Analysis of streaming data), (iii) Variety (Different forms of data) and (iv) Veracity (Uncertainty of data). However, this classification seems too general to guide empirical nowcasting applications. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

10 Introduction: Big Data Types A second option is to focus on numerical data only, which can either be the original big data or the result of a transformation of unstructured data, and refer to the size of the dataset. Unstructured data such as, e.g., credit card transactions or other data that describe the disaggregated actions or characteristics of many agents, are likely to need to be transformed to a two dimensional panel structure where time is usually one dimension. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

11 Introduction: Big Data Types Following, e.g. Doornik and Hendry (2015), we can distinguish three main types of big data: 1 Tall 2 Fat 3 Huge Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

12 Introduction: Big Data Types Tall Tall : not so many variables, N, but many observations, T, with T N. This is for example the case with tick by tick data on selected financial transactions or search queries. In this case T is indeed very large in the original time scale, say seconds, but it should be considered whether it is also large enough in the time scale of the target macroeconomic variable of the nowcasting exercise, say quarters. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

13 Introduction: Big Data Types Tall Tall datasets at very high frequency are not easily obtained, as they are generally owned by private companies. Moreover, they generally require substantial pre-treatment, as indicators typically present particular temporal structures (related, e.g., to market micro-structure) and other types of irregularities, such as outliers, jumps and missing observations. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

14 Introduction: Big Data Types Fat Fat : many variables, but not so many observations, N T. Large cross-sectional databases fall into this category, which is not so interesting from an economic nowcasting point of view, unless either T is also large enough or the variables are homogeneous enough to allow proper model estimation (e.g., by means of panel methods) and nowcast evaluation. However, Fat datasets can be of interest in many other applications of big data, both inside official statistics, e.g., for surveys construction, and outside, e.g., in marketing or medical studies. Also, as the collection of big data started only rather recently, Fat datasets are perhaps the most commonly available type. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

15 Introduction: Big Data Types Fat Actually, statistical methods for big data are mainly meant for Fat datasets, e.g., those developed in the machine learning literature, as they only require a large cross-section of i.i.d. variables. When a (limited) temporal dimension is also present, panel estimation methods are typically adopted in the economic literature, but factor based methods can be also applied. Classical estimation methods are not so suited, as their finite (T) sample properties are generally hardly known, while Bayesian estimation seems more promising, as it can easily handle a fixed T sample size and, with proper priors, also a large cross-sectional dimension. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

16 Introduction: Big Data Types Huge Huge : many variables and many observations, i.e., very large N and T. This is perhaps the most interesting type of data in a nowcasting context even if, unfortunately, it is not so often available. Big data collection started only recently, while collection of the target economic indicators started long ago, generally as far back as the 1950s or 1960s for many developed countries. Google Trends, publicly available summaries of a huge number of specific search queries in Google, are perhaps the best example in this category, and not by chance the most commonly used indicators in economic nowcasting exercises. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

17 Introduction: Big Data Types Huge Contrary to basic econometrics and statistics, in Huge datasets both T and N diverge, and proper techniques must be adopted to take this feature into account at the level of model specification, estimation and evaluation. For example, in principle it is still possible to consider information criteria such as BIC or AIC for model specification (indicator selection for the target variable in the nowcasting equation), although it is the case that modifications may be needed to account for the fact that N is comparable or larger than T, as opposed to much smaller as assumed in the derivations of information criteria. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

18 Introduction: Big Data Types Huge Further, in the case of a linear regression model with N regressors, 2 N alternative models should be compared, which is not computationally feasible when N is very large, so that efficient algorithms that only search specific subsets of the 2 N possible models have been developed. Moreover, the standard properties of the OLS estimator in the regression model are derived assuming that N is fixed and (much) smaller than T. Some properties are preserved, under certain conditions, also when N diverges but empirically the OLS estimator does not perform well due to collinearity problems that require a proper regularization of the second moment matrix of the regressors. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

19 Introduction: Big Data Types Huge This is in turn relevant for nowcasting, as the parameter estimators are used to construct the nowcast (or forecast). As a result of these problems for OLS, a number of regularisation, or penalisation methods have been suggested. An early approach, referred to as Ridge regression, uses shrinkage to ensure a well behaved regressor sample second moment matrix. More recently, other penalisation methods have been developed. A prominent example is LASSO where a penalty is added to the OLS objective function in the form of the sum of the absolute values of the coefficients. Many related penalisation methods have since been proposed and analysed. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

20 Introduction: Big Data Types Huge As an alternative to variable selection, the indicators could be summarized by means of principal components (or estimated factors) or related methods such as dynamic principal components or partial least squares. However, standard principal component analysis is also problematic when N gets very large, but fixes are available, such as the use of sparse principal component analysis. Finally, rather than selecting or summarizing the indicators, they could be all inserted in the nowcasting regression but imposing tight priors on their associated coefficients, which leads to specific Bayesian estimators. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

21 Introduction: Big Data Pros/Cons In a nowcasting context, we think of big data as complements rather than substitutes for more common coincident and leading indicators. Some of the problems discussed for internet based big data also apply to large datasets of conventional indicators. For example, collecting disaggregated macroeconomic and financial variables for an EU country easily leads to a few hundred indicators, and multiplying that for all the EU countries, e.g. in order to conduct a comparative analysis, leads to thousands of variables to be considered in formal econometric models, which can be hardly done with standard techniques. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

22 Introduction: Big Data Issues Data Availability. Most data pass through private providers and are related to personal aspects. Hence, continuity of data provision could not be guaranteed. For example, Google could stop providing Google Trends, or at least no longer make them available for free. Or online retail stores could forbid access to their websites to crawlers for automatic price collection. Or individuals could extend the use of softwares that prevent tracking their internet activities, or tracking could be more tightly regulated by law for privacy reasons. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

23 Introduction: Big Data Issues Digital Divide. The fact that a sizable fraction of the population still has no or limited internet access. This implies that the available data are subject to a sample selection bias, and this can matter for their use. Suppose, for example, that we want to nowcast unemployment at a disaggregate level, either by age or by regions. Internet data relative to older people or people resident in poorer regions could lead to underestimation of their unemployment level, as they have relatively little access to internet based search tools. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

24 Introduction: Big Data Issues Changing Size & Quality. A third issue is that both the size and the quality of internet data keeps changing over time, in general much faster than for standard data collection. For example, applications such as Twitter or WhatsApp were not available just a few years ago, and the number of their users increased exponentially, in particular in the first period after their introduction. Similarly, other applications can be gradually dismissed or used for different uses. For example, the fraction of goods sold by EBay through proper auctions is progressively declining over time, being replaced by other price formation mechanisms. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

25 Introduction: Big Data Issues Bias in Answers. Again more relevant for digital than standard data collection, is that individuals or businesses could not report truthfully their experiences, assessments and opinions. For example, some newspapers and other sites conduct online surveys about the feelings of their readers (happy, tired, angry, etc.) and one could think of using them, for example, to predict election outcomes, as a large fraction of happy people should be good for the ruling political party. But, if respondents are biased, the prediction could be also biased, and a large fraction of non-respondents could lead to substantial uncertainty. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

26 Introduction: Big Data Issues Data Format. A fifth issue is that data could not be available in a numerical format, or not in a directly usable numerical format. A similar issue emerges with standard surveys, for example on economic conditions, where discrete answers from a large number of respondents have to be somewhat summarized and transformed into a continuous index. However, the problem is more common and relevant with internet data. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

27 Introduction: Big Data Issues Irregularities. A final issue, again common also with standard data but more pervasive in internet data due to their high sampling frequency and broad collection set, relates to data irregularities: outliers, working days effects, missing observations, presence of seasonal / periodic patterns, etc. all of which require properly de-noising and smoothing the data. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

28 Introduction: Big Data Advantages Big data provide potentially relevant complementary information with respect to standard data, being based on rather different information sets. Moreover, they are timely available and, generally, they are not subject to subsequent revisions, all relevant features for potential coincident and leading indicators of economic activity. Finally, they could be helpful to provide a more granular perspective on the indicator of interest, both in the temporal and in the cross-sectional dimensions. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

29 Introduction: Big Data Advantages In the temporal dimension, they can used to update nowcasts at a given frequency, such as weekly or even daily, so that the policy and decision makers can promptly update their actions according to the new and more precise estimates. In the cross-sectional dimension, big data could provide relevant information on units, such as regions or sectors, not fully covered by traditional coincident and leading indicators. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

30 Introduction: Summary First, do we get any relevant insights? nowcast precision by using big data? In other words, can we improve Second, do we get a big data hubris? Again as anticipated, we think of big data based indicators as complements to existing soft and hard data-based indicators, and therefore we do not get a big data hubris. Third, do we risk false positives? Namely, can we get some big data based indicators that nowcast well just due to data snooping? Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

31 Introduction: Issues Summary Fourth, do we mistake correlations for causes? Fifth, do we use the proper econometric methods? Sixth, do we have instability due to Algorithm Dynamics or other causes (e.g., the financial crisis, more general institutional changes, the increasing use of internet, discontinuity in data provision, etc.)? Finally, do we allow for variable and model uncertainty? Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

32 Part 1: Literature Review Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

33 Literature Review We review the most important research papers on big data in three distinct areas: Big Data in macroeconomics, Variable selection and dimensional reduction for big data in macroeconomics, Nowcasting in macroeconomics. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

34 Literature Review The aim of the review is to try to answer the following questions: 1 What are possible big data sources in relation to the list of Eurostat s Unit C1 macroeconomic indicators (i.e. GDP, inflation and producer prices, employment and unemployment, industrial production index and retail trade deflated turnover)? 2 What are the advantages and disadvantages of the previously analysed sources? 3 What are the main types of statistical methods used in the big data in macroeconomics literature? 4 What are the possible gains generated either by the use of big data or new statistical methods or both in comparison with existing practices in the field of nowcasting? Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

35 Literature Review Based on the surveyed papers, we can say that the use of Google Trends has been dominant in studies using big data in macroeconomics. There exist some papers based on Twitter data, also reviewed, but they are mainly in finance. Webscrapping and collection of online prices also offer some potential, especially for nowcasting inflation. However, such datasets are very difficult to obtain (and possibly sustain), even more so when many countries and long enough samples are required. A similar comment applies for credit card and financial transactions data, and for data summaries resulting from textual analysis. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

36 Literature Review From the literature it also emerges that the advantages of using data like Google Trends are: more timely forecasts, not subject to data revision, some improvements in forecast accuracy, even though these typically emerge with respect to simple benchmarks (AR models), ease of data access and collection, ease of data management and treatment, expected good data quality, reasonable likelihood that similar data, will be available on a continuous basis and without major definitional changes. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

37 Literature Review There also some disadvantages when using this data source, the main ones being: a typical sole use of such data, which can lead to biased results ( big data hubris ), the impossibility to access the raw data, and the lack of knowledge of the precise algorithms used to pre-treat and summarize them, the possibility that free access will be discontinued by the (private) data provider, or limited due to the introduction of more stringent privacy laws. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

38 Literature Review Varian (2014) provides an intuitive introduction in big data management and manipulation. Big data, possibly after transformation to some kind of numerical format, has to be stored in some sort of database, as it is difficult to be dealt with spreadsheets. A medium sized dataset (i.e. about a million observations) could be stored and manipulated using a relational database, such as MySQL, whereas a dataset of several million observations could be efficiently stored and manipulated by NoSQL databases. Sometimes, and depending on the nature of the research, a carefully selected subsample or summary of the data might be sufficient for analysis. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

39 Literature Review Hence, big data creation, storage and management typically require specific IT skills, software and hardware. The associated costs should be kept into consideration when assessing the potential benefits of big data for nowcasting. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

40 Literature Review In terms of statistics and econometrics, data analysis is typically broken down into four categories: pre-treatment and summarisation, estimation, hypothesis testing and prediction. Since a large amount of data is available, penalised regressions such as LASSO, LARS, and elastic nets can be used instead of the standard linear or logistic regression. These techniques could also be used for variable selection. Then, the choice of the final model should come from forecasting crossvalidation so that the researcher makes sure the model has good out-ofsample predictive ability. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

41 Part 1: Literature Review Main Findings Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

42 Literature Review: Main Findings The purpose of the literature review is to answer four questions: 1 What are possible big data sources in relation to the list of Eurostat s Unit C1 macroeconomic indicators? 2 What are the advantages and disadvantages for each of the previously analysed sources? 3 What are the main types of statistical methods used in the big data in macroeconomics literature? 4 What are the possible gains generated either by the use of big data or new statistical methods or both in comparison with existing practices in the field of nowcasting? Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

43 Literature Review: Main Findings As anticipated in the Introduction, after a careful examination of the most important papers in each area we can say that the majority of big data papers are based on Google Trends as predictors. The advantages of using data like Google Trends include: a the improved timeliness of the forecasts without need for data revision, b the potential improvement of forecasts, c open access to the data, d easy data handling, e good data quality, f reasonable possibility that this sort of data will be released on a continuous basis. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

44 Literature Review: Main Findings However, one must be cautious when using data of this type as it is often associated with the following issues: a the use of Google data as the only data input could lead to biased results (commonly known as big data hubris ), b the restrictions to access the raw data but only the Google index, c the possibility that free access will be discontinued. Therefore, we suggest the use of Google Trends for nowcasting the macroeconomic variables of interest for Eurostat. We strongly believe that such data must be used as a supplement to current forecasting tools and not as a substitute. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

45 Literature Review: Main Findings Regarding questions (3) and (4), it turns out that most of the papers in the literature generate nowcasts based on mixed frequency versions of linear regressions, VARs, (dynamic) factor models or a combination of them, and adopt various strategies for variable selection in the presence of a large set of potential regressors. While no clear-cut ranking of the alternative methodologies emerge, there seems to be consensus about the usefulness of big data for nowcasting variables such as unemployment, GDP, inflation and surveys, even though the gains are often computed with respect to (too) simple benchmarks. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

46 Part 2: Big Data Modelling Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

47 Big Data Modelling Let y t, t = 1,..., T, be the target variable and x t = (x 1t,..., x Nt ) be a set of potential predictors, with N very large. We do not assume a particular data generating process for y t but simply posit the existence of a representation of the form which implies that E(u t x 1t,..., x Nt ) = 0. y t = a + g(x 1t,..., x Nt ) + u t, (1) While the potential nonlinearity in (1) might, in principle, be worth exploring, it is extremely difficult to model nonlinearities in the context of large datasets and no work is available on this in the big data literature. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

48 Big Data Modelling As a result, we consider an approximating linear representation of the form, y t = a + N β i x it + u t, (2) i=1 with u t denoting a martingale difference process and where the set of x it s can also contain products of the original indicators in order to provide a better approximation to (1). Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

49 Big Data Modelling Our main aim is to provide estimates for current and future values of y t, where either no, or only a preliminary, value for y t is available from official statistics. To do so, we can rely on many approaches, which can be categorised in three main strands. The first strand aims to provide estimates for β = (β 1,..., β N ). While ordinary least squares (OLS) is the benchmark method for doing so, it is clear that if N is large this is not optimal or even feasible (when N > T ). Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

50 Big Data Modelling So other methods need to be used. We consider two classes of methods. The first one is sparse regression, with origins in the machine learning literature. A main aim there is to stabilise the variability of the estimated β i. The second class considers the use of a variety of information criteria such as AIC or BIC to select a smaller subset of all the available predictors. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

51 Big Data Modelling The second strand consists of reducing the dimension of x t by producing a much smaller set of generated regressors, which can then be used to produce nowcasts and forecasts in standard ways. The third strand suggests the use of a (possibly very large) set of small models, one for each available indicator or small subset of them, and then the combination of the resulting many nowcasts or forecasts. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

52 Part 2: Big Data Modelling Main Findings Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

53 Big Data Modelling: Main Findings We feel that Multiple Testing (related to Boosting) and, potentially, some variant of LASSO could be the preferred approaches among the set of machine learning techniques. Among the data reduction techniques, PCA and, possibly PLS are promising. And it would be also worthwhile experimenting with Bayesian regression, with substantial shrinkage, and forecast combination, with simple equal weighting. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

54 Big Data Modelling: Main Findings Finally, all these approaches should be also modified to take into account the possibility of a different timing for the target and the indicator variables. We have surveyed a number of alternative methods to handle mixed frequencies, and it turns out that Unrestricted MIDAS or bridge modelling appear as the most promising approaches, as they preserve linearity and do not add an additional layer of computational complexity. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

55 Part 3: Empirical Analysis Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

56 Empirical Analysis: Introduction We consider nowcasting and short-term (one- to twelve- months ahead) forecasting three key macroeconomic variables: 1 inflation (measured by the growth rate in the Consumer Price Index), 2 growth in retail sales (measured by the growth rate of the Retail Trade Index), and 3 the Unemployment Rate. The exercise is conducted recursively in a pseudo out of sample framework, using monthly data for three economies: Germany (DE), Italy (IT) and the UK. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

57 Empirical Analysis: Introduction We assess the relative performance of: Big Data (proxied via weekly Google Trends) and standard indicators (based on a large set of weekly and monthly economic and financial variables) In fact, as we have mentioned several times, we think of Big Data as providing complementary information, and we wish to assess how useful it is in a forecasting context relative to standard indicators. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

58 Empirical Analysis: Introduction We also evaluate the role of several econometric methods and alternative specifications for each of them (with or without big data), also capable of handling the frequency mismatch in our data. Specifically, we consider: Naive and autoregressive (AR) models as benchmarks Dynamic Factor Analysis (DFA) Partial Least Squares (PLS) Bayesian Regression (BR) and LASSO regression. DFA and PLS are representatives of data reduction methods; BR and LASSO are representatives of, respectively, econometric and machine learning techniques. In addition, we also consider model averaging. Overall, we have a total of 255 models and model combinations. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

59 Empirical Analysis: Introduction In general, we find that factors extracted using a combination of standard macroeconomic and Big Data predictors lead to a substantial improvement in the nowcasting and forecasting performance for all variables across the three economies, though the gains are small for unemployment, which is a very persistent variable over our evaluation sample which makes beating a simple AR model difficult, in particular at long horizons. Furthermore, a data-driven automated model selection strategy, where the forecasts from a set of best performing models over the recent past are pooled, performs particularly well, with Big Data present in about 45% of the pooled models. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

60 Empirical Analysis: Conclusions In general, we find the DFA and BR often provide accurate results with and without Google Trends. Most of the suggested models and model averages perform better than the benchmark when nowcasting/forecasting the CPI and RTI, however they fail using the unemployment rate. This is due to structure of the unemployment rate series which is slowly moving in our evaluation months. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

61 Empirical Analysis: Conclusions A longer evaluation period could reverse this finding, and provide more reliable results, but unfortunately we are constrained by the availability of Google Trends, which start in Our suggested data-driven automated strategy, which rotates models based on their forecasting performance in the past 1, 6 and 12 months, seems to work extremely well for CPI and RTI, and decently for unemployment, and therefore it provides a powerful tool for applied forecasting. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

62 Empirical Analysis: Conclusions The detailed analysis of model rotation shows that, on average, 45% of the times the chosen best models for averaging, for the CPI and RTI target variables across all three economies, include Google Trends. Hence, we conclude that Big Data, as proxied via Google Trends in our applications and combined with standard macroeconomic and financial indicators, can indeed improve the nowcasting and short term forecasting performance of econometric models. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

63 Conclusions Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

64 Overall Recommendations Overall, our suggestion is to take a pragmatic approach that balances potential gains and costs from the use of Big Data for nowcasting macroeconomic indicators, in addition to standard indicators. A preliminary step should be an a priori assessment of the potential usefulness of Big Data for a specific indicator of interest, such as GDP growth, inflation or unemployment. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

65 Overall Recommendations This requires to evaluate the quality of the existing nowcasts and whether any identified problems, such as bias or inefficiency or large errors in specific periods, can be fixed by adding information as potentially available in Big Data based indicators. Similarly, it should be considered whether these additional indicators could improve the timeliness, frequency of release and extent of revision of the nowcasts. Relevant information can be gathered by looking at existing empirical studies focusing on similar variables or countries, and in this respect the extensive literature review we presented can be quite helpful. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

66 Overall Recommendations Once Big Data passes the need check in the preliminary step, the first proper step of the Big Data based nowcasting exercise is a careful search for the specific Big Data to be collected. As we have seen, there are many potential providers, which can be grouped into Social Networks, Traditional Business Systems, and the Internet of Things. Naturally, it is not possible to give general guidelines on a preferred data source, as its choice is heavily dependent on the target indicator of the nowcasting exercise. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

67 Overall Recommendations Having identified the preferred source of Big Data, the second step requires to assess the availability and quality of the data. A relevant issue is whether direct data collection is needed, which can be very costly, or a provider makes the data available. In case a provider is available, its reliability (and cost) should be assessed, together with the availability of meta data, the likelihood that continuity of data provision is guaranteed, and the possibility of customization (e.g., make the data available at higher frequency, with a particular disaggregation, for a longer sample, etc.). Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

68 Overall Recommendations All these aspects are particularly relevant in the context of applications in official statistical offices. As the specific goal is nowcasting, it should be also carefully checked that the temporal dimension of the Big Data is long and homogeneous enough to allow for proper model estimation and evaluation of the resulting nowcasts. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

69 Overall Recommendations The third step analyzes specific features of the collected Big Data. A first issue that is sometimes neglected is the amount of the required storage space and the associated need of specific hardware and software for storing and handling the Big Data. A second issue is the type of the Big Data, as it is often unstructured and may require a transformation into cross-sectional or time series observations. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

70 Overall Recommendations Even when already available in numerical format, pre-treatment of the Big Data is often needed to remove deterministic patterns and deal with data irregularities, such as outliers and missing observations. While standard methods can be usually applied, the size of the datasets suggests to resort to robust and computationally simple approaches, applied variable by variable. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

71 Overall Recommendations The fourth step requires to assess the presence of a possible bias in the answers provided by the Big Data, due to the digital divide or the tendency of individuals and businesses not to report truthfully their experiences, assessments and opinions. A related problem, particularly relevant for nowcasting, is the possible instability of the relationship with the target variable. This is a common problem also with standard indicators, as the type and size of economic shocks that hit the economy vary over time. Both issues can be however tackled at the modelling and evaluation stages. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

72 Overall Recommendations The fifth step when nowcasting with Big Data requires to select the proper econometric technique. Here, it is important to be systematic about the correspondence between the nature of the Big Data setting and use under investigation and the method that is used. There is a number of dimensions along which we wish to differentiate. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

73 Overall Recommendations 1 The first choice is between the use of methods suited for large but not huge datasets, and therefore applied to summaries of the Big Data (such as Google Trends, commonly used in nowcasting applications), or of techniques specifically designed for Big Data. For example, nowcasting with large datasets can be based on factor models, large BVARs, or shrinkage regressions. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

74 Overall Recommendations 1 Huge datasets can be handled by sparse principal components, linear models combined with heuristic optimization,or a variety of machine learning methods (which, though, are generally developed assuming i.i.d. variables). It is difficult to provide an a priori ranking of all these techniques and there are few empirical comparisons and even fewer in a nowcasting context, so that is may be appropriate to apply and compare a few of them for nowcasting the specific indicator of interest. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

75 Overall Recommendations 2 A second dimension is the frequency of the available data. If this frequency is mixed then specific techniques for mixed frequency data become relevant. Chief among them is unrestricted MIDAS which provides a very flexible framework of analysis and can be adapted to work together with most if not all Big Data methods be they machine learning of econometric. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

76 Overall Recommendations 3 Yet another dimension relates to the purpose for which large datasets are considered. Possibilities include model or indicator selection, forecasting or a more structural analysis. In this case of course each purpose is best served by different methods and the choice of method crucially depends on the purpose. Most methods can be used for forecasting and so the choice has to be case dependent. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

77 Bayesian VAR models stand out as an appropriate method in this context. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79 Overall Recommendations 3 We recommend that as many methods are possible are evaluated in a forecasting context although past experience suggests that factor analysis and shrinkage methods can be of great use. For model or indicator selection penalised regression and the MT methods seem to be appropriate and also have been reported to have good potential. Finally, for more structural analysis it is clear that it is likely that huge datasets are more difficult to accommodate. In this case, system methods that analyse the whole or a large proportion of the available data simultaneously, seem necessary for a satisfactory analytical outcome.

78 Overall Recommendations The final step consists of a critical and comprehensive assessment of the contribution of Big Data for nowcasting the indicator of interest. In order to avoid, or at least reduce the extent of, data and model snooping, a cross-validation approach should be followed, whereby various models and indicators are estimated over a first sample and they are selected and/or pooled according to their performance, but then the performance of the preferred approaches is re-evaluated over a second sample. This procedure, which we have implemented in the empirical evaluation, provides a reliable assessment of the gains in terms of enhanced nowcasting performance from the use of Big Data. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79

79 Overall Recommendations To conclude, we are very confident that Big Data are precious also in a nowcasting context, not only to reduce the errors but also to improve the timeliness, frequency of release and extent of revision. We hope that the approach we have developed in this project will be useful for many users. Kapetanios, Marcellino, Papailias Big Data & Nowcasting April 7th / 79