Module 1. Classical vs machine learning econometrics

Transcription

1 Module 1. Classical vs machine learning econometrics THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION

2 Overview of classical econometrics: What is econometrics Multiple regression Time series Econometric methods in Official statistics Models of inference 2

3 1. What is econometrics

4 Overview of classical econometrics What is econometrics Econometric is an application of statistics and mathematics aimed at identifying and quantifying the relationship between two sets of variables The predicted variables The predictor variables Y = β 0 + β 1 X β k X k + ɛ 4

5 Overview of classical econometrics What is econometrics Aspects: Uncertainty regarding an outcome Relationships suggested by (economic) theory Assumptions and hypotheses to be specified Sampling process including functional form Obtaining data for the analysis Estimation rule with good statistical properties Fit and test model using software package Analyse and evaluate implications of the results Problems suggest approaches for further research 5

6 Overview of classical econometrics What is econometrics Examples of econometrics models: Demand and supply Models Production Functions Cost Functions Etc. 6

7 Overview of classical econometrics What is econometrics Demand model ln y t d = β 1 + β 2 ln x t + ε t Quantity demanded price Supply model ln y t s Quantity supplied = β 1 + β 2 ln x t + ε t price 7

8 Overview of classical econometrics What is econometrics Production function ln y t = β 1 + β 2 ln x t + ε t output input Cobb-Douglas production function 8

9 Overview of classical econometrics What is econometrics Cost function y t = β 1 + β 2 x t 2 + ε t Total cost output 9

10 Overview of classical econometrics What is econometrics There are also non-lineal models: y = β 1 α β 2x e u And models that can be linearises: y = β 1 x β 2 e u ln y = ln β 1 + β 2 lnx + u= α + β 2 ln x + u 10

11 2. The multiple linear regression model

12 Overview of classical econometrics The multiple linear regression model slopes Error term Y = β 0 + β 1 X β k X k + ɛ Predicted variable, dependent variable intercept Predictor variables, independent variables 12

13 Overview of classical econometrics The multiple linear regression model Ordinary Least Squares method Minimise σn i=1 y i β 0 β 1 x i1... β k x ik 2 OLS estimators of β 0, β 1, β k They give the variation of y i for one-unit variation of x i, mantaining the other variables constants: Δ y = መβ i Δx i 13

14 Overview of classical econometrics The multiple linear regression model Ordinary Least Squares method Predicted value of y i : y i = β 0 + β 1 x i β k x ik Residual or error term : ε i = y i y i 14

15 Overview of classical econometrics The multiple linear regression model Assumptions: E(ε i X i ) = 0 ε i has conditional zero mean (X i,y i ) i.i.d i=1,..n X i and ε i have nonzero finite fourth moment There is no perfect multicollinearity (see later) var(ε i X i ) = σ ε 2 homoschedasticies The conditional distribution of ε i given X i is normal 15

16 Overview of classical econometrics The multiple linear regression model Goodness of Fit Total sum of squares n TSS = i=1 Or y i തy 2 total variation of y TSS= ESS + RSS Explained Sum of Squares: variation explained by the model, i.e. variation of Y explained by X: n i=1 y i തy 2 Residual Sum of Squares: residual variation, i.e. variation explained by the residuals: n 2 n ui 2 i=1 = i=1 y i y i 16

17 Overview of classical econometrics The multiple linear regression model Goodness of Fit R 2 = ESS TSS 0 R 2 1 It can also be written as 1 RSS TSS Adjusted R 2 = 1- n 1 n k (1- R2 ) 17

18 Overview of classical econometrics Collinearity The term independent variable means an explanatory variable is independent of the error term, but not necessarily independent of other explanatory variables. Since economists typically have no control over the implicit experimental design, explanatory variables tend to move together which often makes sorting out their separate influences rather problematic. 18

19 Overview of classical econometrics Collinearity Evidence of high collinearity include: a high pairwise correlation between two explanatory variables a high R-squared when regressing one explanatory variable at a time on each of the remaining explanatory variables a statistically significant F-value when the t-values are statistically insignificant an R 2 that doesn t fall by much when dropping any of the explanatory variables 19

20 Overview of classical econometrics Collinearity Collinearity doesn t mean the model is misspecified Especially common problem in time series regressions It depends on lack of adequate information in the sample 20

21 Overview of classical econometrics Collinearity Some solutions: collect more data with better information impose economic restrictions as appropriate impose statistical restrictions when justified if all else fails at least point out that the poor model performance might be due to the collinearity problem (or it might not). 21

22 3. Time series models

23 Overview of classical econometrics Time series models A collection of observations made sequentially in time (stochastic process) Examples: - Unemployment rate over time - Inflation rate - Production indices - Number of deaths/births - Etc. 23

24 Overview of classical econometrics Time series models Spanish quarterly GDP from 1995 to

25 Overview of classical econometrics Time series models Total employees in Spain, from 1980 to 2004, quarterly variation 25

26 Overview of classical econometrics Time series models Number of births in Spain from 1975 to 2013; monthly data 26

27 Overview of classical econometrics Time series models Total number of population in Spain from 1971 to

28 Overview of classical econometrics Time series models 28

29 Overview of classical econometrics Time series models 29

30 Overview of classical econometrics Time series models Univariate Time Series describe the behaviour of a variable in terms of its own past values Y t = β 0 + β 1 Y t 1 + ε t intercept coefficient Random error (white noise) Multivariate Time Series describe the behaviour of a variable in terms of its own past values and the past values of other variables Y t = β 0 + β 1 Y t 1 + δ 1 X t 1 + ε t 30

31 Overview of classical econometrics Time series models First order autoregression (AR1) Y t = β 0 + β 1 Y t 1 + ε t Second order autoregression (AR2) Y t = β 0 + β 1 Y t 1 + β 2 Y t 2 ε t p th order autoregression (ARp) Y t = β 0 + β 1 Y t 1 + β 2 Y t 2 + β p Y t p + ε t 31

32 Overview of classical econometrics Time series models We use OLS to estimate the coefficients Y t = መβ 0 + መβ 1 Y t 1 + ε t forecast ε t = Y t Y t forecast error The forecast error is not a residual The forecast and the forecast errors pertain to out-of-sample observations (in contrast to in-sample observations) 32

33 Overview of classical econometrics Time series models Lag length selection (choosing the order of p): F-statistics approach BIC (Bayes Information Criterion) AIC (Akaike Information Criterion) 33

34 Overview of classical econometrics Time series models Moving average process Y t = μ + ε t + θε t 1... Y t = μ + ε t + θ 1 ε t 1 + θ 2 ε t 2 + θ q ε t q (MA1) (MAq) MA processes depend not on the level of the last time point, but rather on the level of the last time point s error (ε) 34

35 Overview of classical econometrics Time series models ARMA process ARp Y t = β 0 + β 1 Y t 1 + β 2 Y t β p Y t p + ε t + θ 1 ε t 1 + θ 2 ε t 2 + +θ q ε t q MAq 35

36 Overview of classical econometrics Time series models Nonstationarity Most economic variables (GDP, consumption, price level, etc.) are non-stationary (upward or downward trend over time) Nonstationarity when the probability ditribution of Y t changes over time Many nonstationarity time series can be be made stationary by differencing them one or more times (Integrated processes) 36

37 Overview of classical econometrics Time series models Nonstationarity Deterministic Stochastic Random walk: Y t = Y t 1 + ε t Random walk with drift: Y t = β 0 + Y t 1 + ε t Specific case of AR(1) with β 1 =1 37

38 Overview of classical econometrics Time series models If β 1 = 1 nonstationary time series If β 1 <1 stationary time series β 1 = 1 is called Unit root 38

39 Overview of classical econometrics Time series models If a time series with a stochastic trend (i.e. A unit root), the first difference of the series does not have a trend Y t Y t 1 = β 0 + ε t Y stationary Y t is said to be integrated of order one I(1) 39

40 Overview of classical econometrics Time series models Y t is said to be integrated of order d I(d) if it becomes stationary after being first differenced d times Resulting model is ARIMA model d Y = β 0 + β 1 d Y t 1 + β 2 d Y t β p d Y t p + ε t + θ 1 ε t 1 + θ 2 ε t 2 + +θ q ε t q 40

41 Overview of classical econometrics Time series models The Box-Jenkins approach: Identification Inspect the data for stationarity, identify p and q, take first differences Estimation Apply least squares method (linear or no linear) Validation Check the estimated model fit well with no autocorrelation 41

42 4. Econometric methods in Official statistics

43 Econometric methods in Official statistics Regression methods - Hedonic prices Price comparisons over time and across countries are strongly affected by the statistical treatment of changes in product quality over time and differences in product quality across countries Matching method is not adequate to deal with substantial changes or differences in quality bias in the price index: The inside the sample bias: prices of non-identical products are matched The outside the sample bias: price changes of matched items are not representative of price changes of unmatched items 43

44 Econometric methods in Official statistics Regression methods - Hedonic prices A Hedonic Price Index use a regression analysis to estimate the effect of individual characteristics, the determinants of quality, on a product s price. p i = h z i + ε i Error term Function of the quality characteristics 44

45 Econometric methods in Official statistics Regression methods - Hedonic prices Hedonic modelling Fully linear model p n t = β 0 t + k=1 Logarithmic-linear model K K lnp n t = β 0 t + k=1 β t t k Z nk β t t k Z nk + ε n t + ε n t Multiple linear regression 45

46 Econometric methods in Official statistics Regression methods - Hedonic prices Applications Housing prices ICT- product prices Producer prices 46

47 Econometric methods in Official statistics Regression methods - Hedonic prices Advantages Offer a solution for the quality problem in price indices and international comparison, provided sufficient information on characteristics can be obtained It is used to estimate the willing to pay for, or marginal cost of producing, the characteristics, or the underlying demand or supply functions of these characteristics and corresponding consumer of producer surplus 47

48 Econometric methods in Official statistics Regression methods - Hedonic prices Difficulties Characteristics should represent user value and user cost Needs large datasets BIG DATA Excluded variables Other price determining variables: price mark-ups New features Multicollinearity Small quantities 48

49 Econometric methods in Official statistics Hedonic prices and machine learning Data Sources: GIS Land data (Big Data) Hedonic function: to estimate the value associated with: land characteristics, accessibility, externalities and expectations of future land developments Ln Price = α + Land Characteristics β 1 + Accessibility β 2 +Externalities β 3 + Expectations β 4 + Zoning β 5 +Controls β 5 + ε 49

50 Econometric methods in Official statistics Hedonic prices and machine learning Figure 1. Effect of parcel characteristics on land prices. The map shows log price differentials generated by different values associated with land characteristics. i.e. compare the price predicted by the model for each observation combining all land characteristics and comparing it to the price predicted for an ad-hoc observation with mean values for each explanatory variable corresponding to this group 50

51 Econometric methods in Official statistics Hedonic prices and machine learning Figure 1. Effect of parcel characteristics on land prices. Observations that reduce prices, mainly seen towards the west where price differentials reach - 148% The majority of observations in the city have predicted prices 20% to 57% higher than the observation with average land characteristics in the sample 51

52 Econometric methods in Official statistics Hedonic prices and machine learning Figure 2. Accessibility values. log price differentials generated by different accessibility values i.e. compare the price predicted by the model for each observation combining all accessibility variables and comparing it to the price predicted for an ad-hoc observation with mean values for each explanatory variable corresponding to this group 52

53 Econometric methods in Official statistics Hedonic prices and machine learning Figure 2. Accessibility values. Proximity to the city center adds value to the land 53

54 Econometric methods in Official statistics Deseasonalisation 54

55 Econometric methods in Official statistics Classical vs machine learning econometrics Traditional econometrics Data features - Small to medium size - Monthly, quarterly data (in Official Statistics) - Reasonable number of variables Model definition (usual) Model selection and estimation Assumptions - Model-based relationships between variables, grounded on (economic) theory e.g. Multiple linear regression, time series (ARIMA) - Expert s knowledge - Distribution of asymptotic significance methods - Rigid distributional, independence assumptions Machine Learning econometrics - Large size - High frequency - High dimensional datasets - Algorithm based - Artificial intelligence (but it does not avoid knowing what type of technique to apply!!) - Automatic optimization ( regularization ) of modes - No assumptions 55

56 5. Models of inference

57 Models of inference The context of Official Statistics Budget restrictions to carry out traditional surveys Increasing concern for response burden Increasing non-response New sources of available data: Administrative data Big Data sources: traffic sensors, M2M transactions, social media, satellite images Development of mathematical-statistical methods and IT tools that allow for other forms of data treatment 57

58 Models of inference The objectives of statistical inference The purpose of statistical inference is to obtain information about a population (finite or infinite) from a sample from this population Stochastic assumptions about the individual observations and/or the population are made Statistical information of interest includes totals, means, proportions, ratios, quantiles, etc. or the probability distribution of a random variable 58

59 Models of inference Overview of different modes of inference (paradigms) Design-based Model-assisted Model-based Algorithm-based predictive 59

60 Models of inference Design-based inference Traditionally used by National Statistical Institutes Use of surveys to collect data NSIs prefer not to rely on model assumptions, particularly if they are not verifiable Statistical (mathematical) models may be difficult to understand, communicate or even calculate in a production environment The concepts of random sample, sampling error, weighting observations, etc. are familiar to (educated) users of Official Statistics 60

61 Models of inference Design-based inference: estimation Estimators (of a mean, a total, a proportion) are obtained by expanding or weighting the observations in the sample with survey weights Survey weights are derived from the sample design and available auxiliary information The statistical properties of estimators are based on the probability distributions from the sampling design Design-based estimators have «good» statistical properties such as asymptotic unbiasedness 61

62 Models of inference Design-based inference: theoretical example Horvitz-Thompson estimator of a total Y HT = i S 1 π i y i where π i is the probability of selection of unit i, and 1/π i is the weight of unit i calculated on the basis of the design: Stratification (auxiliary variables that define the strata) Sample size Corrections for non-response, calibration, etc. 62

63 Models of inference Design-based inference: limitations Design-based inference may not be suitable when samples are small in presence of non-sampling errors discontinuities in survey design (e.g. change in data collection mode, new classifications, methodological change of concepts) Design-based estimators do not take into account the changes and cannot separate the «real» change from the methodological change 63

64 Models of inference Model-assisted inference Design-based estimators of the parameter of a variable can be improved by using auxiliary information and modelling the relationship between the variable and the auxiliary information (=model-assisted) 64

65 Models of inference Model-assisted inference: estimation HT estimator obtained from a linear regression model that relates the parameter to auxiliary information Observed (x k ; y k ) for a sample S (e.g. administrative and survey data), x are observed for the whole U universe X HT = σ i S 1 π i x i is the grossed-up total of observed auxiliary x values X = σ i U x i is the known total of auxiliary x values Y HT = σ i S 1 π i y i is the Horvitz-Thompson estimate Y R = Y HT + b X X HT is the regression (=model-based) estimate based on the regression model y = a + b x estimated from the sample of observed (x k ; y k ) 65

66 Models of inference Design-based and model-assisted: examples of application in official statistics Generalised regression estimator (GREG) widely used by NSIs for calibration Adjusts totals for sub-populations (consistency across tables) Adjusts to known totals Small Area Estimation (estimation borrowing strength over space) Surveys based on panels (estimation borrowing strength from the past) Modelling survey discontinuities Integration of sources in National Accounts Hedonic Price Indices Seasonal adjustment of statistical series 66

67 Models of inference Algorithm-based inference In the algorithmic approach, the equivalent of fitting a model is tuning an algorithm, so that it predicts well It is generally impossible to express algorithmic methods analytically in terms of a mathematical expression In the algorithmic approach, the data for which both x and y are known is split into two parts TRAINING SET: part is used to tune the algorithm TEST SET: part used to evaluate or test the predictive capabilities of the trained algorithm 67

68 Models of inference Algorithm-based inference: types of data collected from units through a targeted survey (e.g. Structural Business Survey, Labour Force survey) collected from units in support of some administrative process (e.g. tax records, unemployment benefits) other types, registering events (e.g. a transaction, an e- mail, a Tweet) generated as by-products of processes unrelated to statistics or administration 68

69 Models of inference Algorithm-based inference: types of data Feature Survey data Admin data Other data Records are units of a target population Yes Yes No Target variables are directly available Yes Yes No Auxiliary variables are directly available Yes Often No Data preparation/ conversion is needed No No Yes Data covers the complete target population No Often Rarely Data are (almost) representative Usually Usually No Susceptibility to measurement error High Medium low Source: Buelens et al. (2012) 69

70 Models of inference Algorithm-based inference: theoretical examples Similar to the model-based estimator, the algorithmic estimator is Y Alg = y k + F(x k ) k S For some function F() which maps the observed x to the corresponding y within S (training set of units for which y is known), the set R contains the population units with unknown y. k R Uncertainty of this estimator arises from the imperfect predictive power of the algorithm, and is assessed on the test set using some cost function. 70

71 Models of inference Algorithm-based inference: examples of application in official statistics Central Statistics Office of Ireland: automatic coding system for Classification of Individual Consumption by Purpose (COICOP) assignment for their Household Budget Survey, using previously coded records as training data Statistics New Zealand: Support Vector Machines (SVM) to improve coding of variables Occupation and Post-school Qualification, using two disjoint sets of observations, each of size 10,000, from Census 2013 data for training and testing (50% correctness). Statistics Portugal: classification trees (a type of decision trees whose response variables are categorical) for error detection in foreign trade transaction data. 71

72 Models of inference Algorithm-based inference: examples of application in official statistics (2) US Department of Agriculture: hierarchical clustering to reduce the number of Quarterly Agriculture Survey (QAS) questionnaire versions (states x crops). Italian National Institute of Statistics: substituting (fully or partially) ICT in Enterprises surveys by collecting data via website scraping and extracting information using machine learning methods. Statistics Canada: use of satellite imaging data to assist with estimation of crop yields. Field surveyors were sent to corresponding actual locations to ascertain crop types and yields; these were used as response variables. Probabilistic image processing algorithms were used to learn and predict the field observations based on the satellite data. 72

73 References J.H. Stock and M.W. Watson (2003). Introduction to econometrics, Addison Wesley W.H. Green (2003). Econometrics analysis, Prentice Hall J. van den Brakel and J. Betlehem (2008). Model-based estimation for official statistics. Statistics Netherlands, discussion paper (08002) K. Chu and Cl. Poirier, Statistics Canada (2015). Machine Learning Documentation Initiative. HIGH-LEVEL GROUP FOR THE MODERNISATION OF STATISTICAL PRODUCTION AND SERVICES, Modernisation Committee on Production and Methods Buelens, B. H.J. Boonstra, J. van den Brakel, P. Daas (2012). Shifting paradigms in official statistics. Statistics Netherlands, discussion paper (201218) CROS Portal on MEMOBUST: Generalised Regression Estimator (Method) Calibration (Method) 73

74 References OECD,, ILO, IMF, The World Bank, UNECE (2013). Handbook on Residential Property Price Indices (RPPIs) Peter Hein van Mulligen (2003). Quality aspects in price indices and international comparisons: Applications of the hedonic method C. Goytia and G. Dorna (Universidad Torcuato Di Tella)(2016). Big data and a Spatial Hedonic Approach: Addressing the land market information gap and estimating land prices determinants in metropolitan regions from developing countries 74