Time Series and Forecasting Lecture 4 NonLinear Time Series

Transcription

1 Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

2 Today s Schedule Density Forecasts Threshold Regression Models Nonparametric Regression Models Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

3 Density Forecasts The conditional distribution is F t (y) = P (y t+1 y I t ) The conditional density is f t (y) = d dy P (y t+1 y I t ) Density plots are useful summaries of forecast uncertainty May also be useful as inputs for other purposes Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

4 Density Forecasts y t+1 = µ t + σ t ε t+1 µ t = E (y t+1 I t ) σ 2 t = var (ε t+1 I t ) Assume ε t+1 is independent of I t, with density f ε (u) = d dy F ε (u) Forecast density for y n+1 f n (y) = 1 σ n f ε ( y µn σ n ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

5 Normal Error Model Assume ε t+1 N(0, 1), then f ε (u) = φ(u) f n (y) = 1 σ n φ Probably should not be used ( y β x n σ n ) Contains no information beyond and σ t Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

6 Nonparametric Density Forecast We can estimate f ε (ε) from the normalized residuals ε t+1 using a standard kernel estimator. Discuss this shortly Then the forecast density for y n+1 is f n (y) = 1 σ n f ε (y β x n σ n ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

7 Examples: Interest Rate GDP Nowcast Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

8 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

9 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

10 Nonparametric Density Estimation Let X i be a random variable with density f (x) Observations i = 1,..., n [For example, ε t+1 for t = 0,..., n 1.] The kernel density estimator of f (x) is f (x) = 1 n ( ) Xi x nb k i=1 b where k(u) is a kernel function and b is a bandwidth Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

11 Kernel Functions A kernel function k(u) : R R is any function which satisfies k(u)du = 1. A non-negative kernel satisfies k(u) 0 for all u. In this case, k(u) is a probability density function. A symmetric kernel function satisfies k(u) = k( u) for all u. The order of a kernel, ν, is the first non-zero moment. A standard kernel is non-negative, symmetric, and second-order A kernel is higher-order kernel if ν > 2. These kernels will have negative parts and are not probability densities. They are also referred to as bias-reducing kernels. Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

12 Common Second-Order Kernels Kernel Equation R(k) κ 2 eff Uniform k 0 (u) = ( u 1) 1/2 1/ Epanechnikov k 1 (u) = 3 ( 4 1 u 2 ) 1 ( u 1) 3/5 1/ Biweight k 2 (u) = 15 ( 16 1 u 2 ) 2 1 ( u 1) 5/7 1/ Triweight k 3 (u) = 35 ( 32 1 u 2 ) 3 ( 1 ) ( u 1) 350/429 1/ Gaussian k φ (u) = 1 2π exp 1/2 π u2 2 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

13 Choice of Kernel Not as important as bandwidth Epanechnikov (quadratic) is optimal for minimizing IMSE of f (x) Gaussian is convenient as it is infinitely smooth and has positive support everywhere I am using Gaussian here Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

14 Kernel Density Estimator f (x) = 1 ( ) Xi x nb n i=1 k b f (x)dx = ( ) 1 1 n n i=1 b k Xi x dx = ( ) b 1 1 n n i=1 b k Xi x dx = 1 b n n i=1 1 = 1 since by ( the change ) of variables u = (X i x)/h 1 b k Xi x dx = k b (u) du = 1. Thus f (x) is a density Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

15 First Moment x f (x)dx = 1 n = 1 n = 1 n = 1 n the sample mean of the X i. n i=1 n i=1 n X i i=1 n X i i=1 x 1 ( ) b k Xi x dx b (X i + uhb) k (u) du k (u) du + 1 n n b uk (u) du i=1 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

16 Second Moment x 2 f (x)dx = 1 n = 1 n = 1 n = 1 n n i=1 n i=1 n Xi 2 i=1 x 2 1 ( ) b k Xi x dx b (X i + ub) 2 k (u) du + 2 n n Xi 2 + b 2 κ 2 i=1 where κ 2 = u2 k (u) du (1 in the case of Gaussian) n X i b k(u)du + 1 n i=1 n b 2 u 2 k (u) i=1 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

17 Kernel Density Variance x 2 f ( ) 2 (x)dx x f (x)dx = 1 n ( n Xi 2 + b 2 1 κ 2 i=1 n = σ 2 + b 2 κ 2 ) 2 n X i i=1 where σ 2 is the sample variance of X i In the case of the normalized residuals ε t+1, which have mean zero and sample variance 1, and using a Gaussian kernel: f ε (u) has A mean of zero A variance of 1 + b 2 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

18 Numerical Implementation for Forecast Density Pick a set of grid points for ε, e.g. u 1,..., u G For each ε on grid, evalulate f ε (ε) = 1 n 1 ( ) ε εt+1 φ nb b or t=0 f j ε = 1 n 1 nb t=0 ( ) uj ε t+1 φ b Set the translated gridpoints y j = β x n + σ n u j, for j = 1,..., G, and f j = 1 σ n f ε j The rescaling is the Jacobian of the transformation from u j to y j Plot f j on y-axis against y j on x-axis. This is a plot of ) f n (y) = 1 σ n f ε (y β x n σ n Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

19 Bias of Kernel Estimator E f (x) = E 1 ( ) b k Xi x 1 = b b k ( z x Using the change-of variables u = (z x)/b, this equals k (u) f (x + bu)du Now take a Taylor expansion of f (x + bu) about f (x) : b ) f (z)dz f (x + bu) f (x) + f (1) (x)bu f (2) (x)b 2 u 2 Integrating term-by term, k (u) f (x + bu)du k (u) f (x) + f (1) (x)b k (u) u f (2) (x)b 2 k (u) u2 du = f (x) f (2) (x)b 2 κ 2 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

20 Variance of Kernel Estimator var f (x) = 1 n var 1 ( ) b k Xi x b 1 ( ) z x 2 nb 2 k f (z)dz b = 1 k (u) 2 f (x + bu)du nb f (x) k (u) 2 du nb = f (x)r(k) nb where R(k) = k (u)2 du is called the roughness of the kernel. Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

21 Asymptotic MSE of Kernel Estimator ) AMSE ( f (x)) = Bias( f (x)) 2 + var ( f (x) = κ2 2 4 ( ) 2 f (2) (x) b 4 + f (x) R(k) nb Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

22 Mean Integrated Squared Error (MISE) of Kernel Estimator AMISE = = AMSE ( f (x))dx κ ( 2 f (x)) (2) b 4 dx + = κ2 2 4 R(f )b4 + R(k) nb where R(f ) = ( 2 f (x)) (2) dx is the roughness of f (2) f (x) R(k) dx nb Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

23 Optimal bandwidth The AMISE takes the form Ab 4 + B/nb The bandwidth b which minimizes the AMISE is ( ) 4R(k) 1/5 b = R(f ) 1/5 n 1/5 4κ 2 2 The unknown component is R(f ) 1/5 The rougher is f (x), the larger is R(f ) so the optimal b is smaller Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

24 Rule of Thumb Silverman proposed that we take f = φ as a baseline (or reference) Calculate the optimal bandwidth for this case. The Rule of Thumb b = ˆσCn 1/5 where ( π C = 1/2 2R(k) 2 4!κ 2 2 ) 1/5 Rule of Thumb Constants Epanechnikov 2.34 Biweight 2.78 Triweight 3.15 Gaussian 1.06 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

25 Plug-in bandwidth Methods Estimate R(f ) Use ( ) 4R(k) 1/5 b = R(f ) 1/5 n 1/5 4κ 2 2 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

26 Cross-Validation for Density Bandwidth Mean integrated squared error (MISE). Given b MISE (b) = = ) 2 ( f (x) f (x) dx f (x) 2 dx 2 f (x)f (x)dx + f (x) 2 dx We know the first term, not the second, and the third does not depend on b so we ignore it Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

27 First Term The first term is f (x) 2 dx = = ( 1 bh 1 n 2 b 2 n i=1 n i=1 n j=1 ( ) Xi ) 2 x k dx b ( ) ( Xi x Xj x k k b b Make the change of variables u = X i x, h ( ) ( ) 1 Xi x Xi x k k dx = k (u) k b b b = k ( Xi X j b ) dx ( u X i X j b ) ) du where k (x) = k (u) k (x u) du is the convolution of k with itself. If k(x) = φ(x) then k (x) = 2 1/2 φ(x/ 2) = exp( x 2 /4)/ 4π. Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

28 The first term is thus f (x) 2 dx = 1 n 2 b 2 n i=1 n j=1 k ( Xi X j b ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

29 Second Term The second term is 2 times f (x)f (x)dx an integral with respect to the density of X i, or an expectation with respect to X i We can estimate expectations using sample averages, e.g. 1 n n f i=1 (X i ), but f depends on X i, so this is biased The solution is to use a leave-one-out estimator for f, ( ) 1 f i (x) = (n 1) b Xj x k b j =i Then an unbiased estimate of the second term is n n ( ) 1 1 Xj X i n f i (X i ) = i=1 n (n 1) b k i=1 b j =i Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

30 Cross-Validation Criterion MISE (b) = f (x) 2 dx 2 f (x)f (x)dx + f (x) 2 dx CV (b) = 1 n 2 b 2 n i=1 n j=1 k ( Xi X j b In the case of a Gaussian kernel 1 CV (b) = n 2 b 2 2 n i=1 n j=1 φ CV selected bandwidth ) ( Xi X j 2b ) 2 n (n 1) b 2 n (n 1) b b = argmin CV (b) n i=1 j =i n i=1 j =i ( ) Xj X i k b ( ) Xj X i φ b Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

31 Evaluation Form a grid for b If b r is the rule-of-thumb bandwidth, search over [b r /3, 3b r ] or something similar Many authors define the CV bandwidth as the largest local minimizer In the end, an eyeball reality check of your estimated density is important. Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

32 Theory CV selected bandwidth is consistent Let b 0 minimize the AMISE b b 0 b 0 p 0 But the rate of convergence is slow, n 10 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

33 Examples 10-Year Bond Rate GDP Growth Rate Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

34 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

35 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

36 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

37 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

38 Threshold Models A type of nonlinear time series models Strong nonlinearity Allows for switching effects Most typically univariate (for simplicity) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

39 Threshold Models Threshold Variable q t q t = 100(log(GDP t ) log(gdp t 4 )) = annual growth Threshold γ Split regression Coeffi cients switch if q t γ or q t > γ If growth has been above or below the threshold y t+1 = β 1 x t1 (q t γ) + β 2 x t1 (q t > γ) + e t+1 { β = 1 x t + e t q t γ β 2 x t + e t q t > γ Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

40 Partial Threshold Model y t+1 = β 0 z t + β 1 x t1 (q t γ) + β 2 x t1 (q t > γ) + e t+1 Coeffi cients on z t do not switch More parsimonious Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

41 Estimation y t+1 = β 0 z t + β 1 x t1 (q t γ) + β 2 x t1 (q t > γ) + e t+1 Least Squares ( β 0, β 1, β 2, γ) minimize sum-of-squared errors Equation is non-linear, so NLLS, not OLS Simple to compute by concentration method Given γ, model is linear in β Regressors are z t, x t 1 (q t γ) and x t 1 (q t > γ) Estimate by least-squares Save residuals, sum of squared errors Repeat for all thresholds γ. Find value which minimizes SSE Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

42 Estimation Details For a grid on γ (can use sample values of q t ) Define dummy variables d 1t (γ) = 1 (q t γ) and d 2t (γ) = 1 (q t > γ) Define interaction variables x 1t (γ) = x t d 1t (γ) and x 2t (γ) = x t d 2t (γ) Regress y t+1 on z t, x 1t (γ), x 2t (γ) Sum of squared errors y t+1 = β 0 z t + β 1 x 1t (γ) + β 2 x 2t (γ) + ê t+1 (γ) S(γ) = n ê t+1 (γ) 2 t=1 Write this explicity as a function of γ as the estimates, residuals and SSE vary with γ Find ˆγ which minimizes S(γ) Useful to view plot of S(γ) against γ Given ˆγ, repeat above steps to find estimates ( β 0, β 1, β 2 ) Forecasts made from fitted model Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

43 Example: GDP Forecasting Equation q t = 100(log(GDP t ) log(gdp t 4 )) = annual growth Threshold estimate: ˆγ = 0.18 Splits regression depend if past year s growth is above or below 0.18% 0% Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

44 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

45 Multi-Step Forecasts Nonlinear models (including threshold models) do not have simple iteration method for multi-step forecasts Option 1: Specify direct threshold model Option 2: Iterate one-step threshold model by simulation: Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

46 Multi-Step Simulation Method Take fitted model y t+1 = β 0 z t + β 1 x t1 (q t ˆγ) + β 2 x t1 (q t > ˆγ) + ê t+1 Draw iid errors ê n+1,..., ê n+h from the residuals {ê 1,..., ê n } Create yn+1 (b), y n+2 (b),..., y n+h (b) forward by simulation b indexes the simulation run Repeat B times (a large number) {yn+h (b) : b = 1,..., B} constitute an iid sample from the forecast distribution for y n+h Point forecast f n+h = 1 B B b=1 y n+h (b) Interval forecast: α and 1 α quantiles of y n+h (b) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

47 Testing for a Threshold Null hypothesis: No threshold (linearity) Null Model: No threshold Alternative: Single Threshold y t+1 = β 0 z t + β 1 x t + ê t+1 S 0 = n êt+1 2 t=1 y t+1 = β 0 z t + β 1 x 1t(γ) + β 2 x 2t(γ) + ê t+1 (γ) S 1 (γ) = n ê t+1 (γ) 2 t=1 S 1 = S 1 ( ˆγ) = min S 1 (γ) γ Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

48 Threshold F Test No Threshold against one threshold ( ) S0 S 1 (γ) F (γ) = n S 1 (γ) ( ) S0 S 1 F = n = max γ S 1 F (γ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

49 NonStandard Testing Test is non-standard. Critical values obtained by simulation or bootstrap Fixed Regressor Bootstrap Similar to a bootstrap, a method to simulate the asymptotic null distribution Fix (z t, x t, ê t ), t = 1,..., n Let yt be iid N(0, êt 2 ), t = 1,..., n Estimate the regressions as before S 1 y t+1 = y t+1 = S 0 = β (γ) = min γ 0 z t + n t=1 β 0 z t + β 1 x t + êt+1 n êt+1 2 t=1 β 1 x 1t (γ) + β 2 x 2t (γ) + êt+1 (γ) ê t+1 (γ)2 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

50 Bootstrap Test Statistics S1 = S 1 ( ˆγ) = min S 1 (γ) γ ( S F (γ) = n 0 S1 (γ) ) S1 (γ) ( S F = n 0 S1 ) S1 = max F (γ) γ Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

51 Repeat this B 1000 times. Let F01 (b) denote the b th value Fixed Regressor bootstrap p-value p = 1 B N 1 (F01(b) F 01 ) b=1 Fixed Regressor bootstrap critical values are quantiles of empirical distribution of F 01 (b) Important restriction: Requires serially uncorrelated errors (h = 1) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

52 Example: GDP Forecasting Equation q t = 100(log(GDP t ) log(gdp t 4 )) = annual growth Bootstrap p-value for threshold effect: 10.6% Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

53 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

54 Inference in Threshold Models Threshold Estimate has NonStandard Distribution Confidence intervals by inverting F statistic F Test: Kknown Threshold against Estimated threshold ( ) S1 (γ) S 1 LR(γ) = n S 1 [Call it LR(γ) to distinguish from F (γ) from earlier slide.] Theory: [Hansen, 2000] LR(γ) d ξ = max s [2W (s) s ] P(ξ x) = ( 1 e x /2) 2 Critical values: P(ξ c) c Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

55 Confidence Intervals for Threshold All γ such that LR(γ) c where c is critical value Easy to see in graph of LR(γ) against γ Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

56 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

57 Threshold Estimates Estimate: ˆγ = 0.18 Confidence Interval = [ 1.0, 2.2] Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

58 Inference on Slope Parameters Conventional As if threshold is known Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

59 Threshold Model Estimates q t = 100(log(GDP t ) log(gdp t 4 )) q t 0.18 q t > 0.18 Intercept (4.6) (1.11) log(gdp t ) 0.36 (0.21) 0.16 (0.08) log(gdp t 1 ) (0.21) 0.20 (0.09) Spread t 1.3 (0.8) 0.71 (0.20) Default Spread t (1.26) -2.3 (0.9) Housing Starts t 2.5 (10.6) 4.1 (2.3) Building Permits t 7.8 (10.5) -2.2 (2.0) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

60 NonParametric/NonLinear Time Series Regression Optimal point forecast is g (x n ) where g (x) = E (y t+1 x t = x) and x t are all relevant variables. In general, the form of g (x) is unknown and nonlinear Linear models used for simplicity, but they are not true Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

61 NonParametric/NonLinear Time Series Regression Model y t+1 = g (x t ) + e t+1 E (e t+1 x t ) = 0 The conditional mean zero restriction holds true by construction e t+1 not necessarily iid Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

62 Additively Separable Model x t = (x 1t,..., x pt ) g (x t ) = g 1 (x 1t ) + g 2 (x 2t ) + + g p (x pt ) Then y t+1 = g 1 (x 1t ) + g 2 (x 2t ) + + g p (x pt ) + e t+1 Greatly reduces degree of nonlinearity Useful simplification, but should be viewed as such, not as true Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

63 Partially Linear Model Partition x t = (x 1t, x 2t ) g (x t ) = g 1 (x 1t ) + β x 2t x 2t typically includes dummy variables, controls x 1t main variables of importance For example, if primary dependence through first lag y t+1 = g 1 (y t ) + β 1 y t β p y t p + e t+1 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

64 Sieve Models For simplicity, suppose x t is scalar (real-valued) WLOG in additively separable and partially linear models Approximate g(x) by a sequence g m (x), m = 1, 2,..., of increasing complexity Linear sieves g m (x) = Z m (x) β m where Z m (x) = (z 1m (x),..., z Km (x)) are nonlinear functions of x. Series : Z m (x) = (z 1 (x),..., z K (x)) Sieves : Z m (x) = (z 1m (x),..., z Km (x)) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

65 Polynomial (power series) z j (x) = x j gm(x) = p β j x j j=1 Stone-Weierstrass Theorem: Any continuous function g(x) can be arbitrarily well approximated on a compact set by a polynomial of suffi ciently high order For any ε > 0 there exists coeffi cients p and β j such that X Runge s phenomenon: sup g m (x) g(x) ε x X Polynomials can be poor at interpolation (can be erratic) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

66 Splines Piecewise smooth polynomials Join points are called knots Linear spline with one knot at τ β 00 + β 01 (x τ) g m (x) = β 10 + β 11 (x τ) x < τ x τ To enforce continuity, β 00 = β 10, g m (x) = β 0 + β 1 (x τ) + β 2 (x τ) 1 (x τ) or equivalently g m (x) = β 0 + β 1 x + β 2 (x τ) 1 (x τ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

67 Quadratic Spline with One Knot β 00 + β 01 (x τ) + β 02 (x τ) 2 g m (x) = β 10 + β 11 (x τ) + β 12 (x τ) 2 x < τ x τ Continuous if β 00 = β 10 Continuous first derivative if β 01 = β 11 Imposing these constraints g m (x) = β 0 + β 1 x + β 2 x 2 + β 3 (x τ) 2 1 (x τ). Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

68 Cubic Spline with One Knot g m (x) = β 0 + β 1 x + β 2 x 2 + β 3 x 3 + β 4 (x τ) 3 1 (x τ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

69 General Case Knots at τ 1 < τ 2 < < τ N g m (x) = β 0 + p j=1 β j x j + N k=1 β p+k (x τ k ) p 1 (x τ k ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

70 Uniform Approximation Stone-Weierstrass Theorem: Any continuous function g(x) can be arbitrarily well approximated on a compact set by a polynomial of suffi ciently high order For any ε > 0 there exists coeffi cients p and β j such that X Strengthened Form: sup g m (x) g(x) ε x X if the s th derivative of g(x) is continuous then the uniform approximation error satisfies sup g m (x) g(x) = O ( K α ) m x X where K m is the number of terms in g m (x) This holds for polynomials and splines Runge s phenomenon: Polynomials can be poor at interpolation (can be erratic) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

71 Illustration g(x) = x 1/4 (1 x) 1/2 Polynomials of order K = 3, K = 4, and K = 6 Cubic splines are quite similar Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

72 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

73 Runge s Phenomenon Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

74 Placement of Knots If support of x is [0, 1], typical to set τ j = j/(n + 1) If support of x is [a, b], can set τ j = a + (b a)/(n + 1) Alternatively, can set τ j to equal the j/(n + 1) quantile of the distribution of x Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

75 Estimation Fix number and location of knots Estimate coeffi cients by least-squares Quadratic spline y = β 0 + β 1 x + β 2 x 2 + N k=1 β 2+k (x τ k ) 2 1 (x τ k ) + e Linear model in x, x 2, (x τ 1 ) 2 1 (x τ 1 ),..., (x τ N ) 2 1 (x τ N ) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

76 Selection of Number of Knots Model selection Pick N to minimize Cross-validation function CV is an estimate of MSFE IMSE (integrated mean-squared error) CV selection (and combination) is asymptotically optimal for minimization of the MSFE and IMSE Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

77 Example: GDP Growth y t =GDP Growth x t =Housing Starts Partially Linear Model y t+1 = g(x t ) + β 1 y t 1 + β 2 y t 2 + e t+1 Polynomial Cubic Spline Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

78 CV Selection Polynomial in Housing Starts p CV Cubic Spline in Housing Starts N CV Best fitting regression is quartic polynomial (p = 4) Cubic spline with 1 knot is close Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

79 Polynomial=solid line Cubic Spline=dashed line Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

80 Estimated Cubic Spline Knot=1.5 β s( β) Intercept 29 (8) y t 0.18 (0.08) y t (0.08) HS t 86 (26) HSt 2 79 (23) HSt 3 22 (6) (HS t 1.5) 2 1 (HS t > 1.5) 43 (13) Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

81 New Example: Long and Short Rates Bi-variate model of Long (10-year) and short (3-month) bond rates Key variable: Spread: Long-Short R t =Long Rate r t =Short Rate Z t = R r r t =Spread Model R t+1 = α 0 + α p1 (L) R t + β p1 (L) r t + g 1 (Z t ) + e 1t r t+1 = γ 0 + γ p2 (L) R t + δ p2 (L) r t + g 2 (Z t ) + e 2t Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

82 CV Selection Separately for each equation Long Rate and Short Rate Select over number of lags Number of spline terms for nonlinearity in Spread Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

83 CV Selection: Long Rate Equation p = 0 p = 1 p = 2 p = 3 p = 4 p = 5 p = 6 Linear Quadratic Cubic Knot Knots Knots Selected Model: p = 6, Cubic spline with 1 knot at 1.53 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

84 CV Selection: Short Rate Equation p = 0 p = 1 p = 2 p = 3 p = 4 p = 5 p = 6 Linear Quadratic Cubic Knot Knots Knots Selected Model: p = 1, Cubic spline with 1 knot at 1.53 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

85 Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

86 Forecasting For h > 1, need to use forecast simulation Simulate R n+1, r n+1 forward using iid draws from residuals Create time paths Take means to estimate point forecasts Take quantiles to construct forecats intervals Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87

87 Assignment Construct a nonlinear model to forecast the unemployment rate. Use either a threshold or nonparametric model Use appropriate methods to select the model and variables Make a one-step forecast If time, use simulation to create 1 through 12 step forecast distributions. Use this to calculate point forecasts, intervals and a fan chart. Bruce Hansen (University of Wisconsin) Foundations July 23-27, / 87