Introduction to Smoothing spline ANOVA models (metamodelling)

Transcription

1 Introduction to Smoothing spline ANOVA models (metamodelling) M. Ratto DYNARE Summer School, Paris, June 215. Joint Research Centre Serving society Stimulating innovation Supporting legislation 1

2 Introduction to ANOVA Models Consider the mapping Y f( x) where x is a vector of input variables x ( x1, x2,..., x k ) and Y is the output. x is the i-th element of x varying in 1 i x i x Ω f(x) Y 2

3 Specification of the input factors x Ω Marginal p.d.f. p( x) pi( xi) x 1 x 2 x 3 Marginal p.d.f. + correlation structure x 4 Joint p.d.f. p( x1, x2,..., x k ) Rank correlation, Dependence-trees, diagonal band, copulas MCMC (Gibbs, Metropolis, ) 3

4 Propagation of uncertainty Input Model Output px ( ) Y f( x ) py ( ) x 1 x 2 x 3 x 4 x k 1 2 n x i : input factors y 4

5 Model approximation (meta-modelling) x f(x) Y Realistic Computer models are very costly in terms of run time. We need a way to characterize uncertainty and perform SA based on a limited number of model runs. We aim to identify a 'simple' relationship between Xi's and Y that fits well the original model and is less computationally demanding. 5

6 Model approximation (meta-modelling) A Meta-model is a simpler model that mimics the larger computational model. Evaluations of Meta-Models are much faster. 6

7 Model approximation (meta-modelling) Local approximation methods take f and its derivatives at a base point X and construct a function that matches the properties of f in the nearby region (Taylor series). 7

8 Model approximation (meta-modelling) 1. Linear Regression 2. Quadratic Response Surface Regression Options 1 and 2 work pretty well in some cases. However many realistic models are highly nonlinear and/or periodic and these methods fail. 3. Gaussian Process or Spatial Models 4. Nonparametric Regression Models Both 3 and 4 work fairly well for a small number of inputs. Variable selection is helpful for a modest number of inputs. 8

9 ANOVA Models ANOVA models define a decomposition of Y=f(x) into main effects and interactions k Y f( x) f f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i This is also called the High Dimensional Model Representation (HDMR) E.g., if k=3: f( x) f f ( x ) f ( x ) f ( x ) f ( x, x ) f ( x, x ) f ( x, x ) f ( x, x, x ) This decomposition is non-unique for general joint pdf s of x. The total number of summands in the ANOVA is 2 k 9

10 Properties of the ANOVA decomposition (orthogonal case) k Y f( x) f f x f x, x... f x, x,..., x If each term is chosen with zero mean f ( x ) dp( x ), x i 1,2,..., k... i i i i i i ij i j 1,2,..., k 1 2 k i 1 i j i f ( x, x ) dp( x ) dp( x ), x, x i j ij i j i j i j Ω f ( x, x,..., x ) dp( x ) dp( x )... dp( x ) k 1 2 k 1 2 k 1

11 Properties of the ANOVA decomposition (orthogonal case) k Y f( x) f f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i then the ANOVA decomposition has TWO properties f x dp( x) f E( Y) Ω All the summands are orthogonal: i i j j if,...,,..., 1 s 1 l f f dp( x) k i1,..., is j1,..., jl Ω It follows that the ANOVA decomposition is unique and each term can be defined as 11

12 Properties of the ANOVA decomposition k Y f( x) E( Y) f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i Ω x i f x dp( x x ) f f x E( Y x ) i i i i xi xj Ω, f x dp( x x x ) f f ( x ) f ( x ) f ( x, x ) i j i i j j ij i j E( Y x, x ) i j 12

13 ANOVA k Y E( Y) f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i Being the terms orthogonal, we can square and integrate the eq above over Ω and decompose the variance of f(x) into terms of increasing dimensionality (ANOVA) k V( Y) V V V... V 1,2,..., i ij ijk k i 1 i j i j k V 2 i fi ( xi) dxi 2 V f i1,..., idx dx dx i, i,... i i i... i 1 2 s s 1 2 s k 1 S S S... S 1,2,..., i ij ijk k i 1 i j i j k 13

14 Main effects If we were to approximate f(x) with a function g(x i ) 2 L E[( f( x) g( x i )) ] g( x ) E( Y x ) L L i L E[ Var( Y X )] min i i min f i =E(Y X i ) has the minimum loss L among univariate functions 14

15 Main effects V Var[ E( Y x )] i Var( Y) E[ Var( Y x i )] when we approximate f(x) with a function g(x i ), g*=e(y X i ) has the minimum loss L i 2 L E[( f( x) g( x)) ] g( x) f L V( Y) g( x) f ( x) L g( x ) E( Y x ) L L i L E[ Var( Y X )] min i i min 15

16 Main effects f ij =E(Y X i,x j ) has the minimum loss L among bivariate functions and so on Var(fi)/var(Y): also called non-parameteric R-squared; Pearson s correlation ratio; 16

17 Main effects f ( X, X ) X X X X X i ~ N(,1) 17

18 ANOVA model estimation Tensor product decomposition (reproducing kernel Hilbert space, RKHS) k k F Hj {1} Hj ( Hj Hi ) j 1 j 1 j i Orthogonal functional decomposition F q {1} j 1 F j 18

19 ANOVA model estimation Smoothing methods to estimate ANOVA decompositions, truncated at the 2 nd -3 rd order terms: SMOOTHING SPLINES ANOVA MODELS Smoothing the y vs x mapping (think of an HPfilter), that provides efficient convergence properties to the true ANOVA decomposition. [this is one possible methods, other are RBF s, kernel regressions, ] 19

20 ANOVA model estimation Denote the generic mapping as Y = f(x), where X [, 1] k and k is the number of parameters. The simplest example of smoothing spline mapping estimation of z is the additive model: g( X) g g x k i 1 i i 2

21 ANOVA model estimation To estimate g we can use a multivariate smoothing spline minimization problem, that is, given j, find the minimizer g(x) of: 1 N N 2 k ( y 1 2 n g( Xn)) λj [ gj( X j)] dx j n1 j1 where a Monte Carlo sample of dimension N is assumed. This minimization problem requires the estimation of the k hyperparameters j (also denoted as smoothing parameters): GCV, GML, etc. (see e.g. Wahba, 199; Gu, 22). 21

22 ANOVA model estimation We re-formulate the additive model for the general case with interactions as to find the minimizer g(x) of: 1 1 j y g λ P g N N q 2 ( n ( Xn)) n1 j1θj 2 F where the q-dimensional vector of j smoothing parameters needs to be optimized somehow. 22

23 ANOVA model estimation Wahba (199), Gu (22), Storlie et al. (27): en-bloc (like HP); Ratto et al., 24-27: based on recursive filtering and smoothing estimation: SDP modelling. (like KF-smoothing version of the HP); Liu et al. (22-26): polynomial basis expansion. 23

24 SDP modelling SDP modeling is one class of non-parametric smoothing approach first suggested by Young (1993). The estimation is performed with the help of the `classical' recursive (non-numerical) Kalman filter and associated fixed interval smoothing algorithms and has been applied for sensitivity analysis in Ratto et al. (24-27). 24

25 Mapping/sensitivity strategies OAT (Taylor): truncation; mapping by knowing all derivatives in a base point decomposition with infinite terms GSA (ANOVA): non-parametric regression/smoothing mapping on a spacefilling MC sample decomposition with finite terms 25

26 DSGE models Let us consider a generic DSGE model: y t endogenous variables, u t exogenous shocks X structural parameters. X can be characterised by plausible ranges, expressed in terms of prior distributions, or by a posterior distribution, as a result of an estimation. 26

27 DSGE models The model behaviour is a function of the values assumed for X within the prior or posterior space of structural parameters. Let Y be a generic 'output' of the model: a multiplier, a measure of fit, an IRF. Y depends on the values of X Y=f(X 1,, X k ), f non-linear analytic form is unknown 27

28 Mapping the reduced form of RE models Relationship between the reduced form of a rational expectation model and the structural coefficients. let the reduced form be y t =Ty t-1 +Bu t, 'outputs' Y of our analysis will be the entries in the transition matrix T(X 1,,X k ) or in the matrix B(X 1,,X k ). 28

29 Lubik Schorfheide (25) We analyse the reduced form coefficients describing the relationship between t vs e R,t We sample the structural coefficients from posterior ranges obtained after estimating the model using data for Canada. 29

30 Lubik Schorfheide (25) 3

31 Lubik Schorfheide (25) log( ( vs e )) π t Factorisation R,t Y exp( f f... f e) 1 exp( e) exp( f ) k j j k k k Y exp( e) exp( f ) g ε j j j Correction [new in DYNARE 4.5] j 31

32 LS25: pie vs e_r [-log(y)] 2 rho R, Si=.69 2 psi1, Si=. 1 k, Si=. 1 rho y s, Si= rho q, Si=. 2 tau, Si=. 1 rr, Si=. 1 rho A, Si= rho ies, Si=. p 1 1 psi2, Si=. 1 psi3, Si=. 1 alpha, Si=

33 LS25: R vs e_r [-log(y)].5 k, Si=.72.2 rho R, Si=.12.1 psi1, Si=.8.5 psi2, Si= psi3, Si=.1.2 tau, Si=. rho ies, Si=. p.1.1 rho A, Si= alpha, Si=..1.1 rr, Si=. rho s, Si=. y.1.1 rho q, Si=

34 Lubik Schorfheide (25) 1 Reduced form GSA - Log-transformed elements Values psi1 psi2 psi3 rho_r alpha rr k tau rho_q rho_a rho_ys rho_pies 34

35 Toolbox documentation The mapping of the reduced form soultion forces the use of samples from prior ranges or prior distributions, i.e.: pprior=1 (default); ppost= (default); [unless neighbourhood_width is applied] 35

36 Toolbox documentation 36

37 ss_anova Toolbox Download here the Recursive SS-ANOVA Toolbox for MATLAB. (MATLAB version < 7.5) 37

38 Using Monte Carlo filtering Store the sample of the state space A,B matrices; For 1-step ahead irf, perform the MCF sensitivity tests for B(i,j) within/outside the specified ranges. 38

39 Using Monte Carlo filtering: y ( π vs R ) [ 1,] t t1 rho_r psi1 k 1 rho_r psi1 k inside threshold outside threshold

40 Using Monte Carlo filtering: y ( π vs R ) [, ] t t1 1 rho_r 1 k psi Monte Carlo Filtering for pie vs R 1-dec 2-dec 3-dec 4-dec 5-dec 6-dec 7-dec 8-dec 9-dec 1-dec

41 Mapping reduced form solution 41