Benchmarking a system of time series: Denton s movement preservation principle vs. a data based procedure

Transcription

1 Workshop on Frontiers in Benchmarking Techniques and Their Application to Official Statistics 7 8 April 25 Benchmarking a sstem of time series: Denton s movement preservation principle vs. a data based procedure Tommaso Di Fonzo Marco Marini Dipartimento di Scienze Statistiche Università di Padova difonzo@stat.unipd.it Dip. di Contabilità Nazionale Istat Roma marco.marini@istat.it

2 OUTLINE OF THE PRESENTATION Benchmarking of economic data as a reconciliation problem Reconciliation of economic data as a matrix balancing problem RAS and QPD Least Squares Adjustment (LSA) Benchmarking a sstem of time series in a LSA framework Two benchmarking procedures Movement preservation principle VAR-based benchmarking Applications A simulation exercise Benchmarking monthl Italian industrial value added 2

3 The problem Economic data frequentl produced b different methods, sample surves, pieces of measuring equipment Data are often incomplete at some level of disaggregation SAM, I-O tables, national accounts, balance of paments Consequences Different estimates of the same variables More generall, linear restrictions which the observations should satisf but fail to Possible role of the economic statisticians To make indirect estimates of missing items (reconciliation) Need for using information efficientl to produce fitto-use 3 estimates

4 The target Estimation of a sstem of high-frequenc series subject to contemporaneous (between variables) and temporal (within variables) constraints The available information A table of high-frequenc preliminar series Known - assumed true - temporal aggregates A high frequenc contemporaneousl series, either assumed known or not aggregated 4

5 General considerations Preliminar estimation and benchmarking are considered here as logicall distinct phases (not completel true ) No simultaneous temporal and contemporaneous disaggregation of time series References into the paper (ver large literature for univariate problems and for reconciliation issues) Procedures taking into account the time-series-nature of the items to be reconciled (temporal correlation, Solomou and Weale, 1993, in the LSA framework) 5

6 Year Quarter T 1 A table of M quarterl time series to be reconciled h ,t 1,T 1,1 1,2 1,3 1,4 1,1 j, t j, T j,1 j,2 j,3 j,4 j,1 M, t M, T L L L L L 1 N j N 1,4 3,4 3 M,1 M,2 M,3 M,4 M,4 N 3 M,1 z z t,t z 1 z 2 z 3 z 4 z,1 2 1,4 N 2 j,4 N 2 M,4 N z 2 4 N 3 3 1,4 N 1 j,4 N 1 M,4 N z 1 4 N 2 4 1,4 N j,4 N M,4 N z 4 N 1 N 1, N M, j N, N z z 4 N,N 1 1, n M, n j, n z n 6

7 Reconciliation of economic data as a matrix balancing problem Given an (mxn) matrix Y (prior) determine an (mxn) matrix Y (target) that is close to Y and satisfies a given set of linear restrictions on its entries Problem 1 (exogenous constraints) Given two vectors u and v of dimensions m and n, respectivel, determine n j= 1 ij = u i, i=1,,m, and Y such that m i= 1 ij = v j, j=1,,n Determine Problem 2 (endogenous constraints) Y such that n n ij = ji j= 1 i= 1, i=1,,n 7

8 Exogenous constraints Variable Quarter 1 j M z 1 1,1 j,1 M,1 z 1 2 1,2 j,2 M,2 z 2 3 1,3 j,3 M,3 z 3 4 1,4 j,4 M,4 z 4 z 1 j M Contemporaneous endogenous constraints (A preliminar series z is available) Variable Quarter 1 j M z 1 1,1 j,1 M,1 z 2 1,2 3 1,3 4 1,4 1 j,2 j,3 j,4 j 1 M,2 z 2 M,3 z 3 M,4 z 4 M z More articulated structures for tables of two-wa classified series (Di Fonzo and Marini 23) 8

9 RAS procedure (onl with exogenous constraints) m n ij ij log i= 1 j= 1 ij Bacharach (197) QPD optimization : vectorized preliminar data-matrix : vectorized reconciled data-matrix ( ) 1 ( ' Q ) m n i= 1 j= 1 ( ) 2 ij ij Q= I mn ( ) 2 m n ij ij Almon (1968) Q= diag( ) i= 1 j= 1 ij Deming & Stephan (194), Friedlander (1961) 9

10 Least squares adjustment of economic data subject to linear restrictions (Stone et al., 1942) % : vectorized true (unknown) data-matrix should satisf the sstem of linear independent accounting constraints A% = a The model = % + e E ( e) = ( ee' ) E = V The solution 1 ( ') ( ) = + VA ' AVA a A E ( %)( %)' = '( ') 1 V VA AVA V 1

11 Automaticall balances when row and column totals are endogenous Based on the least-squares principle, leans on a long and solid tradition in statistics Data are adjusted in the light of their relative variances so as to satisf the linear restrictions Under normalit assumptions the reconciled estimates are ML Does not guarantee preservation of sign of the variables Possible lack of an good, objective basis for specifing the variance matrix 11

12 Benchmarking a sstem of time series with exogenous constraints Some notation M: number of series to be benchmarked n: number of high-frequenc periods N: number of low-frequenc periods s: temporal aggregation order (number of intra-annual periods) s=3 quarterl/monthl s=4 annual/quarterl s=12 annual/monthl n > Ns constrained / pure extrapolation 12

13 The available data M high-frequenc (sa, quarterl) preliminar time series, j = 1, K, M j M temporall aggregated (sa, annual) time series, j = 1, K, M j a contemporaneousl aggregated highfrequenc (sa, quarterl) time series z 13

14 The aggregation constraints Contemporaneous M j= 1 j z M j= 1 j = z Temporal J j j j J = j = 1, K, M j ( ) M n ' [ ' M ] J = I j N j' = [1 K 1] (flow series) 1 I = z ( I ) M J = H = a H ' 1M I n = IM J [ n+ NM nm] aggregation matrix ( ) 14

15 Due to contemporaneous aggregation of temporall aggregated series Matrix H has not full row rank N aggregated observations in a are redundant, onl r=n+n(m-1) free H w ' 1M 1 In I n = IM 1 J ' ' w= ' z 1 L M 1 ' Hw = w NB: The results are invariant with respect to the choice of a particular subvector of a, provided it has dimension (Nx1) (technical details in the paper) 15

16 Benchmarking as a LSA problem min ( ) 1 ( ' V ) subject to H = a ( ) ( ) = + VH' HVH' a H ( HVH ') : MP generalized inverse Formulae not involving MP generalized inverse ( ) = + VH V w H ' 1 w w w w = H w 16

17 Two benchmarking approaches 1. mathematical/mechanical solutions: the data-set is balanced b minimizing a penalt criterion which induces a covariance matrix (which is simpl a statistical artifact) 2. data-based solutions: the variabilit of the data to be reconciled is estimated using the available observations 17

18 1. Movement preservation principle (Denton, 1971, Eurostat, 1999, Bloem et al., 21) Proportional First Differences (PFD) 2 2 M n M n jt, jt, jt, 1 jt, 1 jt, jt, 1 j= 1 t= 2 jt, jt, 1 j= 1 t= 2 jt, jt, 1 Preserving growth rates 2 2 M n M n jt, jt, 1 jt, jt, 1 jt, jt, j= 1 t= 2 jt, 1 jt, 1 j= 1 t= 2 jt, 1 jt, 1 1. Data-based benchmarking (Guerrero and Nieto, 1999) Extract information about the variabilit of the series making use of the observed data 18

19 ( n n) Denton s PFD benchmarking Approximate solution Exact solution in the paper (Appendix B) matrix performing (approximate) first differences 1 L 1 1 L D = M M M O M M 1 1 L V = ˆ I ( D ' D ) 1 ˆ ˆ = diag ( ) M Di Fonzo and Marini, 23: Saving of computation time and storage b exploiting the patterns of the involved matrices Solution preserving growth rates ( ideal objective formulation Bloem et al., 21) 19 for

20 Data based benchmarking Guerrero and Nieto (1999) Benchmarking exploiting the autoregressive features of the preliminar series =,, K,, K, ' Assumptions ( ) tg, 1, t 2, t jt, Mt, ( K K ) % = %, %,, %,, % ' tg, 1, t 2, t jt, Mt, 1. Before observing g, % tg, tg, admits a stationar VAR(p) representation Π( L)( ) = ε 2. Once g % tg, tg, tg, Π ( % g g) = ε g ( ' ) E ε ε = I Σ g g n is given, ( ' ) g g g E ε ε = P Σ and % tg, : same VAR representation with different covariance matrix 3. tg, 2

21 For Π, P and Σ known, MMSE for g 1 1 V = Π ( P ΣΠ ) ' % given ( g, a, g) g ( ') (, ) g = g + g ' g a g g V C CV C C C: complete aggregation matrix for dataset vectorized b time Data-set vectorized b variable, onl r linear independent constraints 1 1 V = Π ( Σ P) Π ' ' 1 ( w ) ( ) ' = + w w w VH H VH w H Technical details in the paper (Appendix B) Cov( ) = V VH ( H VH ) V ' ' 1 w w w For 1 1 ˆΠ, ˆP and ˆΣ, ˆ ˆ ( ˆ ˆ) ˆ V = Π Σ P Π ' Data-based LSA 21

22 First stage A two-stage operational procedure - Estimate a VAR(p) for - Get an estimate of Π and Σ for P= I n, then calculate an estimate of V g - Tentative benchmarked series ŷ Second stage - Test whiteness of the residual series ˆ ˆ ˆ Π( ) = ε to verif P = I n - If not, fit a VAR model to ε ˆ tg, and derive the implied estimate of V g - Estimate ŷ 22

23 Empirical validation of the compatibilit between benchmarked and preliminar series A discrepanc measure: Wald statistic DM = ( C )'[ CV C)] ( C ) ' ag, g g ag, g Asmptoticall 2 χ r 1 1' ' r rank[ = CΠ ( Q ΣΠ ) C )] If DM rejects other preliminar series have to be found 23

24 First application: simulated data % =.2 +.5% +.1% + a% 1, t 1, t 1 2, t 1 1, t % =.3+.4% +.5% +.25% + a% 2, t 1, t 1 2, t 1 1, t 2 2, t '.4 E( aa %% t t) = Σ =.1 (orthogonal).4.5 ' E( aa %% t t) = Σ =.5.1 (non-orthogonal) z = H (s=4, q to a) follows the same VAR Preliminar series tg, model with :.5 E( aa t t) = Ψ =.2 (orthogonal).5.5 E( aa t t) = Ψ =.5.2 (non-orthogonal) 24

25 First application: simulated data 1 replications (k=1) relative corrections : absolute corrections : c jt, = % ˆ j, t j, t c = ˆ % j, t j, t j, t j = 1,2, t = 1,...,44 Average and standard deviation across the k experiments on: median, min, max, range, std 25

26 First application: simulated data White noise disturbances (k = 1) relative absolute median min max range mean std PFD mean std Data based mean std Correlated disturbances (k = 1) relative absolute median min max range mean std PFD mean std Data based mean std

27 Second application Benchmarking Italian monthl industrial v.a. Quarterl value added for six industrial sectors from to ( ) Monthl industrial production indices for the same activities ( x j ) The monthl total value added is estimated through Chow-Lin Preliminar series j are derived b regressing (quarterl basis) v.a. on a constant, a trend and x j j We face a benchmarking problem: the preliminar series are not in line with both (quarterl) and the estimated monthl total value added z j 27

28 Second application: real data Quarterl value added for six industrial sectors in Ital. Period 199:1-22:4 1 Food 2 Textiles Chemicals 4 Basic metals Machiner 6 Electrical equipment

29 Temporal discrepancies (%) before reconciliation 1 Food 2 Textiles Chemicals 4 Basic metals Machiner 6 Electrical equipment

30 Discrepancies (in %) between z and the contemporaneous sum of j Estimated coefficients (and t-statistics) of the VAR(1). 1 st step eq. const 1, t 1 2, t 1 3, t 1 4, t 1 5, t 1 6, t 1 2 1,t t-stat ,t t-stat ,t t-stat ,t t-stat ,t t-stat ,t t-stat Gre-shaded background: t-stat < 2 R

31 Preliminar (dotted line) and adjusted series through the data-based benchmarking procedure (solid line) 1 Food 2 Textiles Chemicals 4 Basic metals Machiner 6 Electrical equipment

32 Second application: Benchmarking Italian monthl industrial v.a. Corrections to monthl rates of changes of j PFD GN sect med min max range st.dev med min max range st.dev

33 Estimated residuals from the VAR(1). First step. Sector 1 Sector Sector 3 Sector Sector 5 Sector

34 Estimated residuals from the VAR(3). Second step. 2 Sector 1 2 Sector Sector 3 2 Sector Sector 5 2 Sector Ljung-Box statistics of the estimated residuals first step second step eq. Q(6) p-value Q(6) p-value

35 Inverse roots of AR characteristic polnomial in the VAR estimation 1.5 Inverse Roots of AR Characteristic Polnomial VAR(1) - 1st step

36 Inverse roots of AR characteristic polnomial in the VAR estimation Inverse Roots of AR Characteristic Polnomial 1.5 VAR(3) - 2nd step

37 Conclusions 1 Denton s closeness condition is reasonable but mechanical (let down upon all the series whatever characteristics the possess) No link with the relative accurac of the data to be reconciled Recovering measures of reliabilit from time series variables observed with noise is not straightforward In line with Weale (1992), Solomou and Weale (1993) and Smith et al. (1998), GN procedure gets an estimate of the covariance matrix to be used in the least-squares adjustment Evaluating the compatibilit of the preliminar series: could help in stressing possible deficiencies in (part of) the information basis 37

38 Conclusions 2 Data based techniques might improve the results of a benchmarking problem exploiting the covariance structure of the preliminar series In so far a larger volatilit of the preliminar series is a signal of lower qualit, the data-based benchmarked procedure seems to take into account this characteristics, producing relativel larger corrections to such series than those registered b the other, relativel less variable, series. Naive estimates of the VAR have been used in our examples. Alternative specifications of the VAR can be tested (e.g., coefficient restrictions, VECM). Furthermore, the automatic lag length selection procedure can be improved 38

39 Thank You