SAMPLING DYNAMICAL SYSTEMS Melvin J. Hinich Applied Research Labratries The University f Texas at Austin Austin, TX 78713-8029, USA (512) 835-3278 (Vice) 835-3259 (Fax) hinich@mail.la.utexas.edu ABSTRACT Linear dynamical systems are widely used in many different fields frm engineering t ecnmics. One simple but imprtant class f such systems is called the single input transfer functin mdel. Suppse that all the variables f the system are sampled fr a perid using a fixed sample rate. The central issue f this paper is the determinatin f the smallest sampling rate that will yield a sample that will allw the investigatr t identify the discrete-time representatin f the system. A critical sampling rate exists that will identify the mdel. This rate, called the Nyquist rate, is twice the highest frequency cmpnent f the system. Sampling at a lwer rate will result in an identificatin prblem that is serius. The standard assumptins made abut the mdel and the unbserved innvatin errrs in the mdel prtect the investigatrs frm the identificatin prblem and resulting biases f undersampling. The critical assumptin that is needed t identify an undersampled system is that at least ne f the exgenus time series is white nise. 1. INTRODUCTION Linear dynamical systems are widely used in many different fields frm engineering t ecnmics. One simple but imprtant class f such systems is
called the single input transfer functin mdel. Suppse that all the variables f the system are sampled fr a perid using a fixed sample rate. The central issue f this paper is the determinatin f the smallest sampling rate that will yield a sample that will allw the investigatr t identify the discrete-time representatin f the system. The determinatin f the minimal sufficient sampling rate is a mathematical prblem that was slved years ag using Furier transfrms. A critical sampling rate exists that will identify the mdel. This rate, called the Nyquist rate (p.388, Andersn, 1971) is twice the highest frequency cmpnent f the system. The imprtance f the Nyquist rate fr system identificatin is knwn in the science and engineering spectral analysis literature yet it has been largely ignred in the literature that applies the time dmain methdlgy ppularized by Bx and Jenkins (1970). The standard apprach is t start with a discrete-time linear mdel. An alternative apprach, strngly advcated by Wymer (1972,1997) and Bergstrm (1990), is t start with a linear stchastic differential equatin mdel fr the system. Telser (1967) discusses the identificatin prblem inherent in estimating the parameters f a difference equatin using a data series that is a mving sum f discrete-time bservatins. Telser als recgnized the cnnectin between the parameter identificatin prblem fr discrete-time data and the aliasing f the perid f a sinusid. Phillips (1973) addresses the identificatin f parameters f a cntinuus time differential equatin using discrete-time data. Phillips shws that it is pssible t identify a finite parameter dynamical system by assuming linear cnstraints n the structural matrix even when the stchastic disturbance is aliased. His paper shuld have led t an imprtant set f advances in time series mdel identificatin but it did nt catch n, perhaps because as with Telser s paper it was vershadwed by the ppularity f the Bx and Jenkins pint and click methdlgy. The sampling rate issue is als cnfused r ignred in the ecnmetrics literature extending the Bx and Jenkins methdlgy t ecnmics (see Granger and Newbld, 1976, and Harvey, 1981). The standard reasn usually 2
given by time series ecnmetricians fr ignring the sampling issue is that it is irrelevant fr the identificatin and estimatin prblem fr a discrete-time linear system mdel. The parameters f the mdel are estimated by sample autcrrelatins and the sample autcrrelatins are unbiased. A mre subtle reasn fr ignring the sampling rate is that the sampling rate used t cllect the data was fixed when the data was cllected and it thus can nt be changed. This pint is well made by Telser in the paper previusly cited. These arguments are true but they deflect attentin away frm the fact that the mdels used and the assumptins made abut the unbserved innvatin errrs in the mdel prtect the investigatrs frm the identificatin prblem and resulting biases f undersampling. The critical assumptin needed t identify an undersampled system is that at least ne f the exgenus time series is white nise. The standard frm fr a causal linear transfer functin mdel in cntinuus time is as fllws where x (t) dentes the input time series and y (t) dentes the utput: (1.1) y ( t) = h( s) x( t s) ds 0 The functin h (t) is the impulse respnse f the mdel. In engineering and science applicatins, the time series are called signals and (1.1) is called a filtering peratin where the input signal x (t) is filtering by the impulse respnse t yield the utput signal y (t). Assume that h ( t) = 0 fr t > T. The impulse respnse has finite supprt. The input and utput signals are sampled t prduce a set f data. Since the prblem is mathematical and nt statistical there is n reasn t add a nise signal in (1.1) and the signals are functins f time and nt cntinuus time stchastic prcesses. The sampling issues discussed in this paper apply t any statistical time series mdel. 3
2. BANDLIMITED SAMPLING If x (t) and h (t) are abslutely integrable, the Furier transfrms (2.1) X ( f ) = x( t)exp( i2π t) dt and H ( f ) = h( t)exp( i2π t) dt exist and Y ( f ) = H ( f ) X ( f ). Since x (t) is real X ( f ) = X ( f ), the cmplex cnjugate f X ( f ) and similarly fr the cmplex transfer functin H ( f ). Assume that the set f psitive supprt fr X ( f ) is ( f, f ) fr sme frequency f. This frequency is the bandlimit f x (t). The transfer functin H ( f ) has infinite supprt since h (t) is finite. Suppse that the signal is sampled at the Nyquist rate 2 f, r equivalently at a fixed sampling interval τ = /(2 f ). Then the discrete time versin f the mdel (1.1) is 1 N sin(2π f ( t s)) (2.1) y( tk ) = h ( tn ) x( tk n ) where h ( t) = h( s) ds, π ( t s) n= 0 t k = kτ and N = T / τ (Chapter 10, Bracewell, 1986). If τ is much smaller than T then the impulse respnse parameters h ) h( kτ ) with an errr f rder 1 O ( τ ). The discrete-time cnvlutin f the finite h ) sequence with the x ) sequence t yield the y ) sequence is a set f linear equatins which can be slved t btain the impulse respnse parameters h ) fr k=1,, N. Suppse that the sampling rate used is f rather than the Nyquist rate 2 f. Then every ther h0 (( k n) τ ) and x ) are missing in the system f equatins (2.1). It thus is impssible t slve fr the N values f h ). Fr example, suppse that y ) = x( kτ ) + a x(( k 1) τ ). Then the tw equatins fr k=1 and 2 fr times t 1 = τ and t = 3 2 τ are (2.3) y ( τ ) = x( τ ) + a x(0) and y ( 3τ ) = x(3τ ) + a x(2τ ). 4
These equatins can nt be slved t find a since x (0) and x ( 2τ ) are nt bserved. Using mre equatins is fruitless since the x ) fr even values f k are nt bserved. The parameter a is nt estimable since the system is nt identified. The investigatr must interplate the missing values in rder t estimate a. Interplatin requires sme prir knwledge abut the functinal frm f the input. S far the sampling issue has been separated frm the stchastic linear mdel prblem which is the mtivatin fr this expsitin. Let us turn t the stchastic mdel. 3. STOCHASTIC TRANSFER FUNCTION MODEL Suppse that the { x )} in expressin (2.1) is a sequence f bservatins f a zer mean randm prcess with a given jint distributin. The cvariance functin f { x )} is c xx ) = Ex( nτ ) x(( n + k) τ ). Then the crss cvariance functin fr { x )} and { y )} is N (3.1) cxy ) = h ( nτ ) cxx (( k n) τ ). n= 0 This system f linear equatins is used t slve fr the parameters ). If the system in (2.1) were used t btain an rdinary least squares fit f the parameters, then the slutin wuld be the slutin using (3.1) with sample estimates f the cvariances c xx ) and the crss cvariances c xy ) ignring end effects. Cnsider the cvariances and crss cvariances t be knwn values t simplify expsitin. Once again if the prcesses are sampled at a slwer rate than 2 f then the slutin f the linear system (3.1) will prduce a distrted estimate f the filter parameters unless the investigatr can prduce a valid interplatin fr the missing cvariances. The impulse respnse is nt identified. h An example is helpful here. Suppse that the impulse respnse is ( nτ ) = 10 cs(2πnτ / N) and the input is a first rder autregressive prcess h 5
AR(ρ) whse innvatins variance is ne. Thus the cvariance f the input is kτ 2 1 c xx ) = ρ (1 ρ ). Assume that the prcesses are sampled at the rate f / 2 and thus every furth value f the prcesses is bserved. Figure 1 cmpares the impulse respnse recvered frm a least squares slutin f the under identified system fr ρ =0.9 with the skip sampled true impulse respnse. Figure 2 displays the results fr a sampling rate f f / 6. The undersampling prduces a distrted picture f the respnse f the system. 4. IDENTIFICATION BY WHITE NOISE There is a special case fr which a subset f the impulse respnse parameters will be identified. Suppse that { x )} is white nise, that is c xx ) = 0 fr all k 0 Then c 2 xy ( k ) h ) σ x τ = frm (3.1). In this case, if the prcesses are sampled at a slwer rate than Nyquist, the estimated impulse respnse parameters will be an under-sampled versin f the filter parameters. Fr example if h ) = exp( ckτ ) and the prcess is sampled at a rate f f / 10 the recvered filter parameters will be exp( ck10τ ) fr k N / 10. The recverd impulse respnse will prvide gd shrt-term predictin fr the y ( k10τ ). If ne culd cntrl the input then it is bvius that ne wuld use white nise input. It is the time series equivalent f an rthgnal design in the statistical design f experiments literature. In the mre general dynamical systems the state space representatin is f the frm (4.1) y ) = Ay(( k 1) τ ) + e( kτ ) where y ) is an n dimensinal vectr f bserved exgenus and endgenus time series, A is a nnsingular system matrix and e ) is an n dimensinal vectr f unbserved exgenus randm inputs which are called innvatins in ecnmics. The innvatins sequence mdels the real input t the linear system and are nt a mathematical representatin. If n the ther hand expressin (4.1) is seen as a statistical mdel t represent the crrelatins in the data, then 6
ne shuld questin the validity f using this statistical mdel t make statements abut causal relatinships in the true system. The system matrix is identified if the innvatins are jintly white. This is the generalizatin f the white input in expressin (3.1). If the system is undersampled then the eigenfunctins f the system are similarly under-sampled but their pattern is nt distrted. The identificatin assumptin n the unbserved innvatins time series is the mathematically equivalent t the identificatin f the impulse respnse f a linear transfer functin with an imprtant distinctin. The innvatins are nt bserved while the input f the transfer functin is. If nature is bliging and makes the innvatins white t help the investigatr, then all is well. If nt then the cvariance structure f the input must be mdeled. Anther apprach is t reject the Markv mdel (4.1) and use a prperly sampled multivariate transfer functin mdel fr frecasting. A multivariate transfer functin is an intellectually and technically valid apprach t mdeling and frecasting a linear system where the input can be measured. Transfer functins are widely applied in engineering and science. But it is nt in favr amng mst time series ecnmetricians. 5 CONCLUSIONS The results presented in this paper pse a real prblem fr macrecnmists wh use time series mdels t mdel ecnmics systems. It is impssible t btain high frequency data fr standard macrecnmic series such as interest rates, utput, and prices. The highest frequency macrecnmic data available is mnthly. Thus the analyst must use the dynamical mdel (4.1) and hpe that the innvatins are white. If the mdel is a gd apprximatin t reality then the analyst can get sme sense f the dynamical respnse f the system. Otherwise nly the trends can be analyzed. 7
Acknwledgments I wish t thank Chris Brks, Bart Brnnenberg, Hustn Stkes and Phil Wild fr reading the manuscript and giving me cnstructive criticism REFERENCES Andersn, T.W. (1971), The Statistical Analysis f Time Series. New Yrk: Jhn Wiley & Sns. Bergstrm, A.R. (1990), Cntinuus Time Ecnmetric Mdeling. Oxfrd: Oxfrd University Press. Bx, G.E.P. and G.M. Jenkins (1970), Time Series Analysis: Frecasting and Cntrl, San Francisc: Hlden-Day. Bracewell, R.N. (1986), The Furier Transfrm and Its Applicatins, Secnd Editin, Revised. New Yrk: McGraw-Hill. Granger, C.W.J. and P. Newbld (1976), Frecasting Ecnmic Time Series. New Yrk: Academic Press. Harvey, A.C (1981), Time Series Mdels. Oxfrd: Philip Allan. Phillips, P.C.B (1973), The prblem f identificatin in finite parameter cntinuus time mdels, Jurnal f Ecnmetrics 1, 351-362. Telser, L.G. (1967), Discrete samples and mving sums in statinary stchastic prcesses, Jurnal f the American Statistical Assciatin 62, 484-499. Wymer, C.R. (1972), Ecnmetric estimatin f stchastic differential equatin systems, Ecnmetrica 40, 565-577. Wymer, C.R. (1997), Structural nnlinear cntinuus-time mdels in ecnmetrics, Macrecnmic Dynamics 1 (2), 518-548. 8
Aliased & True Impulse Respnses fr a Sampling Interval f 4τ 120 100 80 Aliased Respnse True Respnse 60 Amplitud 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36-20 -40-60 Time Figure 1 9
120 Aliased & True Impulse Respnses fr a Sampling Interval f 12τ 100 80 60 Aliased Respnse True Respnse Amplitud 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36-20 -40 Time Figure 2 10