Latent Differential Equation Modeling with Multivariate Multi-Occasion Indicators

Transcription

1 Latent Differential Eqation Modeling with Mltivariate Mlti-Occasion Indicators Steven M. Boker University of Notre Dame Michael C. Neale Medical College of Virginia Joseph R. Rasch University of Notre Dame Draft November 17, 23 Please do not cite or qote Abstract A nmber of models for estimating the coefficients of dynamical systems have been proposed. One is the exact discrete model, another is the latent differences model or proportional change model, and a third is a continos time manifest variable differential strctral model. These models differ from standard growth crve modeling in that they intend to illminate processes generating individal trajectories over time rather than jst estimate an aggregate best trajectory. The crrent work proposes a novel approach to the modeling of mltivariate change by first creating a lagged covariance matrix. Then rather than sing factor loadings to estimate average growth characteristics as in other growth crve modeling related methods, confirmatory factor loadings are fixed across the time dimension and freed across the variables dimension, in sch a way as to force the factor scores to estimate instantaneos derivatives. Then, the factor covariances are strctred as a regression that allows the estimation of parameters of differential eqation models of dynamical systems theories proposed to accont for the data. Reslts of a simlation will be presented illstrating strengths and weaknesses of the latent differential eqation model. Fnding for this work was provided in part by NIH Grant R29 AG Correspondence may be addressed to Steven M. Boker, Department of Psychology, The University of Notre Dame, Notre Dame Indiana 466, USA; sent to sboker@nd.ed; or browsers pointed to sboker.

2 LATENT DIFFERENTIAL EQUATION MODELING 2 Introdction In the behavioral sciences, there is increasing interest in nderstanding and characterizing mechanisms of developmental and behavioral processes. Measrements of mltiple indicators obtained on mltiple occasions on a single individal may show some intraindividal change and intraindividal variability. Process oriented theories may predict strctral patterning in these ideographic measrements. Strctral eqations modeling techniqes can be sed to test sch theories by fitting confirmatory models of the implied dynamical systems to repeated observations data. The crrent chapter explores one method for constrcting and testing confirmatory latent variable strctral models in which the latent constrcts (a) evolve continosly over time and (b) have linear relationships between their derivatives. The method appears to generalize well and is expected to be able to be applied to systems well beyond the simple example presented here. The chapter will begin with a rationale for stdying behavioral processes from a dynamical systems perspective, introdce latent differential eqations confirmatory factor models, present the reslts of two simlations testing the viability of this model for estimating parameters of a simple linear system, and then discss ftre sbstantive and methodological qestions that this line of modeling may address. Self Reglating Systems For the prposes of this chapter we will define a psychological process as the time evolving evidence of some behavioral or developmental mechanism. A psychological process will be assmed to have associated with it a set of indicator variables that change over time in a way that is at least partially determined by the behavioral or developmental mechanism. One can consider this definition of a psychological process in terms of one or more latent variables whose vales change in some, at least partially, deterministic way. The lawflness of these changes is then considered to be evidence in spport of a behavioral or developmental mechanism. One form that psychological processes may take is that of a self reglating system. Self reglation involves the notion that changes in a system have orderly and lawfl relations even in the absence of exogenos effects. Many psychological processes may fit within the framework of self reglation. Some possible examples of self reglating psychological processes inclde lifespan development of cognitive abilities (Donaldson & Horn, 1992; McArdle, Hamagami, Meredith, & Bradway, in press), adolescent sbstance abse (Boker & Graham, 1998), self reported mental health in recent widowhood (Bisconti, 21), or anxiety levels of children (Cmmings & Davies, 22). Self reglation can be partitioned into two main categories: active and passive. A physical example of an active self reglating system might be the heating and cooling system in a bilding. This temperatre control system is self reglating and has active elements: one or more temperatre sensors, heat sorces, cooling sorces and perhaps fans for moving the air within the bilding. When temperatre sensors indicate a measrement that differs from the desired eqilibrim state of the system, active elements are engaged sch that a change in temperatre is effected in the bilding. This change is read by the sensor and some frther activity is triggered. Active self reglation generally implies some sort of sensor effector feedback mechanism.

3 LATENT DIFFERENTIAL EQUATION MODELING 3 One physical example of a, passive self reglating system is a pendlm with friction in the earth s gravitational field. If the pendlm is raised to one side and let go, the reslt is a reglar oscillation ntil friction finally damps the velocity of the pendlm to its motionless eqilibrim state along the axis between the pivot of the pendlm and the center of the earth. Althogh lawfl relationships exist relating changes in the pendlm s motion to its crrent distance from eqilibrim, there is no active sensor mechanism trning on and off an effector that pshes the pendlm first in one direction and then the other. The movement of the pendlm is a passive response to the nchanging gravitational field in which it embedded. Consider the behavior of the system shown in Figre 1 a. The vale of the variable (here labeled displacement ) starts as a positive nmber and then as time progresses, the vale first becomes negative, then positive, then negative again; oscillating back and forth arond an eqilibrim point of zero displacement. Eventally, the oscillations are damped to nearly zero. The crve shown in Figre 1 a is called a trajectory; it describes the behavior of a single individal instance of a system. Note that when time is eqal to zero, this trajectory has a vale for its displacement. Also note that at time= there is a slope to the crve. These two vales, the displacement and slope (or first derivative of displacement) are called the initial vales for the trajectory (here x = 1, ẋ = ). Every trajectory, that is every individal instance, of this system has a set of initial vales that is the state of the system at an arbitrarily assigned time of zero. In Figre 1 b, the vale of displacement (here labeled x) and its first derivative (here labeled ẋ) are plotted against one another for the same trajectory shown in Figre 1 a. The oter end of the spiral at (x = 1, ẋ = ) again represents the initial conditions for this trajectory. The reglarity of the spiral demonstrates that there is a relationship between the displacement (x) and its first derivative (ẋ) with respect to time. However, this relationship alone does not determine the system, since for any vale of x there is more than one possible vale of ẋ. In order to completely describe this system at some time t, one needs also to take into accont the crvatre (second derivative of x with respect to time) of the trajectory at time t. One way to visalize this is by the se of a vector field (Boker & McArdle, in press) as shown in Figre 1 c. Here we see an eleven by eleven grid of arrows. The tail of each arrow is located at one point in an eleven by eleven grid of initial conditions of x and ẋ. The head of the arrow points to where a trajectory wold be a short interval τ later in time. Arrows that point straight p or straight down have not changed their vale of ẋ. Since the second derivative of x (written as ẍ) is the change in ẋ, these vertically oriented arrows represent initial conditions in which the trajectory had no crvatre, i.e. initial conditions in which ẍ = dring the interval of time τ. Similarly, horizontal arrows in Figre 1 c represent initial conditions for which there was no change in x dring the interval τ. Ths, these initial conditions had an average derivative, ẋ, of zero. It trns ot that for this system, if we know the vale of x and ẋ at time t then we can exactly calclate the vale for ẍ at time t. The way that these three qantities evolve over time will remain symmetric; that is, when one gets larger another gets proportionally smaller so that a weighted sm of x, ẋ, and ẍ will always be eqal to zero. It is important to note at this point that active and passive mechanisms for self

4 LATENT DIFFERENTIAL EQUATION MODELING 4 a. b. 1 1 Displacement Temperatre X Time Time Ẋ c. d X -1-1 X Ẋ Ẋ Figre 1. For views of a damped linear oscillator. (a) A single trajectory: the evolving vales over time of displacement from eqilibrim for a single set of initial conditions. (b) A phase space: the evolving vales over time of both displacement and the first derivative of displacement plotted against one another for a single set of initial conditions. (c) A vector field: the evolving vales over one time step of an grid of vales of initial conditions of both displacement and the first derivative of displacement, and (d) A sample of initial conditions displayed as a vector field

5 LATENT DIFFERENTIAL EQUATION MODELING reglation do not necessarily reslt in different self reglating processes. For any observed process there might be many self reglating mechanisms which cold have generated the data. For instance, the same set of relationships between derivatives that prodced the trajectories in the graphs in Figre 1 might have reslted from either an active or passive mechanism. The methods discssed in this chapter can be sed to falsify hypotheses abot processes. One mst be carefl to recall that in so doing, we may not necessarily falsify a corresponding hypothesis abot mechanisms since the mapping from nderlying mechanisms to observed processes can be many to one. The methods we discss provide estimates of parameters of intrinsic dynamics of a hypothesized system, in other words parameters of a system that exhibits a self reglating process. a. b. 3 Sample Tre Scores Eta=. Random Normal Score Score Score Score Time Time Occasion Occasion Figre 2. Measred vales from three occasions from 1 instances of two different processes. (a) A random sample of initial conditions and three observations from a damped linear oscillator process. Each observation at occasions 2 and 3 is completely determined by the initial conditions at occasion 1. (b) A random sample from normally distribted random nmbers. All observations at all occasions are independent of one another. In this case, a nll hypothesis of no nderlying process is tre. In Figre 1 d is displayed the trajectory evoltion over a short interval of time τ of a random sample of initial conditions. Essentially, it is a random sample from the vector field displayed in Figre 1 c. This is the sort of data with which we are concerned: relatively few observations per individal and a random sample of individals who may each have their own initial conditions. These initial conditions may be independent of the self reglating psychological process which we wish to nderstand and for which we wish to qantify parameters. Sch data might be represented by the graphs shown in Figre 2. In Figre 2 a are three observations drawn from 1 instances of the system shown Figre 1. In Figre 2 b are three observations drawn from 1 instances of normally distribted random nmbers. All of the observations in Figre 2 b are independent: essentially measrement error. Any method that shows promise, mst be able to reliably distingish between these two cases. That is to say, if factor scores exhibit no deterministic change within individal, we mst not mistake observed change for a self reglating process.

6 LATENT DIFFERENTIAL EQUATION MODELING 6 Differential Eqations Models In the physical sciences, models of processes typically take the form of differential eqations. Differential eqations describe the relationships between the vale of a variable and its derivatives with respect to time. Ths, if there were a proportional relationship between the vale of a variable s displacement from eqilibrim and its instantaneos slope, when displaced from eqilibrim by some exogenos force, the variable wold retrn to eqilibrim with a trajectory that was exponential in shape. This is the differential eqation form of a model also known as the proportional change model (see McArdle & Hamagami, this volme) and in a discrete time form is the atoregressive model with fixed coefficients over time. One benefit to fitting differential eqations models accres in comparison with growth crve models (e.g. McArdle & Epstein, 1987). One way of thinking of growth crve models is that they estimate a single best aggregate trajectory for a differential eqations model. Since each trajectory has its own set of initial conditions, this implies that a growth crve model is estimating a single best set of parameters as well as a single best set of initial conditions. A growth crve approach is sefl if one is interested in finding this best trajectory (growth crve) and if the initial conditions for each individal are in relation to a reference time with a meaningfl zero (sch as date of birth) that can be meaningflly compared across individals. However, when there is no reference time that can be sed to eqate individals with respect to the dynamic of the system of interest, growth crves can confond individal differences in initial conditions with individal differences in the parameters for the crves (see Boker & Bisconti, in press, for examples). Fitting lagged state space models bypasses this problem since initial conditions need not be estimated. Examples of state space methods inclde the method proposed in this chapter, discrete time models (e.g. Jones, 1993), forms of dynamic factor analysis (e.g. Molenaar, 198), or stochastic differential eqations (e.g. Od & Jansen, 2)) In these models individal differences in initial conditions are not confonded with individal differences in the dynamics of the behavior since time is only treated as relative to other measrements. Ths state space models have a distinct advantage when the initial conditions may be de to nknown exogenos inflences: a participant in a stdy may have an essentially random state when he or she begins an experimental protocol, bt changes in behavior dring the protocol may be reliable (see Boker & Nesselroade, 22, for a more complete discssion of this so called phase problem). For the prposes of this chapter, we will consider one of the simplest differential eqations: the damped linear oscillator. Many other examples of this differential eqations models exist (see e.g. Hbbard & West, 1991; Thompson & Stewart, 1986, for introdctions). Sppose there exists a latent variable F with a particlar form of intrinsic dynamics sch that at every time t F = ζf + ηf + e F (1) where the residal e F is normally distribted with mean zero. Frther sppose that the latent variable exhibits a factor strctre sch that x ij = 1F ij + xij (2) y ij = af ij + yij

7 LATENT DIFFERENTIAL EQUATION MODELING 7 z ij = bf ij + zij where x, y, and z are manifest indicators of F measred on individal i at occasion j, 1, a, and b are the factor loadings, and x, y, and z are the corresponding niqenesses. The factor loading for x is constrained to be 1 in order to identify the scale of the latent variable F. In this way, we have defined a factor whose score changes over time sch that it obeys the intrinsic dynamic characteristic of a damped linear oscillator. The two parameters η and ζ have sefl meanings. The parameter η is proportional to the sqare of the freqency ω of oscillation sch that η = (2πω) 2. The parameter ζ is the damping parameter and when ζ is negative, it is similar to the friction of a swinging pendlm. When ζ is positive, it amplifies change in the system sch that small pertrbances at any given time tend to lead to larger changes later a b c d. e. f Figre 3. Six example trajectories for the latent variable F that conform to Eqation 1. Trajectories (a) and (d) have slow and fast freqencies respectively, bt have no damping, i.e. are frictionless. Trajectories (b) and (e) have slow and fast freqencies with moderate damping. Trajectories (c) and (f) have slow and fast freqencies with high damping. Figre 3 plots six possible trajectories that all conform to Eqation 1, bt which have differences in their vales of η and ζ. The graphs in each row of Figre 3 have the same vale of η bt different vales of ζ. The graphs in each colmn of Figre 3 have the same vale of ζ bt different vales of η. The top row has a vale of η that is closer to zero than the bottom row and ths the top row exhibits slower oscillations than the bottom row of graphs. In the leftmost colmn ζ =, the vale of ζ is a small negative nmber in the middle colmn, and a larger negative nmber in the rightmost colmn.

8 LATENT DIFFERENTIAL EQUATION MODELING 8 Of corse factor scores are nobservable, so we are not able to estimate individal trajectories. We do not wish to estimate some optimm aggregate trajectory of these factor scores since that implies an estimation of the nobservable initial conditions of the trajectory. Instead, we wish to estimate the parameters of Eqation 1, which are fnctions of the covariances between the factor and its derivatives. To do so, we will take advantage of the fact that covariances between latent variables can be estimated even when the latent scores themselves are nknown. In the next section, we will place this method in context by first presenting a brief discssion of a two step nivariate manifest variable method called Local Linear Approximation (LLA). We will next extend that to the case of a Univariate Latent Differential Eqation (ULDE). We will then present the Mltivariate Latent Differential Eqation (MLDE) model that can simltaneosly estimate the parameters from Eqations 1 and 2. Reslts of a simlation testing the viability of the MLDE model for estimation of parameters of a known system will be presented. A second simlation will test whether the MLDE model will give an answer that is incorrect when presented with factors that have no time deterministic strctre (as in Figre 2 B). Finally, we will discss potential applications of this method to more complex copled systems and nonlinear models. Local Linear Approximation One approach to estimating the intrinsic dynamics of a psychological process is to first estimate the derivatives of the process for each occasion of measrement and then fit either a regression model (Boker & Nesselroade, 22), a strctral model (Boker, 21), or mltilevel model (Boker & Ghisletta, 21) to those reslting variables. This approach has advantages and disadvantages. The two main advantages to estimating derivatives prior to estimating the dynamical parameters of a system are a great degree of flexibility in specifying the strctre of the differential eqation model and being able to se a wide variety of off the shelf estimation procedres. One method for estimating derivatives from manifest variable processes is Local Linear Approximation (LLA). LLA assmes that the process can be approximated linearly over short intervals of time and that the derivatives at a time t can be estimated from measrements occring within some small interval τ of the chosen time t. First the slopes between measrements at time t τ and t+τ are calclated and then the estimates of the first and second derivatives at time t are given by the average of these two slopes and the change in these two slopes respectively. This simple minded estimation of the derivatives can do remarkably well in estimating dynamical parameters. While LLA has the advantage of sing only three occasions of measrement to estimate the derivatives, it can prove to have biased estimates of the dynamical parameters when the interval τ is not optimal (Boker & Nesselroade, 22). In addition, measrement error can masqerade as high freqency signal. While there are indications that there may be soltions to these technical problems, LLA is at the core a manifest variable method. If one is interested in the dynamics of a variable for which a score cannot be directly observed, then LLA is inappropriate. In this case we wold wish to have a measrement model for sch a latent constrct.

9 LATENT DIFFERENTIAL EQUATION MODELING 9 Univariate Latent Differential Eqation Model There are two ways that measrement models are commonly constrcted, either across variables as in a factor model or across time as in a latent growth crve model. We will start by bilding a measrement model across time, bt with some differences from the standard growth crve approach. In most common forms of latent growth crve modeling, one allows one or more degrees of freedom in the loadings for each latent crve variable and also allows the latent variables to freely covary. In the latent differential eqation approach, we allow no degrees of freedom in the loadings for the latent variables, constraining the loadings in sch a way that the latent variables are estimates of the derivatives of the time series. This part of the approach is similar to Savitzky Golay filtering (Savitzky & Golay, 1964). Then, we se the covariances between these latent derivatives to estimate regression parameters between them. Sppose that we have measred a variable x at for eqally spaced occasions separated by an interval of time τ on a sample of N sbjects reslting in an N 4 data matrix X. Now consider the three matrices L, A, and S defined in Eqations 3,, and 6 below. A loading matrix L is constrcted sch that it will estimate the vale of x, its first derivative, and second derivative at the time midway between the second and third occasions of measrement. L = 1 1.τ ( 1.τ) 2 /2 1.τ (.τ) 2 /2 1.τ (.τ) 2 /2 1 1.τ (1.τ) 2 /2 The matrix L is similar to a growth crve loading matrix in that the first colmn estimates an intercept. Ths the first colmn of L identifies a latent variable which takes on a vale for each row vector of vales {x i1, x i2, x i3, x i4 } where i is a row in the data matrix X. We won t directly observe this latent variable, bt will estimate its covariances with other latent variables. The second colmn estimates a slope sch that the vale of the intercept occrs midway between the second and third occasions and the time scale of the slope is the interval of time τ between occasions. The vale of the latent variable identified by the second colmn is then an estimate of the first derivative of each row of the data matrix X evalated midway between the second and third colmns. The third colmn estimates a qadratic crvatre in a latent variable that is scaled by τ and also scaled by 1/2. This latent variable takes on a vale for each row in the data matrix X that is an estimate of the second derivative of that row at a time midway between the second and third occasions of measrement. To nderstand why the loadings are constrcted in this way, note that each colmn is the indefinite integral of the colmn to its left and centered arond the middle time. Ths since 1dτ = τ and τdτ = ta 2 /2, we can constrct an estimate of the first and second derivatives sing these weights. Then, the covariance strctre between the latent variables is defined sing McArdle and McDonald s RAM notation (McArdle & McDonald, 1984). (3) (4)

10 LATENT DIFFERENTIAL EQUATION MODELING 1 A = S = U = η ζ () V F C F df C F df V df V d2f V x1 V x2 V x3 V x4 (6) (7) The error strctre can then be defined in the matrix U as shown in Eqation 7. In this case we are assming independent error of measrement. Now, we can calclate the expected covariances ˆR between the observed variable x at the for occasions of measrement sing the matrix expression shown in Eqation 8. ˆR = L(I A) 1 S(I A) 1 L + U (8) This model may then be fit sing a strctral modeling program sch as Mx (Neale, Boker, Xie, & Maes, 1999). An Mx script for fitting this model as well as the mltivariate case can be downloaded from the web at A path diagram of the example ULDE model sing RAM diagrammatic conventions (McArdle & Boker, 199; Boker, McArdle, & Neale, 22) is presented in Figre 4. Note that with for occasions of measrement, this model is jst identified, so the predicted covariance matrix can always exactly reprodce the observed covariance matrix. If one has five or more occasions of measrement, the ULDE model can provide a fit statistic estimating how well the dynamics of the process are fit by the model. However, note that at the latent variable level, the example damped linear oscillator differential model is satrated. Ths, there is no opportnity for misfit in this model at the latent variable level. Observed misfit in this example can only be at the measrement level. However, one might test whether either one or both of the parameters η and ζ are zero by fitting a model withot them and testing the difference in fit between that and the latent variable satrated model. If one has very many occasions of measrement on one variable and one individal, the ULDE method may still be sed. The nivariate time series may be embedded into a 4 or dimensional space sing the method of state space embedding (Saer, Yorke, & Casdagli, 1991; Takens, 198; Whitney, 1936) to create a lagged covariance matrix across a chosen interval τ. Frthermore, if one has many observations on many individals, one might create an individal identified state space embedding data matrix and fit a ULDE model sing the raw scores so that individal differences in differential eqations parameters might be estimated.

11 LATENT DIFFERENTIAL EQUATION MODELING 11 η V F V df V d2f F df d2f ζ x 1 x 2 x 3 x x 1 x 2 x 3 x 4 V x1 V x2 V x3 V x4 Figre 4. Path diagram of a Univariate Latent Differential Eqation (ULDE) Model. Mltivariate Latent Differential Eqation Model We next consider the case where a measrement model for a latent variable is indicated both across manifest variables within time as well as within manifest variables across time. In the case of a single occasion of measrement, a measrement model for a latent variable is freqently constrcted across variables as a confirmatory factor model. We present a model in which a confirmatory factor model holds within any single occasion of measrement and covariance between indicators across time is acconted for by the intrinsic dynamics of the latent variable: the differential eqations model at the level of the latent strctre. Sppose we have observed three manifest variables at for occasions. This is the minimm nmber of variables and occasions that will allow the Mltivariate Latent Differential Eqation (MLDE) model to be identified both within occasion and across indicators as well as within indicator and across time. We now bild a data matrix sch that the first for colmns are the for occasions of measrement for the first indicator, the next for colmns are the for occasions of measrement for the second indicator and the last for colmns are the for occasions of measrement for the third indicator. Ths one row of the data matrix wold be {x i1, x i2, x i3, x i4, y i1, y i2, y i3, y i4, z i1, z i2, z i3, z i4 } (9)

12 LATENT DIFFERENTIAL EQUATION MODELING 12 where x i1 is the first occasion of measrement for the first indicator for the ith row of the data matrix. Now the predicted covariance matrix of these indicators can be calclated from the for matrices L, A, S, and U. The first for rows of the loading matrix L shown in Eqation 1 are constrcted as before with three colmns sing the fixed interval τ. The second set of for rows are the same, bt are weighted by an estimated parameter a that is the factor loading for the variable y on the latent variable F. Similarly the third set of for rows are weighted by an estimated parameter b. The strctre of the covariance between the latent variable and its latent derivatives is defined sing the matrices A and S shown in Eqations 11 and 12. The example strctre is again the second order linear differential eqation defining a damped linear oscillator. This is by no means the only dynamical system that can be estimated sing the MLDE approach. In fact, one of the principal advantages of this approach is that the model at the latent level is easily and flexibly specified and estimated for more than one factor. L = A = S = 1 1.τ ( 1.τ) 2 /2 1.τ (.τ) 2 /2 1.τ (.τ) 2 /2 1 1.τ (1.τ) 2 /2 a a( 1.τ) a( 1.τ) 2 /2 a a(.τ) a(.τ) 2 /2 a a(.τ) a(.τ) 2 /2 a a(1.τ) a(1.τ) 2 /2 b b( 1.τ) b( 1.τ) 2 /2 b b(.τ) b(.τ) 2 /2 b b(.τ) b(.τ) 2 /2 b b(1.τ) b(1.τ) 2 /2 η ζ (1) (11) V F C F df C F df V df V d2f (12) We can now fit the model sing a strctral eqations modeling package sch as Mx. The predicted covariance of this model can be calclated in the same way as for the ULDE model as shown in Eqation 13. ˆR = L(I A) 1 S(I A) 1 L + U (13) In this example, the matrix of niqe variances U is specified as a diagonal matrix of free parameters, thereby constraining all of the common variance between indicators within and across time to be acconted for by the differential eqation relating the common factor and its derivatives. Note that while this is the minimm nmber of indicators reqired to identify a single factor within time across indicators and the minimm nmber of occasions

13 LATENT DIFFERENTIAL EQUATION MODELING 13 to identify the model within indicator across time, there are many more degrees of freedom in the covariance matrix of indicators than there are estimated parameters. This is a strong test of the dynamics of a latent variable. Bt note that at the latent variable level, this example model is flly satrated and it is only in the measrement model that we have a potential for misfit to the data. Ths, if we observe sbstantial misfit, we might consider other more complicated relationships sch as across time factors niqe to each indicator. Sch a model wold sggest that there was both a common dynamic to all of the indicators and a separate dynamic within each indicator. V F η V df V d2f F df ζ d2f x 1 x 2 x 3 x 4 y 1 y 2 y 3 y 4 z 1 z 2 z 3 z 4 x 1 x 2 x 3 x 4 y 1 y 2 y 3 y 4 z 1 z 2 z 3 z 4 V x1 V x2 V x3 V x4 V x1 V x2 V x3 V x4 V x1 V x2 V x3 V x4 Figre. Path diagram of the example Mltivariate Latent Differential Eqation (MLDE) model with three indicators observed at for occasions and the latent differential strctre modeled as a damped linear oscillator. A path diagrammatic representation of this example MLDE model is presented in Figre. Again, we have sed the RAM diagramming conventions for representing factor and niqe variances. There are many ways in which this model might seflly be extended. For instance, we cold se more indicators to gain a better estimate of the common factor. We might find that a single factor was insfficient to fit the measred variables. Two or more factors might have independent dynamics, or might be copled over time. A second

14 LATENT DIFFERENTIAL EQUATION MODELING 14 way this model might be extended is that we might have more occasions of measrement and find that higher order derivatives were reqired to fit the observed dynamics of the indicators. This model may also be fit to data comprised of many occasions of measrement on a single individal sing the state space embedding method for bilding a lagged data matrix as mentioned in the previos section. Or one might have mltiple indicators and many occasions of measrement on many individals. In this case a mltiple grop fll information maximm likelihood approach cold be sed that wold allow each individal to take on individal dynamical parameters while loading constraints force metric factor invariance of the measrement model across individals. Simlation of Mltivariate Oscillators In order to test the efficacy of the MLDE modeling procedre, two simlations were implemented. The first simlation tested the behavior of the MLDE model nder a variety of parameter conditions and sampling intervals. The simlated data was calclated sing Mathematica (Wolfram Research, 22) sing Rnge Kttah forth order nmerical integration. We simlated data sing the damped linear oscillator model from Eqation 1. We created 9 instances of the damped linear oscillator each with one η parameter chosen from a set of 1 conditions, η = {.2,.6, 1., 1.4, 1.8, 2.2, 2.6, 3., 3.4, 3.8}, one ζ parameter chosen from a set of 9 conditions, ζ = {.4,.3,.2,.1,.,.1,.2,.3,.4}, and then nmerically integrated the reslting differential eqation to prodce 1, observations of the oscillator. The integration step was chosen to be small (.) so that a wide range of simlated intervals between occasions of measrement cold be tested. Ths, for any chosen combination of η and ζ there was a vector of 1, tre scores from a damped linear oscillator. A sampling interval, τ was chosen from the set of integers from 1 to 16. Then an instance of a 3 individals by 24 observations simlated data matrix was constrcted as follows. The ith row of the simlated data matrix was calclated from a sample individal set of 4 tre scores {F iti, F i(ti +τ), F i(ti +2τ), F i(ti +3τ)} drawn from the selected damped linear oscillator vector where the index of the first occasion t i was a psedorandom nmber niformly distribted on the interval {1... }. Next, six indicator scores were generated for each tre score F ik where k is an element of the set {t i, t i + τ, t i + 2τ, t i + 3τ}. x i1k = 1.F ik +.62 i1k (14) x i2k =.F ik +.82 i2k x i3k =.4F ik +. i3k x i4k =.7F ik i4k x ik =.6F ik +.7 ik x i6k =.3F ik + 1. i6k

15 LATENT DIFFERENTIAL EQUATION MODELING where ijk is a psedorandom nmber drawn from a normal distribtion with zero mean and nit variance. The commnalities of these observed scores depended on the variance of F, and ths on the sampling interval τ and on the selected combination of η and ζ from the tre score vector. In this way, 1 data matrices were generated for each combination of η, ζ and τ, reslting in a total of 14,4 data matrices. An MLDE model with 6 indicators, 4 occasions and a latent differential strctre of a damped linear oscillator were fit to each of these data matrices sing Mx. Reslts were aggregated over the 1 replications within each η ζ τ condition cell. The factor loadings from Eqation 14 were recovered well when the model fit the simlated data reasonably well. There were some conditions of η and τ that violated the Nyqist limit (see e.g. Hamming, 1977, for an introdction) for or for occasion samples and ths models fit to those data matrices performed extremely poorly if they converged at all. Figre 6 presents the reslts of fitting the MLDE model to all conditions of η and τ when ζ =. This is the condition where there is no damping, i.e. similar to a frictionless pendlm. As can be seen in Figres 6 a and b, there is a reasonably good correspondence between the tre vale of η and the mean estimated vale of η except when large vales of τ are combined with large negative vales of η (seen as the cliff on the right side of the graph). In fact, when the vale of τ is sch that 4τ are greater than one half the period of the oscillation, the model does not fit. This is exactly the Nyqist limit which states that the sampling interval mst be less than one half the period of the oscillation one wishes to estimate. The mean bias in estimation of η is plotted in Figre 6 c. The bias is low except near the Nyqist limit and when the sampling interval is short. The median bias as a percentage of the tre vale of η over all cells that did not violate the Nyqist limit was 6.2%. This small positive bias was primarily de to cases approaching the Nyqist limit bt which were not formal violations. When the sampling interval is short there is little change in the tre score in comparison with the added measrement error. Ths, the commnalities of the observed scores are qite low in this combination of conditions, reslting in highly variable estimates of η. Figre 6 d plots the mean likelihood ratio fit statistic (χ 2 ) over the 1 replications in each condition cell. Note that the fit statistic does a good job of flagging bias de to the Nyqist limit. Of corse in real data, one does not know where this limit might be and ths having the χ 2 as a diagnostic is sefl. Unfortnately, the χ 2 does not help in recognizing low commnalities, bt commnalities can be estimated in other ways, so this is not problematic. Figre 7 presents the reslts of the simlation for a chosen vale of η =.6, the slowest freqency shown in Figre 6. For vales of τ > 8, the estimated vale of ζ is a reasonably good approximation of the tre vale of ζ, exhibiting low bias, althogh the variability of estimation is higher than for that of η. However, for vales of τ < 8, the estimated vale of ζ shows very high variability. Again, for small vales of τ, with a very small time step constant (.) in the nmerical integration sed for the simlation, there will be little variance in the derivatives in comparison with the added measrement error. Ths with low commnalities, it is not nexpected that the standard error of estimation of ζ will be large when τ is small.

16 LATENT DIFFERENTIAL EQUATION MODELING 16 τ Estimated Eta When Zeta =. 1 a. τ 1 b. Tre Eta When Zeta =. 2 2 η -2 η tre η -2-3 tre η Difference between Estimated and Tre Eta When Zeta =. τ 1 c. τ 1 d. Chi Sqare When Zeta = bias χ tre η -2 tre η Figre 6. Reslts of fitting an MLDE model for ζ = over all conditions of η and τ. (a) The mean estimated vale of η verss τ and the tre vale of η. (b) The tre vale of η verss τ and the tre vale of η. (c) Mean bias of estimation (Figre a mins Figre b). (d) The likelihood ratio fit fnction (χ 2 ). Simlation with No Time Dependence In discssing Figre 2 we sggested that a sefl method wold need to be able to distingish random flctations in a score from flctations de to a self reglatory process. The reason this can be problematic can be nderstood by considering Figre 2 b in which all observations are independent draws from a normal distribtion. Sppose one drew an extreme observation at the second occasion. The expected vale of the first and third observation given the extreme vale are still zero. Ths, the more extreme the vale of the second occasion, the greater the expected bend at the second occasion. Ths there is a bilt in correlation between the second derivative at the second occasion and the vale of the variable at that occasion. In fact, sing Local Linear Approximation, there is a calclated correlation of -.8 between the vale of the variable and its estimated second derivative. This is why it is difficlt to distingish between the cases in Figres 2 a and

17 LATENT DIFFERENTIAL EQUATION MODELING 17 τ 1 a. Estimated Zeta When Eta= -.6 τ 1 b. Tre Zeta When Eta= ζ. -.. ζ tre ζ -.2 tre ζ Difference between Estimated and Tre Zeta When Eta= -.6 τ 1 c. τ 1 d. Chi Sqare When Eta= bias χ tre ζ -.2 tre ζ Figre 7. Reslts of fitting an MLDE model for η =.6 over all conditions of ζ and τ. (a) The mean estimated vale of ζ verss τ and the tre vale of ζ. (b) The tre vale of ζ verss τ and the tre vale of ζ. (c) Mean bias of estimation (Figre a mins Figre b). (d) The likelihood ratio fit fnction (χ 2 ). 2 b. We simlated a vector of factor scores as independent draws from a normal distribtion and repeated the simlation experiment described above. There was an immediately obvios difference between the reslts of this simlation and the simlation with a tre linear oscillator generating the factor scores. The vales of the χ 2 fit fnction for the case with no time dependence in the factors were on average approximately times larger than the fit fnction vales of for the simlation with tre linear oscillation. In the first simlation, maximm χ 2 fit fnction vale for the tre linear oscillator across all conditions was 33 when τ was small enogh that the 4 occasions were within the Nyqist limit. Even when Nyqist limit violations are inclded, the maximm χ 2 vale was 1699 for the tre linear oscillator data. By contrast, the minimm vale of χ 2 in the case in which the factor scores were independent across time was 1,294 (16 simlated data sets). In every one of the simlated

18 LATENT DIFFERENTIAL EQUATION MODELING 18 data sets with no time dependence, the linear oscillator was clearly rejected by the vale of the fit fnction. It shold be noted that the within occasion factor loadings were, in almost every case, correctly calclated for the time independent data. Ths it was not simply that the models did not converge correctly for the time independent data. Discssion Data sets were simlated that mimicked an experimental design in which 3 independent individals were measred on six indicator variables on for occasions separated by eqal intervals of time. All individals in each simlated data set had a single factor that had an intrinsic dynamic in accord with a damped linear oscillator differential eqation model with fixed parameters. Independent identically distribted measrement error was added to each observation. The MLDE model was able to simltaneosly recover low bias estimates of the factor loadings and differential eqations parameters if the total interval between the first and last observation was(a) smaller than the Nyqist limit of one half the period of the tre oscillation and (b) large enogh that the commnalities of the indicators was greater than.. The MLDE method appears to be mch less sensitive to measrement interval than other methods for direct estimation of differential eqations parameters from derivatives explored by the athors (Boker & Nesselroade, 22; Boker, 21). This means that the MLDE method does not reqire more than a gross estimate of the period of any hypothesized cyclicity in the process nder analysis. In addition, the inclsion of a measrement model opens p a variety of possibilities for extensions to the model. For instance, many longitdinal designs are not eqal interval. If all sbjects are measred on the same schedle, the time interval τ sed in the matrix L can take on appropriate vales for each row so that estimates of the covariance between latent derivatives can still be obtained and ths the covariance between their derivatives can still be obtained. This applies even when some indicators are measred on a different schedle than others as long as there is a centering time arond which all observations can be synchronized. This cold be very sefl when attempting to estimate the latent dynamic strctre of a variable measred, for instance, with pencil and paper as well as physiological instrments. Physiological measres are likely to be observed mch more freqently than qestionnaire or psychometric measres. Being able to correct for these time scale differences is critical to estimating copling between psychological and physiological measres. A frther extension of the model cold se latent indicators for missing observations (as in e.g. McArdle & Hamagami, 1992). When many observations are available on each individal (or pre defined grop), it is possible to bild a mltigrop model where each individal s (or grop s) dynamic parameters cold be estimated while constraints on factor invariance were maintained. Weaknesses There are several weaknesses of the LDE methods. First, the data reqirements are sbstantial. One mst have measred several variables on many occasions on many individals in order to both estimate within and between individal parameters. The model as presented here also reqires that the process be stationary, that is, the estimated parameters

19 LATENT DIFFERENTIAL EQUATION MODELING 19 may not change over time. A windowed extension to this method (perhaps along the lines of Boker, X, Rotondo, & King, 22) may be able to relax the stationarity reqirement. The chosen interval between occasions of measrement mst be short enogh so that at least 8 observations can occr within a single period of oscillation when the latent strctre incldes a second derivative. This means that a process mst be measred relatively often in comparison with its period of flctation. Finally, when the rows of the lagged data matrix are not independent observations, that is when single individals may contribte more than one row to the lagged data matrix, the standard errors for the dynamic parameters calclated by maximm likelihood will be smaller than the tre variability in these parameters. Resampling by blocks may be one soltion, bt as far as is known to the athors, this is still an open problem. Ftre Directions While adding a mltivariate measrement model to direct differential eqations estimation has benefits in and of itself, the main benefit foreseen by the athors is extension to dynamical systems with mltiple factors and copling between factors. As a first step in this direction, consider the model shown in the path diagram in Figre 8. Here, two latent variables each have a separate dynamic strctre and each of them inflences the dynamic of the other. At the latent level, this is a system of two second order differential eqations. Each latent variable has its own set of indicators and these indicators need not be measred at the same intervals as long as there is a synchronizing time t that is the same at the center of each grop of observations. This type of model cold easily be extended to systems of three or more (possibly nonlinear) differential eqations. Finally, we see promise for the MLDE method to be applied to behavioral genetics data. A reasonable hypothesis might be that a dynamical process has both additive genetic as well as environmental inflences. The MLDE method seems likely to be able to be adapted to estimate these inflences. Sibling interaction may possibly be constrctively modeled sing copled systems similar to the copled oscillator shown in Figre 8. If one is attempting to design experiments that can take advantage of methods in dynamical systems modeling, we have the following recommendations. It goes withot saying that yo shold measre as often as yo can afford, obtain as many occasions per individal as yo can afford, and measre as many individals as yo can afford. When bdgets are limited, trade offs mst be made. The absolte minimm nmber of occasions per individal is for in order to se an ULDE or MLDE model. Bt if at all possible try to obtain ten so that yo will have access to more options and better parameter stability. In this case yo will still be making an assmption that all individals dynamic parameters are eqal. If yo wish to relax that assmption yo will need a minimm of arond 3 observations per individal and will have mch better power with 1 observations per individal. In order to get stable estimates of the factor loadings, yo will need at least 3 qadrples of measrements. This sonds forbidding, bt remember that this need not be 3 individals if yo have many qadrples of observations from each individal. Finally, it is extremely important to remember that if yo expect cyclicity in yor data yo mst measre often enogh that the interval between the first and last observation in a qadrple of occasions is less than one half the period of the cycle.

20 LATENT DIFFERENTIAL EQUATION MODELING 2 x1 a1 x2 a2 x3 a3 x4 a4 y1 f1 f2 b1 y2 b2 df1 df2 y3 b3 y4 d2f1 d2f2 b4 e e z1 c1 z2 c2 z3 c3 z4 c4 Figre 8. variables. Path diagram of a mltivariate latent differential eqations model with two copled latent References Bisconti, T. L. (21). Widowhood in later life: A dynamical systems approach to emotion reglation. Unpblished doctoral dissertation, University of Notre Dame. (Unpblished doctoral dissertation) Boker, S. M. (21). Differential strctral modeling of intraindividal variability. In L. Collins & A. Sayer (Eds.), New methods for the analysis of change (pp. 3 28). Washington, DC: APA. Boker, S. M., & Bisconti, T. L. (in press). Dynamical systems modeling in aging research. In C. S. Bergeman & S. M. Boker (Eds.), Qantitative methodology in aging research. Mahwah, NJ: Lawrence Erlbam Associates. Boker, S. M., & Ghisletta, P. (21). Random coefficients models for control parameters in dynamical systems. Mltilevel Modelling Newsletter, 13 (1), Boker, S. M., & Graham, J. (1998). A dynamical systems analysis of adolescent sbstance abse. Mltivariate Behavioral Research, 33 (4),

21 LATENT DIFFERENTIAL EQUATION MODELING 21 Boker, S. M., & McArdle, J. J. (in press). Slope fields, vector fields, and statistical vector fields for longitdinal data. In P. Armitage & T. Colton (Eds.), Encyclopedia of biostatistics. Chichester, UK: Wiley. Boker, S. M., McArdle, J. J., & Neale, M. C. (22). An algorithm for the hierarchical organization of path diagrams and calclation of components of covariance between variables. Strctral Eqation Modeling, 9 (2), Boker, S. M., & Nesselroade, J. R. (22). A method for modeling the intrinsic dynamics of intraindividal variability: Recovering the parameters of simlated oscillators in mlti wave panel data. Mltivariate Behavioral Research, 37 (1), Boker, S. M., X, M., Rotondo, J. L., & King, K. (22). Windowed cross correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7 (1), Cmmings, E. M., & Davies, P. T. (22). Effects of marital conflict on children: Recent advances and emerging themes in process oriented research. Jornal of Child Psychology and Psychiatry, 43 (1), Donaldson, G., & Horn, J. L. (1992). Age, cohort and time developmental mddles: Easy in practice, hard in theory. Experimental Aging Research, 18, Hamming, R. (1977). Digital filters. Englewood Cliffs, NJ: Prentice Hall. Hbbard, J. H., & West, B. H. (1991). Differential eqations: A dynamical systems approach. New York: Springer Verlag. Jones, R. H. (1993). Longitdinal data with serial correlation: A state space approach. Boca Raton, FL: Chapman & Hall/CRC. McArdle, J., & Epstein, D. (1987). Latent growth crves within developmental strctral eqation models. Child Development, 8, McArdle, J. J., & Boker, S. M. (199). Rampath. Hillsdale, NJ: Lawrence Erlbam. McArdle, J. J., & Hamagami, F. (1992). Modeling incomplete longitdinal and cross sectional data sing latent growth strctral models. Experimental Aging Research, 18 (3), McArdle, J. J., Hamagami, F., Meredith, W., & Bradway, K. P. (in press). Modeling the dynamic hypotheses of gf-gc theory sing longitdinal life-span data. Learning and Individal Differences. McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the Reticlar Action Model for moment strctres. British Jornal of Mathematical and Statistical Psychology, 87, Molenaar, P. C. M. (198). A dynamic factor model for the analysis of mltivariate time series. Psychometrika,, Neale, M. C., Boker, S. M., Xie, G., & Maes, H. H. (1999). Mx: Statistical modeling. (Box 126 MCV, Richmond, VA 23298: Department of Psychiatry, th Edition) Od, J. H. L., & Jansen, R. A. R. G. (2). Continos time state space modeling of panel data by means of SEM. Psychometrica, 6 (2), Saer, T., Yorke, J., & Casdagli, M. (1991). Embedology. Jornal of Statistical Physics, 6 (3,4), Savitzky, A., & Golay, M. J. E. (1964). Smoothing and differentiation of data by simplified least sqares. Analytical Chemistry, 3,