Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements

Transcription

1 Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Jeffrey N. Rouder Francis Tuerlinckx Paul L. Speckman Jun Lu & Pablo Gomez May The Weibull regression model The Weibull regression model for RT is given as T ij ψ i λ ij β i iid Weibull(ψ i λ ij β i (1 log λ ij α i θ i γ j = α i + θ i c j + γ j ( α i α 0 σ α θ i θ 0 σ θ γ j σ γ The probability density function of the Weibull is iid Normal(α 0 σα (3 iid Normal(θ 0 σθ (4 iid Normal(0 σγ. (5 f(t ψ λ β = λβ(t ψ β 1 exp ( λ(t ψ β t ψ. Priors are needed for (ψ i η 1 η α 0 θ 0 σ α σ θ σ γ. We have previously provided the following priors and have shown they provide for good estimation across a wide range of experimental outcomes (Lu 004 Rouder et al : ψ i Uniform(0 min j T ij η 1 Gamma(c 1 d 1 η Gamma(c d with values of c 1 = d 1 =.0 c = and d =.04. The density of the gamma distribution is ηη1 η1 1 f(x; η 1 η = x exp( η x Γ(η 1 where Γ is the gamma function(abramowitz & Stegun Flat priors are convenient and appropriate for location parameters (α 0 θ 0 ; inverse gamma 1

2 priors are convenient and appropriate for variance parameters (σα σ γ σ θ. The inverse gamma has a probability density function f(x; a b = ba Γ(a x a 1 exp ( b x Parameters a and b are set to values of.1. With this choice the resulting prior coarsely approximates the Jeffrey s noninformative prior on variance (Jeffreys The target of analysis in Bayesian statistics is the derivation of the marginal posterior distribution for each parameter. Often these marginal posteriors cannot be derived as closed-form expressions. We follow the common approach of deriving closed-form expressions for full conditional posterior distributions and sampling these with the Markov chain Monte Carlo (MCMC technique (Gelfand & Smith It is convenient to refer to vectors of parameters and these will be denoted with boldface typeset; e.g. α = (α 1 α... α I. The following full conditional distributions are well-known random variables: ( α 0 α σα; i T Normal α i σ α I I ( θ 0 θ σθ ; T Normal i θ i σ θ I I ( i σα α α 0; T Inverse Gamma a + I/ b + (α i α 0 ( σ γ γ; T Inverse Gamma a + J/ b + ( σθ θ θ 0 ; T Inverse Gamma a + I/ b + η β η 1 ; T Gamma(Iη 1 + c d + i β i. j γ j i (θ i θ 0 The derivation of the first five full conditionals is standard (see Rouder & Lu 005. The derivation of the last one is provided in Rouder et al. (003. The remaining full conditional posteriors have closed form probability density functions but are not well-known random variables. These are expressed conveniently as the logarithms of density functions and expressed up to an additive constant: log f(ψ i α i γ θ i β i ; T = (β i 1 j J i log(t ij ψ i j J i exp(α i + γ j + θ i c j (T ij ψ i βi ψ i < min j T ij log f(α i ψ i γ θ i β i α 0 σα; T = J i α i exp(α i + γ j + θ i c j (T ij ψ i βi (α i α 0 j J i σ α

3 log f(γ j ψ α θ β σ γ; T log f(θ i ψ i α i γ β i θ 0 σ θ; T = I j γ j exp(α i + γ j + θ i c j (T ij ψ i βi γ j σ i I γ j = θ i c j exp(α i + γ j + θ i c j (T ij ψ i βi (θ i θ 0 j J i j log f(β i ψ i α i γ θ i η 1 η ; T = ( J i + η 1 1logβ i + (β i 1 j J i log(t ij ψ i σ θ exp(α i j J i exp(γ j + θc j (T ij ψ i βi β i η log f(η 1 β η ; T = η 1 (I log η d 1 + i log β i I log Γ(η 1 + (c 1 1log(η 1. where I j is the set of participants that was presented item j and J i is the set of items presented to participant i. The derivations of these log posterior densities are straightforward. Full conditional posteriors that are distributed as normals inverse gammas and gammas are conveniently sampled in many packages (we used the Gnu Scientific Library for C. The remaining full conditionals are sampled with the Metropolis-Hastings algorithm (Gelman et al. 004 with the exception of parameter ψ. Efficient sampling of ψ i is possible with rejection sampling (Tanner For these shift parameters a shifted rightward-facing exponential serves as a suitable candidate distribution. While sampling is convenient in this framework excessive autocorrelation of MCMC chains results. To mitigate this autocorrelation we implemented a series of decorrelating Metropolis steps (Graves Speckman & Sun submitted; Liu & Sabatti 000. A sample of the resulting chains are shown in Figure 1. There is some autocorrelation in the chains though it is not excessive. To mitigate this remaining autocorrelation we used a large number of iterations in application (50000 iterations with a burn-in of iterations. Derivation of Expected Values We derive the expected value of T ij for any participant observing an item with covariate value c j that is E(T ij ψ i α i θ i β i : E(T ij ψ i α i θ i β i = E γj E(T ij ψ i α i θ i β i γ j = E γj [ ψ i + Γ(1 + 1/β j e (αi+γj+θicj/βi ] = ψ i + Γ(1 + 1/β j e (αi+γj+θicj/βi E γj [ e γj/βi ]. The factor E γj [ e γ j/β i ] is the moment generating function of γj. Random variable γ j is distributed as a normal and its moment generating function is 3

4 Parameter Value Shape(β Interation Parameter Value WF Effect (θ Interation Parameter Value Variance(σ γ Interation Figure 1: MCMC chain values for selected parameters. Top to bottom panels show samples for β 0 θ 0 and σγ respectively. These chains are typical for other parameters. As can be seen mixing is not ideal but posterior quantities may nonetheless be estimated with long chains. well known (e.g. Hogg & Craig 1978: ( ] ( γj σ γ E γj [exp = exp. β i Substituting in this equality and rearranging terms yields ( E(T ij ψ i α i θ i β i = ψ i + K i exp θ ic j (6 β i β i where K i is the constant ( ( βi + 1 K i = Γ exp α i + σ γ β i β i βi. 3 Robustness to Misspecification The hierarchical Weibull regression model assumes that RT is distributed as a Weibull. The Weibull is a sufficiently simple distribution that this assumption may be assessed. To do so Rouder et al. (005 recommend transforming the data: Q ij = (T ij ˆψ i ˆβ iˆλij. (7 With this transformation the distribution of each Q ij is independent and follows a standard exponential (rate of 1.0. Consequently the Weibull assumption may be assessed by plotting the distribution of Q ij and comparing it to a standard 4

5 Density Exponential Residuals Density Value Standard Exponential Time (ms Figure : Misspecification of the Weibull distribution. Left: Histogram of transformed data Q ij. The line is the standard exponential distribution. Middle: Quantile-quantile plot of the transformed data against the standard exponential quantiles. Right: A comparison of the inverse Gaussian (dashed and Weibull (solid distributions. exponential. The left panel of Figure shows the histogram of Q ij for the data from Gomez Perea & Ratcliff s (007 Experiment 1; the line is the density of the standard exponential. The misfit is obvious. The data have more mass in the left tail than the Weibull. The center plot shows a quantile-quantile plot of Q ij vs. the standard exponential. This plot reveals that the data have a heavier right tail than the Weibull model. The right panel shows the comparison of an inverse Gaussian distribution 1 (dashed line and a Weibull distribution (solid line. The inverse Gaussian has heavier left and right tails than the Weibull and is therefore a more realistic distribution. Although the inverse-gaussian is more realistic it is more difficult to fit. Unfortunately small deviations in data have big effects on parameter estimates (Rouder & Speckman 004. Given the evidence for misspecification it is desirable to test whether the discrimination of functional forms is robust in the hierarchical Weibull model. We explored how well the Weibull regression model recovered the functional form when (simulated data were distributed as an inverse Gaussian. To make the simulations realistic we used the design matrix from the Gomez et al. experiment; that is each simulated participant corresponded to a real participant and each simulated item corresponded to a real item. We simulated two data sets: one following a power function (logarithm of the inverse Gaussian scale 1 The probability density function of the inverse Gaussian distribution is ( ξ ((x ψφ ξ f(x; ψ ξ φ = π (x ψ 3/ exp. ξ(x ψ 5

6 was linear in log-wf and one following an exponential function (logarithm of inverse Gaussian scale was linear in WF. The true inverse Gaussian parameters were chosen such that the resulting means and variances of simulated data matched those estimated from Gomez et al. s data. Each of the two inverse Gaussian simulated data sets was analyzed with two hierarchical Weibull models: one in which WF served as the covariate and a second in which log-wf served as the covariate. Plots of item residuals (γ j as a function of the covariate are shown in Figure 3. The top row shows analyses when inverse Gaussian log-scale follows WF (an exponential function; the bottom row shows the same when inverse Gaussian log-scale follows log-wf (a power function. The left column is for Weibull analysis with a WF covariate; the right column is the same with a log-wf covariate. Item residuals γ j are flat when the covariate matches the generating function and curved otherwise. The Weibull model therefore appears to recover the appropriate functional form even when the data are from an inverse Gaussian. 6

7 Linear Covariate Logarithmic Covariate Item Residuals Item Residuals Word Frequency Log Word Frequency Exponential Law Serves As Truth Power Law Serves as Truth Word Frequency Log Word Frequency Figure 3: Item residuals (γ j as a function of covariate for data simulated from an inverse Gaussian. Top and Bottom Rows: True relationship follows an exponential and power function respectively. Left and Right Columns: Weibull analysis with covariates linear and logarithmic in word frequency respectively. Lines are nonparametric smooths. The Weibull successfully recovers the parametric relationship even when the distribution is misspecified. 7

8 References [1] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions: with Formulas Graphs and Mathematical Tables. Dover New York [] A. Gelfand and A. F. M. Smith. Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association 85: [3] A. Gelman J. B. Carlin H. S. Stern and D. B. Rubin. Bayesian data analysis (nd edition. Chapman and Hall London 004. [4] P. Gomez and M. Ratcliff R.and Perea. Diffusion model of the go/no-go task. Journal of Experimental Psychology: General 136: [5] T. L. Graves P. L. Speckman and D. Sun. Improved mixing in MCMC algorithms for linear models. submitted. [6] R. V. Hogg and A. T. Craig. Introduction to mathematical statistics. MacMillan New York [7] H. Jeffreys. Theory of Probability (3rd Edition. Oxford University Press New York [8] S. J. Liu and C. Sabatti. Generalised Gibbs sampler and multigrid Monte Carlo for Bayesian computation. Biometrika 87: [9] J. Lu. Bayesian hierarchical models and applications in psychology research. Unpublished dissertation. University of Missouri-Columbia [10] J. N. Rouder and J. Lu. An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin and Review 1: [11] J. N. Rouder J. Lu P. L. Speckman D. Sun and Y. Jiang. A hierarchical model for estimating response time distributions. Psychonomic Bulletin and Review 1: [1] J. N. Rouder and P. L. Speckman. An evlatuation of the Vincentizing method of forming group-level response time distributions. Psychonomic Bulletin and Review 11: [13] J. N. Rouder D. Sun P. L. Speckman J. Lu and D. Zhou. A hierarchical Bayesian statistical framework for response time distributions. Psychometrika 68: [14] J. N. Rouder F. Tuerlinckx P. L. Speckman J. Lu and P. Gomez. A hierarchical approach for fitting curves to response time measurements. submitted. 8

9 [15] Martin A. Tanner. Tools for statistical inference: Methods for the exploration of posterior distributions and likelihood functions. Springer Berlin