Covariance modelling for longitudinal randomised controlled trials

Size: px
Start display at page:

Download "Covariance modelling for longitudinal randomised controlled trials"

Transcription

1 Covariance modelling for longitudinal randomised controlled trials G. MacKenzie 1,2 1 Centre of Biostatistics, University of Limerick, Ireland. 2 CREST, ENSAI, Rennes, France. BSU, Cambridge, UK, March 2014 BSU, Cambridge, March /25

2 Longitudinal Setting: MRC s ARMD Study SFRADS 2.0 Distance Visual Acuity (Y1) 1.5 DVA (Visit - Baseline) Group -1.5 Treated Control Time in Years BSU, Cambridge, March /25

3 First choice model Consider a longitudinal Gaussian response, Y, with µ(t) = x(t) β Σ(t) = Σ(t, θ) where, x(t) is a polynomial in t & possibly baseline covariates θ measures the dependence of Σ(t) on t BSU, Cambridge, March /25

4 Basic Thesis From a statistical standpoint inference for the two parameters µ and Σ should, in principle, be symmetric as the two parameters are only Cox-Reid orthogonal. However, often µ is thought to be more important than Σ - Σ being treated a nuisance parameter!! This latter idea is now rather passe and with it much current statistical practice, and in RCTS. Rather, the goal should be to find the joint (µ, Σ) model which is optimal in the joint mean-covariance model space, {M x C} BSU, Cambridge, March /25

5 A Key Result In general the mles of the regression coefficients, take the form When is ˆβ, Σ invariant? ˆβ Σ = (X Σ 1 X) 1 X Σ 1 Y (1) An obvious case arises when Σ I, i.e., for i.i.d. errors. More importantly, Rao (1965) showed that ˆβ is Σ invariant when Σ = XΓX + QΘQ (2) where Γ of order (p p) and Θ of order ((p m) (p m)) are p.d. and Q is a (p (p m)) matrix orthogonal to X, i.e., Q X = 0. We have assumed exactly m repeated measures over time. BSU, Cambridge, March /25

6 Covariance Model The technique is based on a modified Cholesky decomposition of the marginal covariance Σ(t, θ), viz: T ΣT = D (3) leading to Σ(t, ς, φ), in which the new parameters are the natural logarithms of the innovation variances, ς, and generalized autoregressive coefficients, φ We model these as different polynomial functions of lag and time φ(t) = z(t) γ ς(t) = h(t) λ (4) Here, γ and λ are regression parameters of primary scientific interest while z(t) and h(t) are typically particular polynomials in lag (l) and time(t), respectively (MacKenzie & Pan, 2004). BSU, Cambridge, March /25

7 Structure of the T & D matrices T D = diag ( 2 1,, 2 j,, 2 m) BSU, Cambridge, March /25

8 Regressograms BSU, Cambridge, March /25

9 Joint Mean-Covariance Model We can now put everything together into one general joint mean-covariance model which has the following structure µ(t) = x(t) β φ(t) = z(t) γ ς(t) = h(t) λ which we can view as a basic representation for any set of longitudinal data. All of the scientific and inferential interest is focussed on the regression parameters: β, γ and λ BSU, Cambridge, March /25

10 Major univariate extensions Baseline covariates in the cov. mod. ie, the indicator in a RCT (Pan & MacKenzie, 2006) Linear Mixed Models - modified Cholesky transform to within subject cov. matrix (Pan & MacKenzie, 2007) Change to GEE (Ye and Pan, 2006) Constrained mean cov. mod. (Xu & MacKenzie, 2013) BSU, Cambridge, March /25

11 Bivariate Model - 2 dependent responses Consider a longitudinal bivariate Gaussian response (y 1, y 2 ). Such as DVA and NVA in one eye or DVA in each eye. Consider m repeated measures over time. Covariance structure: within subject over time in y 1 and separately in y 2, cross correlation as in y 1 (t) on say y 2 (t 1) and y 1 (t) on y 2 (t) at time t. Leads to modelling innovation matrices - key point. BSU, Cambridge, March /25

12 Bivariate Response Covariance Modelling Let and T = D = diag(d 1,..., D m ) I Φ 21 I Φ m1 Φ m(m 1) I, where: D l is a 2 2 innovation covariance matrix with innovation variances on the diagonal, I is a 2 2 identity sub-matrix and 0 is a 2 2 zero sub-matrix and Φ jk (k < j, j = 2,..., m) are 2 2 matrices. BSU, Cambridge, March /25

13 Bivariate Covariance Models Now we model the innovation matrices D j via matrix logarithm A j matrices. Let ( ) aj1 a A j = log D j = j2 = C j log(g j )C j. a j2 a j3 Thus A j = log D j is a symmetric matrix. By the definition of the matrix exponential it follows that D j = exp(a j ) and the positive definiteness of the D j and hence Σ is guaranteed. The 2 2 Φ jk (k < j, j = 2,..., m) matrices are asymmetric and can be modelled directly - no time to show. BSU, Cambridge, March /25

14 Bivariate Response, Polynomial Covariance Models As a first step, we illustrate the use of polynomial modelling of the innovation matrices. Let ( aj1 a A j = log D j = j2 Recall that j = 1,..., m whence we have m elements over time, then the polynomial models can be written for each s as a j2 a j3 ). a js = λ (s) 0 + λ (s) 1 j + + λ(s) d s j ds (s = 1, 2, 3) for each element in A j and d = d 1 + d 2 + d is the number of parameters fitted (d=degree). BSU, Cambridge, March /25

15 Bivariate Response - ARMD RCT Data RCT of teletherapy in ARMD 3 UK hospital centres Radiotherapy v Observation Examined at Baseline and 3, 6, 12 & 24 months Two outcomes (y 1, y 2 ) DVA and NVA 100 patients per group Only Treatment group analysed - just to illustrate ideas. BSU, Cambridge, March /25

16 ARMD RCT Data - directed simulation Parameter Truth Ave. Est. St. Dev Root. MSE Coverage freq. β % β % β % β % γ % γ % γ % γ % γ % γ % γ % γ % λ % λ % λ % λ % λ % λ % Simple model: linear 2 x 2 = 4 parameters; linear for covariance model; autoregressive, 4 x 2=8 parameters; log-innovation variances, 3 x 2 parameters. BSU, Cambridge, March /25

17 Modelling Innovation Matrices - Results from ARMD Study (i) Upper curve = element (1,2), (ii) Solid circles = element (1,1), (iii) Empty circles = element(2,2) BSU, Cambridge, March /25

18 Conclusions This paper opens the door on MV covariance modelling in RCTs. It solves the key problem of maintaining a PD covariance matrix. The Gaussian case is merely the exemplar case. But can now also apply all of the Univariate methods - Heterogeneity, GEE, Mixed models, Bayesian methods easily to MV response RCTSs. Bodes well for the impact of covariance modelling in longitudinal RCTs with multiple responses. Forthcoming paper on the ARMD RCT with trivariate response (Contrast Sensitivity). BSU, Cambridge, March /25

19 Key References -1 Daniels M. J. and Pourahmadi, M. (2002). Bayesian analysis of covariance matrices and dynamic models for longitudinal data. Biometrika 89, Jones, R. H. (1993). Longitudinal Data with Serial Correlation: A State-space approach, London: Chapman and Hall. Kenward, M. G. (1987). A method for comparing profiles of repeated measurements. Appl. Statist. 36, Laird, N. M.& Ware, J.J. (1982). Random-effects models for longitudinal data. Biometrics, 38, MacKenzie, G and Pan, J. (2001). Modelling marginal covariance structures in linear mixed models. In: Proceedings of the 16th International Workshop on Statistical Modelling. Editors. B. Klein and L. Korsholm , Odense, Denmark. MacKenzie, G. & Pan J. (2007) Optimal joint-mean covariance modelling. In: Proceedings of the 2nd International Workshop on Correlated Data Modelling. Editors: MacKenzie G. and Gregario D.. BSU, Cambridge, March /25

20 Key References -2 MacKenzie G & Reeves JAF. (2002). Modelling bivariate longitudinal data with serial correlation. In: Correlated Data Modelling. Editors: Dario Gregori, et al. Franco Angeli, 19-26, Milano, Italy. Pan JX & MacKenzie G. (2001). Modelling conditional covariance structure in linear mixed models. In: Proceedings of the 16th International Workshop on Statistical Modelling. Editors. B. Klein and L. Korsholm , Odense, Denmark. Pan JX & MacKenzie G. (2003). On modelling mean-covariance structures in longitudinal studies Biometrika 90,1, Pan JX & MacKenzie G. (2006). Regression models for covariance structures in longitudinal studies Statistical Modelling 6, (1): Pan JX & MacKenzie G. (2007). On modelling conditional mean-covariance structures in longitudinal studies. Statistical Modelling 7, (1): Pourahmadi, M. (1999). Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika 86, Pourahmadi, M. (2000). Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika 87, Rao, C. R. (1965). The theory of least squares when the parameters are stochastic and its application to the analysis of growth curves. Biometrika 52,1, BSU, Cambridge, March /25

21 Key References -3 Reeves, JAF. & MacKenzie, G. (1998). A bivariate regression with serial correlation. The Statistician, Vol 47, No. 4, pp Xu, J. & MacKenzie, G. (2012). Modelling covariance structure in bivariate marginal models for longitudinal data. Biometrika, 99, 3, BSU, Cambridge, March /25

22 Joint Model Space BSU, Cambridge, March /25

23 ARMD Study - Cross Correlations Estimated correlation and cross-correlation matrix between Distance visual acuity and Near visual acuity. Estimated correlations are along the main diagonal, estimated cross-correlations at lag k are above the main diagonal, estimated correlations at lag k are below the main diagonal. NVA t DVA BSU, Cambridge, March /25

24 Current SFI Supported Work Science Foundation Ireland s BIO-SI research project Creating R software for our standard methods Developing other classes of covariance models (smoother(s)) Extending to GLMMs Developing multivariate versions (Xu & MacK, 2012) Developing design (MacKenzie & Xu) Developing missingness (Xu & MacK, 2013, forthcoming) BSU, Cambridge, March /25

25 Some History 1999 The Trieste Meeting, WCDM01, November, The Keele Meeting, June 2000 Pourahmadi s 1999 & 2000 Biometrika papers Seminal years: Keele Bayesian ideas - Daniels & Pourahmadi, Biometrika 2002 Pan & MacKenzie, Biometrika 2003 Pan & MacKenzie, 2006, 2007 and Pan The modern era BSU, Cambridge, March /25

Modelling covariance structure in bivariate marginal models for longitudinal data

Modelling covariance structure in bivariate marginal models for longitudinal data Modelling covariance structure in bivariate marginal models for longitudinal data By JING XU Centre of Biostatistics, Department of Mathematics and Statistics, University of Limerick, Ireland jing.xu@ul.ie

More information

Modelling Mean-Covariance Structures in the Growth Curve Models

Modelling Mean-Covariance Structures in the Growth Curve Models Modelling Mean-Covariance Structures in the Growth Curve Models Jianxin Pan and Dietrich von Rosen Research Report Centre of Biostochastics Swedish University of Report 004:4 Agricultural Sciences ISSN

More information

A Cautionary Note on Generalized Linear Models for Covariance of Unbalanced Longitudinal Data

A Cautionary Note on Generalized Linear Models for Covariance of Unbalanced Longitudinal Data A Cautionary Note on Generalized Linear Models for Covariance of Unbalanced Longitudinal Data Jianhua Z. Huang a, Min Chen b, Mehdi Maadooliat a, Mohsen Pourahmadi a a Department of Statistics, Texas A&M

More information

Multivariate Survival Analysis

Multivariate Survival Analysis Multivariate Survival Analysis Previously we have assumed that either (X i, δ i ) or (X i, δ i, Z i ), i = 1,..., n, are i.i.d.. This may not always be the case. Multivariate survival data can arise in

More information

Figure 36: Respiratory infection versus time for the first 49 children.

Figure 36: Respiratory infection versus time for the first 49 children. y BINARY DATA MODELS We devote an entire chapter to binary data since such data are challenging, both in terms of modeling the dependence, and parameter interpretation. We again consider mixed effects

More information

,..., θ(2),..., θ(n)

,..., θ(2),..., θ(n) Likelihoods for Multivariate Binary Data Log-Linear Model We have 2 n 1 distinct probabilities, but we wish to consider formulations that allow more parsimonious descriptions as a function of covariates.

More information

Mohsen Pourahmadi. 1. A sampling theorem for multivariate stationary processes. J. of Multivariate Analysis, Vol. 13, No. 1 (1983),

Mohsen Pourahmadi. 1. A sampling theorem for multivariate stationary processes. J. of Multivariate Analysis, Vol. 13, No. 1 (1983), Mohsen Pourahmadi PUBLICATIONS Books and Editorial Activities: 1. Foundations of Time Series Analysis and Prediction Theory, John Wiley, 2001. 2. Computing Science and Statistics, 31, 2000, the Proceedings

More information

Sample Size and Power Considerations for Longitudinal Studies

Sample Size and Power Considerations for Longitudinal Studies Sample Size and Power Considerations for Longitudinal Studies Outline Quantities required to determine the sample size in longitudinal studies Review of type I error, type II error, and power For continuous

More information

Discussion of Missing Data Methods in Longitudinal Studies: A Review by Ibrahim and Molenberghs

Discussion of Missing Data Methods in Longitudinal Studies: A Review by Ibrahim and Molenberghs Discussion of Missing Data Methods in Longitudinal Studies: A Review by Ibrahim and Molenberghs Michael J. Daniels and Chenguang Wang Jan. 18, 2009 First, we would like to thank Joe and Geert for a carefully

More information

Robust Estimation of the Correlation Matrix of Longitudinal Data

Robust Estimation of the Correlation Matrix of Longitudinal Data Statistics and Computing manuscript No. (will be inserted by the editor Robust Estimation of the Correlation Matrix of Longitudinal Data Mehdi Maadooliat, Mohsen Pourahmadi, and Jianhua Z. Huang Abstract

More information

Now consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.

Now consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown. Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)

More information

PQL Estimation Biases in Generalized Linear Mixed Models

PQL Estimation Biases in Generalized Linear Mixed Models PQL Estimation Biases in Generalized Linear Mixed Models Woncheol Jang Johan Lim March 18, 2006 Abstract The penalized quasi-likelihood (PQL) approach is the most common estimation procedure for the generalized

More information

Longitudinal data analysis using generalized linear models

Longitudinal data analysis using generalized linear models Biomttrika (1986). 73. 1. pp. 13-22 13 I'rinlfH in flreal Britain Longitudinal data analysis using generalized linear models BY KUNG-YEE LIANG AND SCOTT L. ZEGER Department of Biostatistics, Johns Hopkins

More information

Repeated ordinal measurements: a generalised estimating equation approach

Repeated ordinal measurements: a generalised estimating equation approach Repeated ordinal measurements: a generalised estimating equation approach David Clayton MRC Biostatistics Unit 5, Shaftesbury Road Cambridge CB2 2BW April 7, 1992 Abstract Cumulative logit and related

More information

Longitudinal + Reliability = Joint Modeling

Longitudinal + Reliability = Joint Modeling Longitudinal + Reliability = Joint Modeling Carles Serrat Institute of Statistics and Mathematics Applied to Building CYTED-HAROSA International Workshop November 21-22, 2013 Barcelona Mainly from Rizopoulos,

More information

SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati

SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Regressione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida

More information

Generalized Linear Models For The Covariance Matrix of Longitudinal Data. How To Lift the Curses of Dimensionality and Positive-Definiteness?

Generalized Linear Models For The Covariance Matrix of Longitudinal Data. How To Lift the Curses of Dimensionality and Positive-Definiteness? Generalized Linear Models For The Covariance Matrix of Longitudinal Data How To Lift the Curses of Dimensionality and Positive-Definiteness? Mohsen Pourahmadi Division of Statistics Northern Illinois University

More information

Using Estimating Equations for Spatially Correlated A

Using Estimating Equations for Spatially Correlated A Using Estimating Equations for Spatially Correlated Areal Data December 8, 2009 Introduction GEEs Spatial Estimating Equations Implementation Simulation Conclusion Typical Problem Assess the relationship

More information

Bayesian Inference for the Multivariate Normal

Bayesian Inference for the Multivariate Normal Bayesian Inference for the Multivariate Normal Will Penny Wellcome Trust Centre for Neuroimaging, University College, London WC1N 3BG, UK. November 28, 2014 Abstract Bayesian inference for the multivariate

More information

Modeling Correlation in Incomplete Longitudinal Data: The Case of Fruit Fly Mortality Data

Modeling Correlation in Incomplete Longitudinal Data: The Case of Fruit Fly Mortality Data Modeling Correlation in Incomplete Longitudinal Data: The Case of Fruit Fly Mortality Data Tanya Garcia, Priya Kohli, and Mohsen Pourahmadi 1 Abstract Longitudinal studies are prevalent in clinical trials,

More information

Gauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA

Gauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA JAPANESE BEETLE DATA 6 MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA Gauge Plots TuscaroraLisa Central Madsen Fairways, 996 January 9, 7 Grubs Adult Activity Grub Counts 6 8 Organic Matter

More information

Gaussian processes. Basic Properties VAG002-

Gaussian processes. Basic Properties VAG002- Gaussian processes The class of Gaussian processes is one of the most widely used families of stochastic processes for modeling dependent data observed over time, or space, or time and space. The popularity

More information

CTDL-Positive Stable Frailty Model

CTDL-Positive Stable Frailty Model CTDL-Positive Stable Frailty Model M. Blagojevic 1, G. MacKenzie 2 1 Department of Mathematics, Keele University, Staffordshire ST5 5BG,UK and 2 Centre of Biostatistics, University of Limerick, Ireland

More information

Whether to use MMRM as primary estimand.

Whether to use MMRM as primary estimand. Whether to use MMRM as primary estimand. James Roger London School of Hygiene & Tropical Medicine, London. PSI/EFSPI European Statistical Meeting on Estimands. Stevenage, UK: 28 September 2015. 1 / 38

More information

Generalized Linear Models For Covariances : Curses of Dimensionality and PD-ness

Generalized Linear Models For Covariances : Curses of Dimensionality and PD-ness Generalized Linear Models For Covariances : Curses of Dimensionality and PD-ness Mohsen Pourahmadi Division of Statistics Northern Illinois University MSU November 1, 2005 Outline 1 Prevalence of Covariance

More information

Latent Variable Model for Weight Gain Prevention Data with Informative Intermittent Missingness

Latent Variable Model for Weight Gain Prevention Data with Informative Intermittent Missingness Journal of Modern Applied Statistical Methods Volume 15 Issue 2 Article 36 11-1-2016 Latent Variable Model for Weight Gain Prevention Data with Informative Intermittent Missingness Li Qin Yale University,

More information

Practical considerations for survival models

Practical considerations for survival models Including historical data in the analysis of clinical trials using the modified power prior Practical considerations for survival models David Dejardin 1 2, Joost van Rosmalen 3 and Emmanuel Lesaffre 1

More information

Multivariate Regression Generalized Likelihood Ratio Tests for FMRI Activation

Multivariate Regression Generalized Likelihood Ratio Tests for FMRI Activation Multivariate Regression Generalized Likelihood Ratio Tests for FMRI Activation Daniel B Rowe Division of Biostatistics Medical College of Wisconsin Technical Report 40 November 00 Division of Biostatistics

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Introduction to mtm: An R Package for Marginalized Transition Models

Introduction to mtm: An R Package for Marginalized Transition Models Introduction to mtm: An R Package for Marginalized Transition Models Bryan A. Comstock and Patrick J. Heagerty Department of Biostatistics University of Washington 1 Introduction Marginalized transition

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

On Fitting Generalized Linear Mixed Effects Models for Longitudinal Binary Data Using Different Correlation

On Fitting Generalized Linear Mixed Effects Models for Longitudinal Binary Data Using Different Correlation On Fitting Generalized Linear Mixed Effects Models for Longitudinal Binary Data Using Different Correlation Structures Authors: M. Salomé Cabral CEAUL and Departamento de Estatística e Investigação Operacional,

More information

1 Data Arrays and Decompositions

1 Data Arrays and Decompositions 1 Data Arrays and Decompositions 1.1 Variance Matrices and Eigenstructure Consider a p p positive definite and symmetric matrix V - a model parameter or a sample variance matrix. The eigenstructure is

More information

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall 1 Structural Nested Mean Models for Assessing Time-Varying Effect Moderation Daniel Almirall Center for Health Services Research, Durham VAMC & Dept. of Biostatistics, Duke University Medical Joint work

More information

Estimation of a multivariate normal covariance matrix with staircase pattern data

Estimation of a multivariate normal covariance matrix with staircase pattern data AISM (2007) 59: 211 233 DOI 101007/s10463-006-0044-x Xiaoqian Sun Dongchu Sun Estimation of a multivariate normal covariance matrix with staircase pattern data Received: 20 January 2005 / Revised: 1 November

More information

Nonparametric estimation of large covariance matrices of longitudinal data

Nonparametric estimation of large covariance matrices of longitudinal data Nonparametric estimation of large covariance matrices of longitudinal data By WEI BIAO WU Department of Statistics, The University of Chicago, Chicago, Illinois 60637, U.S.A. wbwu@galton.uchicago.edu and

More information

Simulating Longer Vectors of Correlated Binary Random Variables via Multinomial Sampling

Simulating Longer Vectors of Correlated Binary Random Variables via Multinomial Sampling Simulating Longer Vectors of Correlated Binary Random Variables via Multinomial Sampling J. Shults a a Department of Biostatistics, University of Pennsylvania, PA 19104, USA (v4.0 released January 2015)

More information

Course topics (tentative) The role of random effects

Course topics (tentative) The role of random effects Course topics (tentative) random effects linear mixed models analysis of variance frequentist likelihood-based inference (MLE and REML) prediction Bayesian inference The role of random effects Rasmus Waagepetersen

More information

An R # Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM

An R # Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM An R Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM Lloyd J. Edwards, Ph.D. UNC-CH Department of Biostatistics email: Lloyd_Edwards@unc.edu Presented to the Department

More information

Computer Vision Group Prof. Daniel Cremers. 3. Regression

Computer Vision Group Prof. Daniel Cremers. 3. Regression Prof. Daniel Cremers 3. Regression Categories of Learning (Rep.) Learnin g Unsupervise d Learning Clustering, density estimation Supervised Learning learning from a training data set, inference on the

More information

Introduction An approximated EM algorithm Simulation studies Discussion

Introduction An approximated EM algorithm Simulation studies Discussion 1 / 33 An Approximated Expectation-Maximization Algorithm for Analysis of Data with Missing Values Gong Tang Department of Biostatistics, GSPH University of Pittsburgh NISS Workshop on Nonignorable Nonresponse

More information

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives

More information

Conditional Inference Functions for Mixed-Effects Models with Unspecified Random-Effects Distribution

Conditional Inference Functions for Mixed-Effects Models with Unspecified Random-Effects Distribution Conditional Inference Functions for Mixed-Effects Models with Unspecified Random-Effects Distribution Peng WANG, Guei-feng TSAI and Annie QU 1 Abstract In longitudinal studies, mixed-effects models are

More information

On Properties of QIC in Generalized. Estimating Equations. Shinpei Imori

On Properties of QIC in Generalized. Estimating Equations. Shinpei Imori On Properties of QIC in Generalized Estimating Equations Shinpei Imori Graduate School of Engineering Science, Osaka University 1-3 Machikaneyama-cho, Toyonaka, Osaka 560-8531, Japan E-mail: imori.stat@gmail.com

More information

CHOICE OF REFERENCE SUBCLASS IN REGRESSION MODELS

CHOICE OF REFERENCE SUBCLASS IN REGRESSION MODELS CHOICE OF REFERENCE SUBCLASS IN REGRESSION MODELS Gilbert MacKenzie 1,2 & Defen Peng 2,3 1 ENSAI, Rennes & 2 Centre of Biostatistics University of Limerick, Ireland 3 UBC,Vancouver, Canada. CASI, Templepatrick,

More information

PROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception

PROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception PROBABILITY DISTRIBUTIONS Credits 2 These slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Parametric Distributions 3 Basic building blocks: Need to determine given Representation:

More information

CMPE 58K Bayesian Statistics and Machine Learning Lecture 5

CMPE 58K Bayesian Statistics and Machine Learning Lecture 5 CMPE 58K Bayesian Statistics and Machine Learning Lecture 5 Multivariate distributions: Gaussian, Bernoulli, Probability tables Department of Computer Engineering, Boğaziçi University, Istanbul, Turkey

More information

Cross-sectional space-time modeling using ARNN(p, n) processes

Cross-sectional space-time modeling using ARNN(p, n) processes Cross-sectional space-time modeling using ARNN(p, n) processes W. Polasek K. Kakamu September, 006 Abstract We suggest a new class of cross-sectional space-time models based on local AR models and nearest

More information

6 Pattern Mixture Models

6 Pattern Mixture Models 6 Pattern Mixture Models A common theme underlying the methods we have discussed so far is that interest focuses on making inference on parameters in a parametric or semiparametric model for the full data

More information

The Surprising Conditional Adventures of the Bootstrap

The Surprising Conditional Adventures of the Bootstrap The Surprising Conditional Adventures of the Bootstrap G. Alastair Young Department of Mathematics Imperial College London Inaugural Lecture, 13 March 2006 Acknowledgements Early influences: Eric Renshaw,

More information

Continuous Time Survival in Latent Variable Models

Continuous Time Survival in Latent Variable Models Continuous Time Survival in Latent Variable Models Tihomir Asparouhov 1, Katherine Masyn 2, Bengt Muthen 3 Muthen & Muthen 1 University of California, Davis 2 University of California, Los Angeles 3 Abstract

More information

The Restricted Likelihood Ratio Test at the Boundary in Autoregressive Series

The Restricted Likelihood Ratio Test at the Boundary in Autoregressive Series The Restricted Likelihood Ratio Test at the Boundary in Autoregressive Series Willa W. Chen Rohit S. Deo July 6, 009 Abstract. The restricted likelihood ratio test, RLRT, for the autoregressive coefficient

More information

Imputation Algorithm Using Copulas

Imputation Algorithm Using Copulas Metodološki zvezki, Vol. 3, No. 1, 2006, 109-120 Imputation Algorithm Using Copulas Ene Käärik 1 Abstract In this paper the author demonstrates how the copulas approach can be used to find algorithms for

More information

Labor-Supply Shifts and Economic Fluctuations. Technical Appendix

Labor-Supply Shifts and Economic Fluctuations. Technical Appendix Labor-Supply Shifts and Economic Fluctuations Technical Appendix Yongsung Chang Department of Economics University of Pennsylvania Frank Schorfheide Department of Economics University of Pennsylvania January

More information

Analysing geoadditive regression data: a mixed model approach

Analysing geoadditive regression data: a mixed model approach Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression

More information

The impact of covariance misspecification in multivariate Gaussian mixtures on estimation and inference

The impact of covariance misspecification in multivariate Gaussian mixtures on estimation and inference The impact of covariance misspecification in multivariate Gaussian mixtures on estimation and inference An application to longitudinal modeling Brianna Heggeseth with Nicholas Jewell Department of Statistics

More information

Bayesian Graphical Models for Structural Vector AutoregressiveMarch Processes 21, / 1

Bayesian Graphical Models for Structural Vector AutoregressiveMarch Processes 21, / 1 Bayesian Graphical Models for Structural Vector Autoregressive Processes Daniel Ahelegbey, Monica Billio, and Roberto Cassin (2014) March 21, 2015 Bayesian Graphical Models for Structural Vector AutoregressiveMarch

More information

Single Equation Linear GMM with Serially Correlated Moment Conditions

Single Equation Linear GMM with Serially Correlated Moment Conditions Single Equation Linear GMM with Serially Correlated Moment Conditions Eric Zivot November 2, 2011 Univariate Time Series Let {y t } be an ergodic-stationary time series with E[y t ]=μ and var(y t )

More information

Charles E. McCulloch Biometrics Unit and Statistics Center Cornell University

Charles E. McCulloch Biometrics Unit and Statistics Center Cornell University A SURVEY OF VARIANCE COMPONENTS ESTIMATION FROM BINARY DATA by Charles E. McCulloch Biometrics Unit and Statistics Center Cornell University BU-1211-M May 1993 ABSTRACT The basic problem of variance components

More information

Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes

Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes TTU, October 26, 2012 p. 1/3 Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes Hao Zhang Department of Statistics Department of Forestry and Natural Resources Purdue University

More information

MODELING COVARIANCE STRUCTURE IN UNBALANCED LONGITUDINAL DATA. A Dissertation MIN CHEN

MODELING COVARIANCE STRUCTURE IN UNBALANCED LONGITUDINAL DATA. A Dissertation MIN CHEN MODELING COVARIANCE STRUCTURE IN UNBALANCED LONGITUDINAL DATA A Dissertation by MIN CHEN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for

More information

Group Sequential Tests for Delayed Responses. Christopher Jennison. Lisa Hampson. Workshop on Special Topics on Sequential Methodology

Group Sequential Tests for Delayed Responses. Christopher Jennison. Lisa Hampson. Workshop on Special Topics on Sequential Methodology Group Sequential Tests for Delayed Responses Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Lisa Hampson Department of Mathematics and Statistics,

More information

Multivariate Bayesian Linear Regression MLAI Lecture 11

Multivariate Bayesian Linear Regression MLAI Lecture 11 Multivariate Bayesian Linear Regression MLAI Lecture 11 Neil D. Lawrence Department of Computer Science Sheffield University 21st October 2012 Outline Univariate Bayesian Linear Regression Multivariate

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information

401 Review. 6. Power analysis for one/two-sample hypothesis tests and for correlation analysis.

401 Review. 6. Power analysis for one/two-sample hypothesis tests and for correlation analysis. 401 Review Major topics of the course 1. Univariate analysis 2. Bivariate analysis 3. Simple linear regression 4. Linear algebra 5. Multiple regression analysis Major analysis methods 1. Graphical analysis

More information

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score

More information

Approximation of Survival Function by Taylor Series for General Partly Interval Censored Data

Approximation of Survival Function by Taylor Series for General Partly Interval Censored Data Malaysian Journal of Mathematical Sciences 11(3): 33 315 (217) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Journal homepage: http://einspem.upm.edu.my/journal Approximation of Survival Function by Taylor

More information

GARCH Models Estimation and Inference

GARCH Models Estimation and Inference GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in

More information

Bayesian Multivariate Logistic Regression

Bayesian Multivariate Logistic Regression Bayesian Multivariate Logistic Regression Sean M. O Brien and David B. Dunson Biostatistics Branch National Institute of Environmental Health Sciences Research Triangle Park, NC 1 Goals Brief review of

More information

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2 Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate

More information

Multivariate Normal-Laplace Distribution and Processes

Multivariate Normal-Laplace Distribution and Processes CHAPTER 4 Multivariate Normal-Laplace Distribution and Processes The normal-laplace distribution, which results from the convolution of independent normal and Laplace random variables is introduced by

More information

Pubh 8482: Sequential Analysis

Pubh 8482: Sequential Analysis Pubh 8482: Sequential Analysis Joseph S. Koopmeiners Division of Biostatistics University of Minnesota Week 7 Course Summary To this point, we have discussed group sequential testing focusing on Maintaining

More information

Asymptotic standard errors of MLE

Asymptotic standard errors of MLE Asymptotic standard errors of MLE Suppose, in the previous example of Carbon and Nitrogen in soil data, that we get the parameter estimates For maximum likelihood estimation, we can use Hessian matrix

More information

A Fully Nonparametric Modeling Approach to. BNP Binary Regression

A Fully Nonparametric Modeling Approach to. BNP Binary Regression A Fully Nonparametric Modeling Approach to Binary Regression Maria Department of Applied Mathematics and Statistics University of California, Santa Cruz SBIES, April 27-28, 2012 Outline 1 2 3 Simulation

More information

Introduction to lnmle: An R Package for Marginally Specified Logistic-Normal Models for Longitudinal Binary Data

Introduction to lnmle: An R Package for Marginally Specified Logistic-Normal Models for Longitudinal Binary Data Introduction to lnmle: An R Package for Marginally Specified Logistic-Normal Models for Longitudinal Binary Data Bryan A. Comstock and Patrick J. Heagerty Department of Biostatistics University of Washington

More information

Multivariate Regression

Multivariate Regression Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the

More information

Probability and Estimation. Alan Moses

Probability and Estimation. Alan Moses Probability and Estimation Alan Moses Random variables and probability A random variable is like a variable in algebra (e.g., y=e x ), but where at least part of the variability is taken to be stochastic.

More information

Introduction to Maximum Likelihood Estimation

Introduction to Maximum Likelihood Estimation Introduction to Maximum Likelihood Estimation Eric Zivot July 26, 2012 The Likelihood Function Let 1 be an iid sample with pdf ( ; ) where is a ( 1) vector of parameters that characterize ( ; ) Example:

More information

Statistical Inference and Methods

Statistical Inference and Methods Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 31st January 2006 Part VI Session 6: Filtering and Time to Event Data Session 6: Filtering and

More information

Generalized Linear Models (GLZ)

Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the

More information

Heteroskedasticity; Step Changes; VARMA models; Likelihood ratio test statistic; Cusum statistic.

Heteroskedasticity; Step Changes; VARMA models; Likelihood ratio test statistic; Cusum statistic. 47 3!,57 Statistics and Econometrics Series 5 Febrary 24 Departamento de Estadística y Econometría Universidad Carlos III de Madrid Calle Madrid, 126 2893 Getafe (Spain) Fax (34) 91 624-98-49 VARIANCE

More information

Computationally efficient banding of large covariance matrices for ordered data and connections to banding the inverse Cholesky factor

Computationally efficient banding of large covariance matrices for ordered data and connections to banding the inverse Cholesky factor Computationally efficient banding of large covariance matrices for ordered data and connections to banding the inverse Cholesky factor Y. Wang M. J. Daniels wang.yanpin@scrippshealth.org mjdaniels@austin.utexas.edu

More information

Multivariate Time Series: VAR(p) Processes and Models

Multivariate Time Series: VAR(p) Processes and Models Multivariate Time Series: VAR(p) Processes and Models A VAR(p) model, for p > 0 is X t = φ 0 + Φ 1 X t 1 + + Φ p X t p + A t, where X t, φ 0, and X t i are k-vectors, Φ 1,..., Φ p are k k matrices, with

More information

Geostatistical Modeling for Large Data Sets: Low-rank methods

Geostatistical Modeling for Large Data Sets: Low-rank methods Geostatistical Modeling for Large Data Sets: Low-rank methods Whitney Huang, Kelly-Ann Dixon Hamil, and Zizhuang Wu Department of Statistics Purdue University February 22, 2016 Outline Motivation Low-rank

More information

Outline. Overview of Issues. Spatial Regression. Luc Anselin

Outline. Overview of Issues. Spatial Regression. Luc Anselin Spatial Regression Luc Anselin University of Illinois, Urbana-Champaign http://www.spacestat.com Outline Overview of Issues Spatial Regression Specifications Space-Time Models Spatial Latent Variable Models

More information

Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation

Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation Patrick Breheny February 8 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/27 Introduction Basic idea Standardization Large-scale testing is, of course, a big area and we could keep talking

More information

Modeling the scale parameter ϕ A note on modeling correlation of binary responses Using marginal odds ratios to model association for binary responses

Modeling the scale parameter ϕ A note on modeling correlation of binary responses Using marginal odds ratios to model association for binary responses Outline Marginal model Examples of marginal model GEE1 Augmented GEE GEE1.5 GEE2 Modeling the scale parameter ϕ A note on modeling correlation of binary responses Using marginal odds ratios to model association

More information

ST 740: Linear Models and Multivariate Normal Inference

ST 740: Linear Models and Multivariate Normal Inference ST 740: Linear Models and Multivariate Normal Inference Alyson Wilson Department of Statistics North Carolina State University November 4, 2013 A. Wilson (NCSU STAT) Linear Models November 4, 2013 1 /

More information

On prediction and density estimation Peter McCullagh University of Chicago December 2004

On prediction and density estimation Peter McCullagh University of Chicago December 2004 On prediction and density estimation Peter McCullagh University of Chicago December 2004 Summary Having observed the initial segment of a random sequence, subsequent values may be predicted by calculating

More information

Machine learning - HT Maximum Likelihood

Machine learning - HT Maximum Likelihood Machine learning - HT 2016 3. Maximum Likelihood Varun Kanade University of Oxford January 27, 2016 Outline Probabilistic Framework Formulate linear regression in the language of probability Introduce

More information

Integrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University

Integrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University Integrated Likelihood Estimation in Semiparametric Regression Models Thomas A. Severini Department of Statistics Northwestern University Joint work with Heping He, University of York Introduction Let Y

More information

Bayesian (conditionally) conjugate inference for discrete data models. Jon Forster (University of Southampton)

Bayesian (conditionally) conjugate inference for discrete data models. Jon Forster (University of Southampton) Bayesian (conditionally) conjugate inference for discrete data models Jon Forster (University of Southampton) with Mark Grigsby (Procter and Gamble?) Emily Webb (Institute of Cancer Research) Table 1:

More information

A Sampling of IMPACT Research:

A Sampling of IMPACT Research: A Sampling of IMPACT Research: Methods for Analysis with Dropout and Identifying Optimal Treatment Regimes Marie Davidian Department of Statistics North Carolina State University http://www.stat.ncsu.edu/

More information

[y i α βx i ] 2 (2) Q = i=1

[y i α βx i ] 2 (2) Q = i=1 Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation

More information

Regression diagnostics

Regression diagnostics Regression diagnostics Kerby Shedden Department of Statistics, University of Michigan November 5, 018 1 / 6 Motivation When working with a linear model with design matrix X, the conventional linear model

More information

Richard D Riley was supported by funding from a multivariate meta-analysis grant from

Richard D Riley was supported by funding from a multivariate meta-analysis grant from Bayesian bivariate meta-analysis of correlated effects: impact of the prior distributions on the between-study correlation, borrowing of strength, and joint inferences Author affiliations Danielle L Burke

More information

Inference on the Univariate Frailty Model: An Approach Bayesian Reference Analysis

Inference on the Univariate Frailty Model: An Approach Bayesian Reference Analysis Inference on the Univariate Frailty Model: An Approach Bayesian Reference Analysis Vera Lucia D. Tomazella and Camila Bertini Martins Universidade Federal de São Carlos, SP-Brasil (vera@ufscar.br; cacamila-bertini@yahoo.com.br)

More information

STA 2201/442 Assignment 2

STA 2201/442 Assignment 2 STA 2201/442 Assignment 2 1. This is about how to simulate from a continuous univariate distribution. Let the random variable X have a continuous distribution with density f X (x) and cumulative distribution

More information

PROD. TYPE: COM. Simple improved condence intervals for comparing matched proportions. Alan Agresti ; and Yongyi Min UNCORRECTED PROOF

PROD. TYPE: COM. Simple improved condence intervals for comparing matched proportions. Alan Agresti ; and Yongyi Min UNCORRECTED PROOF pp: --2 (col.fig.: Nil) STATISTICS IN MEDICINE Statist. Med. 2004; 2:000 000 (DOI: 0.002/sim.8) PROD. TYPE: COM ED: Chandra PAGN: Vidya -- SCAN: Nil Simple improved condence intervals for comparing matched

More information

Random vectors X 1 X 2. Recall that a random vector X = is made up of, say, k. X k. random variables.

Random vectors X 1 X 2. Recall that a random vector X = is made up of, say, k. X k. random variables. Random vectors Recall that a random vector X = X X 2 is made up of, say, k random variables X k A random vector has a joint distribution, eg a density f(x), that gives probabilities P(X A) = f(x)dx Just

More information