Integrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University
|
|
- Zoe Wells
- 5 years ago
- Views:
Transcription
1 Integrated Likelihood Estimation in Semiparametric Regression Models Thomas A. Severini Department of Statistics Northwestern University Joint work with Heping He, University of York
2 Introduction Let Y 1, Y 2,..., Y n denote real-valued random variables of the form Y j = x T j β + γ(z j ) + ϵ j, j = 1,..., n where x 1,..., x n are constants in R p ; z 1,..., z n are constants, taking values in a set Z ϵ 1,..., ϵ n are unobserved mean-0 r.v.s such that ϵ = (ϵ 1,..., ϵ n ) T has a multivariate normal distribution covariance matrix Ω ϕ, ϕ Φ and β R p are unknown parameters γ is an unknown real-valued function on Z, taking values in a set of functions Γ Our goal is inference about the parameter β in the presence of the nuisance parameters γ and ϕ
3 The likelihood function for this model is given by Ω ϕ exp{ 2 (Y Xβ g))t Ω 1 ϕ (Y Xβ g)} where Y = (y 1,..., y n ) T, X is the n p matrix with jth row x j, and g = (γ(z 1 ),..., γ(z n )) T. Hence, in order to proceed with likelihood inference for β some method of dealing with the nuisance parameters γ, ϕ is needed. Many methods of estimation have been proposed for this model: Engle, Granger, Rice, and Weiss (1986), Hastie and Tibshirani (1990), Heckman (1986), Ruppert, Wand, and Carroll (2003), Severini and Staniswalis (1994), and Speckman (1988). Most involve eliminating γ using some modification of the profile likelihood idea.
4 An alternative approach is to use an integrated likelihood in which γ is removed by averaging with respect to some weight function. Suppose that Z R and Γ is a set of differentiable functions on Z. Consider a weight function for γ corresponding to a mean-zero Gaussian stochastic process with covariance function K λ (, ) where λ is a parameter. Under this distribution, the vector (γ(z 1 ),..., γ(z n )) T has a multivariate normal distribution with mean vector 0 and covariance matrix Σ λ. The integrated likelihood is given by Ω ϕ + Σ λ 1 2 exp{ 1 2 (y xβ)t (Ω ϕ + Σ λ ) 1 (y xβ)}.
5 The integrated likelihood approach has several advantages: Restrictions on γ are often easy to impose by using a covariance function that respects the restrictions More complicated models in which the parameters of interest are intertwined with the unknown function are often easier to handle through the covariance structure than through the mean function of the observations It is straightforward to incorporate a parametric model for the covariance matrix of the errors
6 Inference based on an integrated likelihood is related to Bayesian inference in nonparametric and semiparametric regression models. Much of the Bayesian work in this area has made use of the fact that smoothing splines have a Bayesian interpretation (Wahba, 1990) and the covariance function is chosen so that spline estimation can be used (see below). Here the covariance function is chosen to reflect our assumptions about γ and the model Also we consider non-bayesian methods of inference and consider standard frequentist properties such as consistency and asymptotic distribution theory. However, the basic approach could also be applied to Bayesian inference.
7 Estimation The integrated likelihood is a normal likelihood with mean vector Xβ and covariance matrix V (θ) = Ω ϕ + Σ λ, θ = (ϕ, λ). Given the covariance parameter θ, β can be estimated by generalized least-squares: ˆβ(θ) = X T (X T V 1 X) 1 X T V 1 Y, V V (θ). When θ is unknown, it can be replaced by an estimator. To estimate θ, we can use the restricted maximum likelihood (REML) estimator, l p (θ) 1 2 log X T V (θ)x where l p is the profile integrated likelihood. Given the REML estimator ˆθ of θ, an estimator of β is given by ˆβ(ˆθ).
8 Note that standard methods of computation for mixed models can be used. To estimate γ, we can use the Best Linear Unbiased Predictor (BLUP) based on the assumption that γ is a random function. Let z denote an element of Z and consider estimation of γ(z ). The BLUP of γ(z ) is Σ (ˆθ)V (ˆθ) 1 (Y X ˆβ(ˆθ)). To use this approach, the covariance function K λ must be chosen; to do this, we consider the properties of {γ(z) : z Z} as a random process.
9 Models with an Unknown Continuous Function on the Real Line Suppose Z R and γ is a smooth function. It is often reasonable to assume that the covariance of γ(z) and γ( z) is a decreasing function of z z so that K λ (z, z) = τ 2 Kν ( z z /α) where K ν is a decreasing, positive definite function on [0, ) with K ν (0) = 1. Here τ > 0 is the standard deviation of γ(z), α > 0 represents a scale parameter, and ν represents a shape parameter (if present). One choice for K ν is the Gaussian covariance function K(t) = exp( 1 2 t2 ); then {γ(z) : z Z} is a stationary, infinitely-differentiable random process.
10 As noted earlier, the IL approach is related to spline estimation. There are at least two spline methods that can be used here: smoothing splines (e.g., Wahba, 1990) and penalized splines (e.g., Ruppert, Wand, and Carroll, 2003). Smoothing splines: γ is a mean-zero Gaussian process with covariance function 1 + z z [(1 + z 2 )(1 + z 2 )] 1 2, z, z [0, 1]. This process is nonstationary and highly correlated. Penalized splines: γ is a Gaussian stochastic process with mean δ 0 + δ 1 z + δ 2 z 2 and covariance function k K P (z, z) = τ 2 (z d j ) 2 ( z d j ) 2 for d k < z d k+1 and z z, where 0 < d 1 < d 2 <... < d r < 1 are given. Under K P, the correlation of γ(z), γ( z) is generally small. j=1
11 Incorporating Assumptions about γ( ) in the Model A main advantage of the IL approach is in models with additional assumptions on γ. Linear constraints on γ Suppose γ is subject to a constraint of the form T γ = 0 where T is a known, realvalued, affine function on L 2 (Z). In carrying out the IL approach, we need a distribution for {γ(z) : z Z} that respects the condition T γ = 0. First consider a mean-zero Gaussian process {γ 0 (z) : z Z} with Gaussian covariance function H λ and take {γ(z) : z Z} to have the the conditional distribution of γ 0 given that T γ 0 = 0. This conditional distribution is identical to the distribution of (Janson, 1997). γ 0 (z) Cov[γ 0(z), T γ 0 ] T γ 0 Var(T γ 0 )
12 It follows that {γ(z) : z Z} is a mean-zero Gaussian process with covariance function K λ (t, s) = H λ (t, s) Cov[γ 0(t), T γ 0 ; λ]cov[γ 0 (s), T γ 0 ; λ]. Var(T γ 0 ; λ) Thus, the restriction can be taken into account by simply modifying the covariance function of the process. For instance, suppose that T γ 0 = Z γ 0 (t)w(t)dt c where w is a given element of L 2 (Z) and c is a constant. Then K λ (t, s) = H λ (t, s) Z H λ(s, t)w(t)dt Z H λ(s, t)w(s)ds Z Z H. λ(s, t)w(s)w(t)ds dt
13 Asymptotic Properties of the Estimator Suppose that θ satisfies ˆθ = θ + O p (1/ n). Recall that θ = (ϕ, λ) where ϕ is a parameter of the error covariance matrix and λ is a parameter of the covariance function of γ( ). Therefore ϕ = ϕ 0, the true value of ϕ. However, there is no conventional true value of λ. ˆβ has the same asymptotic distribution as ˆβ (X T (V ) 1 X) 1 X T (V ) 1 Y, V = V (θ ). Note that ˆβ is normally distributed but it has bias (X T (V ) 1 X) 1 X T (V ) 1 g, g = (γ(z 1 ),..., γ(z n )) T.
14 The key idea in showing that the bias is asymptotically negligible is that Σ λ properties similar to a covariance function of g. has E.g., suppose that Ω ϕ = I and Σ λ g. Then (V ) 1 g = = gg T, the sample covariance function based on g g g = O(n 1 ). Under fairly general conditions on γ, it can be shown that n( ˆβ β0 ) D N(0, M ) as n where M lim n n[(xt V 1 (θ )X) 1 X T V 1 (θ )Ω ϕ0 V 1 (θ )X(X T V 1 (θ )X) 1 ].
15 Examples Example 1: Semiparametric regression model with independent errors Bowman and Azzalini (1997) present data taken taken from a survey of the fauna on the sea bed lying between the coast of northern Queensland and the Great Barrier Reef. Let Y denote catch score 1 and let x and z denote the latitude and longitude, respectively, of the sampling position. Here we use the data from zone 1; the sample size is n = 42. An appropriate model for these data is Y j = β 0 + β 1 x j + γ(z j ) + ϵ j, j = 1,..., n where ϵ 1,..., ϵ n are independent error terms with mean 0 and constant variance.
16 This model was fit using the IL method with a Gaussian covariance function. For comparison, the model was also fit using the generalized additive model approach of Hastie & Tibshirani (smoothing splines), the penalized spline method described in Semiparametric Regression by Ruppert, Wand, & Carroll and a kernel-based estimator (Speckman, 1988 and many others). Estimates of β 1 (reported SE): IL: 1.020(0.356) GAM: 1.153(0.371) Pen Spline: 1.098(0.368) Kernel: 1.203(0.371) The estimates of γ are also in close agreement.
17 Estimates of gamma in the reef example gamma(z) Int Like SPM GAM Kernel z
18 A small simulation study was conducted in which data were simulated from the model described here, with the parameter values taken to be the estimates based on the integrated likelihood method. A Monte Carlo sample size of 5000 was used. Comparison of Estimators in the Reef Example Method Int Lik GAM Pen Spline Kernel Bias SD MSE Est SE Cov Prob
19 Example 2: A shape-invariant model Hastie, Tibshirani, and Friedman (2001) describe data on bone mineral density (BMD) in adolescents. The response variable Y j is relative change in spinal BMD, which is modeled as a function of age and gender. Preliminary analysis suggests that the relationship between Y j and age is different for males and females, with the function relating Y j and age for males being a scaled and shifted version of the corresponding function for females. This observation suggests a model in which the mean of Y j is of the form β 0 + β x j 1 γ(z j + β 2 x j ) where z j denotes age and x j = 1 is subject j is male and 0 otherwise.
20 It follows that the mean function for males is β 0 +β 1 γ(z j +β 2 ) while the mean function for females is β 0 + γ(z j ). To compute the IL, we use a weight function based on taking γ to be a mean 0 Gaussian process with a Gaussian covariance function. Then Cov(β x j 1 γ(z j + β 2 x j ), β x k 1 γ(z k + β 2 x k )) = β x j+x k 1 K λ ( z j z k + β 2 (x j x k ) ). There is a further complication to this data set some of the subjects are tested multiple times (485 observations on 261 subjects). To account for this, the model was modified to include subject-specific intercept terms, taken to be normally distributed random effects.
21 Thus, the model has 7 parameters: β 0, the mean of the subject-specific intercepts β 1 and β 2, which describe how males and females differ the variances of the error term and of the random interecepts two parameters for the Gaussian covariance function. Note that the parameters of primary interest, β 1 and β 2, appear in the covariance matrix of Y, rather than in the mean function. The estimate of the shift is 2.1 years (SE = 0.19); the estimate of the scaling factor is 0.79 (SE = 0.068). The plot of the estimated model describes the differences between the relationship between change in BMD and age for males and females.
22 Comparison of Males and Females in the BMD Example Relative Change in Spinal BMD Age
23 Summary The IL method provides a conceptually easy approach to estimation in models with an unknown function In simple models, the IL method works (nearly) as well as standard methods In more complicated settings, it is often straightforward to modify the covariance function used to form the IL Computation: standard methods work surprisingly well in the normal case; for non-normal errors more sophisticated methods will be needed Current proofs of asymptotic properties require stronger conditions than other methods; examples suggest that weaker conditions would suffice
Likelihood-Based Methods
Likelihood-Based Methods Handbook of Spatial Statistics, Chapter 4 Susheela Singh September 22, 2016 OVERVIEW INTRODUCTION MAXIMUM LIKELIHOOD ESTIMATION (ML) RESTRICTED MAXIMUM LIKELIHOOD ESTIMATION (REML)
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationNonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University
Nonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University this presentation derived from that presented at the Pan-American Advanced
More informationWeb Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.
Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we
More informationModelling geoadditive survival data
Modelling geoadditive survival data Thomas Kneib & Ludwig Fahrmeir Department of Statistics, Ludwig-Maximilians-University Munich 1. Leukemia survival data 2. Structured hazard regression 3. Mixed model
More informationSpatially Adaptive Smoothing Splines
Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia speckman@statmissouriedu September 11, 23 Banff 9/7/3 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, =
More informationA Modern Look at Classical Multivariate Techniques
A Modern Look at Classical Multivariate Techniques Yoonkyung Lee Department of Statistics The Ohio State University March 16-20, 2015 The 13th School of Probability and Statistics CIMAT, Guanajuato, Mexico
More informationInversion Base Height. Daggot Pressure Gradient Visibility (miles)
Stanford University June 2, 1998 Bayesian Backtting: 1 Bayesian Backtting Trevor Hastie Stanford University Rob Tibshirani University of Toronto Email: trevor@stat.stanford.edu Ftp: stat.stanford.edu:
More informationSome Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model
Some Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model 1. Introduction Varying-coefficient partially linear model (Zhang, Lee, and Song, 2002; Xia, Zhang, and Tong, 2004;
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis
More informationREGRESSION WITH SPATIALLY MISALIGNED DATA. Lisa Madsen Oregon State University David Ruppert Cornell University
REGRESSION ITH SPATIALL MISALIGNED DATA Lisa Madsen Oregon State University David Ruppert Cornell University SPATIALL MISALIGNED DATA 10 X X X X X X X X 5 X X X X X 0 X 0 5 10 OUTLINE 1. Introduction 2.
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationProfessors Lin and Ying are to be congratulated for an interesting paper on a challenging topic and for introducing survival analysis techniques to th
DISCUSSION OF THE PAPER BY LIN AND YING Xihong Lin and Raymond J. Carroll Λ July 21, 2000 Λ Xihong Lin (xlin@sph.umich.edu) is Associate Professor, Department ofbiostatistics, University of Michigan, Ann
More informationA general mixed model approach for spatio-temporal regression data
A general mixed model approach for spatio-temporal regression data Thomas Kneib, Ludwig Fahrmeir & Stefan Lang Department of Statistics, Ludwig-Maximilians-University Munich 1. Spatio-temporal regression
More informationPenalized Splines, Mixed Models, and Recent Large-Sample Results
Penalized Splines, Mixed Models, and Recent Large-Sample Results David Ruppert Operations Research & Information Engineering, Cornell University Feb 4, 2011 Collaborators Matt Wand, University of Wollongong
More informationRegularization in Cox Frailty Models
Regularization in Cox Frailty Models Andreas Groll 1, Trevor Hastie 2, Gerhard Tutz 3 1 Ludwig-Maximilians-Universität Munich, Department of Mathematics, Theresienstraße 39, 80333 Munich, Germany 2 University
More informationNonparametric Small Area Estimation Using Penalized Spline Regression
Nonparametric Small Area Estimation Using Penalized Spline Regression 0verview Spline-based nonparametric regression Nonparametric small area estimation Prediction mean squared error Bootstrapping small
More informationSome properties of Likelihood Ratio Tests in Linear Mixed Models
Some properties of Likelihood Ratio Tests in Linear Mixed Models Ciprian M. Crainiceanu David Ruppert Timothy J. Vogelsang September 19, 2003 Abstract We calculate the finite sample probability mass-at-zero
More informationExact Likelihood Ratio Tests for Penalized Splines
Exact Likelihood Ratio Tests for Penalized Splines By CIPRIAN CRAINICEANU, DAVID RUPPERT, GERDA CLAESKENS, M.P. WAND Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe Street, Baltimore,
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationFREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE
FREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE Donald A. Pierce Oregon State Univ (Emeritus), RERF Hiroshima (Retired), Oregon Health Sciences Univ (Adjunct) Ruggero Bellio Univ of Udine For Perugia
More informationBayesian Estimation and Inference for the Generalized Partial Linear Model
Bayesian Estimation Inference for the Generalized Partial Linear Model Haitham M. Yousof 1, Ahmed M. Gad 2 1 Department of Statistics, Mathematics Insurance, Benha University, Egypt. 2 Department of Statistics,
More informationAnalysing geoadditive regression data: a mixed model approach
Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression
More informationSemiparametric Regression of Multi-Dimensional Genetic Pathway Data: Least Squares Kernel Machines and Linear Mixed Models
Semiparametric Regression of Multi-Dimensional Genetic Pathway Data: Least Squares Kernel Machines and Linear Mixed Models Dawei Liu 1, Xihong Lin 2, Debashis Ghosh 3 1 Center for Statistical Sciences,
More informationAn Introduction to GAMs based on penalized regression splines. Simon Wood Mathematical Sciences, University of Bath, U.K.
An Introduction to GAMs based on penalied regression splines Simon Wood Mathematical Sciences, University of Bath, U.K. Generalied Additive Models (GAM) A GAM has a form something like: g{e(y i )} = η
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationRestricted Likelihood Ratio Tests in Nonparametric Longitudinal Models
Restricted Likelihood Ratio Tests in Nonparametric Longitudinal Models Short title: Restricted LR Tests in Longitudinal Models Ciprian M. Crainiceanu David Ruppert May 5, 2004 Abstract We assume that repeated
More informationGaussian Graphical Models and Graphical Lasso
ELE 538B: Sparsity, Structure and Inference Gaussian Graphical Models and Graphical Lasso Yuxin Chen Princeton University, Spring 2017 Multivariate Gaussians Consider a random vector x N (0, Σ) with pdf
More informationAsymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands
Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Elizabeth C. Mannshardt-Shamseldin Advisor: Richard L. Smith Duke University Department
More informationHierarchical Modeling for Univariate Spatial Data
Hierarchical Modeling for Univariate Spatial Data Geography 890, Hierarchical Bayesian Models for Environmental Spatial Data Analysis February 15, 2011 1 Spatial Domain 2 Geography 890 Spatial Domain This
More informationLikelihood Ratio Tests. that Certain Variance Components Are Zero. Ciprian M. Crainiceanu. Department of Statistical Science
1 Likelihood Ratio Tests that Certain Variance Components Are Zero Ciprian M. Crainiceanu Department of Statistical Science www.people.cornell.edu/pages/cmc59 Work done jointly with David Ruppert, School
More informationBIOS 2083 Linear Models c Abdus S. Wahed
Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 9: Basis Expansions Department of Statistics & Biostatistics Rutgers University Nov 01, 2011 Regression and Classification Linear Regression. E(Y X) = f(x) We want to learn
More informationRestricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model
Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives
More informationSEMI-LINEAR LINEAR INDEX MODEL WHEN THE LINEAR COVARIATES AND INDICES ARE INDEPENDENT
SEMI-LINEAR LINEAR INDEX MODEL WHEN THE LINEAR COVARIATES AND INDICES ARE INDEPENDENT By Yun Sam Chong, Jane-Ling Wang and Lixing Zhu Summary Wecker Associate, University of California at Davis, and The
More informationSpatial smoothing using Gaussian processes
Spatial smoothing using Gaussian processes Chris Paciorek paciorek@hsph.harvard.edu August 5, 2004 1 OUTLINE Spatial smoothing and Gaussian processes Covariance modelling Nonstationary covariance modelling
More informationMotivational Example
Motivational Example Data: Observational longitudinal study of obesity from birth to adulthood. Overall Goal: Build age-, gender-, height-specific growth charts (under 3 year) to diagnose growth abnomalities.
More informationSemiparametric Mixed Model for Evaluating Pathway-Environment Interaction
Semiparametric Mixed Model for Evaluating Pathway-Environment Interaction arxiv:1206.2716v1 [stat.me] 13 Jun 2012 Zaili Fang 1, Inyoung Kim 1, and Jeesun Jung 2 June 14, 2012 1 Department of Statistics,
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Institute of Statistics and Econometrics Georg-August-University Göttingen Department of Statistics
More informationGaussian processes and bayesian optimization Stanisław Jastrzębski. kudkudak.github.io kudkudak
Gaussian processes and bayesian optimization Stanisław Jastrzębski kudkudak.github.io kudkudak Plan Goal: talk about modern hyperparameter optimization algorithms Bayes reminder: equivalent linear regression
More informationIllustration of the Varying Coefficient Model for Analyses the Tree Growth from the Age and Space Perspectives
TR-No. 14-06, Hiroshima Statistical Research Group, 1 11 Illustration of the Varying Coefficient Model for Analyses the Tree Growth from the Age and Space Perspectives Mariko Yamamura 1, Keisuke Fukui
More informationRidge Estimation and its Modifications for Linear Regression with Deterministic or Stochastic Predictors
Ridge Estimation and its Modifications for Linear Regression with Deterministic or Stochastic Predictors James Younker Thesis submitted to the Faculty of Graduate and Postdoctoral Studies in partial fulfillment
More informationThe Poisson transform for unnormalised statistical models. Nicolas Chopin (ENSAE) joint work with Simon Barthelmé (CNRS, Gipsa-LAB)
The Poisson transform for unnormalised statistical models Nicolas Chopin (ENSAE) joint work with Simon Barthelmé (CNRS, Gipsa-LAB) Part I Unnormalised statistical models Unnormalised statistical models
More informationLOCAL POLYNOMIAL AND PENALIZED TRIGONOMETRIC SERIES REGRESSION
Statistica Sinica 24 (2014), 1215-1238 doi:http://dx.doi.org/10.5705/ss.2012.040 LOCAL POLYNOMIAL AND PENALIZED TRIGONOMETRIC SERIES REGRESSION Li-Shan Huang and Kung-Sik Chan National Tsing Hua University
More informationMIXED MODELS THE GENERAL MIXED MODEL
MIXED MODELS This chapter introduces best linear unbiased prediction (BLUP), a general method for predicting random effects, while Chapter 27 is concerned with the estimation of variances by restricted
More informationModeling Real Estate Data using Quantile Regression
Modeling Real Estate Data using Semiparametric Quantile Regression Department of Statistics University of Innsbruck September 9th, 2011 Overview 1 Application: 2 3 4 Hedonic regression data for house prices
More information6 Pattern Mixture Models
6 Pattern Mixture Models A common theme underlying the methods we have discussed so far is that interest focuses on making inference on parameters in a parametric or semiparametric model for the full data
More informationLocal regression I. Patrick Breheny. November 1. Kernel weighted averages Local linear regression
Local regression I Patrick Breheny November 1 Patrick Breheny STA 621: Nonparametric Statistics 1/27 Simple local models Kernel weighted averages The Nadaraya-Watson estimator Expected loss and prediction
More informationHypothesis Testing in Smoothing Spline Models
Hypothesis Testing in Smoothing Spline Models Anna Liu and Yuedong Wang October 10, 2002 Abstract This article provides a unified and comparative review of some existing test methods for the hypothesis
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationTwo Applications of Nonparametric Regression in Survey Estimation
Two Applications of Nonparametric Regression in Survey Estimation 1/56 Jean Opsomer Iowa State University Joint work with Jay Breidt, Colorado State University Gerda Claeskens, Université Catholique de
More informationA Framework for Daily Spatio-Temporal Stochastic Weather Simulation
A Framework for Daily Spatio-Temporal Stochastic Weather Simulation, Rick Katz, Balaji Rajagopalan Geophysical Statistics Project Institute for Mathematics Applied to Geosciences National Center for Atmospheric
More informationTopic 12 Overview of Estimation
Topic 12 Overview of Estimation Classical Statistics 1 / 9 Outline Introduction Parameter Estimation Classical Statistics Densities and Likelihoods 2 / 9 Introduction In the simplest possible terms, the
More informationGeneralized Elastic Net Regression
Abstract Generalized Elastic Net Regression Geoffroy MOURET Jean-Jules BRAULT Vahid PARTOVINIA This work presents a variation of the elastic net penalization method. We propose applying a combined l 1
More informationSIMULTANEOUS CONFIDENCE INTERVALS FOR SEMIPARAMETRIC LOGISTICS REGRESSION AND CONFIDENCE REGIONS FOR THE MULTI-DIMENSIONAL EFFECTIVE DOSE
Statistica Sinica 20 (2010), 637-659 SIMULTANEOUS CONFIDENCE INTERVALS FOR SEMIPARAMETRIC LOGISTICS REGRESSION AND CONFIDENCE REGIONS FOR THE MULTI-DIMENSIONAL EFFECTIVE DOSE Jialiang Li 1, Chunming Zhang
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationConsistent high-dimensional Bayesian variable selection via penalized credible regions
Consistent high-dimensional Bayesian variable selection via penalized credible regions Howard Bondell bondell@stat.ncsu.edu Joint work with Brian Reich Howard Bondell p. 1 Outline High-Dimensional Variable
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationThe linear model is the most fundamental of all serious statistical models encompassing:
Linear Regression Models: A Bayesian perspective Ingredients of a linear model include an n 1 response vector y = (y 1,..., y n ) T and an n p design matrix (e.g. including regressors) X = [x 1,..., x
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationASSESSING A VECTOR PARAMETER
SUMMARY ASSESSING A VECTOR PARAMETER By D.A.S. Fraser and N. Reid Department of Statistics, University of Toronto St. George Street, Toronto, Canada M5S 3G3 dfraser@utstat.toronto.edu Some key words. Ancillary;
More informationKneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach"
Kneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach" Sonderforschungsbereich 386, Paper 43 (25) Online unter: http://epub.ub.uni-muenchen.de/
More informationInference with few assumptions: Wasserman s example
Inference with few assumptions: Wasserman s example Christopher A. Sims Princeton University sims@princeton.edu October 27, 2007 Types of assumption-free inference A simple procedure or set of statistics
More informationCointegrating Regressions with Messy Regressors: J. Isaac Miller
NASMES 2008 June 21, 2008 Carnegie Mellon U. Cointegrating Regressions with Messy Regressors: Missingness, Mixed Frequency, and Measurement Error J. Isaac Miller University of Missouri 1 Messy Data Example
More informationInvariant HPD credible sets and MAP estimators
Bayesian Analysis (007), Number 4, pp. 681 69 Invariant HPD credible sets and MAP estimators Pierre Druilhet and Jean-Michel Marin Abstract. MAP estimators and HPD credible sets are often criticized in
More informationOn fixed effects estimation in spline-based semiparametric regression for spatial data
Libraries Conference on Applied Statistics in Agriculture 015-7th Annual Conference Proceedings On fixed effects estimation in spline-based semiparametric regression for spatial data Guilherme Ludwig University
More informationSpatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields
Spatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields 1 Introduction Jo Eidsvik Department of Mathematical Sciences, NTNU, Norway. (joeid@math.ntnu.no) February
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationMS&E 226: Small Data. Lecture 11: Maximum likelihood (v2) Ramesh Johari
MS&E 226: Small Data Lecture 11: Maximum likelihood (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 18 The likelihood function 2 / 18 Estimating the parameter This lecture develops the methodology behind
More informationCorrelated Spatiotemporal Data Modeling Using Generalized Additive Mixed Model and Bivariate Smoothing Techniques
Science Journal of Applied Mathematics and Statistics 2018; 6(2): 49-57 http://www.sciencepublishinggroup.com/j/sjams doi: 10.11648/j.sjams.20180602.11 ISSN: 2376-9491 (Print); ISSN: 2376-9513 (Online)
More informationStatistics for analyzing and modeling precipitation isotope ratios in IsoMAP
Statistics for analyzing and modeling precipitation isotope ratios in IsoMAP The IsoMAP uses the multiple linear regression and geostatistical methods to analyze isotope data Suppose the response variable
More informationMathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationAPTS course: 20th August 24th August 2018
APTS course: 20th August 24th August 2018 Flexible Regression Preliminary Material Claire Miller & Tereza Neocleous The term flexible regression refers to a wide range of methods which provide flexibility
More informationIntroduction. Chapter 1
Chapter 1 Introduction In this book we will be concerned with supervised learning, which is the problem of learning input-output mappings from empirical data (the training dataset). Depending on the characteristics
More informationMultivariate Survival Analysis
Multivariate Survival Analysis Previously we have assumed that either (X i, δ i ) or (X i, δ i, Z i ), i = 1,..., n, are i.i.d.. This may not always be the case. Multivariate survival data can arise in
More informationSTAT331. Cox s Proportional Hazards Model
STAT331 Cox s Proportional Hazards Model In this unit we introduce Cox s proportional hazards (Cox s PH) model, give a heuristic development of the partial likelihood function, and discuss adaptations
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationMaximum Smoothed Likelihood for Multivariate Nonparametric Mixtures
Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures David Hunter Pennsylvania State University, USA Joint work with: Tom Hettmansperger, Hoben Thomas, Didier Chauveau, Pierre Vandekerkhove,
More informationEstimating prediction error in mixed models
Estimating prediction error in mixed models benjamin saefken, thomas kneib georg-august university goettingen sonja greven ludwig-maximilians-university munich 1 / 12 GLMM - Generalized linear mixed models
More informationOn prediction and density estimation Peter McCullagh University of Chicago December 2004
On prediction and density estimation Peter McCullagh University of Chicago December 2004 Summary Having observed the initial segment of a random sequence, subsequent values may be predicted by calculating
More informationNow consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.
Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)
More informationFunctional Latent Feature Models. With Single-Index Interaction
Generalized With Single-Index Interaction Department of Statistics Center for Statistical Bioinformatics Institute for Applied Mathematics and Computational Science Texas A&M University Naisyin Wang and
More informationBayesian Linear Regression
Bayesian Linear Regression Sudipto Banerjee 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. September 15, 2010 1 Linear regression models: a Bayesian perspective
More informationWU Weiterbildung. Linear Mixed Models
Linear Mixed Effects Models WU Weiterbildung SLIDE 1 Outline 1 Estimation: ML vs. REML 2 Special Models On Two Levels Mixed ANOVA Or Random ANOVA Random Intercept Model Random Coefficients Model Intercept-and-Slopes-as-Outcomes
More informationModel Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao
Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley
More information9. Model Selection. statistical models. overview of model selection. information criteria. goodness-of-fit measures
FE661 - Statistical Methods for Financial Engineering 9. Model Selection Jitkomut Songsiri statistical models overview of model selection information criteria goodness-of-fit measures 9-1 Statistical models
More informationVariable Selection for Generalized Additive Mixed Models by Likelihood-based Boosting
Variable Selection for Generalized Additive Mixed Models by Likelihood-based Boosting Andreas Groll 1 and Gerhard Tutz 2 1 Department of Statistics, University of Munich, Akademiestrasse 1, D-80799, Munich,
More informationChapter 17: Undirected Graphical Models
Chapter 17: Undirected Graphical Models The Elements of Statistical Learning Biaobin Jiang Department of Biological Sciences Purdue University bjiang@purdue.edu October 30, 2014 Biaobin Jiang (Purdue)
More informationA Bayesian Treatment of Linear Gaussian Regression
A Bayesian Treatment of Linear Gaussian Regression Frank Wood December 3, 2009 Bayesian Approach to Classical Linear Regression In classical linear regression we have the following model y β, σ 2, X N(Xβ,
More informationIssues on quantile autoregression
Issues on quantile autoregression Jianqing Fan and Yingying Fan We congratulate Koenker and Xiao on their interesting and important contribution to the quantile autoregression (QAR). The paper provides
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationGeneralized Linear Models. Kurt Hornik
Generalized Linear Models Kurt Hornik Motivation Assuming normality, the linear model y = Xβ + e has y = β + ε, ε N(0, σ 2 ) such that y N(μ, σ 2 ), E(y ) = μ = β. Various generalizations, including general
More informationMinimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model.
Minimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model By Michael Levine Purdue University Technical Report #14-03 Department of
More informationGaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency
Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency Chris Paciorek March 11, 2005 Department of Biostatistics Harvard School of
More informationNonparametric Small Area Estimation via M-quantile Regression using Penalized Splines
Nonparametric Small Estimation via M-quantile Regression using Penalized Splines Monica Pratesi 10 August 2008 Abstract The demand of reliable statistics for small areas, when only reduced sizes of the
More informationHierarchical Modelling for Univariate Spatial Data
Hierarchical Modelling for Univariate Spatial Data Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department
More informationReduced-rank hazard regression
Chapter 2 Reduced-rank hazard regression Abstract The Cox proportional hazards model is the most common method to analyze survival data. However, the proportional hazards assumption might not hold. The
More informationShort Questions (Do two out of three) 15 points each
Econometrics Short Questions Do two out of three) 5 points each ) Let y = Xβ + u and Z be a set of instruments for X When we estimate β with OLS we project y onto the space spanned by X along a path orthogonal
More information