Semiparametric Methods for Mapping Brain Development
|
|
- Virginia McDowell
- 6 years ago
- Views:
Transcription
1 University of Haifa From the SelectedWorks of Philip T. Reiss May, 2012 Semiparametric Methods for Mapping Brain Development Philip T. Reiss Yin-Hsiu Chen Lan Huo Available at:
2 1 / 40 Semiparametric methods for mapping brain development Philip T. Reiss New York University and Nathan Kline Institute phil.reiss@nyumc.org Thomas R. Ten Have Symposium on Statistics in Psychiatry University of Pennsylvania May 23, 2012 Research supported in part by grant DMS , US National Science Foundation
3 2 / 40 Acknowledgments This is joint work with Yin-Hsiu Chen and Lan Huo (NYU) building on earlier joint work with Lei Huang (NYU, now at Johns Hopkins), Thad Tarpey (Wright State), and Maarten Mennes (NYU, now at Radboud University Nijmegen). Aaron Alexander-Bloch, Armin Raznahan and Jay Giedd (NIMH) supplied the data; and they, along with Eva Petkova and Xavier Castellanos (NYU), provided valuable feedback.
4 3 / 40 Cortical thickness The cerebral cortex, a convoluted sheet of neurons, is the grey matter area of the brain responsible for information processing. Cortical thickness (CT), which varies from about mm, has been linked to cognitive function. Maps of CT can be derived from magnetic resonance images, by measuring distance from the pial surface to the grey/white matter boundary at points (vertices).
5 4 / 40 An example cortical thickness map
6 5 / 40 NIMH cortical thickness data set The Brain Imaging Unit at the NIMH Child Psychiatry Branch, headed by Dr. Jay Giedd, has been conducting longitudinal MRI studies for about two decades. They have kindly provided us with 1181 CT maps from 615 typically developing controls representing 398 families: Number of scans Frequency Age at time of scan ranges from Each map consists of CT values at vertices.
7 Neurodevelopmental trajectories Overarching scientific question: Characterizing the course of development of CT throughout the brain in normal development ( implications for how we view the teen brain ) in neurological or psychiatric disorders Key finding: CT has an inverted U-shaped trajectory, attributed to pruning of neurons Maturation may be related to attainment of peak thickness ( delay theory of ADHD) CT Age Control ADHD 6 / 40
8 7 / 40 Standard way to map neurodevelopmental trajectories: Polynomial functions of age (e.g., Zuo et al., 2010)
9 More formally: In a cross-sectional study of n individuals, for i = 1,..., n, we have collected age x i ; CT values y i1,..., y iv ; other demographic / clinical covariates (but we ll omit these for simplicity). Standard approach for developmental neuroimaging data: for v = 1,..., V, use stepwise testing or AIC to choose among M 0v : E(y iv ) = β 0v M 1v : E(y iv ) = β 0v + β 1v x i M 2v : E(y iv ) = β 0v + β 1v x i + β 2v xi 2 M 3v : E(y iv ) = β 0v + β 1v x i + β 2v xi 2 + β 3v xi 3 Similarly, for a longitudinal study, we choose among M u 0v, Mu 1v, M u 2v, Mu 3v, where M u 1v : E(y iv u iv ) = β 0v + β 1v x i + u iv, etc. 8 / 40
10 9 / 40 Some advantages of polynomial models: familiar relatively straightforward computationally fast age of peak CT falls out of quadratic model: Some disadvantages: β 0 + β 1 x i + β 2 x 2 i peaks at x = β 1 2β 2. polynomials may not describe the trajectory well (e.g., plateau) estimated peak age is highly sensitive to sampled age range (Fjell et al., 2010) Spline smoothing can remove both of these limitations.
11 So... why don t we just model vertex-wise developmental trajectories by penalized B-splines instead of polynomials? Three issues: 1. Conceptual complexity 2. Computational expense 3. Not clear how to map the results 10 / 40
12 11 / 40 Goals for the rest of the talk Consider how we can address these 3 issues Present preliminary applications of semiparametric regression to inverted-u-shaped developmental trajectories of cortical thickness
13 12 / 40 Issue 1: Conceptual complexity To mitigate this problem, we adopt a minimalist approach to semiparametric regression (Ruppert, Wand and Carroll, 2003). A key tool is the mixed-model formulation of penalized splines, which offers a unifying, simplifying framework for modeling and inference.
14 13 / 40 Penalized B-splines Given data (x 1, y 1 ),..., (x n, y n) (univariate responses), we fit the model as follows. y i = g(x i ) + ε i, E(ε i ) = 0 1. Assume that g(x) = θ T b(x) where b( ) = [b 1 ( ),..., b K ( )] T, where the b j s are B-spline basis functions. 2. Estimate θ by penalized least squares: ˆθ = arg min θ R K y Bθ 2 }{{} sum of squared errors where y = (y 1,..., y n) T, b 1 (x 1 )... b K (x 1 ) B = λ θ } T {{ Pθ }, roughness functional b 1 (x n)... b K (x n) λ is a tuning parameter, and P is a K K matrix. ;
15 Roughness penalty λ θ T Pθ in criterion y Bθ 2 + λ θ T Pθ prevents overfitting by shrinking ĝ( ) = ˆθ T b( ) toward {θ T b( ) : θ T Pθ = 0} = {θ T b( ) : θ nullp}; smoothing parameter λ 0 controls the degree of shrinkage. Examples: P = ( b i b j ) 1 i,j K θ T Pθ = g (x) 2 dx shrink toward linear g x y λ small x y x y λ large P = ( b i b j ) 1 i,j K θ T Pθ = g (x) 2 dx shrink toward constant g x y x y x y 14 / 40
16 15 / 40 Connection with linear mixed models With a change of basis B (X Z ) where colx = nullp (e.g., Wand and Ormerod, 2008), criterion y Bθ 2 + λθ T Pθ becomes y Xβ Z u 2 + λu T u, which is log[f (y u)f (u)] for the linear mixed model y = Xβ + Z u + ε with u N[0, (σ 2 /λ)i], ε N(0, σ 2 I) (cf. Speed, 1991). Can reduce nonparametric inference to mixed model inference, with smoothing parameter λ recast as a variance parameter (e.g., Ruppert, Wand and Carroll, 2003; Wood, 2011).
17 More specifically, the correspondence between the penalized spline problem ( ) ˆθ = arg min y Bθ 2 + λθ T Pθ (1) θ R K and a linear mixed model y = Xβ + Z u + ε with u N[0, (σ 2 /λ)i], ε N(0, σ 2 I), (2) motivates basing inference on the modified profile likelihood, aka the restricted maximum likelihood (REML) criterion l R (λ y) = 1 2 (n p) log[y T {V 1 λ V 1 λ X(X T V 1 X) 1 X T V 1 }y] 1 2 log V λ 1 2 log X T V 1 λ X (with V λ I n + λ 1 Z Z T ) : λ λ 1. Testing H 0 : θ nullp (e.g., a linear fit vs. a nonparametric alternative) reduces to testing λ = (zero random effect variance); can use the Crainiceanu and Ruppert (2004) restricted likelihood ratio test statistic RLRT(y) = sup λ 0 2l R (λ y) 2l R ( y). 2. Do optimal smoothing by taking λ = arg max λ 0 l R (λ y) in (1). 16 / 40
18 17 / 40 Upshot regarding issue 1 (conceptual complexity) Linear mixed models are already in use by neuroimagers (for studies with longitudinal and/or family data). With penalized splines, estimation and testing for semiparametric regression can be reduced to corresponding questions for mixed models.
19 18 / 40 Issue 2: Computational expense For v = 1,..., V 80000, the penalized spline problem ( ) ˆθ v = arg min y v Bθ 2 + λθ T Pθ θ R K gives rise to REML criterion l R (λ y v ) and hence to 1. a problem of testing parametric null H 0v : θ nullp via RLRT(y v ) = sup λ 0 2l R (λ y v ) 2l R ( y v ). 2. an optimal smoothing problem of finding λ v = arg max λ 0 l R (λ y v ). Question: How to solve each of these problems efficiently V times??
20 19 / 40 Massively parallel RLRT: naïve approach To approximate RLRT(y v ) = sup 2l R (λ y v ) 2l R ( y v ) for v = 1,..., V, λ 0 1. choose a grid λ (1) <... < λ (G) ; 2. find the maximum of each column of the G V matrix l R (λ (1) y 1 ) l R (λ (1) y 2 )... l R (λ (1) y V ) l R (λ (G) y 1 ) l R (λ (G) y 2 )... l R (λ (G) y V )
21 20 / 40 Massively parallel RLRT: proposed fast approach Computational shortcut: That big matrix [ ] l R (λ (g) y v ) 1 g G,1 v V can be shown to equal y T 1 M λ (1)y 1... y T V M λ (1)y V..... y T 1 M λ (G)y 1... y T V M λ (G)y V = 1 T [Y (M λ (1)Y )]. 1 T [Y (M λ (G)Y )], where Y = (y 1... y V ) and M λ = V 1 λ X(X T V 1 X) 1 X T V 1 λ λ V 1 λ.
22 21 / 40 Massively parallel smoothing At vth point (vertex), the smooth function g v(x) = θ T v b(x) is estimated via ( ) ˆθ v = arg min y v Bθ 2 + λ vθ T Pθ, θ R K where B = [b j (x i )] 1 i n,1 j K. Two tasks must be performed for v = 1,..., V with huge V : 1. Find the optimal λ v, i.e., λ v = arg max λ 0 l R (λ y v ). Use same trick as for RLRT. 2. Given the optimal λ v, compute ˆθ v = (B T B + λ vp) 1 B T y v. Demmler-Reinsch algorithm lets us do this in one big matrix multiplication.
23 22 / 40 Approximate timing comparisons (in minutes) for a real-data example RLRT Smoothing Naïve Proposed
24 23 / 40 RLRT for null hypothesis of linear development
25 24 / 40 Issue 3: Mapping the estimates Massively parallel smoothing yields an estimate ĝ v ( ) of the mean developmental trajectory for vertex v (v = 1,..., V ). How to display a succinct summary of all these function estimates? Solution: Treat them as functional data, and apply functional data clustering methodology (e.g., Tarpey and Kinateder, 2003).
26 25 / 40 Functional data clustering algorithm 1. Interested in shape, rather than height, of trajectories cluster estimated first derivative of trajectories... ĝ 1 ( ) = ˆθ T 1 b( ). ĝ V ( ) = ˆθ T V b( ) ĝ 1 ( ) = ˆθ T 1 b ( ). ĝ V ( ) = ˆθ T V b ( ) 2. Truncated Karhunen-Loève expansion ĝ v( ) M m=1 c vmφ m ( ) reduce vth function to functional principal component scores c v1,..., c vm (e.g., Silverman, 1996; Ramsay and Silverman, 2005). 3. Apply k-means clustering to the c vm s.
27 26 / 40 2-cluster solution
28 27 / 40 3-cluster solution
29 28 / 40 4-cluster solution
30 29 / 40 5-cluster solution
31 30 / 40 Age of peak CT: some preliminary work The proposed fast methods work mainly for cross-sectional analyses. Whole-brain longitudinal analyses still require hours (even for polynomial regression). But... we can apply our methods to repeated cross-sectional subsamples from the NIMH data set to study properties of cross-sectional analyses (e.g., variability). Sample Families Individuals Scans Implementation Full Unrelated longitudinal gamm4 loop Cross-sectional Proposed Cross-sectional methods....
32 LH vertex 4124 Age Cortical thickness With random effects subtracted off Age Cortical thickness 31 / 40
33 32 / 40 Example: Sex differences in age of peak CT Repeated cross-sectional sex-specific curves for vertex Cortical thickness Female Male Age
34 33 / 40 Age of peak thickness: median over cross-sectional samples Male Female
35 34 / 40 Vertex CDF of age of peak CT Cortical thickness Female Male Cumulative distribution Female Male Age Age
36 35 / 40 Vertex 65953: 2nd derivative penalty Vertex 65953: 3rd derivative penalty Cortical thickness Female Male Cortical thickness Female Male Age Age
37 36 / 40 2nd derivative penalty 3rd derivative penalty Cumulative distribution Female Male Cumulative distribution Female Male Age Age
38 37 / 40 Discussion We have outlined three challenges for semiparametric mapping of neurodevelopmental trajectories, and progress toward addressing them. In the coming years, longitudinal imaging data will become much more common, but most data sets will still be cross-sectional. Much work is needed in many areas e.g., massively parallel varying-coefficient models for differently-shaped trajectories in different demographic or diagnostic groups.
39 38 / 40 Thank you! Ben Franklin s 13 Virtues 1. Temperance 2. Silence 3. Order 4. Resolution 5. Computational Frugality 6. Industry 7. Sincerity 8. Justice 9. Moderation 10. Cleanliness 11. Tranquility 12. Chastity 13. Humility
40 References I Crainiceanu, C. M., and Ruppert, D. (2004). Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society, Series B 66, Fjell, A. M., Walhovd, K. M., Westlye, L. T., Østby, Y. Tamnes, C. K., Jernigan T. L., Gamst, A., and Dale, A. M. (2010). When does brain aging accelerate? Dangers of quadratic fits in cross-sectional studies. NeuroImage 50, Ramsay, J. O., and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. New York: Springer. Reiss, P. T., Huang, L., Chen, Y.-H., Tarpey, T., and Mennes, M. (2011). Massively parallel nonparametrics. Submitted. Available at Ruppert, D., Wand, M. P., and Carroll, R. J. (2003). Semiparametric Regression. Cambridge and New York: Cambridge University Press. Silverman, B. W. (1996). Smoothed functional principal components analysis by choice of norm. Annals of Statistics 24, Speed, T. (1991). Discussion of That BLUP is a good thing: the estimation of random effects by G. K. Robinson. Statistical Science 6, Tarpey, T., and Kinateder, K. K. J. (2003). Clustering functional data. Journal of Classification 20, Wand, M. P., and Ormerod, J. T. (2008). On semiparametric regression with O Sullivan penalized splines. Australian & New Zealand Journal of Statistics 50, Wood, S. N. (2011). Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Statistical Society, Series B 73, / 40
41 References II Zuo, X.-N., Kelly, A. M. C., Di Martino, A., Mennes, M., Margulies, D., Bangaru, S., Grzadzinski, R., Evans, A., Zang, Y., Castellanos, F. X., and Milham, M. P. (2010). Growing together and growing apart: regional differences in the lifespan developmental trajectories of functional homotopy. Journal of Neuroscience 30, / 40
Function-on-Scalar Regression with the refund Package
University of Haifa From the SelectedWorks of Philip T. Reiss July 30, 2012 Function-on-Scalar Regression with the refund Package Philip T. Reiss, New York University Available at: https://works.bepress.com/phil_reiss/28/
More informationSUPPLEMENTARY APPENDICES FOR WAVELET-DOMAIN REGRESSION AND PREDICTIVE INFERENCE IN PSYCHIATRIC NEUROIMAGING
Submitted to the Annals of Applied Statistics SUPPLEMENTARY APPENDICES FOR WAVELET-DOMAIN REGRESSION AND PREDICTIVE INFERENCE IN PSYCHIATRIC NEUROIMAGING By Philip T. Reiss, Lan Huo, Yihong Zhao, Clare
More informationCross-sectional versus longitudinal designs for function estimation, with an application to cerebral cortex development
University of Haifa From the SelectedWorks of Philip T. Reiss 2018 Cross-sectional versus longitudinal designs for function estimation, with an application to cerebral cortex development Philip T. Reiss
More informationSimultaneous Confidence Bands for the Coefficient Function in Functional Regression
University of Haifa From the SelectedWorks of Philip T. Reiss August 7, 2008 Simultaneous Confidence Bands for the Coefficient Function in Functional Regression Philip T. Reiss, New York University Available
More informationLikelihood ratio testing for zero variance components in linear mixed models
Likelihood ratio testing for zero variance components in linear mixed models Sonja Greven 1,3, Ciprian Crainiceanu 2, Annette Peters 3 and Helmut Küchenhoff 1 1 Department of Statistics, LMU Munich University,
More informationRegularization in Cox Frailty Models
Regularization in Cox Frailty Models Andreas Groll 1, Trevor Hastie 2, Gerhard Tutz 3 1 Ludwig-Maximilians-Universität Munich, Department of Mathematics, Theresienstraße 39, 80333 Munich, Germany 2 University
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Institute of Statistics and Econometrics Georg-August-University Göttingen Department of Statistics
More informationIntegrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University
Integrated Likelihood Estimation in Semiparametric Regression Models Thomas A. Severini Department of Statistics Northwestern University Joint work with Heping He, University of York Introduction Let Y
More informationFlexible penalized regression for functional data...and other complex data objects
University of Haifa From the SelectedWorks of Philip T. Reiss November, 2015 Flexible penalized regression for functional data...and other complex data objects Philip T. Reiss Available at: https://works.bepress.com/phil_reiss/41/
More informationLikelihood Ratio Tests. that Certain Variance Components Are Zero. Ciprian M. Crainiceanu. Department of Statistical Science
1 Likelihood Ratio Tests that Certain Variance Components Are Zero Ciprian M. Crainiceanu Department of Statistical Science www.people.cornell.edu/pages/cmc59 Work done jointly with David Ruppert, School
More informationThe Hodrick-Prescott Filter
The Hodrick-Prescott Filter A Special Case of Penalized Spline Smoothing Alex Trindade Dept. of Mathematics & Statistics, Texas Tech University Joint work with Rob Paige, Missouri University of Science
More informationLecture 5: A step back
Lecture 5: A step back Last time Last time we talked about a practical application of the shrinkage idea, introducing James-Stein estimation and its extension We saw our first connection between shrinkage
More informationMaximum Smoothed Likelihood for Multivariate Nonparametric Mixtures
Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures David Hunter Pennsylvania State University, USA Joint work with: Tom Hettmansperger, Hoben Thomas, Didier Chauveau, Pierre Vandekerkhove,
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of
More informationRegression, Ridge Regression, Lasso
Regression, Ridge Regression, Lasso Fabio G. Cozman - fgcozman@usp.br October 2, 2018 A general definition Regression studies the relationship between a response variable Y and covariates X 1,..., X n.
More informationEfficient Estimation for the Partially Linear Models with Random Effects
A^VÇÚO 1 33 ò 1 5 Ï 2017 c 10 Chinese Journal of Applied Probability and Statistics Oct., 2017, Vol. 33, No. 5, pp. 529-537 doi: 10.3969/j.issn.1001-4268.2017.05.009 Efficient Estimation for the Partially
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationRestricted Likelihood Ratio Tests in Nonparametric Longitudinal Models
Restricted Likelihood Ratio Tests in Nonparametric Longitudinal Models Short title: Restricted LR Tests in Longitudinal Models Ciprian M. Crainiceanu David Ruppert May 5, 2004 Abstract We assume that repeated
More informationResearch Projects. Hanxiang Peng. March 4, Department of Mathematical Sciences Indiana University-Purdue University at Indianapolis
Hanxiang Department of Mathematical Sciences Indiana University-Purdue University at Indianapolis March 4, 2009 Outline Project I: Free Knot Spline Cox Model Project I: Free Knot Spline Cox Model Consider
More informationMinimum Hellinger Distance Estimation in a. Semiparametric Mixture Model
Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.
More informationTowards a Regression using Tensors
February 27, 2014 Outline Background 1 Background Linear Regression Tensorial Data Analysis 2 Definition Tensor Operation Tensor Decomposition 3 Model Attention Deficit Hyperactivity Disorder Data Analysis
More informationRejoinder. 1 Phase I and Phase II Profile Monitoring. Peihua Qiu 1, Changliang Zou 2 and Zhaojun Wang 2
Rejoinder Peihua Qiu 1, Changliang Zou 2 and Zhaojun Wang 2 1 School of Statistics, University of Minnesota 2 LPMC and Department of Statistics, Nankai University, China We thank the editor Professor David
More informationSome properties of Likelihood Ratio Tests in Linear Mixed Models
Some properties of Likelihood Ratio Tests in Linear Mixed Models Ciprian M. Crainiceanu David Ruppert Timothy J. Vogelsang September 19, 2003 Abstract We calculate the finite sample probability mass-at-zero
More informationVarying-smoother models for functional responses
Varying-smoother models for functional responses Philip T. Reiss 1,2,, Lei Huang 3, Huaihou Chen 4, and Stan Colcombe 2 arxiv:1412.0778v1 stat.me 2 Dec 2014 1 Department of Child and Adolescent Psychiatry
More informationAn Introduction to Functional Data Analysis
An Introduction to Functional Data Analysis Chongzhi Di Fred Hutchinson Cancer Research Center cdi@fredhutch.org Biotat 578A: Special Topics in (Genetic) Epidemiology November 10, 2015 Textbook Ramsay
More informationFUNCTIONAL DATA ANALYSIS. Contribution to the. International Handbook (Encyclopedia) of Statistical Sciences. July 28, Hans-Georg Müller 1
FUNCTIONAL DATA ANALYSIS Contribution to the International Handbook (Encyclopedia) of Statistical Sciences July 28, 2009 Hans-Georg Müller 1 Department of Statistics University of California, Davis One
More informationExact Likelihood Ratio Tests for Penalized Splines
Exact Likelihood Ratio Tests for Penalized Splines By CIPRIAN CRAINICEANU, DAVID RUPPERT, GERDA CLAESKENS, M.P. WAND Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe Street, Baltimore,
More informationRegression When the Predictors Are Images
University of Haifa From the SelectedWorks of Philip T. Reiss May, 2009 Regression When the Predictors Are Images Philip T. Reiss, New York University Available at: https://works.bepress.com/phil_reiss/15/
More informationModeling Multiple Correlated Functional Outcomes with Spatially. Heterogeneous Shape Characteristics
Biometrics 000, 1 25 DOI: 000 000 0000 Modeling Multiple Correlated Functional Outcomes with Spatially Heterogeneous Shape Characteristics Kunlaya Soiaporn 1,, David Ruppert 2,, and Raymond J. Carroll
More informationMultidimensional Density Smoothing with P-splines
Multidimensional Density Smoothing with P-splines Paul H.C. Eilers, Brian D. Marx 1 Department of Medical Statistics, Leiden University Medical Center, 300 RC, Leiden, The Netherlands (p.eilers@lumc.nl)
More informationHST 583 FUNCTIONAL MAGNETIC RESONANCE IMAGING DATA ANALYSIS AND ACQUISITION A REVIEW OF STATISTICS FOR FMRI DATA ANALYSIS
HST 583 FUNCTIONAL MAGNETIC RESONANCE IMAGING DATA ANALYSIS AND ACQUISITION A REVIEW OF STATISTICS FOR FMRI DATA ANALYSIS EMERY N. BROWN AND CHRIS LONG NEUROSCIENCE STATISTICS RESEARCH LABORATORY DEPARTMENT
More informationPenalized Splines, Mixed Models, and Recent Large-Sample Results
Penalized Splines, Mixed Models, and Recent Large-Sample Results David Ruppert Operations Research & Information Engineering, Cornell University Feb 4, 2011 Collaborators Matt Wand, University of Wollongong
More informationEfficient Bayesian Multivariate Surface Regression
Efficient Bayesian Multivariate Surface Regression Feng Li (joint with Mattias Villani) Department of Statistics, Stockholm University October, 211 Outline of the talk 1 Flexible regression models 2 The
More informationSelection of Smoothing Parameter for One-Step Sparse Estimates with L q Penalty
Journal of Data Science 9(2011), 549-564 Selection of Smoothing Parameter for One-Step Sparse Estimates with L q Penalty Masaru Kanba and Kanta Naito Shimane University Abstract: This paper discusses the
More informationPiotr Majer Risk Patterns and Correlated Brain Activities
Alena My²i ková Piotr Majer Song Song Alena Myšičková Peter N. C. Mohr Peter N. C. Mohr Wolfgang K. Härdle Song Song Hauke R. Heekeren Wolfgang K. Härdle Hauke R. Heekeren C.A.S.E. Centre C.A.S.E. for
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 6: Model complexity scores (v3) Ramesh Johari ramesh.johari@stanford.edu Fall 2015 1 / 34 Estimating prediction error 2 / 34 Estimating prediction error We saw how we can estimate
More informationLecture 2 Machine Learning Review
Lecture 2 Machine Learning Review CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago March 29, 2017 Things we will look at today Formal Setup for Supervised Learning Things
More informationNonparametric Small Area Estimation Using Penalized Spline Regression
Nonparametric Small Area Estimation Using Penalized Spline Regression 0verview Spline-based nonparametric regression Nonparametric small area estimation Prediction mean squared error Bootstrapping small
More informationOn the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models
On the Behavior of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models Thomas Kneib Department of Mathematics Carl von Ossietzky University Oldenburg Sonja Greven Department of
More informationRLRsim: Testing for Random Effects or Nonparametric Regression Functions in Additive Mixed Models
RLRsim: Testing for Random Effects or Nonparametric Regression Functions in Additive Mixed Models Fabian Scheipl 1 joint work with Sonja Greven 1,2 and Helmut Küchenhoff 1 1 Department of Statistics, LMU
More informationDESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA
Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University SMAC, November 6, 2015 E. Christou, M. G. Akritas (PSU) SIQR
More informationPaper Review: Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties by Jianqing Fan and Runze Li (2001)
Paper Review: Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties by Jianqing Fan and Runze Li (2001) Presented by Yang Zhao March 5, 2010 1 / 36 Outlines 2 / 36 Motivation
More informationRegularization and Variable Selection via the Elastic Net
p. 1/1 Regularization and Variable Selection via the Elastic Net Hui Zou and Trevor Hastie Journal of Royal Statistical Society, B, 2005 Presenter: Minhua Chen, Nov. 07, 2008 p. 2/1 Agenda Introduction
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationRegression I: Mean Squared Error and Measuring Quality of Fit
Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving
More informationWeb Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.
Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More informationMissing Data Issues in the Studies of Neurodegenerative Disorders: the Methodology
Missing Data Issues in the Studies of Neurodegenerative Disorders: the Methodology Sheng Luo, PhD Associate Professor Department of Biostatistics & Bioinformatics Duke University Medical Center sheng.luo@duke.edu
More informationSome Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model
Some Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model 1. Introduction Varying-coefficient partially linear model (Zhang, Lee, and Song, 2002; Xia, Zhang, and Tong, 2004;
More informationThe General Linear Model (GLM)
he General Linear Model (GLM) Klaas Enno Stephan ranslational Neuromodeling Unit (NU) Institute for Biomedical Engineering University of Zurich & EH Zurich Wellcome rust Centre for Neuroimaging Institute
More informationTwo Applications of Nonparametric Regression in Survey Estimation
Two Applications of Nonparametric Regression in Survey Estimation 1/56 Jean Opsomer Iowa State University Joint work with Jay Breidt, Colorado State University Gerda Claeskens, Université Catholique de
More informationA NOTE ON A NONPARAMETRIC REGRESSION TEST THROUGH PENALIZED SPLINES
Statistica Sinica 24 (2014), 1143-1160 doi:http://dx.doi.org/10.5705/ss.2012.230 A NOTE ON A NONPARAMETRIC REGRESSION TEST THROUGH PENALIZED SPLINES Huaihou Chen 1, Yuanjia Wang 2, Runze Li 3 and Katherine
More informationGeneral Functional Concurrent Model
General Functional Concurrent Model Janet S. Kim Arnab Maity Ana-Maria Staicu June 17, 2016 Abstract We propose a flexible regression model to study the association between a functional response and multiple
More informationSparse Linear Models (10/7/13)
STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationSmooth Common Principal Component Analysis
1 Smooth Common Principal Component Analysis Michal Benko Wolfgang Härdle Center for Applied Statistics and Economics benko@wiwi.hu-berlin.de Humboldt-Universität zu Berlin Motivation 1-1 Volatility Surface
More informationGWAS IV: Bayesian linear (variance component) models
GWAS IV: Bayesian linear (variance component) models Dr. Oliver Stegle Christoh Lippert Prof. Dr. Karsten Borgwardt Max-Planck-Institutes Tübingen, Germany Tübingen Summer 2011 Oliver Stegle GWAS IV: Bayesian
More informationEstimation of cumulative distribution function with spline functions
INTERNATIONAL JOURNAL OF ECONOMICS AND STATISTICS Volume 5, 017 Estimation of cumulative distribution function with functions Akhlitdin Nizamitdinov, Aladdin Shamilov Abstract The estimation of the cumulative
More informationDivide-and-combine Strategies in Statistical Modeling for Massive Data
Divide-and-combine Strategies in Statistical Modeling for Massive Data Liqun Yu Washington University in St. Louis March 30, 2017 Liqun Yu (WUSTL) D&C Statistical Modeling for Massive Data March 30, 2017
More informationOutline Introduction OLS Design of experiments Regression. Metamodeling. ME598/494 Lecture. Max Yi Ren
1 / 34 Metamodeling ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 1, 2015 2 / 34 1. preliminaries 1.1 motivation 1.2 ordinary least square 1.3 information
More informationNonparametric Estimation of Luminosity Functions
x x Nonparametric Estimation of Luminosity Functions Chad Schafer Department of Statistics, Carnegie Mellon University cschafer@stat.cmu.edu 1 Luminosity Functions The luminosity function gives the number
More informationModel Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model
Model Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School of Population
More informationHigh-dimensional regression
High-dimensional regression Advanced Methods for Data Analysis 36-402/36-608) Spring 2014 1 Back to linear regression 1.1 Shortcomings Suppose that we are given outcome measurements y 1,... y n R, and
More informationGeneralized Functional Concurrent Model
Biometrics, 1 25 2015 Generalized Functional Concurrent Model Janet S. Kim, Arnab Maity, and Ana-Maria Staicu Department of Statistics, North Carolina State University, Raleigh, North Carolina 27695, U.S.A.
More informationECE521 lecture 4: 19 January Optimization, MLE, regularization
ECE521 lecture 4: 19 January 2017 Optimization, MLE, regularization First four lectures Lectures 1 and 2: Intro to ML Probability review Types of loss functions and algorithms Lecture 3: KNN Convexity
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationLinear regression methods
Linear regression methods Most of our intuition about statistical methods stem from linear regression. For observations i = 1,..., n, the model is Y i = p X ij β j + ε i, j=1 where Y i is the response
More informationLinear Model Selection and Regularization
Linear Model Selection and Regularization Recall the linear model Y = β 0 + β 1 X 1 + + β p X p + ɛ. In the lectures that follow, we consider some approaches for extending the linear model framework. In
More informationLecture 3: Statistical Decision Theory (Part II)
Lecture 3: Statistical Decision Theory (Part II) Hao Helen Zhang Hao Helen Zhang Lecture 3: Statistical Decision Theory (Part II) 1 / 27 Outline of This Note Part I: Statistics Decision Theory (Classical
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationRegularization Methods for Additive Models
Regularization Methods for Additive Models Marta Avalos, Yves Grandvalet, and Christophe Ambroise HEUDIASYC Laboratory UMR CNRS 6599 Compiègne University of Technology BP 20529 / 60205 Compiègne, France
More information4. Distributions of Functions of Random Variables
4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n
More informationSVAN 2016 Mini Course: Stochastic Convex Optimization Methods in Machine Learning
SVAN 2016 Mini Course: Stochastic Convex Optimization Methods in Machine Learning Mark Schmidt University of British Columbia, May 2016 www.cs.ubc.ca/~schmidtm/svan16 Some images from this lecture are
More informationOptimization Problems
Optimization Problems The goal in an optimization problem is to find the point at which the minimum (or maximum) of a real, scalar function f occurs and, usually, to find the value of the function at that
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University JSM, 2015 E. Christou, M. G. Akritas (PSU) SIQR JSM, 2015
More informationModel Comparison. Course on Bayesian Inference, WTCN, UCL, February Model Comparison. Bayes rule for models. Linear Models. AIC and BIC.
Course on Bayesian Inference, WTCN, UCL, February 2013 A prior distribution over model space p(m) (or hypothesis space ) can be updated to a posterior distribution after observing data y. This is implemented
More informationEnsemble estimation and variable selection with semiparametric regression models
Ensemble estimation and variable selection with semiparametric regression models Sunyoung Shin Department of Mathematical Sciences University of Texas at Dallas Joint work with Jason Fine, Yufeng Liu,
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 9: Basis Expansions Department of Statistics & Biostatistics Rutgers University Nov 01, 2011 Regression and Classification Linear Regression. E(Y X) = f(x) We want to learn
More informationEstimation for nonparametric mixture models
Estimation for nonparametric mixture models David Hunter Penn State University Research supported by NSF Grant SES 0518772 Joint work with Didier Chauveau (University of Orléans, France), Tatiana Benaglia
More informationGLAM An Introduction to Array Methods in Statistics
GLAM An Introduction to Array Methods in Statistics Iain Currie Heriot Watt University GLAM A Generalized Linear Array Model is a low-storage, high-speed, method for multidimensional smoothing, when data
More informationAnalysis Methods for Supersaturated Design: Some Comparisons
Journal of Data Science 1(2003), 249-260 Analysis Methods for Supersaturated Design: Some Comparisons Runze Li 1 and Dennis K. J. Lin 2 The Pennsylvania State University Abstract: Supersaturated designs
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationCOS 424: Interacting with Data
COS 424: Interacting with Data Lecturer: Rob Schapire Lecture #14 Scribe: Zia Khan April 3, 2007 Recall from previous lecture that in regression we are trying to predict a real value given our data. Specically,
More informationMorphometrics with SPM12
Morphometrics with SPM12 John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK. What kind of differences are we looking for? Usually, we try to localise regions of difference.
More informationKneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach"
Kneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach" Sonderforschungsbereich 386, Paper 43 (25) Online unter: http://epub.ub.uni-muenchen.de/
More informationISyE 691 Data mining and analytics
ISyE 691 Data mining and analytics Regression Instructor: Prof. Kaibo Liu Department of Industrial and Systems Engineering UW-Madison Email: kliu8@wisc.edu Office: Room 3017 (Mechanical Engineering Building)
More informationTolerance Bands for Functional Data
Tolerance Bands for Functional Data Lasitha N. Rathnayake and Pankaj K. Choudhary 1 Department of Mathematical Sciences, FO 35 University of Texas at Dallas Richardson, TX 75080-3021, USA Abstract Often
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationGaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency
Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency Chris Paciorek March 11, 2005 Department of Biostatistics Harvard School of
More informationDirect Learning: Linear Regression. Donglin Zeng, Department of Biostatistics, University of North Carolina
Direct Learning: Linear Regression Parametric learning We consider the core function in the prediction rule to be a parametric function. The most commonly used function is a linear function: squared loss:
More information( t) Cox regression part 2. Outline: Recapitulation. Estimation of cumulative hazards and survival probabilites. Ørnulf Borgan
Outline: Cox regression part 2 Ørnulf Borgan Department of Mathematics University of Oslo Recapitulation Estimation of cumulative hazards and survival probabilites Assumptions for Cox regression and check
More informationLecture 1: Bayesian Framework Basics
Lecture 1: Bayesian Framework Basics Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de April 21, 2014 What is this course about? Building Bayesian machine learning models Performing the inference of
More informationRecap. HW due Thursday by 5 pm Next HW coming on Thursday Logistic regression: Pr(G = k X) linear on the logit scale Linear discriminant analysis:
1 / 23 Recap HW due Thursday by 5 pm Next HW coming on Thursday Logistic regression: Pr(G = k X) linear on the logit scale Linear discriminant analysis: Pr(G = k X) Pr(X G = k)pr(g = k) Theory: LDA more
More informationStatistical inference on Lévy processes
Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop 2013 26th March Outline
More informationPart 2: Multivariate fmri analysis using a sparsifying spatio-temporal prior
Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 2: Multivariate fmri analysis using a sparsifying spatio-temporal prior Tom Heskes joint work with Marcel van Gerven
More informationBAYESIAN DECISION THEORY
Last updated: September 17, 2012 BAYESIAN DECISION THEORY Problems 2 The following problems from the textbook are relevant: 2.1 2.9, 2.11, 2.17 For this week, please at least solve Problem 2.3. We will
More informationPart 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
More informationAdaptive Piecewise Polynomial Estimation via Trend Filtering
Adaptive Piecewise Polynomial Estimation via Trend Filtering Liubo Li, ShanShan Tu The Ohio State University li.2201@osu.edu, tu.162@osu.edu October 1, 2015 Liubo Li, ShanShan Tu (OSU) Trend Filtering
More information