Recap. Vector observation: Y f (y; θ), Y Y R m, θ R d. sample of independent vectors y 1,..., y n. pairwise log-likelihood n m. weights are often 1
|
|
- Benjamin Goodman
- 5 years ago
- Views:
Transcription
1 Recap Vector observation: Y f (y; θ), Y Y R m, θ R d sample of independent vectors y 1,..., y n pairwise log-likelihood n m i=1 r=1 s>r w rs log f 2 (y ir, y is ; θ) weights are often 1 more generally, cl(θ; y) = n w k l k (θ; y i ) i=1 k K l k (θ; y i ) = log f (y i A k ; θ) STA 4508: Topics in Likelihood Inference February 2, /23
2 Inference Sample: Y 1,..., Y n, i.i.d., cl(θ; y) = n i=1 cl(θ; y i) ˆθ CL θ. N{0, G 1 (θ)} G n (θ) = H(θ)J(θ) 1 H(θ) w(θ) = 2{cl(ˆθ CL ) cl(θ)}. d a=1 µ az 2 a Z a N(0, 1) µ 1,..., µ d eigenvalues of J(θ)H(θ) 1 = H(θ)G(θ) 1 Nuisance parameters θ = (ψ, λ) n( ˆψ CL ψ). N{0, G ψψ (θ)} w(ψ) = 2{cl(ˆθ CL ) cl( θ ψ )}. d 0 a=1 µ az 2 a µ 1,..., µ d0 eigenvalues of (H ψψ ) 1 G ψψ STA 4508: Topics in Likelihood Inference February 2, /23
3 Example: dichotomized MV Normal Y ir = 1{Z ir > 0} Z N(0, R) r = 1,..., m; i = 1, l 2 (ρ) = n {y ir y is log P(y r = 1, y s = 1) + y ir (1 y is ) log P 10 i=1 s<r + (1 y ir )y is log P 01 + (1 y ir )(1 y is ) log P 00 } a.var(ˆρ CL ) = 1 n 4π 2 m 2 (1 ρ 2 ) (m 1) 2 var(t ) T = i (2y ir y is y ir y is ) s<r var(t ) = nm 4 (p p p 11 p )+ m 3 ( 6p ) + m 2 (...) + m(...) STA 4508: Topics in Likelihood Inference February 2, /23
4 a. variance pairwise full rho ρ ARE ρ ARE Numbers changed from last week. Also incorrect in Cox & Reid 2004 Table 1
5 Example: longitudinal count data subjects i = 1,..., n observations counts y ir, r = 1,... m i model y ir Poisson(u ir x T ir β) u i1,..., u imi gamma-distributed random effects but correlated corr(u ir, u is ) = ρ r s joint density has combinatorial number of terms in m i ; impractical weighted pairwise composite likelihood cl pair (β) = n i=1 1 m i 1 m i m i r=1 s=r+1 f (y ir, y is ; β) weights chosen so that L pair = full likelihood if ρ = 0 Henderson & Shimura, 2003 STA 4508: Topics in Likelihood Inference February 2, /23
6 Example: Varin & Czado 2010 pain severity scores recorded at four time points morning, noon, evening, bed 119 patients; varying number of days per patient covariates: personal and weather response: pain score y ij response at time t ij for observation j on subject i yij a latent variable, continuous y ij = k a k 1 < yij < a k y ij = x T ij β + u i + ɛ ij if u i N(0, σ 2 ) and ɛ ij N(0, 1) f (y i1,..., y imi ) = ni j=1 {Φ(a y ij xij Tβ u i) Φ(a yij 1 xij Tβ u i)}φ( u i σ )du i L(θ; y) = n i=1 STA 4508: Topics in Likelihood Inference February 2, /23
7 ... pain severity scores yij and yij have constant correlation σ 2 /(σ 2 + 1) points nearer in time might be expected to have higher correlation change ɛ ij i.i.d. N(0, 1) to corr(ɛ ij, ɛ ij ) = exp(δ t ij t ij ) now n ãyi1 ãyimi L(θ; y) =... φ ni (z i1,..., z imi ; R i )dz i1... dz imi ã yimi 1 i=1 ã yi1 1 R ijj = σ2 tij σ e δ tij σ pairwise log-likelihood: n m i cl(θ; y) = log f 2 (y ij, y ij ; θ)1 [ q,q] (t ij t ij ) j<j i=1 weights are 1 or 0, depending on distance between time points STA 4508: Topics in Likelihood Inference February 2, /23
8 ... pain severity scores Table 4 Varin & Czado Supplementary material online STA 4508: Topics in Likelihood Inference February 2, /23
9 Applications: spatial and space-time data conditional approaches seem more natural condition on neighbours in space condition on small number of lags (in time) some form of blockwise components often proposed Stein et al, 04; Caragea and Smith, 07 fmri time series Kang et al, 12 air pollution and health effects Bai et al, 12 computer experiments: Gaussian process models Xi, 12 spatially correlated extremes joint tail probability known joint density requires combinatorial effort (partial derivatives) composite likelihood based on joint distribution of pairs, triples seems to work well Davison et al, (2012); Genton et al., 12, Ribatet, 12 STA 4508: Topics in Likelihood Inference February 2, /23
10 Spatial extremes vector observations (X 1i,..., X mi ), i = 1,..., n example rainfall at each of m locations component-wise maxima Z 1,..., Z m ; Z j = max(x j1,..., X jn ) Z j are transformed (centered and scaled) general theory says Pr(Z 1 z 1,..., Z m z m ) = exp{ V (z 1,..., z m )} function V ( ) can be parameterized via Gaussian process models example V (z 1, z 2 ) = z 1 1 Φ{(1/2)a(h) + a 1 (h) log(z 2 /z 1 )} + z 1 2 Φ{(1/2)a(h) + a 1 (h) log(z 1 /z 2 )} Z (h) = (z 1, z 2 ), Z (0) = (0, 0), a(h) = h T Ω 1 h STA 4508: Topics in Likelihood Inference February 2, /23
11 ... spatial extremes Pr(Z 1 z 1,..., Z d z m ) = exp{ V (z 1,..., z m )} to compute log-likelihood function, need the density combinatorial explosion in computing joint derivatives of V ( ) Davison et al. (2012, Statistical Science) used pairwise composite likelihood compared the fits of several competing models, using AIC analogue described above applied to annual maximum rainfall at several stations near Zurich STA 4508: Topics in Likelihood Inference February 2, /23
12 Davison et al, 2012 STA 4508: Topics in Likelihood Inference February 2, /23
13 ... Davison et al, 2012 STA 4508: Topics in Likelihood Inference February 2, /23
14 ... applications time series a case of large m, fixed n need new arguments re consistency, asymptotic normality consecutive pairs: consistent, not always asy. normal AR(1): consecutive pairs fully efficient; all pairs terrible (consistent, highly variable) MA(1): consecutive pairs consistent but very inefficient Davis and Yau (2011) genetics: estimation of recombination rate somewhat similar to time series but correlation may not decrease with increasing length suggesting all possible pairs may be inconsistent joint blocks of short sequences seems preferable linkage disequilibrium family based sampling Larribe and Fearnhead (2011); Choi and Briollais, 12 STA 4508: Topics in Likelihood Inference February 2, /23
15 Example: Ising model Ising model: f (y; θ) = exp( (j,k) E 1 θ jk y j y k ) Z (θ) j, k = 1,..., K neighbourhood contributions f (y j y ( j) ; θ) = exp(2y j k j θ jky k ) exp(2y j k j θ jky k ) + 1 = exp l j(θ; y) penalized CL estimation based on sample y (1),..., y (n) n K max l θ j (θ; y (i) ) P λ ( θ jk ) j<k i=1 j=1 Xue et al., 2012 Ravikumar et al., 2010 STA 4508: Topics in Likelihood Inference February 2, /23
16 Misspecified models: general theory y 1,..., y n independent observations with distribution function G( ) and density g( ) we fit the incorrect model f (y; θ) Kullback-Liebler (KL) divergence between f (y; θ) and g(y) is defined as { } g(y) KL(θ) = log g(y)dy f (y; θ) Wikipedia writes D KL (G F) or more precisely D KL (P G P F ) KL(θ) 0, and KL(θ) = 0 f (y; θ) g(y) define θ = arg min KL(θ) f (y; θ ) is closest to G( ) in the family {f ( ; θ), θ Θ} STA 4508: Topics in Likelihood Inference February 2, /23
17 ... misspecified models KL(θ) = { } g(y) log g(y)dy f (y; θ) θ = arg min θ KL(θ) θ = arg max θ log{f (y; θ)}g(y)dy = arg maxθ E G {l(θ; y)} leads to a proof that the maximum likelihood estimator converges to θ if g(y) = f (y; θ 0 ), then θ = θ 0 otherwise θ is the least false parameter value STA 4508: Topics in Likelihood Inference February 2, /23
18 ... misspecified models Example true model G is log-normal log y N(µ, σ 2 ) g(y) =? Model f (y; θ) = 1 θ exp( y θ ) E G {l(θ; y)} = log θ + E G ( y θ ) θ = E G (y) = exp(µ + σ 2 /2) reminder: MLE solves θ l(θ; y) = 0 y = (y 1,..., y n) in the example, ˆθ = ȳ p θ STA 4508: Topics in Likelihood Inference February 2, /23
19 ... misspecified models θ l(ˆθ; y) = 0 inference? E G θ l(θ; y) = θ l(θ; y)g(y)dy E G θθ l(θ; y) = θθ l(θ; y)g(y)dy H(θ) E G θ l(θ; y) 2 = θ l(θ; y) 2 g(y)dy J(θ) (ˆθ θ ) = H(θ ) 1 U(θ ){1 + o p (1)} = 0, at θ U(θ) = θ l(θ) ˆθ. N{θ, G 1 (θ )} G(θ) = H(θ)J 1 (θ)h(θ) STA 4508: Topics in Likelihood Inference February 2, /23
20 Examples y 1,..., y n i.i.d. G; we assume f (y; θ) is N(µ, σ 2 ) ˆµ = ȳ, ˆσ 2 = 1 n (yi ȳ) 2 θ l(θ; y) = [ Σ(y i µ)/σ 2, (n/2σ 2 ) + Σ(y i µ) 2 /(2σ 4 ) ] [ ] 1/(σ 2 H(θ) = n 0 ) 0 0 1/(2σ0 4 [ ) 1/(σ 2 J(θ) = n 0 ) µ 3 /(2σ0 6) µ 3 /(2σ0 6 (µ 4 σ0 4)/(4σ8 0 ) σ 2 0 = var G(y 1 ), µ 3 = E G (y 1 µ 0 ) 3, etc. ] H(θ) = E G l (θ; y); J(θ) = Var G l (θ; y) STA 4508: Topics in Likelihood Inference February 2, /23
21 ... examples y 1,..., y n i.i.d. G; we assume f (y; θ) is N(µ, σ 2 ) a.var(ˆθ) = [ σ 2 0 /n γ 1 σ0 3/n γ 1 σ0 3/n γ 2σ0 4/n ] γ 3 = E(y 1 µ 0 ) 3 /σ 3 0, γ 4 = E(y 1 µ 0 ) 4 /(σ 4 0 ) 1 note that ˆµ, ˆσ 2 not uncorrelated unless γ 3 = 0 ˆθ = (ˆµ, ˆσ 2 ) = {ȳ, Σ(y i ȳ) 2 /n} STA 4508: Topics in Likelihood Inference February 2, /23
22 ... examples linear regression model G: y = Xβ 0 + ɛ, ɛ (0, σ 2 R) linear regression; correlated errors working model f (y; β) = N(Xβ, σ 2 I) uncorrelated errors ˆβ = (X T X) 1 X T y least squares estimator; mle under normality E G ( ˆβ) = β 0 LSE unbiased (consistent) var G ( ˆβ) = σ 2 (X T X) 1 X T RX(X T X) 1 sandwich variance working model variance is incorrect σ 2 (X T X) 1 STA 4508: Topics in Likelihood Inference February 2, /23
23 ... examples linear regression model G: y = Xβ 0 + ɛ, ɛ (0, σ 2 R) linear regression; correlated errors var G ( ˆβ) = σ 2 (X T X) 1 X T RX(X T X) 1 sandwich variance working model variance is incorrect σ 2 (X T X) 1 single intercept model, constant correlation ρ: ˆβ = ȳ true standard error of ȳ is σ{1 + ρ(n 1)} 1/2 / n compare to σ/ n ρ = 0.1, n = 100, ratio is 3.3 variance inflation STA 4508: Topics in Likelihood Inference February 2, /23
Composite Likelihood
Composite Likelihood Nancy Reid January 30, 2012 with Cristiano Varin and thanks to Don Fraser, Grace Yi, Ximing Xu Background parametric model f (y; θ), y R m ; θ R d likelihood function L(θ; y) f (y;
More informationVarious types of likelihood
Various types of likelihood 1. likelihood, marginal likelihood, conditional likelihood, profile likelihood, adjusted profile likelihood 2. semi-parametric likelihood, partial likelihood 3. empirical likelihood,
More informationComposite likelihood methods
1 / 20 Composite likelihood methods Nancy Reid University of Warwick, April 15, 2008 Cristiano Varin Grace Yun Yi, Zi Jin, Jean-François Plante 2 / 20 Some questions (and answers) Is there a drinks table
More informationComposite Likelihood Estimation
Composite Likelihood Estimation With application to spatial clustered data Cristiano Varin Wirtschaftsuniversität Wien April 29, 2016 Credits CV, Nancy M Reid and David Firth (2011). An overview of composite
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Light touch assessment One or two problems assigned weekly graded during Reading Week http://www.utstat.toronto.edu/reid/4508s14.html
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Problems assigned weekly, due the following week http://www.utstat.toronto.edu/reid/4508s16.html Various types of likelihood 1. likelihood,
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationApproximate Likelihoods
Approximate Likelihoods Nancy Reid July 28, 2015 Why likelihood? makes probability modelling central l(θ; y) = log f (y; θ) emphasizes the inverse problem of reasoning y θ converts a prior probability
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationLikelihood inference in the presence of nuisance parameters
Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationLikelihood and p-value functions in the composite likelihood context
Likelihood and p-value functions in the composite likelihood context D.A.S. Fraser and N. Reid Department of Statistical Sciences University of Toronto November 19, 2016 Abstract The need for combining
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationVarious types of likelihood
Various types of likelihood 1. likelihood, marginal likelihood, conditional likelihood, profile likelihood, adjusted profile likelihood 2. semi-parametric likelihood, partial likelihood 3. empirical likelihood,
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
More informationAN OVERVIEW OF COMPOSITE LIKELIHOOD METHODS
Statistica Sinica 21 (2011), 5-42 AN OVERVIEW OF COMPOSITE LIKELIHOOD METHODS Cristiano Varin, Nancy Reid and David Firth Università Ca Foscari Venezia, University of Toronto and University of Warwick
More informationF & B Approaches to a simple model
A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 215 http://www.astro.cornell.edu/~cordes/a6523 Lecture 11 Applications: Model comparison Challenges in large-scale surveys
More informationBIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation
BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationAssociation studies and regression
Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration
More informationGeneral Linear Model: Statistical Inference
Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter 4), least
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationMathematics Ph.D. Qualifying Examination Stat Probability, January 2018
Mathematics Ph.D. Qualifying Examination Stat 52800 Probability, January 2018 NOTE: Answers all questions completely. Justify every step. Time allowed: 3 hours. 1. Let X 1,..., X n be a random sample from
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Merlise Clyde STA721 Linear Models Duke University August 31, 2017 Outline Topics Likelihood Function Projections Maximum Likelihood Estimates Readings: Christensen Chapter
More informationLikelihood inference for complex data
1 / 52 Likelihood inference for complex data Nancy Reid Kuwait Foundation Lecture DPMMS, University of Cambridge May 5, 2009 2 / 52 Models, data and likelihood Likelihood inference Theory Examples Composite
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationEcon 583 Final Exam Fall 2008
Econ 583 Final Exam Fall 2008 Eric Zivot December 11, 2008 Exam is due at 9:00 am in my office on Friday, December 12. 1 Maximum Likelihood Estimation and Asymptotic Theory Let X 1,...,X n be iid random
More informationLikelihood Inference in the Presence of Nuisance Parameters
PHYSTAT2003, SLAC, September 8-11, 2003 1 Likelihood Inference in the Presence of Nuance Parameters N. Reid, D.A.S. Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe
More informationA Model for Correlated Paired Comparison Data
Working Paper Series, N. 15, December 2010 A Model for Correlated Paired Comparison Data Manuela Cattelan Department of Statistical Sciences University of Padua Italy Cristiano Varin Department of Statistics
More informationMultiple Comparisons using Composite Likelihood in Clustered Data
Multiple Comparisons using Composite Likelihood in Clustered Data Mahdis Azadbakhsh, Xin Gao and Hanna Jankowski York University May 29, 15 Abstract We study the problem of multiple hypothesis testing
More informationThe Slow Convergence of OLS Estimators of α, β and Portfolio. β and Portfolio Weights under Long Memory Stochastic Volatility
The Slow Convergence of OLS Estimators of α, β and Portfolio Weights under Long Memory Stochastic Volatility New York University Stern School of Business June 21, 2018 Introduction Bivariate long memory
More informationBIOS 2083 Linear Models c Abdus S. Wahed
Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Guy Lebanon February 19, 2011 Maximum likelihood estimation is the most popular general purpose method for obtaining estimating a distribution from a finite sample. It was
More information2018 2019 1 9 sei@mistiu-tokyoacjp http://wwwstattu-tokyoacjp/~sei/lec-jhtml 11 552 3 0 1 2 3 4 5 6 7 13 14 33 4 1 4 4 2 1 1 2 2 1 1 12 13 R?boxplot boxplotstats which does the computation?boxplotstats
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationLikelihood Inference in the Presence of Nuisance Parameters
Likelihood Inference in the Presence of Nuance Parameters N Reid, DAS Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe some recent approaches to likelihood based
More informationECON 4160, Autumn term Lecture 1
ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationFractional Imputation in Survey Sampling: A Comparative Review
Fractional Imputation in Survey Sampling: A Comparative Review Shu Yang Jae-Kwang Kim Iowa State University Joint Statistical Meetings, August 2015 Outline Introduction Fractional imputation Features Numerical
More informationtesting MAHDIS AZADBAKHSH A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN
Composite likelihood: Multiple comparisons and non-standard conditions in hypothesis testing MAHDIS AZADBAKHSH A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE
More informationEstimation and Model Selection in Mixed Effects Models Part I. Adeline Samson 1
Estimation and Model Selection in Mixed Effects Models Part I Adeline Samson 1 1 University Paris Descartes Summer school 2009 - Lipari, Italy These slides are based on Marc Lavielle s slides Outline 1
More informationAn Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data
An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data Jae-Kwang Kim 1 Iowa State University June 28, 2012 1 Joint work with Dr. Ming Zhou (when he was a PhD student at ISU)
More informationGeneralized Linear Models Introduction
Generalized Linear Models Introduction Statistics 135 Autumn 2005 Copyright c 2005 by Mark E. Irwin Generalized Linear Models For many problems, standard linear regression approaches don t work. Sometimes,
More informationLikelihood based Statistical Inference. Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona
Likelihood based Statistical Inference Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona L. Pace, A. Salvan, N. Sartori Udine, April 2008 Likelihood: observed quantities,
More informationAn Extended BIC for Model Selection
An Extended BIC for Model Selection at the JSM meeting 2007 - Salt Lake City Surajit Ray Boston University (Dept of Mathematics and Statistics) Joint work with James Berger, Duke University; Susie Bayarri,
More informationLinear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2016 Julien Berestycki (University of Oxford) SB2a MT 2016 1 / 32 Lecture 14 : Variational Bayes
More informationAspects of Likelihood Inference
Submitted to the Bernoulli Aspects of Likelihood Inference NANCY REID 1 1 Department of Statistics University of Toronto 100 St. George St. Toronto, Canada M5S 3G3 E-mail: reid@utstat.utoronto.ca, url:
More informationCentral Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
More informationGraduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationMathematics Qualifying Examination January 2015 STAT Mathematical Statistics
Mathematics Qualifying Examination January 2015 STAT 52800 - Mathematical Statistics NOTE: Answer all questions completely and justify your derivations and steps. A calculator and statistical tables (normal,
More information,..., θ(2),..., θ(n)
Likelihoods for Multivariate Binary Data Log-Linear Model We have 2 n 1 distinct probabilities, but we wish to consider formulations that allow more parsimonious descriptions as a function of covariates.
More informationStatistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
More informationi=1 h n (ˆθ n ) = 0. (2)
Stat 8112 Lecture Notes Unbiased Estimating Equations Charles J. Geyer April 29, 2012 1 Introduction In this handout we generalize the notion of maximum likelihood estimation to solution of unbiased estimating
More informationLecture 14 More on structural estimation
Lecture 14 More on structural estimation Economics 8379 George Washington University Instructor: Prof. Ben Williams traditional MLE and GMM MLE requires a full specification of a model for the distribution
More informationModeling Real Estate Data using Quantile Regression
Modeling Real Estate Data using Semiparametric Quantile Regression Department of Statistics University of Innsbruck September 9th, 2011 Overview 1 Application: 2 3 4 Hedonic regression data for house prices
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More information1 One-way analysis of variance
LIST OF FORMULAS (Version from 21. November 2014) STK2120 1 One-way analysis of variance Assume X ij = µ+α i +ɛ ij ; j = 1, 2,..., J i ; i = 1, 2,..., I ; where ɛ ij -s are independent and N(0, σ 2 ) distributed.
More informationFlat and multimodal likelihoods and model lack of fit in curved exponential families
Mathematical Statistics Stockholm University Flat and multimodal likelihoods and model lack of fit in curved exponential families Rolf Sundberg Research Report 2009:1 ISSN 1650-0377 Postal address: Mathematical
More informationMLES & Multivariate Normal Theory
Merlise Clyde September 6, 2016 Outline Expectations of Quadratic Forms Distribution Linear Transformations Distribution of estimates under normality Properties of MLE s Recap Ŷ = ˆµ is an unbiased estimate
More informationDefault priors and model parametrization
1 / 16 Default priors and model parametrization Nancy Reid O-Bayes09, June 6, 2009 Don Fraser, Elisabeta Marras, Grace Yun-Yi 2 / 16 Well-calibrated priors model f (y; θ), F(y; θ); log-likelihood l(θ)
More informationStatistics and Econometrics I
Statistics and Econometrics I Point Estimation Shiu-Sheng Chen Department of Economics National Taiwan University September 13, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I September 13,
More informationGauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA
JAPANESE BEETLE DATA 6 MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA Gauge Plots TuscaroraLisa Central Madsen Fairways, 996 January 9, 7 Grubs Adult Activity Grub Counts 6 8 Organic Matter
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationFinal Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given.
1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given. (a) If X and Y are independent, Corr(X, Y ) = 0. (b) (c) (d) (e) A consistent estimator must be asymptotically
More informationProbability and Statistics Notes
Probability and Statistics Notes Chapter Seven Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Seven Notes Spring 2011 1 / 42 Outline
More informationMaximum Likelihood Estimation
Chapter 8 Maximum Likelihood Estimation 8. Consistency If X is a random variable (or vector) with density or mass function f θ (x) that depends on a parameter θ, then the function f θ (X) viewed as a function
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationPh.D. Qualifying Exam Monday Tuesday, January 4 5, 2016
Ph.D. Qualifying Exam Monday Tuesday, January 4 5, 2016 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Find the maximum likelihood estimate of θ where θ is a parameter
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid February 26, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Nancy Reid Accurate directional inference for vector parameters York University
More informationLast week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36
Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) =
More informationSTAT Financial Time Series
STAT 6104 - Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR
More informationGeneralized Linear Models. Kurt Hornik
Generalized Linear Models Kurt Hornik Motivation Assuming normality, the linear model y = Xβ + e has y = β + ε, ε N(0, σ 2 ) such that y N(μ, σ 2 ), E(y ) = μ = β. Various generalizations, including general
More information2. Inference method for margins and jackknife.
The Estimation Method of Inference Functions for Margins for Multivariate Models Harry Joe and James J. Xu Department of Statistics, University of British Columbia ABSTRACT An estimation approach is proposed
More informationLikelihood-Based Methods
Likelihood-Based Methods Handbook of Spatial Statistics, Chapter 4 Susheela Singh September 22, 2016 OVERVIEW INTRODUCTION MAXIMUM LIKELIHOOD ESTIMATION (ML) RESTRICTED MAXIMUM LIKELIHOOD ESTIMATION (REML)
More informationNotes on the Multivariate Normal and Related Topics
Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More information8. Hypothesis Testing
FE661 - Statistical Methods for Financial Engineering 8. Hypothesis Testing Jitkomut Songsiri introduction Wald test likelihood-based tests significance test for linear regression 8-1 Introduction elements
More informationBayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference
1 The views expressed in this paper are those of the authors and do not necessarily reflect the views of the Federal Reserve Board of Governors or the Federal Reserve System. Bayesian Estimation of DSGE
More informationBAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN
BAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C., U.S.A. J. Stuart Hunter Lecture TIES 2004
More informationSTAT 525 Fall Final exam. Tuesday December 14, 2010
STAT 525 Fall 2010 Final exam Tuesday December 14, 2010 Time: 2 hours Name (please print): Show all your work and calculations. Partial credit will be given for work that is partially correct. Points will
More informationGARCH Models Estimation and Inference
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in
More informationGibbs Sampling in Endogenous Variables Models
Gibbs Sampling in Endogenous Variables Models Econ 690 Purdue University Outline 1 Motivation 2 Identification Issues 3 Posterior Simulation #1 4 Posterior Simulation #2 Motivation In this lecture we take
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationEstimation of spatial max-stable models using threshold exceedances
Estimation of spatial max-stable models using threshold exceedances arxiv:1205.1107v1 [stat.ap] 5 May 2012 Jean-Noel Bacro I3M, Université Montpellier II and Carlo Gaetan DAIS, Università Ca Foscari -
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationIntroduction to Normal Distribution
Introduction to Normal Distribution Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 17-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Introduction
More informationStatistics 3858 : Maximum Likelihood Estimators
Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,
More informationNotes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed
18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,
More informationRegression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood
Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 Minimize this by maximizing
More information2.6.3 Generalized likelihood ratio tests
26 HYPOTHESIS TESTING 113 263 Generalized likelihood ratio tests When a UMP test does not exist, we usually use a generalized likelihood ratio test to verify H 0 : θ Θ against H 1 : θ Θ\Θ It can be used
More informationChapter 17: Undirected Graphical Models
Chapter 17: Undirected Graphical Models The Elements of Statistical Learning Biaobin Jiang Department of Biological Sciences Purdue University bjiang@purdue.edu October 30, 2014 Biaobin Jiang (Purdue)
More informationGeneralized Estimating Equations
Outline Review of Generalized Linear Models (GLM) Generalized Linear Model Exponential Family Components of GLM MLE for GLM, Iterative Weighted Least Squares Measuring Goodness of Fit - Deviance and Pearson
More information