Default priors and model parametrization
|
|
- Valentine Byrd
- 5 years ago
- Views:
Transcription
1 1 / 16 Default priors and model parametrization Nancy Reid O-Bayes09, June 6, 2009 Don Fraser, Elisabeta Marras, Grace Yun-Yi
2 2 / 16 Well-calibrated priors model f (y; θ), F(y; θ); log-likelihood l(θ) = log f (y; θ) can we find priors that are guaranteed to be well-calibrated, at least approximately what structure do such priors need to have can we do this for all component parameters simultaneously Welch & Peers, 1963: π(θ)dθ i 1/2 (θ)dθ expected Fisher information (matrix) i(θ) = E{ l (θ)} Peers, 1965; Tibshirani, 1989 parameter of interest θ = (ψ, λ): π(θ)dθ i 1/2 ψψ (θ)g(λ)dθ matching priors using Edgeworth expansion
3 3 / 16 Well-calibrated priors model f (y; θ), F(y; θ); log-likelihood l(θ) = log f (y; θ) can we find priors that are guaranteed to be well-calibrated, at least approximately what structure do such priors need to have can we do this for all component parameters simultaneously Welch & Peers, 1963: π(θ)dθ i 1/2 (θ)dθ expected Fisher information (matrix) i(θ) = E{ l (θ)} Peers, 1965; Tibshirani, 1989 parameter of interest θ = (ψ, λ): π(θ)dθ i 1/2 ψψ (θ)g(λ)dθ matching priors using Edgeworth expansion
4 4 / 16 Approximate location models Location model: Y f (y θ) = π(θ)dθ dθ Location model: θ θ + dθ, y y + dθ F(y; θ) unchanged, i.e. df(y; θ) = 0 scalar Y, θ General model: require df (y 0 ; θ) = 0 y 0 sample point F y (y 0 ; θ)dy + F ;θ (y 0 ; θ)dθ = 0 sample y 1,..., y n : dy = F ;θ(y 0 ; θ) F y (y 0 dθ = V (θ)dθ ; θ) dy i = V i (θ)dθ scalar Y, vector θ i.i.d Y i, vector θ
5 Default prior dy = V (θ)dθ; V (θ) = V 1 (θ). V n (θ) possible default prior π(θ)dθ V (θ)v (θ) 1/2 dθ convert to maximum likelihood coordinates l θ (ˆθ; y) = 0 = l θθ (ˆθ; y)d ˆθ + l θ;y (ˆθ; y)dy = 0 n p matrix d ˆθ = ĵ 1 Hdy = ĵ 1 HV (θ)dθ proposed default prior ĵ = j(ˆθ 0 ): observed Fisher information π(θ)dθ ĵ 1 HV (θ) dθ 5 / 16
6 6 / 16 V (θ) links dy to dθ ĵ 1 H links dy to d ˆθ... default prior π(θ)dθ ĵ 1 HV (θ) dθ gives right invariant prior for transformation parameter in transformation models provides extension of right invariant prior to general (continuous) models V (θ) = dy dθ = F θ(y; θ) y=y 0 f (y; θ) y=y 0 H = H(y 0 ; ˆθ 0 ) = 2 l(θ; y) θ y ĵ = j(ˆθ 0 ) = l θθ (ˆθ 0 ) ˆθ 0,y 0
7 7 / 16 Example: regression model y 1 = X 1 β + σɛ 1,..., y n = X n β + σɛ n V (θ) = dy dθ = {X, (y 0 Xβ)/(2σ)} y 0 ( X /ˆσ 2 ) H = ẑ 0 /ˆσ 3 ĵ = diag{x X/ˆσ 2, n/(2ˆσ 4 )} W (θ) = = { { I (X X) 1 X z 0 (θ)/(2σ) 2ẑ 0 ˆσX/n ẑ 0 z 0 (θ)ˆσ/(nσ) I ( β ˆ } 0 β)/2σ 2 0 ˆσ 2 /σ 2. }
8 8 / regression y N(Xβ, σ 2 ), θ = (β, σ 2 ) V (θ) = ( X y 0 ) Xβ σ y = Xβ + σɛ design residuals d ˆβ = dβ + ( ˆβ 0 β)dσ 2 /2σ 2 d ˆσ 2 = ˆσ 2 dσ 2 /σ 2. π(θ)dθ dβdσ 2 /σ 2 dβdσ/σ nonlinear regression, y N(x(β), σ 2 ), leads to d βdσ/σ β coordinates for tangent plane ẋ( ˆβ 0 )(β ˆβ 0 )
9 Scalar parameter default prior is data dependent, changes with y 0 based on approximate local location model Jeffreys prior π J (θ)dθ i 1/2 (θ)dθ gives frequentist matching to second order depends on model, not data unconditional prior Flat priors for Gamma shape default Jeffreys θ 9 / 16
10 Targetted priors If parameter of interest is curved, then prior needs to be targetted on the parameter of interest marginalization paradox (Dawid, Stone and Zidek) Example: Normal circle y i N(µ i, 1/n), i = 1,..., k; ψ = (µ 2 i )1/2 Stein Default prior π(µ)dµ dµ Posterior χ 2 k (n y 2 ) Exact χ 2 k (n ψ 2 ) Normal Circle, k=2, 5, 10 pvalue posterior p-value ψ 10 / 16
11 Normal Circle, k=2, 5, 10 pvalue posterior p-value actual coverage ψ nominal probability Bayes - frequentist Φ { } (k 1) ψ n not fixed by hierarchy of priors
12 12 / 16 Targetted priors: strong matching use Laplace approximations to posterior and to frequentist p-value structure of approximations makes comparison easy s(ψ). = Φ(r + 1 r log q B r ): Bayesian survivor value p(ψ). = Φ(r + 1 r log q F r ): Frequentist p-value r = r(ψ) = ± 2{l(ˆθ) l(ψ, ˆλ ψ )} q B contains the prior; q F various information functions and sample space derivatives q B = q F π(ˆθ) π(ˆθ ψ ) =... default prior along the curve θ = ˆθ ψ F&R, 2002 need to extend to full parameter space
13 13 / 16 q B = l ψ (ˆθ ψ ) j λλ(ˆθ ψ ) 1/2 j(ˆθ) 1/2... details π(ˆθ) π(ˆθ ψ ) q F = l ;V (ˆθ) l ;V (ˆθ ψ ) l λ;v (ˆθ ψ ) l θ;v (ˆθ) j(ˆθ) 1/2 j λλ (ˆθ ψ ) 1/2 π(ˆθ ψ ) π(ˆθ) l ψ (ˆθ ψ ) l θ;v (ˆθ) j λλ (ˆθ ψ ) l ;V (ˆθ) l ;V (ˆθ ψ ) l λ;v (ˆθ ψ ) j(ˆθ) along the profile curve C ψ = {θ : θ = (ψ, ˆλ ψ )} based on exponential family approximation at y 0 use observed information off the curve to extend prior to parameter space to get a version that when integrated via Laplace, brings us back to rf approximation π(θ) j (θθ) (ˆθ ψ ) 1/2 j (λλ) (θ) 1/2
14 14 / details any continuous model can be approximated to O(n 1 ) by an exponential family model (at y 0 ) canonical parameter ϕ(θ) = l ;V (θ; y 0 ) = l yi (θ; y 0 )V i (ˆθ 0 ) V (θ) the same matrix as in the default prior connection through location model approximation ancillarity flat prior
15 15 / 16 Conclusions calibrated priors are data dependent focus motivated by asymptotic theory for likelihood inference reference priors also targetted on parameter of interest marginalization to curved parameters using flat priors may lead to poorly calibrated inferences hierarchical Poisson models: E(y ij ) = c 0 x ij exp(µ + α i + β j + γ ij ) non-informative uniform priors on µ, α, σ β, σ γ difficulties and opportunities with new large data sets checking sensitivity to prior specification can be done simply using asymptotic approximation Φ(r B ) connections to Empirical Bayes?
16 16 / 16 Some references Fraser, D.A.S., Marras, E., Reid, N. and Yi, G. (2009). Default priors for Bayesian and frequentist inference. Fraser, D.A.S. and Reid, N. (2002). Strong matching of frequentist and Bayesian inference. J. Statist. Plan. Infer. 103, Berger, J.O., Bernardo, J. and Sun, D. (2009).The formal definition of reference priors. Ann. Statist. 37, Datta, G.S. and Ghosh, M. (1995). Some remarks on noninformative priors. J. Amer. Statist. Assoc. 90, Dawid, A.P., Stone, M., and Zidek, J.V. (1973). Marginalization paradoxes in Bayesian and structural inference. J. Roy. Statist. Soc. B 35, Bernardo, J.M. (1979). Reference posterior distributions for Bayesian inference. J. Roy. Statist. Soc. B 41, (with discussion).
Default priors for Bayesian and frequentist inference
Default priors for Bayesian and frequentist inference D.A.S. Fraser and N. Reid University of Toronto, Canada E. Marras Centre for Advanced Studies and Development, Sardinia University of Rome La Sapienza,
More informationBayesian and frequentist inference
Bayesian and frequentist inference Nancy Reid March 26, 2007 Don Fraser, Ana-Maria Staicu Overview Methods of inference Asymptotic theory Approximate posteriors matching priors Examples Logistic regression
More informationLikelihood inference in the presence of nuisance parameters
Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference
More informationApproximating models. Nancy Reid, University of Toronto. Oxford, February 6.
Approximating models Nancy Reid, University of Toronto Oxford, February 6 www.utstat.utoronto.reid/research 1 1. Context Likelihood based inference model f(y; θ), log likelihood function l(θ; y) y = (y
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Light touch assessment One or two problems assigned weekly graded during Reading Week http://www.utstat.toronto.edu/reid/4508s14.html
More informationLast week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36
Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) =
More informationMeasuring nuisance parameter effects in Bayesian inference
Measuring nuisance parameter effects in Bayesian inference Alastair Young Imperial College London WHOA-PSI-2017 1 / 31 Acknowledgements: Tom DiCiccio, Cornell University; Daniel Garcia Rasines, Imperial
More informationLikelihood Inference in the Presence of Nuisance Parameters
Likelihood Inference in the Presence of Nuance Parameters N Reid, DAS Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe some recent approaches to likelihood based
More informationASYMPTOTICS AND THE THEORY OF INFERENCE
ASYMPTOTICS AND THE THEORY OF INFERENCE N. Reid University of Toronto Abstract Asymptotic analysis has always been very useful for deriving distributions in statistics in cases where the exact distribution
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Problems assigned weekly, due the following week http://www.utstat.toronto.edu/reid/4508s16.html Various types of likelihood 1. likelihood,
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid February 26, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Nancy Reid Accurate directional inference for vector parameters York University
More informationLikelihood Inference in the Presence of Nuisance Parameters
PHYSTAT2003, SLAC, September 8-11, 2003 1 Likelihood Inference in the Presence of Nuance Parameters N. Reid, D.A.S. Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe
More informationDEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx ISSN 0256-422 X DEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY D. A. S. FRASER Department of Statistical Sciences,
More informationBFF Four: Are we Converging?
BFF Four: Are we Converging? Nancy Reid May 2, 2017 Classical Approaches: A Look Way Back Nature of Probability BFF one to three: a look back Comparisons Are we getting there? BFF Four Harvard, May 2017
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,
More informationSome Curiosities Arising in Objective Bayesian Analysis
. Some Curiosities Arising in Objective Bayesian Analysis Jim Berger Duke University Statistical and Applied Mathematical Institute Yale University May 15, 2009 1 Three vignettes related to John s work
More informationASSESSING A VECTOR PARAMETER
SUMMARY ASSESSING A VECTOR PARAMETER By D.A.S. Fraser and N. Reid Department of Statistics, University of Toronto St. George Street, Toronto, Canada M5S 3G3 dfraser@utstat.toronto.edu Some key words. Ancillary;
More informationPARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE.
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx Bangladesh ISSN 0256-422 X PARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE. A.M. FRASER Department of Mathematics, University
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationApplied Asymptotics Case studies in higher order inference
Applied Asymptotics Case studies in higher order inference Nancy Reid May 18, 2006 A.C. Davison, A. R. Brazzale, A. M. Staicu Introduction likelihood-based inference in parametric models higher order approximations
More informationStaicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors.
Staicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors. Early version, also known as pre-print Link to publication record in Explore Bristol Research PDF-document University
More informationLinear Models A linear model is defined by the expression
Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose
More informationBayesian Asymptotics
BS2 Statistical Inference, Lecture 8, Hilary Term 2008 May 7, 2008 The univariate case The multivariate case For large λ we have the approximation I = b a e λg(y) h(y) dy = e λg(y ) h(y ) 2π λg (y ) {
More informationCOMBINING p-values: A DEFINITIVE PROCESS. Galley
0 Journal of Statistical Research ISSN 0 - X 00, Vol., No., pp. - Bangladesh COMBINING p-values: A DEFINITIVE PROCESS D.A.S. Fraser Department of Statistics, University of Toronto, Toronto, Canada MS G
More informationANCILLARY STATISTICS: A REVIEW
Statistica Sinica 20 (2010), 1309-1332 ANCILLARY STATISTICS: A REVIEW M. Ghosh 1, N. Reid 2 and D. A. S. Fraser 2 1 University of Florida and 2 University of Toronto Abstract: In a parametric statistical
More informationIntroduction to Bayesian Methods
Introduction to Bayesian Methods Jessi Cisewski Department of Statistics Yale University Sagan Summer Workshop 2016 Our goal: introduction to Bayesian methods Likelihoods Priors: conjugate priors, non-informative
More informationNoninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions
Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted
More informationANCILLARY STATISTICS: A REVIEW
1 ANCILLARY STATISTICS: A REVIEW M. Ghosh, N. Reid and D.A.S. Fraser University of Florida and University of Toronto Abstract: In a parametric statistical model, a function of the data is said to be ancillary
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationInvariant HPD credible sets and MAP estimators
Bayesian Analysis (007), Number 4, pp. 681 69 Invariant HPD credible sets and MAP estimators Pierre Druilhet and Jean-Michel Marin Abstract. MAP estimators and HPD credible sets are often criticized in
More informationBayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University
Bayesian Inference and the Parametric Bootstrap Bradley Efron Stanford University Importance Sampling for Bayes Posterior Distribution Newton and Raftery (1994 JRSS-B) Nonparametric Bootstrap: good choice
More informationModule 22: Bayesian Methods Lecture 9 A: Default prior selection
Module 22: Bayesian Methods Lecture 9 A: Default prior selection Peter Hoff Departments of Statistics and Biostatistics University of Washington Outline Jeffreys prior Unit information priors Empirical
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationOverall Objective Priors
Overall Objective Priors Jim Berger, Jose Bernardo and Dongchu Sun Duke University, University of Valencia and University of Missouri Recent advances in statistical inference: theory and case studies University
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationThird-order inference for autocorrelation in nonlinear regression models
Third-order inference for autocorrelation in nonlinear regression models P. E. Nguimkeu M. Rekkas Abstract We propose third-order likelihood-based methods to derive highly accurate p-value approximations
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 425: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 2017 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 2017 1 / 10 Lecture 7: Prior Types Subjective
More information1 Hypothesis Testing and Model Selection
A Short Course on Bayesian Inference (based on An Introduction to Bayesian Analysis: Theory and Methods by Ghosh, Delampady and Samanta) Module 6: From Chapter 6 of GDS 1 Hypothesis Testing and Model Selection
More informationModern likelihood inference for the parameter of skewness: An application to monozygotic
Working Paper Series, N. 10, December 2013 Modern likelihood inference for the parameter of skewness: An application to monozygotic twin studies Mameli Valentina Department of Mathematics and Computer
More informationAsymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands
Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Elizabeth C. Mannshardt-Shamseldin Advisor: Richard L. Smith Duke University Department
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationRemarks on Improper Ignorance Priors
As a limit of proper priors Remarks on Improper Ignorance Priors Two caveats relating to computations with improper priors, based on their relationship with finitely-additive, but not countably-additive
More informationBayesian Model Comparison
BS2 Statistical Inference, Lecture 11, Hilary Term 2009 February 26, 2009 Basic result An accurate approximation Asymptotic posterior distribution An integral of form I = b a e λg(y) h(y) dy where h(y)
More informationApproximate Inference for the Multinomial Logit Model
Approximate Inference for the Multinomial Logit Model M.Rekkas Abstract Higher order asymptotic theory is used to derive p-values that achieve superior accuracy compared to the p-values obtained from traditional
More informationIntegrated Objective Bayesian Estimation and Hypothesis Testing
Integrated Objective Bayesian Estimation and Hypothesis Testing José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es 9th Valencia International Meeting on Bayesian Statistics Benidorm
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationStats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions
Stats 579 Intermediate Bayesian Modeling Assignment # 2 Solutions 1. Let w Gy) with y a vector having density fy θ) and G having a differentiable inverse function. Find the density of w in general and
More informationImproved Inference for First Order Autocorrelation using Likelihood Analysis
Improved Inference for First Order Autocorrelation using Likelihood Analysis M. Rekkas Y. Sun A. Wong Abstract Testing for first-order autocorrelation in small samples using the standard asymptotic test
More informationChapter 4 HOMEWORK ASSIGNMENTS. 4.1 Homework #1
Chapter 4 HOMEWORK ASSIGNMENTS These homeworks may be modified as the semester progresses. It is your responsibility to keep up to date with the correctly assigned homeworks. There may be some errors in
More informationExponential Models: Approximations for Probabilities
JIRSS (2011) Vol. 10, No. 2, pp 95-107 Exponential Models: Approximations for Probabilities D. A. S. Fraser 1,2,A.Naderi 3, Kexin Ji 1,WeiLin 1, Jie Su 1 1 Department of Statistics, University of Toronto,
More informationA simple analysis of the exact probability matching prior in the location-scale model
A simple analysis of the exact probability matching prior in the location-scale model Thomas J. DiCiccio Department of Social Statistics, Cornell University Todd A. Kuffner Department of Mathematics, Washington
More informationUSEFUL PROPERTIES OF THE MULTIVARIATE NORMAL*
USEFUL PROPERTIES OF THE MULTIVARIATE NORMAL* 3 Conditionals and marginals For Bayesian analysis it is very useful to understand how to write joint, marginal, and conditional distributions for the multivariate
More informationA BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain
A BAYESIAN MATHEMATICAL STATISTICS PRIMER José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es Bayesian Statistics is typically taught, if at all, after a prior exposure to frequentist
More informationBootstrap and Parametric Inference: Successes and Challenges
Bootstrap and Parametric Inference: Successes and Challenges G. Alastair Young Department of Mathematics Imperial College London Newton Institute, January 2008 Overview Overview Review key aspects of frequentist
More informationNew Bayesian methods for model comparison
Back to the future New Bayesian methods for model comparison Murray Aitkin murray.aitkin@unimelb.edu.au Department of Mathematics and Statistics The University of Melbourne Australia Bayesian Model Comparison
More informationImproper mixtures and Bayes s theorem
and Bayes s theorem and Han Han Department of Statistics University of Chicago DASF-III conference Toronto, March 2010 Outline Bayes s theorem 1 Bayes s theorem 2 Bayes s theorem Non-Bayesian model: Domain
More informationBAYESIAN ANALYSIS OF BIVARIATE COMPETING RISKS MODELS
Sankhyā : The Indian Journal of Statistics 2000, Volume 62, Series B, Pt. 3, pp. 388 401 BAYESIAN ANALYSIS OF BIVARIATE COMPETING RISKS MODELS By CHEN-PIN WANG University of South Florida, Tampa, U.S.A.
More informationBIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation
BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)
More informationAn Extended BIC for Model Selection
An Extended BIC for Model Selection at the JSM meeting 2007 - Salt Lake City Surajit Ray Boston University (Dept of Mathematics and Statistics) Joint work with James Berger, Duke University; Susie Bayarri,
More informationLikelihood based Statistical Inference. Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona
Likelihood based Statistical Inference Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona L. Pace, A. Salvan, N. Sartori Udine, April 2008 Likelihood: observed quantities,
More informationChapter 4 - Fundamentals of spatial processes Lecture notes
Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative
More informationStatistical Theory MT 2007 Problems 4: Solution sketches
Statistical Theory MT 007 Problems 4: Solution sketches 1. Consider a 1-parameter exponential family model with density f(x θ) = f(x)g(θ)exp{cφ(θ)h(x)}, x X. Suppose that the prior distribution has the
More informationg-priors for Linear Regression
Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 g-priors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and g-priors In the last lecture,
More informationAn Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models
An Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models Pierre Nguimkeu Georgia State University Abstract This paper proposes an improved likelihood-based method
More informationThe binomial model. Assume a uniform prior distribution on p(θ). Write the pdf for this distribution.
The binomial model Example. After suspicious performance in the weekly soccer match, 37 mathematical sciences students, staff, and faculty were tested for the use of performance enhancing analytics. Let
More informationComparing Non-informative Priors for Estimation and Prediction in Spatial Models
Environmentrics 00, 1 12 DOI: 10.1002/env.XXXX Comparing Non-informative Priors for Estimation and Prediction in Spatial Models Regina Wu a and Cari G. Kaufman a Summary: Fitting a Bayesian model to spatial
More informationOverall Objective Priors
Overall Objective Priors James O. Berger Duke University, USA Jose M. Bernardo Universitat de València, Spain and Dongchu Sun University of Missouri-Columbia, USA Abstract In multi-parameter models, reference
More informationSpatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields
Spatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields 1 Introduction Jo Eidsvik Department of Mathematical Sciences, NTNU, Norway. (joeid@math.ntnu.no) February
More informationST 740: Linear Models and Multivariate Normal Inference
ST 740: Linear Models and Multivariate Normal Inference Alyson Wilson Department of Statistics North Carolina State University November 4, 2013 A. Wilson (NCSU STAT) Linear Models November 4, 2013 1 /
More informationNow consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.
Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)
More informationA Bayesian Treatment of Linear Gaussian Regression
A Bayesian Treatment of Linear Gaussian Regression Frank Wood December 3, 2009 Bayesian Approach to Classical Linear Regression In classical linear regression we have the following model y β, σ 2, X N(Xβ,
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationA Very Brief Summary of Bayesian Inference, and Examples
A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X
More informationMarginal Posterior Simulation via Higher-order Tail Area Approximations
Bayesian Analysis (2014) 9, Number 1, pp. 129 146 Marginal Posterior Simulation via Higher-order Tail Area Approximations Erlis Ruli, Nicola Sartori and Laura Ventura Abstract. A new method for posterior
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationPart III. A Decision-Theoretic Approach and Bayesian testing
Part III A Decision-Theoretic Approach and Bayesian testing 1 Chapter 10 Bayesian Inference as a Decision Problem The decision-theoretic framework starts with the following situation. We would like to
More informationBayesian inference for factor scores
Bayesian inference for factor scores Murray Aitkin and Irit Aitkin School of Mathematics and Statistics University of Newcastle UK October, 3 Abstract Bayesian inference for the parameters of the factor
More informationFiducial Inference and Generalizations
Fiducial Inference and Generalizations Jan Hannig Department of Statistics and Operations Research The University of North Carolina at Chapel Hill Hari Iyer Department of Statistics, Colorado State University
More informationImproved Inference for Moving Average Disturbances in Nonlinear Regression Models
Improved Inference for Moving Average Disturbances in Nonlinear Regression Models Pierre Nguimkeu Georgia State University November 22, 2013 Abstract This paper proposes an improved likelihood-based method
More informationPredictive Distributions
Predictive Distributions October 6, 2010 Hoff Chapter 4 5 October 5, 2010 Prior Predictive Distribution Before we observe the data, what do we expect the distribution of observations to be? p(y i ) = p(y
More informationIntroduction to Bayesian Statistics with WinBUGS Part 4 Priors and Hierarchical Models
Introduction to Bayesian Statistics with WinBUGS Part 4 Priors and Hierarchical Models Matthew S. Johnson New York ASA Chapter Workshop CUNY Graduate Center New York, NY hspace1in December 17, 2009 December
More informationAdvanced Quantitative Methods: maximum likelihood
Advanced Quantitative Methods: Maximum Likelihood University College Dublin 4 March 2014 1 2 3 4 5 6 Outline 1 2 3 4 5 6 of straight lines y = 1 2 x + 2 dy dx = 1 2 of curves y = x 2 4x + 5 of curves y
More informationModern Methods of Statistical Learning sf2935 Auxiliary material: Exponential Family of Distributions Timo Koski. Second Quarter 2016
Auxiliary material: Exponential Family of Distributions Timo Koski Second Quarter 2016 Exponential Families The family of distributions with densities (w.r.t. to a σ-finite measure µ) on X defined by R(θ)
More informationHarrison B. Prosper. CMS Statistics Committee
Harrison B. Prosper Florida State University CMS Statistics Committee 08-08-08 Bayesian Methods: Theory & Practice. Harrison B. Prosper 1 h Lecture 3 Applications h Hypothesis Testing Recap h A Single
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationLikelihood and p-value functions in the composite likelihood context
Likelihood and p-value functions in the composite likelihood context D.A.S. Fraser and N. Reid Department of Statistical Sciences University of Toronto November 19, 2016 Abstract The need for combining
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationIntegrated Objective Bayesian Estimation and Hypothesis Testing
BAYESIAN STATISTICS 9, J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West (Eds.) c Oxford University Press, 2010 Integrated Objective Bayesian Estimation
More informationChapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)
Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December
More information7. Estimation and hypothesis testing. Objective. Recommended reading
7. Estimation and hypothesis testing Objective In this chapter, we show how the election of estimators can be represented as a decision problem. Secondly, we consider the problem of hypothesis testing
More informationLinear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
More informationMore on nuisance parameters
BS2 Statistical Inference, Lecture 3, Hilary Term 2009 January 30, 2009 Suppose that there is a minimal sufficient statistic T = t(x ) partitioned as T = (S, C) = (s(x ), c(x )) where: C1: the distribution
More informationNuisance parameters and their treatment
BS2 Statistical Inference, Lecture 2, Hilary Term 2008 April 2, 2008 Ancillarity Inference principles Completeness A statistic A = a(x ) is said to be ancillary if (i) The distribution of A does not depend
More informationBayesian Econometrics
Bayesian Econometrics Christopher A. Sims Princeton University sims@princeton.edu September 20, 2016 Outline I. The difference between Bayesian and non-bayesian inference. II. Confidence sets and confidence
More informationChapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
More informationBayesian Statistics from Subjective Quantum Probabilities to Objective Data Analysis
Bayesian Statistics from Subjective Quantum Probabilities to Objective Data Analysis Luc Demortier The Rockefeller University Informal High Energy Physics Seminar Caltech, April 29, 2008 Bayes versus Frequentism...
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationAspects of Likelihood Inference
Submitted to the Bernoulli Aspects of Likelihood Inference NANCY REID 1 1 Department of Statistics University of Toronto 100 St. George St. Toronto, Canada M5S 3G3 E-mail: reid@utstat.utoronto.ca, url:
More informationThe formal relationship between analytic and bootstrap approaches to parametric inference
The formal relationship between analytic and bootstrap approaches to parametric inference T.J. DiCiccio Cornell University, Ithaca, NY 14853, U.S.A. T.A. Kuffner Washington University in St. Louis, St.
More information