Approximating models. Nancy Reid, University of Toronto. Oxford, February 6.
|
|
- Sherman Dickerson
- 5 years ago
- Views:
Transcription
1 Approximating models Nancy Reid, University of Toronto Oxford, February 6 1
2 1. Context Likelihood based inference model f(y; θ), log likelihood function l(θ; y) y = (y 1,..., y n ), l(θ; y) = log f(y i ; θ) assume l(θ; y) = O(n), θ ˆθ = O(n 1/2 ) goal to improve the approximation given by the limiting distribution e.g. r = ±[2{l(ˆθ) l(θ)}] 1/2 d N(0, 1) E(r) = an 1/2 + O(n 3/2 ) var(r) = 1 + bn 1 + O(n 3/2 ) implies r an 1/2. (1 + bn 1 ) 1/2 N(0, 1) is better than r. N(0, 1) 2
3 2. Local exponential family models f(x; ϕ) = exp{ϕx c(ϕ) d(x)} l ;x = l ϕ;x = Start with arbitrary f(y; θ), y, θ scalar Expand l(θ; y) about (θ 0 ; y 0 ) θ 0 = Represent coefficients l(θ 0 ; y 0 ), l ;y (θ 0 ; y 0 ), etc. by a ij a 00 a 01 a 02 a 03 a 04 0 a 11 a 12 a 13 a 20 a 21 a 22 a 30 a 31 a 40 3
4 1. Standardize θ (θ θ 0 )ĵ 1/2 y (y y 0 )a 11 ĵ 1/2 a 20 1, a 11 1, a ij ã ij 2. Reparametrize θ θ + ã 21 θ 2 /2 + ã 31 θ 3 /6 ã 21 0, ã New variable ỹ ỹ + ã 12 ỹ 2 /2 + ã 13 ỹ 3 /6 ã 12 0, ã Notation a 30 = α 3 n 1/2, a 40 = a 22 = a 00 a 01 a 02 a 03 a γ/n α 3 /n 1/2 0 α 4 /n 4
5 5. Density must integrate to 1 3α 4 5α γ 24n α 3 2 n 1 + α 4 2α 2 3 5γ 2n α 3 n α 4 3α 2 3 n γ/n α 3 /n 1/2 0 α 4 /n α 3 = α 4 = γ =
6 First row is log(2π) + (3α 4 5α γ)/24n, α 3/2n 1/2, 1 + (α 4 2α 2 3 5γ)/2n, α 3/n 1/2, (α 4 3α 2 3 6γ)/n 4-1
7 ...2 Local exponential family models New density looks like with cdf f(x, ϕ). = φ(x ϕ) exp{...}. = φ(x ϕ){1 +...} F (x, ϕ) = Φ(x ϕ) + [ α3 φ(x ϕ) 6 n {...} + α 4 24n {...} + α2 3 72n + γ { 2x + ϕx 2 + x 3}] 4n {...} Free of γ at x = 0 (y = y 0 ) p-value does not depend on γ Andrews, Fraser, Wong,
8 3. Tangent exponential model p T EM (x; θ) = c j(ˆϕ) 1/2 exp[l(θ; y 0 ) l(ˆθ 0 ; y 0 ) +, {ϕ(θ) ϕ(ˆθ 0 )}x] ϕ = l(θ, y) y y=y 0 x = l(θ; y) θ θ=ˆθ 0 j(ˆϕ) = 2 l(ϕ) ϕ 2 ˆϕ 6
9 l(θ; y 0 ) is first column (ignoring (0,0) entry) ϕ(θ) is second column (ignoring (0,1) entry) These 2 columns determine the rest of the array, except the γ/n term Easy to use p T EM to get a p-value (saddlepoint type approximation) 6-1
10 ...3 Tangent exponential model How to get a scalar variable y? Condition on an (approximate) ancillary, so l ;y is taken for fixed ancillary a(y). This can be computed by finding a vector V = (V 1,..., V n ) T tangent to the ancillary at y 0 : ϕ(θ) = l ;V (θ; y) y 0 = l ;yi (θ; y 0 i )V i Example y i f(y i µ) a i = y i ˆµ, say, V i = 1 ϕ(θ) = log f(y i µ) y i y 0 = l θ (θ; y 0 ) 7
11 Example f(y 1, y 2 ; θ) = e y θ {1 + e (y θ) } 2 exp[γ(θ)(y θ) c{γ(θ)}], 1 θ 1 γ(θ) = 0.5 tanh(θ) c(θ) = log{(πθ)/ sin(πθ)} y density y y2 y1 8
12 ...3 Tangent exponential model Vector θ? Use the same approach, now V = (V 1, V 2,..., V n ) T V i is 1 d, l ;V (θ) is also 1 d Example Example y i = x T i β + σe i V i = (x T i ê i ) y i = µ i (β) + σe i V i = {µ i (ˆβ) ê i } Inference re nuisance parameters uses p T EM twice to get a marginal distribution Example House price data (Srivastava and Sen); 4 covariates, 26 observations, model y i = x T i β + σe i, e i t 5 9
13 marginal inference for β 4 and for log σ, (conditional on usual ancillary), uses Alessandra Brazzale s Splus library HOA Lugannani-Rice tail approximations 95 % Confidence Intervals MLE normal approximation cond. MLE normal approximation marg. MLE normal approximation directed deviance modified directed deviance conditional marginal approximate marginal deviance Coefficient of front Coefficient of front 10
14 Profile and modified profile log-liks profile log-likelihood modified profile log-likelihood approximate marginal log-likelihood Coefficient of front
15 Lugannani-Rice tail approximations 95 % Confidence Intervals MLE normal approximation cond. MLE normal approximation marg. MLE normal approximation directed deviance modified directed deviance conditional marginal approximate marginal deviance log(scale) scale
16 Profile and modified profile log-liks profile log-likelihood modified profile log-likelihood approximate marginal log-likelihood log(scale)
17 > houses.marg.front <- cond.rsm(mod.obj=houses.rsm,offset=fron > summary(houses.marg.front) FORMULA: FAMILY : OFFSET : price ~ bdroom + floor + rooms + front student front COEFFICIENTS Value Std. Error uncond cond marg CONFIDENCE INTERVALS level = 95 % lower two-sided MLE normal approx Cond. MLE normal approx Marg. MLE normal approx Directed deviance Modified directed deviance Marginal directed deviance
18 4. Local location models f(x; β) = f(x β) l β = l ;x l ββ = l βx = l ;xx 4.1 If y f(y; θ) then x = y F y (y; θ 0 ) F θ (y; θ 0 ) dy has a density which is a location model near θ 0, g(x ), say. Satisfies l = l ;x, but not higher order. This model has an exact ancillary This ancillary can be used for the original model, for computing p-values. (This is where V above came from.) 11
19 ...4. Local location models 4.2 As with exponential model we can carry this further to get an array of coefficients for the double expansion about (y 0, ˆθ 0 ) of the form: a a 3 /n 1/2 a 4 /n 0 1 a 3 /n 1/2 a 4 /n 1 a 3 /n 1/2 a 4 /n+ a 3 /n 1/2 a 4 /n a 4 /n Andrews, Fraser, Wong, 2003 A more compact notation f{x β(θ)}, β(θ) = θ l θ (θ) ϕ(θ) dθ Existence (algorithm) for vector θ Fraser, Yi,
20 a + 3α 4 5α γ 24n γ 2n α 3 n 1/2 α 4 6γ n 0 1 α 3 /n 1/2 α 4 /n 1 α 3 /n 1/2 α 4+γ n α 4 /n x = y F y(y; 0) dy, G(x; θ) = F {y(x); θ} F ;θ (y; 0) G x (x; 0) = F y {y(x); 0} = F ;θ (y; 0) = G ;θ (x; 0) { F ;θ(y; 0) F y (y; 0) } 12-1
21 ...4 Local location model Bayesian analysis of location model uses flat prior for location parameter, in our case π(θ) dβ(θ) and this will give posterior p-values equal to those from tangent exponential model to O(n 3/2 ) if non-location term γ = 0, to O(n 1 ) if γ 0 With nuisance parameters, can only obtain strong matching priors for a single parameter of interest, using π(ψ, ˆλ ψ ) ψ β 1 (ψ,ˆλ ψ ) j λλ(ˆθ ψ ) ϕ λ (ˆθ ψ ) Fraser & Reid,
22 Example Location model with curved parameter of interest Y 1 N(θ 1, 1), Y 2 N(θ 2, 1) independent ψ 2 = (R + θ 1 ) 2 + θ 2 2 ; R known r 2 = {(R + y 1 ) 2 + y 2 2 } Bayesian posterior under usual flat prior (θ 1, θ 2 y) N(y 1, y 2 ) frequentist p-value (marginal) Pr{r r 0 ; ψ 0 ) Bayesian p-value Pr{ψ ψ 0 y) Will be quite different: matching prior using information adjustment gives π(θ) r ψ 14
23 frequentist = Pr{χ 2 2 (ψ0 ) 2 (y 1 + R) 2 + y 2 2 } Bayesian = Pr{χ 2 2 ((y 1 + R) 2 + y 2 2 ) ψ02 } Bayesian frequentist = Pr{X 1 X 2 = 0} X 1 P o((y 1 + R) 2 + y 2 2 ), X 2 P o(ψ 02 ) 14-1
24 R=1 R= p-value 0.4 p-value psi psi R=3 R= p-value 0.4 p-value psi psi 15
25 16
26 References Andrews, D.A., Fraser, D.A.S., Wong, A. Computation of distribution functions from likelihood information near observed data. Brazzale, A. brazzale Fraser, D.A.S., Reid, N. Strong matching of frequentist and Bayesian parametric inference. Fraser, D.A.S., Yi, G. Location reparametrization and default priors for statistical analyses. Reid, N. Asymptotics and the theory of inference. 17
Last week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36
Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) =
More informationASSESSING A VECTOR PARAMETER
SUMMARY ASSESSING A VECTOR PARAMETER By D.A.S. Fraser and N. Reid Department of Statistics, University of Toronto St. George Street, Toronto, Canada M5S 3G3 dfraser@utstat.toronto.edu Some key words. Ancillary;
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid February 26, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Nancy Reid Accurate directional inference for vector parameters York University
More informationBayesian and frequentist inference
Bayesian and frequentist inference Nancy Reid March 26, 2007 Don Fraser, Ana-Maria Staicu Overview Methods of inference Asymptotic theory Approximate posteriors matching priors Examples Logistic regression
More informationDefault priors and model parametrization
1 / 16 Default priors and model parametrization Nancy Reid O-Bayes09, June 6, 2009 Don Fraser, Elisabeta Marras, Grace Yun-Yi 2 / 16 Well-calibrated priors model f (y; θ), F(y; θ); log-likelihood l(θ)
More informationDEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx ISSN 0256-422 X DEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY D. A. S. FRASER Department of Statistical Sciences,
More informationLikelihood inference in the presence of nuisance parameters
Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,
More informationPARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE.
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx Bangladesh ISSN 0256-422 X PARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE. A.M. FRASER Department of Mathematics, University
More informationLikelihood Inference in the Presence of Nuisance Parameters
PHYSTAT2003, SLAC, September 8-11, 2003 1 Likelihood Inference in the Presence of Nuance Parameters N. Reid, D.A.S. Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe
More informationLikelihood Inference in the Presence of Nuisance Parameters
Likelihood Inference in the Presence of Nuance Parameters N Reid, DAS Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe some recent approaches to likelihood based
More informationCOMBINING p-values: A DEFINITIVE PROCESS. Galley
0 Journal of Statistical Research ISSN 0 - X 00, Vol., No., pp. - Bangladesh COMBINING p-values: A DEFINITIVE PROCESS D.A.S. Fraser Department of Statistics, University of Toronto, Toronto, Canada MS G
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationApproximate Inference for the Multinomial Logit Model
Approximate Inference for the Multinomial Logit Model M.Rekkas Abstract Higher order asymptotic theory is used to derive p-values that achieve superior accuracy compared to the p-values obtained from traditional
More informationApplied Asymptotics Case studies in higher order inference
Applied Asymptotics Case studies in higher order inference Nancy Reid May 18, 2006 A.C. Davison, A. R. Brazzale, A. M. Staicu Introduction likelihood-based inference in parametric models higher order approximations
More informationASYMPTOTICS AND THE THEORY OF INFERENCE
ASYMPTOTICS AND THE THEORY OF INFERENCE N. Reid University of Toronto Abstract Asymptotic analysis has always been very useful for deriving distributions in statistics in cases where the exact distribution
More informationBayesian Asymptotics
BS2 Statistical Inference, Lecture 8, Hilary Term 2008 May 7, 2008 The univariate case The multivariate case For large λ we have the approximation I = b a e λg(y) h(y) dy = e λg(y ) h(y ) 2π λg (y ) {
More informationDefault priors for Bayesian and frequentist inference
Default priors for Bayesian and frequentist inference D.A.S. Fraser and N. Reid University of Toronto, Canada E. Marras Centre for Advanced Studies and Development, Sardinia University of Rome La Sapienza,
More informationNew Bayesian methods for model comparison
Back to the future New Bayesian methods for model comparison Murray Aitkin murray.aitkin@unimelb.edu.au Department of Mathematics and Statistics The University of Melbourne Australia Bayesian Model Comparison
More informationExponential Models: Approximations for Probabilities
JIRSS (2011) Vol. 10, No. 2, pp 95-107 Exponential Models: Approximations for Probabilities D. A. S. Fraser 1,2,A.Naderi 3, Kexin Ji 1,WeiLin 1, Jie Su 1 1 Department of Statistics, University of Toronto,
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationA Very Brief Summary of Bayesian Inference, and Examples
A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Light touch assessment One or two problems assigned weekly graded during Reading Week http://www.utstat.toronto.edu/reid/4508s14.html
More informationStaicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors.
Staicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors. Early version, also known as pre-print Link to publication record in Explore Bristol Research PDF-document University
More informationThird-order inference for autocorrelation in nonlinear regression models
Third-order inference for autocorrelation in nonlinear regression models P. E. Nguimkeu M. Rekkas Abstract We propose third-order likelihood-based methods to derive highly accurate p-value approximations
More informationThe formal relationship between analytic and bootstrap approaches to parametric inference
The formal relationship between analytic and bootstrap approaches to parametric inference T.J. DiCiccio Cornell University, Ithaca, NY 14853, U.S.A. T.A. Kuffner Washington University in St. Louis, St.
More informationBayesian Model Comparison
BS2 Statistical Inference, Lecture 11, Hilary Term 2009 February 26, 2009 Basic result An accurate approximation Asymptotic posterior distribution An integral of form I = b a e λg(y) h(y) dy where h(y)
More informationImproved Inference for First Order Autocorrelation using Likelihood Analysis
Improved Inference for First Order Autocorrelation using Likelihood Analysis M. Rekkas Y. Sun A. Wong Abstract Testing for first-order autocorrelation in small samples using the standard asymptotic test
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationCONVERTING OBSERVED LIKELIHOOD FUNCTIONS TO TAIL PROBABILITIES. D.A.S. Fraser Mathematics Department York University North York, Ontario M3J 1P3
CONVERTING OBSERVED LIKELIHOOD FUNCTIONS TO TAIL PROBABILITIES D.A.S. Fraser Mathematics Department York University North York, Ontario M3J 1P3 N. Reid Department of Statistics University of Toronto Toronto,
More informationhoa: An R Package Bundle for Higher Order Likelihood Inference
hoa: An R Package Bundle for Higher Order Likelihood Inference by Alessandra R. Brazzale Rnews, 5/1 May 2005, pp. 20 27 Introduction The likelihood function represents the basic ingredient of many commonly
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Problems assigned weekly, due the following week http://www.utstat.toronto.edu/reid/4508s16.html Various types of likelihood 1. likelihood,
More informationNuisance parameters and their treatment
BS2 Statistical Inference, Lecture 2, Hilary Term 2008 April 2, 2008 Ancillarity Inference principles Completeness A statistic A = a(x ) is said to be ancillary if (i) The distribution of A does not depend
More informationPrinciples of Statistics
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 81 Paper 4, Section II 28K Let g : R R be an unknown function, twice continuously differentiable with g (x) M for
More informationStats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions
Stats 579 Intermediate Bayesian Modeling Assignment # 2 Solutions 1. Let w Gy) with y a vector having density fy θ) and G having a differentiable inverse function. Find the density of w in general and
More informationModern likelihood inference for the parameter of skewness: An application to monozygotic
Working Paper Series, N. 10, December 2013 Modern likelihood inference for the parameter of skewness: An application to monozygotic twin studies Mameli Valentina Department of Mathematics and Computer
More informationANCILLARY STATISTICS: A REVIEW
1 ANCILLARY STATISTICS: A REVIEW M. Ghosh, N. Reid and D.A.S. Fraser University of Florida and University of Toronto Abstract: In a parametric statistical model, a function of the data is said to be ancillary
More informationFrequentist-Bayesian Model Comparisons: A Simple Example
Frequentist-Bayesian Model Comparisons: A Simple Example Consider data that consist of a signal y with additive noise: Data vector (N elements): D = y + n The additive noise n has zero mean and diagonal
More information1. Fisher Information
1. Fisher Information Let f(x θ) be a density function with the property that log f(x θ) is differentiable in θ throughout the open p-dimensional parameter set Θ R p ; then the score statistic (or score
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationMeasuring nuisance parameter effects in Bayesian inference
Measuring nuisance parameter effects in Bayesian inference Alastair Young Imperial College London WHOA-PSI-2017 1 / 31 Acknowledgements: Tom DiCiccio, Cornell University; Daniel Garcia Rasines, Imperial
More informationLikelihood and p-value functions in the composite likelihood context
Likelihood and p-value functions in the composite likelihood context D.A.S. Fraser and N. Reid Department of Statistical Sciences University of Toronto November 19, 2016 Abstract The need for combining
More information2018 2019 1 9 sei@mistiu-tokyoacjp http://wwwstattu-tokyoacjp/~sei/lec-jhtml 11 552 3 0 1 2 3 4 5 6 7 13 14 33 4 1 4 4 2 1 1 2 2 1 1 12 13 R?boxplot boxplotstats which does the computation?boxplotstats
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2
Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationANCILLARY STATISTICS: A REVIEW
Statistica Sinica 20 (2010), 1309-1332 ANCILLARY STATISTICS: A REVIEW M. Ghosh 1, N. Reid 2 and D. A. S. Fraser 2 1 University of Florida and 2 University of Toronto Abstract: In a parametric statistical
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationStatistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
More information7. Estimation and hypothesis testing. Objective. Recommended reading
7. Estimation and hypothesis testing Objective In this chapter, we show how the election of estimators can be represented as a decision problem. Secondly, we consider the problem of hypothesis testing
More informationLecture 1: Introduction
Principles of Statistics Part II - Michaelmas 208 Lecturer: Quentin Berthet Lecture : Introduction This course is concerned with presenting some of the mathematical principles of statistical theory. One
More informationSTAT215: Solutions for Homework 2
STAT25: Solutions for Homework 2 Due: Wednesday, Feb 4. (0 pt) Suppose we take one observation, X, from the discrete distribution, x 2 0 2 Pr(X x θ) ( θ)/4 θ/2 /2 (3 θ)/2 θ/4, 0 θ Find an unbiased estimator
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationStatistics 135 Fall 2008 Final Exam
Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations
More informationSecond order ancillary: A differential view from continuity
Second order ancillary: A differential view from continuity BY AILANA M. FRASER Department of Mathematics, University of British Columbia, Vancouver, Canada V6T 1Z2 afraser@math.ubc.ca DONALD A.S. FRASER*
More informationFractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling
Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling Jae-Kwang Kim 1 Iowa State University June 26, 2013 1 Joint work with Shu Yang Introduction 1 Introduction
More informationf(x θ)dx with respect to θ. Assuming certain smoothness conditions concern differentiating under the integral the integral sign, we first obtain
0.1. INTRODUCTION 1 0.1 Introduction R. A. Fisher, a pioneer in the development of mathematical statistics, introduced a measure of the amount of information contained in an observaton from f(x θ). Fisher
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationResearch Article Inference for the Sharpe Ratio Using a Likelihood-Based Approach
Journal of Probability and Statistics Volume 202 Article ID 87856 24 pages doi:0.55/202/87856 Research Article Inference for the Sharpe Ratio Using a Likelihood-Based Approach Ying Liu Marie Rekkas 2 and
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2016 Julien Berestycki (University of Oxford) SB2a MT 2016 1 / 32 Lecture 14 : Variational Bayes
More informationMAS3301 Bayesian Statistics Problems 5 and Solutions
MAS3301 Bayesian Statistics Problems 5 and Solutions Semester 008-9 Problems 5 1. (Some of this question is also in Problems 4). I recorded the attendance of students at tutorials for a module. Suppose
More informationTopic 12 Overview of Estimation
Topic 12 Overview of Estimation Classical Statistics 1 / 9 Outline Introduction Parameter Estimation Classical Statistics Densities and Likelihoods 2 / 9 Introduction In the simplest possible terms, the
More informationStat 451 Lecture Notes Numerical Integration
Stat 451 Lecture Notes 03 12 Numerical Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 5 in Givens & Hoeting, and Chapters 4 & 18 of Lange 2 Updated: February 11, 2016 1 / 29
More informationBootstrap and Parametric Inference: Successes and Challenges
Bootstrap and Parametric Inference: Successes and Challenges G. Alastair Young Department of Mathematics Imperial College London Newton Institute, January 2008 Overview Overview Review key aspects of frequentist
More informationFREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE
FREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE Donald A. Pierce Oregon State Univ (Emeritus), RERF Hiroshima (Retired), Oregon Health Sciences Univ (Adjunct) Ruggero Bellio Univ of Udine For Perugia
More informationAccurate directional inference for vector parameters in linear exponential families
Accurate directional inference for vector parameters in linear exponential families A. C. Davison, D. A. S. Fraser, N. Reid and N. Sartori August 27, 2013 Abstract We consider inference on a vector-valued
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Jonathan Marchini Department of Statistics University of Oxford MT 2013 Jonathan Marchini (University of Oxford) BS2a MT 2013 1 / 27 Course arrangements Lectures M.2
More informationMarginal Posterior Simulation via Higher-order Tail Area Approximations
Bayesian Analysis (2014) 9, Number 1, pp. 129 146 Marginal Posterior Simulation via Higher-order Tail Area Approximations Erlis Ruli, Nicola Sartori and Laura Ventura Abstract. A new method for posterior
More informationAsymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands
Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Elizabeth C. Mannshardt-Shamseldin Advisor: Richard L. Smith Duke University Department
More informationMore on nuisance parameters
BS2 Statistical Inference, Lecture 3, Hilary Term 2009 January 30, 2009 Suppose that there is a minimal sufficient statistic T = t(x ) partitioned as T = (S, C) = (s(x ), c(x )) where: C1: the distribution
More informationImproved Inference for Moving Average Disturbances in Nonlinear Regression Models
Improved Inference for Moving Average Disturbances in Nonlinear Regression Models Pierre Nguimkeu Georgia State University November 22, 2013 Abstract This paper proposes an improved likelihood-based method
More informationInformation in a Two-Stage Adaptive Optimal Design
Information in a Two-Stage Adaptive Optimal Design Department of Statistics, University of Missouri Designed Experiments: Recent Advances in Methods and Applications DEMA 2011 Isaac Newton Institute for
More informationFor iid Y i the stronger conclusion holds; for our heuristics ignore differences between these notions.
Large Sample Theory Study approximate behaviour of ˆθ by studying the function U. Notice U is sum of independent random variables. Theorem: If Y 1, Y 2,... are iid with mean µ then Yi n µ Called law of
More informationAn Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models
An Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models Pierre Nguimkeu Georgia State University Abstract This paper proposes an improved likelihood-based method
More informationLecture 26: Likelihood ratio tests
Lecture 26: Likelihood ratio tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f θ0 (X) > c 0 for
More information1. INTRODUCTION We examine the agreement between frequentist and Bayesian methods for parametric inference using recent methods from higher order like
211:1999nov4 Strong matching of frequentist and Bayesian parametric inference By D.A.S. Fraser and N. Reid Department of Statistics University of Toronto Toronto, Canada, M5S 3G3 SUMMARY We dene a notion
More informationLoglikelihood and Confidence Intervals
Stat 504, Lecture 2 1 Loglikelihood and Confidence Intervals The loglikelihood function is defined to be the natural logarithm of the likelihood function, l(θ ; x) = log L(θ ; x). For a variety of reasons,
More informationBayesian Analysis (Optional)
Bayesian Analysis (Optional) 1 2 Big Picture There are two ways to conduct statistical inference 1. Classical method (frequentist), which postulates (a) Probability refers to limiting relative frequencies
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationNoninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions
Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted
More informationBayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University
Bayesian Inference and the Parametric Bootstrap Bradley Efron Stanford University Importance Sampling for Bayes Posterior Distribution Newton and Raftery (1994 JRSS-B) Nonparametric Bootstrap: good choice
More informationJoint work with Nottingham colleagues Simon Preston and Michail Tsagris.
/pgf/stepx/.initial=1cm, /pgf/stepy/.initial=1cm, /pgf/step/.code=1/pgf/stepx/.expanded=- 10.95415pt,/pgf/stepy/.expanded=- 10.95415pt, /pgf/step/.value required /pgf/images/width/.estore in= /pgf/images/height/.estore
More informationCh. 5 Hypothesis Testing
Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,
More informationMultistate Modeling and Applications
Multistate Modeling and Applications Yang Yang Department of Statistics University of Michigan, Ann Arbor IBM Research Graduate Student Workshop: Statistics for a Smarter Planet Yang Yang (UM, Ann Arbor)
More informationP n. This is called the law of large numbers but it comes in two forms: Strong and Weak.
Large Sample Theory Large Sample Theory is a name given to the search for approximations to the behaviour of statistical procedures which are derived by computing limits as the sample size, n, tends to
More informationIntroduction to Bayesian Methods
Introduction to Bayesian Methods Jessi Cisewski Department of Statistics Yale University Sagan Summer Workshop 2016 Our goal: introduction to Bayesian methods Likelihoods Priors: conjugate priors, non-informative
More informationParametric fractional imputation for missing data analysis
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Biometrika (????),??,?, pp. 1 15 C???? Biometrika Trust Printed in
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationA BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain
A BAYESIAN MATHEMATICAL STATISTICS PRIMER José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es Bayesian Statistics is typically taught, if at all, after a prior exposure to frequentist
More informationBayesian inference for factor scores
Bayesian inference for factor scores Murray Aitkin and Irit Aitkin School of Mathematics and Statistics University of Newcastle UK October, 3 Abstract Bayesian inference for the parameters of the factor
More informationStat 5102 Lecture Slides Deck 3. Charles J. Geyer School of Statistics University of Minnesota
Stat 5102 Lecture Slides Deck 3 Charles J. Geyer School of Statistics University of Minnesota 1 Likelihood Inference We have learned one very general method of estimation: method of moments. the Now we
More informationBFF Four: Are we Converging?
BFF Four: Are we Converging? Nancy Reid May 2, 2017 Classical Approaches: A Look Way Back Nature of Probability BFF one to three: a look back Comparisons Are we getting there? BFF Four Harvard, May 2017
More information