Bayesian and frequentist inference
|
|
- Sheena Sabrina Black
- 5 years ago
- Views:
Transcription
1 Bayesian and frequentist inference Nancy Reid March 26, 2007 Don Fraser, Ana-Maria Staicu
2 Overview Methods of inference Asymptotic theory Approximate posteriors matching priors Examples Logistic regression normal circle Constructing priors Conclusions
3 A Basic Structure data y = (y 1,..., y n ) R n model f (y; θ) probability density parameters θ = (θ 1,..., θ k ) R k inference about θ (or components of θ) an interval of values of θ consistent with the data a probability distribution for θ, conditional on the data
4
5 The average 12 month weight loss on the Atkins diet was 10.3 lbs, compared to 3.5, 5.7 and 4.8 lbs on the Zone, Learn and Ornish diets, respectively. 1 mean 12 month weight loss on the Atkins diet was 4.7 kg (95% confidence interval 3.1 to 6.3 kg) mean 12 month weight loss was significantly different between the Atkins and the Zone diets (p < 0.05) The probability that the average weight loss on the Atkins diet is greater than that on the Zone diet is JAMA, Mar 7
6
7
8 π(θ y) f (y; θ)π(θ) f (y; θ)π(θ) π(θ y) = f (y; θ)π(θ)dθ parameter of interest ψ(θ) Bayesian inference π m (ψ y) = {ψ(θ)=ψ} π(θ y)dθ integrals often computed using simulation methods marginal posterior density summarizes all information about ψ available from the data
9 nonbayesian inference find a function of the data t(y), say, with 1. distribution depending only on ψ 2. distribution that is known usually based on the log-likelihood function l(θ; y) = log f (y; θ) ˆθ arg sup l(θ; y) ˆθ ψ arg sup ψ(θ)=ψ l(θ; y) likelihood ratio statistic: 2{l(ˆθ; y) l(ˆθ ψ ; y)} d χ 2 d 1 Wald statistic ˆΣ 1/2 d ( ˆψ ψ) N(0, I) if ψ is scalar: r (ψ). N(0, 1) statistic is derived from the model (via the likelihood function) distribution comes from asymptotic theory likelihood is not (usually) integrated but is (usually) maximized
10 ... Bayesian inference well defined basis for inference internally consistent leads to optimal results from one point of view requires a probability distribution for θ ( a prior distribution) not necessarily calibrated not always clear how much answers depend on the choice of prior requires (high-) dimensional integration, or simulation, or approximation useful in applications
11 ... nonbayesian inference properties well understood calibrated: properties correspond to long-run frequency choice of function t(y) may be non-optimal approximation to distribution of t(y ) may be poor approximation to distribution of t(y ) may be excellent useful in applications
12 Fisher (1920s, frequentist): I know only one case in mathematics of a doctrine which has been accepted and developed by the most eminent men of their time, and is now perhaps accepted by men now living, which at the same time has appeared to a succession of sound writers to be fundamentally false and devoid of foundation Lindley (1950s, Bayesian): Every statistician would be a Bayesian if he took the trouble to read the literature thoroughly and was honest enough to admit that he might have been wrong
13 Approximate posteriors ψ π m (ψ y)dψ = Φ(r B ){1 + O(n 3/2 )} r B = r + 1 r log q B r r = [2{l p ( ˆψ) l p (ψ)}] 1/2 q B = l p(ψ)j 1/2 p ( ˆψ) j λλ(ˆθ) 1/2 π(ˆθ ψ ) j λλ (ˆθ ψ ) 1/2 π(ˆθ) l p (ψ) = l(ψ, ˆλ ψ ) = l(ˆθ ψ )
14 Bayesian and frequentist inference Asymptotic theory Approximate posteriors Approximate posteriors Approximate posteriors ψ πm(ψ y)dψ = Φ(r B ){1 + O(n 3/2 )} rb = r + 1 r log q B r r = [2{lp( ˆψ) lp(ψ)}] 1/2 q B = l p(ψ)j 1/2 p ( ˆψ) jλλ(ˆθ) 1/2 π(ˆθψ) jλλ(ˆθψ) 1/2 π(ˆθ) lp(ψ) = l(ψ, ˆλψ) = l(ˆθψ) marginal posterior density: π m (ψ y) = {ψ(θ)=ψ} exp{l(θ; y}π(θ)dθ/ exp{l(θ; y)}π(θ)dθ = c exp{l(ˆθ ψ )} j λλ (ˆθ ψ ) 1/2 π(ˆθ ψ ) {1 + O(n 3/2 )} exp{l(ˆθ)} j(ˆθ) 1/2 π(ˆθ). j λλ (ˆθ ψ ) 1/2 π(ˆθ ψ ) = c exp{l(ˆθ ψ ) l(ˆθ)} j p ( ˆψ) 1/2 j λλ (ˆθ ψ ) 1/2 π(ˆθ) π m (r y) = c exp( 1 2 r 2 ) r q B = c exp( 1 2 r 2 log r ) q B
15 ... approximation to the posterior is really good Example: y i N(θ, I), y i = (y i1,..., y ik ), θ = (θ 1,..., θ k ) ψ = (θ θ2 k ) flat prior π(θ)dθ dθ posterior is proper, in fact is χ 2 k (n y 2 ) { rb = ) } 1 (k 1)/2 n( ˆψ ψ) + n( ˆψ ψ) (ψˆψ log
16 ... normal circle, k=2 normal circle, n=1 normal circle posterior marginal difference psi psi
17 ... normal circle, increasing k changing k increasing k posterior marginal difference k=5 k=10 k= psi psi
18 Are Bayesian and nonbayesian solutions really that different? posterior probability limit Pr θ Y {θ θ (1 α) y} = 1 α upper confidence bound Pr Y θ {θ θ (1 α) (π, Y )} = 1 α first depends on the prior, second is evaluated under the model is there a prior for which both limits are equal? no 2, but there might be a prior for which they are approximately equal: Pr Y θ {θ θ (1 α) (π, Y )} = 1 α + O(1/n) matching prior 2 except in a location model
19 ... matching priors if θ R, then π(θ) i 1/2 (θ) (Welch & Peers, 1963) if θ = (ψ, λ) then π(θ) i 1/2 ψψ (θ)g(λ) (Peers, 1965; Tibshirani, 1989) where we reparametrize if necessary so that i ψλ (θ) = 0 matching priors are one version of objective priors they depend on the model reference priors of Berger and Bernardo are a different version of objective priors with a slightly different goal: maximize the distance from prior to posterior good news: Bayesian inference with matching priors is calibrated bad news: there is a whole family of such priors, even in relatively simple situations
20 ... good news: matching priors are unique when used in the r approximation(diciccio & Martin 93, Staicu 07) ψ π m (ψ y)dψ = Φ(r B ){1 + O(n 3/2 )} r B = r + 1 r log q B r r = [2{l p ( ˆψ) l p (ψ)}] 1/2 q B = l p(ψ)j 1/2 p ( ˆψ) j λλ(ˆθ) 1/2 π(ψ, ˆλ ψ ) j λλ (ˆθ ψ ) 1/2 π( ˆψ, ˆλ) = i 1/2 when ψ orthogonal to λ, ˆλ ψ = ˆλ{1 + O p (n 1 )} ψψ (ψ, ˆλ ψ ) g(ˆλ ψ ) i 1/2 ψψ ( ˆψ, ˆλ) g(ˆλ)
21 Logistic regression The first ten out of 79 sets of observations on the physical characteristics of urine. Presence/absence of calcium oxalate crystals is indicated by 1/0. Two cases with missing values. 3 Case Crystals Specific gravity ph Osmolarity Conductivity Urea Calcium Andrews & Herzberg, 1985
22 ... logistic regression Model: Independent binary responses Y 1,..., Y n with Pr(Y i = 1) = linear exponential family: exp(x T i β) 1 + exp(x T i β) l(β; y) = exp{β 0 Σy i + β 1 Σx 1i y i + + β 6 Σx 6i y i c(β) d(y)} parameter of interest ψ = β 6, say orthogonal nuisance parameter λ j = E(yx j ) ˆλ ψ ˆλ (special to exponential families) π(ψ, λ) i 1/2 ψψ (ψ, λ), unique
23 BN and Laplace approximation beta_p
24 ... logistic regression Confidence/posterior interval for parameter of interest method lower bound upper bound p-value for 0 Φ(r ) Φ(rB ) st order First line uses a (very accurate) approximation to the conditional distribution of Σy i x 6i, given sufficient statistics for nuisance parameters (exponential family) Fitted using cond library of package hoa
25 Bayesian and frequentist inference Examples Logistic regression... logistic regression... logistic regression Confidence/posterior interval for parameter of interest method lower bound upper bound p-value for 0 Φ(r ) Φ(rB ) st order First line uses a (very accurate) approximation to the conditional distribution of Σy ix 6i, given sufficient statistics for nuisance parameters (exponential family) Fitted using cond library of package hoa n = 77, d = 7, but the information in 77 binary observations seems to be comparable to the information in about 10 continuous observations
26 Matching priors need to be targetted on the parameter of interest as do all objective priors flat priors for vector parameters don t lead to calibrated inferences Example: y i N(θ i, 1), i = 1,..., k exact and approximate posterior for θi 2 nearly identical
27 changing k increasing k posterior marginal difference k=5 k=10 k= psi psi
28 Matching priors need to be targetted on the parameter of interest as do all objective priors flat priors for vector parameters don t lead to calibrated inferences Example: y i N(θ i, 1), i = 1,..., k exact and approximate posterior for θ 2 i nearly identical posterior used prior π(θ) 1, marginalize to ψ = θ 2 i posterior limits are not probability limits there is an exact nonbayesian solution, based on marginal density of y 2 i (noncentral χ 2 ) Dawid, Stone, Zidek, 1974
29 dependence on dimension difference dim = 10 dim = 5 dim = psi y1
30 dependence on n difference y1 psi
31 nonbayesian asymptotics find a function of the data with known distribution depending on ψ our candidate: rf = r F (ψ; y) with N(0, 1) distribution r F = r + 1 r log q F r q F = {χ(ˆθ) χ(ˆθ ψ )}/ˆσ χ a maximum likelihood type departure χ(θ) = scalar function of ϕ(θ) θ ϕ(θ) = ϕ(θ; y 0 ): a re-parametrization of the model
32 ... nonbayesian asymptotics where does this come from and why does it work using ϕ we can construct an approximate exponential family model this is the route to accurate density approximation using saddlepoint-type arguments to get ϕ(θ) differentiate the log-likelihood on the sample space ϕ j (θ) = d dt l(θ; y 0 + tv j ) t=0 jiggle the observed data in directions v j, and record the change in the log likelihood gives a way to implement approximate conditioning leads to quite accurate frequentist solutions
33 ... does this help with finding priors? 1. strong matching priors (FR, 2002) posterior marginal distribution s(ψ) = Φ(r B ) nonbayesian p-value function p(ψ) = Φ(r F ) r F = r + 1 r log q F r r B = r + 1 r log q B r Set q F = q B, and solve for prior π (if possible) guarantees matching to 3rd order leads to a data dependent prior automatically targetted on parameter of interest only a function of the parameter of interest
34 Bayesian and frequentist inference Constructing priors... does this help with finding priors?... does this help with finding priors? 1. strong matching priors (FR, 2002) posterior marginal distribution s(ψ) = Φ(rB ) nonbayesian p-value function p(ψ) = Φ(rF ) rf = r + 1 r log q F r rb = r + 1 r log q B r Set q F = q B, and solve for prior π (if possible) guarantees matching to 3rd order leads to a data dependent prior automatically targetted on parameter of interest only a function of the parameter of interest q B = l p(ψ)j 1/2 p ( ˆψ) blue prior q F = {χ(ˆθ) χ(ˆθ ψ )}/ˆσ χ
35 ... does this help 2. approximate location models every model f (y; θ) can be approximated by a location model (locally) location models have automatic prior for vector θ: flat change at θ that generates an increment d ˆθ at observed data, W (θ), say flat prior is π(θ) W (θ) = AV (θ) = d ˆθ dθ V is a n k matrix with ith row V i (θ) = F ;θ (y 0 i ; θ) F y (y 0 i ; θ) gives a natural default prior for vector parameters still need to deal with marginalization to target parameters of interest
36 Bayesian and frequentist inference Constructing priors... does this help 2. approximate location models... does this help every model f (y; θ) can be approximated by a location model (locally) location models have automatic prior for vector θ: flat change at θ that generates an increment d ˆθ at observed data, W (θ), say flat prior is π(θ) W (θ) = AV (θ) = d ˆθ dθ V is a n k matrix with ith row V i(θ) = F;θ (y i 0 ; θ) Fy(yi 0 ; θ) gives a natural default prior for vector parameters still need to deal with marginalization to target parameters of interest approximate location model f (y β(θ)), say factors as f 1 (a)f 2 (ˆθ β(θ)) if f 2 ( ) = 0 then flat prior is β θ (θ) d ˆθ = β θ (θ)dθ
37 Summarizing asymptotic theory is useful for obtaining (very) good approximations to distributions can be used in Bayesian or nonbayesian contexts can be used to derive posterior intervals that are calibrated can be used to simplify the choice of matching priors finding priors is not straightforward likelihood plus first derivative change in the sample space is instrinsic to inference, at least asymptotically to O(n 3/2 ) flat priors for a location parameter will not lead to calibrated posteriors for curved parameters
38 Economist Jan 14, 2006 We re all Bayesians really
39 ... Bayesian Griffiths and Tenenbaum Psychological Science asked subjects to make predictions based on limited evidence concluded that the predictions were consistent with Bayesian methods of inference the results of our experiment reveal a far closer correspondence between optimal statistical inference and everyday cognition than that suggested by previous research
40 ... Bayesian Goldstein (2006): we argue first that the subjectivist Bayes approach is the only feasible method for tackling many scientific problems Neal (1998): Using technological or reference priors chosen solely for convenience, or out of a mis-guided desire for pseudo-objectivity... [or] using priors that vary with the amount of data that you have collected... have no real Bayesian justification, and since they are usually offered with no other justification either, I consider them to be highly dubious. Wasserman (2006): I think it would be hard to defend a sequence of subjective analyses that yield poor frequency coverage
41 References 1. ba.stat.cmu Volume 2 # 3 Berger 2006, Goldstein 2006, Wasserman DiCiccio and Martin Fraser Reid Little Tibshirani Welch and Peers 1963
Applied Asymptotics Case studies in higher order inference
Applied Asymptotics Case studies in higher order inference Nancy Reid May 18, 2006 A.C. Davison, A. R. Brazzale, A. M. Staicu Introduction likelihood-based inference in parametric models higher order approximations
More informationDefault priors and model parametrization
1 / 16 Default priors and model parametrization Nancy Reid O-Bayes09, June 6, 2009 Don Fraser, Elisabeta Marras, Grace Yun-Yi 2 / 16 Well-calibrated priors model f (y; θ), F(y; θ); log-likelihood l(θ)
More informationLast week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36
Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) =
More informationLikelihood inference in the presence of nuisance parameters
Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Light touch assessment One or two problems assigned weekly graded during Reading Week http://www.utstat.toronto.edu/reid/4508s14.html
More informationApproximating models. Nancy Reid, University of Toronto. Oxford, February 6.
Approximating models Nancy Reid, University of Toronto Oxford, February 6 www.utstat.utoronto.reid/research 1 1. Context Likelihood based inference model f(y; θ), log likelihood function l(θ; y) y = (y
More informationNancy Reid SS 6002A Office Hours by appointment
Nancy Reid SS 6002A reid@utstat.utoronto.ca Office Hours by appointment Problems assigned weekly, due the following week http://www.utstat.toronto.edu/reid/4508s16.html Various types of likelihood 1. likelihood,
More informationDefault priors for Bayesian and frequentist inference
Default priors for Bayesian and frequentist inference D.A.S. Fraser and N. Reid University of Toronto, Canada E. Marras Centre for Advanced Studies and Development, Sardinia University of Rome La Sapienza,
More informationLikelihood Inference in the Presence of Nuisance Parameters
PHYSTAT2003, SLAC, September 8-11, 2003 1 Likelihood Inference in the Presence of Nuance Parameters N. Reid, D.A.S. Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationStaicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors.
Staicu, A-M., & Reid, N. (2007). On the uniqueness of probability matching priors. Early version, also known as pre-print Link to publication record in Explore Bristol Research PDF-document University
More informationLikelihood Inference in the Presence of Nuisance Parameters
Likelihood Inference in the Presence of Nuance Parameters N Reid, DAS Fraser Department of Stattics, University of Toronto, Toronto Canada M5S 3G3 We describe some recent approaches to likelihood based
More informationMeasuring nuisance parameter effects in Bayesian inference
Measuring nuisance parameter effects in Bayesian inference Alastair Young Imperial College London WHOA-PSI-2017 1 / 31 Acknowledgements: Tom DiCiccio, Cornell University; Daniel Garcia Rasines, Imperial
More informationASYMPTOTICS AND THE THEORY OF INFERENCE
ASYMPTOTICS AND THE THEORY OF INFERENCE N. Reid University of Toronto Abstract Asymptotic analysis has always been very useful for deriving distributions in statistics in cases where the exact distribution
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationApproximate Inference for the Multinomial Logit Model
Approximate Inference for the Multinomial Logit Model M.Rekkas Abstract Higher order asymptotic theory is used to derive p-values that achieve superior accuracy compared to the p-values obtained from traditional
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid February 26, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Nancy Reid Accurate directional inference for vector parameters York University
More informationASSESSING A VECTOR PARAMETER
SUMMARY ASSESSING A VECTOR PARAMETER By D.A.S. Fraser and N. Reid Department of Statistics, University of Toronto St. George Street, Toronto, Canada M5S 3G3 dfraser@utstat.toronto.edu Some key words. Ancillary;
More informationBayesian Asymptotics
BS2 Statistical Inference, Lecture 8, Hilary Term 2008 May 7, 2008 The univariate case The multivariate case For large λ we have the approximation I = b a e λg(y) h(y) dy = e λg(y ) h(y ) 2π λg (y ) {
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,
More informationLikelihood and Asymptotic Theory for Statistical Inference
Likelihood and Asymptotic Theory for Statistical Inference Nancy Reid 020 7679 1863 reid@utstat.utoronto.ca n.reid@ucl.ac.uk http://www.utstat.toronto.edu/reid/ltccf12.html LTCC Likelihood Theory Week
More informationModern likelihood inference for the parameter of skewness: An application to monozygotic
Working Paper Series, N. 10, December 2013 Modern likelihood inference for the parameter of skewness: An application to monozygotic twin studies Mameli Valentina Department of Mathematics and Computer
More informationDEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx ISSN 0256-422 X DEFNITIVE TESTING OF AN INTEREST PARAMETER: USING PARAMETER CONTINUITY D. A. S. FRASER Department of Statistical Sciences,
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationPrinciples of Statistical Inference
Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical
More informationA Very Brief Summary of Bayesian Inference, and Examples
A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X
More informationNoninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions
Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted
More informationPARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE.
Journal of Statistical Research 200x, Vol. xx, No. xx, pp. xx-xx Bangladesh ISSN 0256-422 X PARAMETER CURVATURE REVISITED AND THE BAYES-FREQUENTIST DIVERGENCE. A.M. FRASER Department of Mathematics, University
More informationSome Curiosities Arising in Objective Bayesian Analysis
. Some Curiosities Arising in Objective Bayesian Analysis Jim Berger Duke University Statistical and Applied Mathematical Institute Yale University May 15, 2009 1 Three vignettes related to John s work
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationMarginal Posterior Simulation via Higher-order Tail Area Approximations
Bayesian Analysis (2014) 9, Number 1, pp. 129 146 Marginal Posterior Simulation via Higher-order Tail Area Approximations Erlis Ruli, Nicola Sartori and Laura Ventura Abstract. A new method for posterior
More informationImproved Inference for First Order Autocorrelation using Likelihood Analysis
Improved Inference for First Order Autocorrelation using Likelihood Analysis M. Rekkas Y. Sun A. Wong Abstract Testing for first-order autocorrelation in small samples using the standard asymptotic test
More informationThird-order inference for autocorrelation in nonlinear regression models
Third-order inference for autocorrelation in nonlinear regression models P. E. Nguimkeu M. Rekkas Abstract We propose third-order likelihood-based methods to derive highly accurate p-value approximations
More informationCOMBINING p-values: A DEFINITIVE PROCESS. Galley
0 Journal of Statistical Research ISSN 0 - X 00, Vol., No., pp. - Bangladesh COMBINING p-values: A DEFINITIVE PROCESS D.A.S. Fraser Department of Statistics, University of Toronto, Toronto, Canada MS G
More informationMore on nuisance parameters
BS2 Statistical Inference, Lecture 3, Hilary Term 2009 January 30, 2009 Suppose that there is a minimal sufficient statistic T = t(x ) partitioned as T = (S, C) = (s(x ), c(x )) where: C1: the distribution
More informationBayesian Model Comparison
BS2 Statistical Inference, Lecture 11, Hilary Term 2009 February 26, 2009 Basic result An accurate approximation Asymptotic posterior distribution An integral of form I = b a e λg(y) h(y) dy where h(y)
More informationIntroduction to Bayesian Methods
Introduction to Bayesian Methods Jessi Cisewski Department of Statistics Yale University Sagan Summer Workshop 2016 Our goal: introduction to Bayesian methods Likelihoods Priors: conjugate priors, non-informative
More informationPart III. A Decision-Theoretic Approach and Bayesian testing
Part III A Decision-Theoretic Approach and Bayesian testing 1 Chapter 10 Bayesian Inference as a Decision Problem The decision-theoretic framework starts with the following situation. We would like to
More informationLikelihood and p-value functions in the composite likelihood context
Likelihood and p-value functions in the composite likelihood context D.A.S. Fraser and N. Reid Department of Statistical Sciences University of Toronto November 19, 2016 Abstract The need for combining
More informationANCILLARY STATISTICS: A REVIEW
Statistica Sinica 20 (2010), 1309-1332 ANCILLARY STATISTICS: A REVIEW M. Ghosh 1, N. Reid 2 and D. A. S. Fraser 2 1 University of Florida and 2 University of Toronto Abstract: In a parametric statistical
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationhoa: An R Package Bundle for Higher Order Likelihood Inference
hoa: An R Package Bundle for Higher Order Likelihood Inference by Alessandra R. Brazzale Rnews, 5/1 May 2005, pp. 20 27 Introduction The likelihood function represents the basic ingredient of many commonly
More informationSpatial Statistics Chapter 4 Basics of Bayesian Inference and Computation
Spatial Statistics Chapter 4 Basics of Bayesian Inference and Computation So far we have discussed types of spatial data, some basic modeling frameworks and exploratory techniques. We have not discussed
More informationLinear Models A linear model is defined by the expression
Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose
More informationBFF Four: Are we Converging?
BFF Four: Are we Converging? Nancy Reid May 2, 2017 Classical Approaches: A Look Way Back Nature of Probability BFF one to three: a look back Comparisons Are we getting there? BFF Four Harvard, May 2017
More informationBootstrap and Parametric Inference: Successes and Challenges
Bootstrap and Parametric Inference: Successes and Challenges G. Alastair Young Department of Mathematics Imperial College London Newton Institute, January 2008 Overview Overview Review key aspects of frequentist
More informationLikelihood based Statistical Inference. Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona
Likelihood based Statistical Inference Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona L. Pace, A. Salvan, N. Sartori Udine, April 2008 Likelihood: observed quantities,
More informationANCILLARY STATISTICS: A REVIEW
1 ANCILLARY STATISTICS: A REVIEW M. Ghosh, N. Reid and D.A.S. Fraser University of Florida and University of Toronto Abstract: In a parametric statistical model, a function of the data is said to be ancillary
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationBayesian Inference in Astronomy & Astrophysics A Short Course
Bayesian Inference in Astronomy & Astrophysics A Short Course Tom Loredo Dept. of Astronomy, Cornell University p.1/37 Five Lectures Overview of Bayesian Inference From Gaussians to Periodograms Learning
More informationMarginal posterior simulation via higher-order tail area approximations
Bayesian Analysis (2004) 1, Number 1, pp. 1 13 Marginal posterior simulation via higher-order tail area approximations Erlis Ruli, Nicola Sartori and Laura Ventura Abstract. A new method for posterior
More information1 Hypothesis Testing and Model Selection
A Short Course on Bayesian Inference (based on An Introduction to Bayesian Analysis: Theory and Methods by Ghosh, Delampady and Samanta) Module 6: From Chapter 6 of GDS 1 Hypothesis Testing and Model Selection
More informationA simple analysis of the exact probability matching prior in the location-scale model
A simple analysis of the exact probability matching prior in the location-scale model Thomas J. DiCiccio Department of Social Statistics, Cornell University Todd A. Kuffner Department of Mathematics, Washington
More informationAspects of Likelihood Inference
Submitted to the Bernoulli Aspects of Likelihood Inference NANCY REID 1 1 Department of Statistics University of Toronto 100 St. George St. Toronto, Canada M5S 3G3 E-mail: reid@utstat.utoronto.ca, url:
More informationNuisance parameters and their treatment
BS2 Statistical Inference, Lecture 2, Hilary Term 2008 April 2, 2008 Ancillarity Inference principles Completeness A statistic A = a(x ) is said to be ancillary if (i) The distribution of A does not depend
More informationFREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE
FREQUENTIST BEHAVIOR OF FORMAL BAYESIAN INFERENCE Donald A. Pierce Oregon State Univ (Emeritus), RERF Hiroshima (Retired), Oregon Health Sciences Univ (Adjunct) Ruggero Bellio Univ of Udine For Perugia
More information1. Fisher Information
1. Fisher Information Let f(x θ) be a density function with the property that log f(x θ) is differentiable in θ throughout the open p-dimensional parameter set Θ R p ; then the score statistic (or score
More informationImproved Inference for Moving Average Disturbances in Nonlinear Regression Models
Improved Inference for Moving Average Disturbances in Nonlinear Regression Models Pierre Nguimkeu Georgia State University November 22, 2013 Abstract This paper proposes an improved likelihood-based method
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationModule 22: Bayesian Methods Lecture 9 A: Default prior selection
Module 22: Bayesian Methods Lecture 9 A: Default prior selection Peter Hoff Departments of Statistics and Biostatistics University of Washington Outline Jeffreys prior Unit information priors Empirical
More informationOverall Objective Priors
Overall Objective Priors Jim Berger, Jose Bernardo and Dongchu Sun Duke University, University of Valencia and University of Missouri Recent advances in statistical inference: theory and case studies University
More informationAn Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models
An Improved Specification Test for AR(1) versus MA(1) Disturbances in Linear Regression Models Pierre Nguimkeu Georgia State University Abstract This paper proposes an improved likelihood-based method
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationLecture 2: Statistical Decision Theory (Part I)
Lecture 2: Statistical Decision Theory (Part I) Hao Helen Zhang Hao Helen Zhang Lecture 2: Statistical Decision Theory (Part I) 1 / 35 Outline of This Note Part I: Statistics Decision Theory (from Statistical
More informationBayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University
Bayesian Inference and the Parametric Bootstrap Bradley Efron Stanford University Importance Sampling for Bayes Posterior Distribution Newton and Raftery (1994 JRSS-B) Nonparametric Bootstrap: good choice
More informationA BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain
A BAYESIAN MATHEMATICAL STATISTICS PRIMER José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es Bayesian Statistics is typically taught, if at all, after a prior exposure to frequentist
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationDecision theory. 1 We may also consider randomized decision rules, where δ maps observed data D to a probability distribution over
Point estimation Suppose we are interested in the value of a parameter θ, for example the unknown bias of a coin. We have already seen how one may use the Bayesian method to reason about θ; namely, we
More informationNew Bayesian methods for model comparison
Back to the future New Bayesian methods for model comparison Murray Aitkin murray.aitkin@unimelb.edu.au Department of Mathematics and Statistics The University of Melbourne Australia Bayesian Model Comparison
More informationSTAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01
STAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01 Nasser Sadeghkhani a.sadeghkhani@queensu.ca There are two main schools to statistical inference: 1-frequentist
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationSeminar über Statistik FS2008: Model Selection
Seminar über Statistik FS2008: Model Selection Alessia Fenaroli, Ghazale Jazayeri Monday, April 2, 2008 Introduction Model Choice deals with the comparison of models and the selection of a model. It can
More informationInvariant HPD credible sets and MAP estimators
Bayesian Analysis (007), Number 4, pp. 681 69 Invariant HPD credible sets and MAP estimators Pierre Druilhet and Jean-Michel Marin Abstract. MAP estimators and HPD credible sets are often criticized in
More informationSTAT 830 Bayesian Estimation
STAT 830 Bayesian Estimation Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Bayesian Estimation STAT 830 Fall 2011 1 / 23 Purposes of These
More informationStat 535 C - Statistical Computing & Monte Carlo Methods. Arnaud Doucet.
Stat 535 C - Statistical Computing & Monte Carlo Methods Arnaud Doucet Email: arnaud@cs.ubc.ca 1 Suggested Projects: www.cs.ubc.ca/~arnaud/projects.html First assignement on the web: capture/recapture.
More informationThe formal relationship between analytic and bootstrap approaches to parametric inference
The formal relationship between analytic and bootstrap approaches to parametric inference T.J. DiCiccio Cornell University, Ithaca, NY 14853, U.S.A. T.A. Kuffner Washington University in St. Louis, St.
More informationOverview. Probabilistic Interpretation of Linear Regression Maximum Likelihood Estimation Bayesian Estimation MAP Estimation
Overview Probabilistic Interpretation of Linear Regression Maximum Likelihood Estimation Bayesian Estimation MAP Estimation Probabilistic Interpretation: Linear Regression Assume output y is generated
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationBayesian Statistics from Subjective Quantum Probabilities to Objective Data Analysis
Bayesian Statistics from Subjective Quantum Probabilities to Objective Data Analysis Luc Demortier The Rockefeller University Informal High Energy Physics Seminar Caltech, April 29, 2008 Bayes versus Frequentism...
More informationTHE FORMAL DEFINITION OF REFERENCE PRIORS
Submitted to the Annals of Statistics THE FORMAL DEFINITION OF REFERENCE PRIORS By James O. Berger, José M. Bernardo, and Dongchu Sun Duke University, USA, Universitat de València, Spain, and University
More informationTwo examples of the use of fuzzy set theory in statistics. Glen Meeden University of Minnesota.
Two examples of the use of fuzzy set theory in statistics Glen Meeden University of Minnesota http://www.stat.umn.edu/~glen/talks 1 Fuzzy set theory Fuzzy set theory was introduced by Zadeh in (1965) as
More information7. Estimation and hypothesis testing. Objective. Recommended reading
7. Estimation and hypothesis testing Objective In this chapter, we show how the election of estimators can be represented as a decision problem. Secondly, we consider the problem of hypothesis testing
More informationParameter Estimation
Parameter Estimation Chapters 13-15 Stat 477 - Loss Models Chapters 13-15 (Stat 477) Parameter Estimation Brian Hartman - BYU 1 / 23 Methods for parameter estimation Methods for parameter estimation Methods
More informationDepartment of Statistics
Research Report Department of Statistics Research Report Department of Statistics No. 208:4 A Classroom Approach to the Construction of Bayesian Credible Intervals of a Poisson Mean No. 208:4 Per Gösta
More informationChapter 5. Bayesian Statistics
Chapter 5. Bayesian Statistics Principles of Bayesian Statistics Anything unknown is given a probability distribution, representing degrees of belief [subjective probability]. Degrees of belief [subjective
More informationExponential Models: Approximations for Probabilities
JIRSS (2011) Vol. 10, No. 2, pp 95-107 Exponential Models: Approximations for Probabilities D. A. S. Fraser 1,2,A.Naderi 3, Kexin Ji 1,WeiLin 1, Jie Su 1 1 Department of Statistics, University of Toronto,
More informationBayesian tests of hypotheses
Bayesian tests of hypotheses Christian P. Robert Université Paris-Dauphine, Paris & University of Warwick, Coventry Joint work with K. Kamary, K. Mengersen & J. Rousseau Outline Bayesian testing of hypotheses
More informationLecture 1: Introduction
Principles of Statistics Part II - Michaelmas 208 Lecturer: Quentin Berthet Lecture : Introduction This course is concerned with presenting some of the mathematical principles of statistical theory. One
More informationStandard Errors & Confidence Intervals. N(0, I( β) 1 ), I( β) = [ 2 l(β, φ; y) β i β β= β j
Standard Errors & Confidence Intervals β β asy N(0, I( β) 1 ), where I( β) = [ 2 l(β, φ; y) ] β i β β= β j We can obtain asymptotic 100(1 α)% confidence intervals for β j using: β j ± Z 1 α/2 se( β j )
More informationEmpirical Likelihood Based Deviance Information Criterion
Empirical Likelihood Based Deviance Information Criterion Yin Teng Smart and Safe City Center of Excellence NCS Pte Ltd June 22, 2016 Outline Bayesian empirical likelihood Definition Problems Empirical
More informationLECTURE 5 NOTES. n t. t Γ(a)Γ(b) pt+a 1 (1 p) n t+b 1. The marginal density of t is. Γ(t + a)γ(n t + b) Γ(n + a + b)
LECTURE 5 NOTES 1. Bayesian point estimators. In the conventional (frequentist) approach to statistical inference, the parameter θ Θ is considered a fixed quantity. In the Bayesian approach, it is considered
More informationApproximate Likelihoods
Approximate Likelihoods Nancy Reid July 28, 2015 Why likelihood? makes probability modelling central l(θ; y) = log f (y; θ) emphasizes the inverse problem of reasoning y θ converts a prior probability
More informationLikelihood and Fairness in Multidimensional Item Response Theory
Likelihood and Fairness in Multidimensional Item Response Theory or What I Thought About On My Holidays Giles Hooker and Matthew Finkelman Cornell University, February 27, 2008 Item Response Theory Educational
More informationAccurate directional inference for vector parameters in linear exponential families
Accurate directional inference for vector parameters in linear exponential families A. C. Davison, D. A. S. Fraser, N. Reid and N. Sartori August 27, 2013 Abstract We consider inference on a vector-valued
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationIntegrated likelihoods in survival models for highlystratified
Working Paper Series, N. 1, January 2014 Integrated likelihoods in survival models for highlystratified censored data Giuliana Cortese Department of Statistical Sciences University of Padua Italy Nicola
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 425: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 2017 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 2017 1 / 10 Lecture 7: Prior Types Subjective
More informationIntegrated Objective Bayesian Estimation and Hypothesis Testing
Integrated Objective Bayesian Estimation and Hypothesis Testing José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es 9th Valencia International Meeting on Bayesian Statistics Benidorm
More information