On the Conditional Distribution of the Multivariate t Distribution

Size: px
Start display at page:

Download "On the Conditional Distribution of the Multivariate t Distribution"

Transcription

1 On the Conditional Distribution of the Multivariate t Distribution arxiv: v [math.st] 2 Apr 206 Peng Ding Abstract As alternatives to the normal distributions, t distributions are widely applied in robust analysis for data with outliers or heavy tails. The properties of the multivariate t distribution are well documented in Kotz and Nadarajah s book, which, however, states a wrong conclusion about the conditional distribution of the multivariate t distribution. Previous literature has recognized that the conditional distribution of the multivariate t distribution also follows the multivariate t distribution. We provide an intuitive proof without directly manipulating the complicated density function of the multivariate t distribution. Key Words: Bayes Theorem; Data augmentation; Mahalanobis distance; Normal mixture; Representation. Introduction The conventional version of the multivariate t MVT distribution X t p µ,σ,ν, with location µ, scale matrix Σ, and degrees of freedom ν, has the probability density function fx= Γν+p/2} ν+p/2. Γν/2νπ p/2 Σ /2 +ν x µ Σ x µ} We can see from the above that the tail probability of the MVT distribution decays at a polynomial rate, resulting in heavier tails than the multivariate normal distribution. Because of this property, the MVT distribution is widely applied in robust data analysis including linear and nonlinear regressions Lange et al. 988; Liu 994; Liu 2004, linear mixed effects models Pinheiro et al. 200, and sample selection models Marchenko and Genton 202; Ding 204. Peng Ding is Assistant Professor pengdingpku@gmail.com, Department of Statistics, University of California, Berkeley, 425 Evans Hall, Berkeley, CA 94720, USA. The author thanks the Editor, Associate Editor and three reviewers for their helpful comments.

2 In the following discussion, we use W χ 2 b /c to denote the scaled χ2 distribution, with density proportional to w b/2 e cw/2. We exploit the following representation of the MVT distribution: X =µ+σ /2 Z/ q, 2 where Z follows a p dimensional standard normal distribution, q χ 2 ν/ν, and Z is independent of q Kotz and Nadarajah 2004; Nadarajah and Kotz It differs from the multivariate normal distribution N p µ,σ only by the random scaling factor q. The above representation 2 implies X q N p µ,σ/q, i.e., X follows a multivariate normal distribution given the latent variable q. If we partition X into two parts, X andx 2, with dimensions p and p 2, we obtain the following normal mixture representation conditional on q: X = X X 2 µ Σ Σ q N p +p 2, 2 µ 2 Σ 2 Σ 22 } /q, 3 where the location and scale parameters are partitioned corresponding to the partition of X. Marginally, we have X q N p µ,σ /q, and therefore X t p µ,σ,ν, which follows a p dimensional MVT distribution with degrees of freedom ν. Although we can obtain the marginal distribution in an obvious way, the conditional distribution ofx 2 givenx is less transparent. In fact, the conditional distribution of the MVT distribution is also a MVT distribution, with degrees of freedom different from the original distribution. 2 Conditional Distribution Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 claimed that the conditional distribution of the MVT distribution is not a MVT distribution except for some extreme cases. For the case with µ = 0, define x 2 = x Σ 2 Σ x, and define Σ 22 = Σ 22 Σ 2 Σ Σ 2 as the Schur complement of the block Σ in matrix Σ. They derived the conditional density of X 2 given 2

3 X by calculating fx/ f x using the probability density function in. Ignoring the normalizing constant in formula 5 of Nadarajah and Kotz 2005, we present only the key term: f 2 x 2 x +ν+p x ν+x Σ x 2 Σ ν+p 22 x 2 ν+p +p 2 /2. 4 Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 obtained the correct conditional density function, but due to the complex form of their formula 5, they did not recognize that the key term in 4 is essentially the unnormalized probability density function of a p 2 dimensional MVT distribution with location Σ 2 Σ x, scale matrix ν+x Σ x /ν+p Σ 22, and degrees of freedom ν+p. In the literature, some authors stated the right conclusion about the conditional distribution without providing a proof e.g., Liu 994, page 5, DeGroot 2005, page 6, and some other authors re-derived the correct conditional density function in 4 and pointed out that it is another MVT distribution e.g., Roth 203b. Instead of directly calculating the conditional density or using the general theory of elliptically contoured distributions Cambanis et al. 98; Fang et al. 990, Theorems 2.8 and 3.8; Kibria and Joarder 2006, we provide an elementary and thus more transparent proof of the conditional distribution based on the normal mixture representations in 3. 3 Proof via Representation Our proof proceeds in two steps: we first condition on q, and then average over q. First, we condition on both X and q. According to the property of the multivariate normal distribution, X 2 follows a multivariate normal distribution conditional on X,q, i.e., X 2 X,q N p2 µ2 +Σ 2 Σ X µ,σ 22 Σ 2 Σ Σ 2/q } = N p2 µ2,σ 22 /q, where µ 2 =µ 2 +Σ 2 Σ X µ is the linear regression ofx 2 on X. Second, we obtain the conditional distribution of q given X. According to Bayes Theorem, the 3

4 conditional probability density function of q given X is f q q X f q X qπq Σ /q /2 exp q } 2 X µ Σ X µ q ν/2 e νq/2 q ν+p/2 exp q } 2 ν+ d, where d = X µ Σ X µ is the squared Mahalanobis distance of X from µ with scale matrix Σ. Therefore, the conditional distribution of q given X is q X χ 2 ν+p /ν+ d, according to the density of the scaled χ 2 distribution. Finally, we can represent the conditional distribution ofx 2 given X as X 2 X µ 2 +Σ /2 22 Z 2/ q X / µ 2 +Σ /2 22 Z χν+p 2 2 ν+ d ν+ d / µ 2 + Σ /2 χν+p ν+p 22 Z 2 2, 5 ν+p where Z 2 is a p 2 dimensional standard multivariate normal vector, independent of χ 2 ν+p /ν + p. From the normal mixture representation of the MVT distribution in 2, we can obtain the following conditional distribution. Conclusion One: The conditional distribution of X 2 given X is X 2 X t p2 µ 2, ν+ d Σ ν+p 22,ν+p. The conditional distribution of the multivariate t distribution is very similar to that of the multivariate normal distribution. The conditional location parameter is the linear regression of X 2 on X. The conditional scale matrix is Σ 22 inflated or deflated by the factor ν+ d /ν + p. Because d p Fp,ν with mean being p ν/ν 2 or infinity according to ν > 2 or ν 2 Roth 203a, page 83, on average the factor ν+ d /ν + p is larger than one. With more extreme values of X, the conditional distributions ofx 2 are more disperse. More interestingly, the conditional degrees 4

5 of freedom increase to ν + p. The more dimensions we condition on, the less heavy-tailedness we have. When ν =, the MVT distribution is also called the multivariate Cauchy distribution. The marginal distributions are still Cauchy, but the conditional distributions are MVT instead of Cauchy. Although the marginal means of the multivariate Cauchy distribution do not exist for any dimensions, all the conditional means exist because the conditional distributions of the multivariate Cauchy distribution follow MVT distributions with degrees of freedom at least as large as two. As a byproduct, we can easily obtain from the representation 5 that ν+p ν+ d X2 µ 2 X t p2 0,Σ22,ν+p, which does not depend on X. The above result further implies the following conclusion. Conclusion Two: X and ν+p ν+d X2 µ 2 are independent. It is straightforward to show that X and X 2 µ 2 are uncorrelated for both multivariate normal and MVT distributions. If X follows the multivariate normal distribution, this also implies independence between X and X 2 µ 2. If X follows the MVT distribution, we need only to adjust the linear regression residual X 2 µ 2 by the factor ν+p /ν+ d and the independence result still holds. 4 Discussion Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 failed to recognize that the conditional distribution of the MVT distribution is also a MVT distribution due to the complexity of the conditional density function. Our proof, based on the normal mixture representation of the MVT distribution, offers a more direct and transparent way to revealing the property of the conditional distribution of the MVT distribution. Our representation in Section 3 implicitly exploits the data augmentation formula f 2 x 2 x = f 2,q x 2 x,q f q q x dq Tanner and Wong 987. To make our proof more intuitive, we used formula 5 to avoid the integral, which is essentially a Monte 5

6 Carlo version of the data augmentation formula. Conclusion One could also follow from the general theory of elliptically contoured distributions, including the MVT distribution as a special case Cambanis et al. 980; Fang et al Conditional distributions of elliptically contoured distributions are also elliptically contoured distributions. But this does not immediately guarantee that conditional distributions of the MVT distributions are also MVT distributions without some further algebra. The MVT distribution has the property of having MVT marginal and conditional distributions. It will be interesting to find other sub-classes of elliptically contoured distributions that have the same property. References Cambanis, S., Huang, S. and Simons, G. 98. On the theory of elliptically contoured distributions. Journal of Multivariate Analysis,, Ding, P Bayesian robust inference of sample selection using selection-t models. Journal of Multivariate Analysis, 24, DeGroot, M. H Optimal Statistical Decisions. New York: John Wiley & Sons. Fang, K. T., Kotz, S., and Ng, K. W Symmetric Multivariate and Related Distributions. London: Chapman & Hall. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D., Vehtari, A., and Rubin, D. B Bayesian Data Analysis, 2nd edition. London: Chapman & Hall/CRC. Kibria, B. G., and Joarder, A. H A short review of multivariate t-distribution. Journal of Statistical Research, 40, Kotz, S. and Nadarajah, S Multivariate t Distributions and Their Applications. New York: Cambridge University Press. Lange, K. L., Little, R. J., and Taylor, J. M Robust statistical modeling using the t distribution. 6

7 Journal of the American Statistical Association, 84, Liu, C Statistical Analysis Using the Multivariate t Distribution. Doctoral dissertation, Harvard University. Liu, C Robit regression: a simple robust alternative to logistic and probit regression. In Gelman, A. and Meng, X.-L. Eds. Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives pp New York: John Wiley & Sons. Marchenko, Y. V., and Genton, M. G A Heckman selection-t model. Journal of the American Statistical Association, 07, Nadarajah, S. and Kotz, S Mathematical properties of the multivariate t distribution. Acta Applicandae Mathematica, 89, Pinheiro, J. C., Liu, C., and Wu, Y. N Efficient algorithms for robust estimation in linear mixed-effects models using the multivariate t distribution. Journal of Computational and Graphical Statistics, 0, Roth, M. 203a. Kalman Filters for Nonlinear Systems and Heavy-Tailed Noise, Licentiate of Engineering Thesis, Linköping University, Linköping. Roth, M. 203b. On the multivariate t distribution. Technical report from Automatic Control at Linköpings Universitet. Report no.: LiTH-ISY-R Tanner, M. A., and Wong, W. H The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82,

Bayesian Inference for the Multivariate Normal

Bayesian Inference for the Multivariate Normal Bayesian Inference for the Multivariate Normal Will Penny Wellcome Trust Centre for Neuroimaging, University College, London WC1N 3BG, UK. November 28, 2014 Abstract Bayesian inference for the multivariate

More information

RECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T DISTRIBUTION

RECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T DISTRIBUTION 1 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 3 6, 1, SANTANDER, SPAIN RECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T

More information

Online tests of Kalman filter consistency

Online tests of Kalman filter consistency Tampere University of Technology Online tests of Kalman filter consistency Citation Piché, R. (216). Online tests of Kalman filter consistency. International Journal of Adaptive Control and Signal Processing,

More information

Tools for Parameter Estimation and Propagation of Uncertainty

Tools for Parameter Estimation and Propagation of Uncertainty Tools for Parameter Estimation and Propagation of Uncertainty Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu Outline Models, parameters, parameter estimation,

More information

On Bayesian Inference with Conjugate Priors for Scale Mixtures of Normal Distributions

On Bayesian Inference with Conjugate Priors for Scale Mixtures of Normal Distributions Journal of Applied Probability & Statistics Vol. 5, No. 1, xxx xxx c 2010 Dixie W Publishing Corporation, U. S. A. On Bayesian Inference with Conjugate Priors for Scale Mixtures of Normal Distributions

More information

Bayesian linear regression

Bayesian linear regression Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding

More information

Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements

Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Jeffrey N. Rouder Francis Tuerlinckx Paul L. Speckman Jun Lu & Pablo Gomez May 4 008 1 The Weibull regression model

More information

Discussion on Fygenson (2007, Statistica Sinica): a DS Perspective

Discussion on Fygenson (2007, Statistica Sinica): a DS Perspective 1 Discussion on Fygenson (2007, Statistica Sinica): a DS Perspective Chuanhai Liu Purdue University 1. Introduction In statistical analysis, it is important to discuss both uncertainty due to model choice

More information

σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =

σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) = Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,

More information

STA 216, GLM, Lecture 16. October 29, 2007

STA 216, GLM, Lecture 16. October 29, 2007 STA 216, GLM, Lecture 16 October 29, 2007 Efficient Posterior Computation in Factor Models Underlying Normal Models Generalized Latent Trait Models Formulation Genetic Epidemiology Illustration Structural

More information

eqr094: Hierarchical MCMC for Bayesian System Reliability

eqr094: Hierarchical MCMC for Bayesian System Reliability eqr094: Hierarchical MCMC for Bayesian System Reliability Alyson G. Wilson Statistical Sciences Group, Los Alamos National Laboratory P.O. Box 1663, MS F600 Los Alamos, NM 87545 USA Phone: 505-667-9167

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information

Parameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1

Parameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1 Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data

More information

Testing Restrictions and Comparing Models

Testing Restrictions and Comparing Models Econ. 513, Time Series Econometrics Fall 00 Chris Sims Testing Restrictions and Comparing Models 1. THE PROBLEM We consider here the problem of comparing two parametric models for the data X, defined by

More information

Research Article The Laplace Likelihood Ratio Test for Heteroscedasticity

Research Article The Laplace Likelihood Ratio Test for Heteroscedasticity International Mathematics and Mathematical Sciences Volume 2011, Article ID 249564, 7 pages doi:10.1155/2011/249564 Research Article The Laplace Likelihood Ratio Test for Heteroscedasticity J. Martin van

More information

STATISTICAL INFERENCE WITH DATA AUGMENTATION AND PARAMETER EXPANSION

STATISTICAL INFERENCE WITH DATA AUGMENTATION AND PARAMETER EXPANSION STATISTICAL INFERENCE WITH arxiv:1512.00847v1 [math.st] 2 Dec 2015 DATA AUGMENTATION AND PARAMETER EXPANSION Yannis G. Yatracos Faculty of Communication and Media Studies Cyprus University of Technology

More information

Bagging During Markov Chain Monte Carlo for Smoother Predictions

Bagging During Markov Chain Monte Carlo for Smoother Predictions Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods

More information

Bayesian Regression Linear and Logistic Regression

Bayesian Regression Linear and Logistic Regression When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we

More information

Uniform Correlation Mixture of Bivariate Normal Distributions and. Hypercubically-contoured Densities That Are Marginally Normal

Uniform Correlation Mixture of Bivariate Normal Distributions and. Hypercubically-contoured Densities That Are Marginally Normal Uniform Correlation Mixture of Bivariate Normal Distributions and Hypercubically-contoured Densities That Are Marginally Normal Kai Zhang Department of Statistics and Operations Research University of

More information

Bayesian data analysis in practice: Three simple examples

Bayesian data analysis in practice: Three simple examples Bayesian data analysis in practice: Three simple examples Martin P. Tingley Introduction These notes cover three examples I presented at Climatea on 5 October 0. Matlab code is available by request to

More information

Quantile POD for Hit-Miss Data

Quantile POD for Hit-Miss Data Quantile POD for Hit-Miss Data Yew-Meng Koh a and William Q. Meeker a a Center for Nondestructive Evaluation, Department of Statistics, Iowa State niversity, Ames, Iowa 50010 Abstract. Probability of detection

More information

Chris Bishop s PRML Ch. 8: Graphical Models

Chris Bishop s PRML Ch. 8: Graphical Models Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular

More information

Bayesian Inference. p(y)

Bayesian Inference. p(y) Bayesian Inference There are different ways to interpret a probability statement in a real world setting. Frequentist interpretations of probability apply to situations that can be repeated many times,

More information

A Note on Bayesian Inference After Multiple Imputation

A Note on Bayesian Inference After Multiple Imputation A Note on Bayesian Inference After Multiple Imputation Xiang Zhou and Jerome P. Reiter Abstract This article is aimed at practitioners who plan to use Bayesian inference on multiplyimputed datasets in

More information

Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo

Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo Michael Betancourt arxiv:1604.00695v1 [stat.me] 3 Apr 2016 Abstract. When properly tuned, Hamiltonian Monte Carlo scales to some

More information

Contents. Part I: Fundamentals of Bayesian Inference 1

Contents. Part I: Fundamentals of Bayesian Inference 1 Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method

Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Madeleine B. Thompson Radford M. Neal Abstract The shrinking rank method is a variation of slice sampling that is efficient at

More information

The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study

The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study MATEMATIKA, 2012, Volume 28, Number 1, 35 48 c Department of Mathematics, UTM. The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study 1 Nahdiya Zainal Abidin, 2 Mohd Bakri Adam and 3 Habshah

More information

Some Approximations of the Logistic Distribution with Application to the Covariance Matrix of Logistic Regression

Some Approximations of the Logistic Distribution with Application to the Covariance Matrix of Logistic Regression Working Paper 2013:9 Department of Statistics Some Approximations of the Logistic Distribution with Application to the Covariance Matrix of Logistic Regression Ronnie Pingel Working Paper 2013:9 June

More information

Markov Chain Monte Carlo methods

Markov Chain Monte Carlo methods Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As

More information

arxiv: v1 [math.st] 7 Jan 2014

arxiv: v1 [math.st] 7 Jan 2014 Three Occurrences of the Hyperbolic-Secant Distribution Peng Ding Department of Statistics, Harvard University, One Oxford Street, Cambridge 02138 MA Email: pengding@fas.harvard.edu arxiv:1401.1267v1 [math.st]

More information

On Reparametrization and the Gibbs Sampler

On Reparametrization and the Gibbs Sampler On Reparametrization and the Gibbs Sampler Jorge Carlos Román Department of Mathematics Vanderbilt University James P. Hobert Department of Statistics University of Florida March 2014 Brett Presnell Department

More information

McGill University. Department of Epidemiology and Biostatistics. Bayesian Analysis for the Health Sciences. Course EPIB-682.

McGill University. Department of Epidemiology and Biostatistics. Bayesian Analysis for the Health Sciences. Course EPIB-682. McGill University Department of Epidemiology and Biostatistics Bayesian Analysis for the Health Sciences Course EPIB-682 Lawrence Joseph Intro to Bayesian Analysis for the Health Sciences EPIB-682 2 credits

More information

The Variational Gaussian Approximation Revisited

The Variational Gaussian Approximation Revisited The Variational Gaussian Approximation Revisited Manfred Opper Cédric Archambeau March 16, 2009 Abstract The variational approximation of posterior distributions by multivariate Gaussians has been much

More information

Computation of Signal to Noise Ratios

Computation of Signal to Noise Ratios MATCH Communications in Mathematical in Computer Chemistry MATCH Commun. Math. Comput. Chem. 57 7) 15-11 ISS 34-653 Computation of Signal to oise Ratios Saralees adarajah 1 & Samuel Kotz Received May,

More information

Identifiability problems in some non-gaussian spatial random fields

Identifiability problems in some non-gaussian spatial random fields Chilean Journal of Statistics Vol. 3, No. 2, September 2012, 171 179 Probabilistic and Inferential Aspects of Skew-Symmetric Models Special Issue: IV International Workshop in honour of Adelchi Azzalini

More information

A Bayesian Treatment of Linear Gaussian Regression

A Bayesian Treatment of Linear Gaussian Regression A Bayesian Treatment of Linear Gaussian Regression Frank Wood December 3, 2009 Bayesian Approach to Classical Linear Regression In classical linear regression we have the following model y β, σ 2, X N(Xβ,

More information

Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions

Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions R U T C O R R E S E A R C H R E P O R T Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions Douglas H. Jones a Mikhail Nediak b RRR 7-2, February, 2! " ##$%#&

More information

Sampling Methods (11/30/04)

Sampling Methods (11/30/04) CS281A/Stat241A: Statistical Learning Theory Sampling Methods (11/30/04) Lecturer: Michael I. Jordan Scribe: Jaspal S. Sandhu 1 Gibbs Sampling Figure 1: Undirected and directed graphs, respectively, with

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate

More information

A Semi-parametric Bayesian Framework for Performance Analysis of Call Centers

A Semi-parametric Bayesian Framework for Performance Analysis of Call Centers Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session STS065) p.2345 A Semi-parametric Bayesian Framework for Performance Analysis of Call Centers Bangxian Wu and Xiaowei

More information

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

More information

Prior Distributions for Variance Parameters in Hierarchical Models. Andrew Gelman. EERI Research Paper Series No 6/2004

Prior Distributions for Variance Parameters in Hierarchical Models. Andrew Gelman. EERI Research Paper Series No 6/2004 EERI Economics and Econometrics Research Institute Prior Distributions for Variance Parameters in Hierarchical Models Andrew Gelman EERI Research Paper Series No 6/2004 Copyright 2004 by Andrew Gelman

More information

Ronald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California

Ronald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California Texts in Statistical Science Bayesian Ideas and Data Analysis An Introduction for Scientists and Statisticians Ronald Christensen University of New Mexico Albuquerque, New Mexico Wesley Johnson University

More information

Statistical Practice

Statistical Practice Statistical Practice A Note on Bayesian Inference After Multiple Imputation Xiang ZHOU and Jerome P. REITER This article is aimed at practitioners who plan to use Bayesian inference on multiply-imputed

More information

Biostat 2065 Analysis of Incomplete Data

Biostat 2065 Analysis of Incomplete Data Biostat 2065 Analysis of Incomplete Data Gong Tang Dept of Biostatistics University of Pittsburgh October 20, 2005 1. Large-sample inference based on ML Let θ is the MLE, then the large-sample theory implies

More information

Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation

Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation COMPSTAT 2010 Revised version; August 13, 2010 Michael G.B. Blum 1 Laboratoire TIMC-IMAG, CNRS, UJF Grenoble

More information

The Mixture Approach for Simulating New Families of Bivariate Distributions with Specified Correlations

The Mixture Approach for Simulating New Families of Bivariate Distributions with Specified Correlations The Mixture Approach for Simulating New Families of Bivariate Distributions with Specified Correlations John R. Michael, Significance, Inc. and William R. Schucany, Southern Methodist University The mixture

More information

Learning Energy-Based Models of High-Dimensional Data

Learning Energy-Based Models of High-Dimensional Data Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal

More information

Bayesian inference for multivariate skew-normal and skew-t distributions

Bayesian inference for multivariate skew-normal and skew-t distributions Bayesian inference for multivariate skew-normal and skew-t distributions Brunero Liseo Sapienza Università di Roma Banff, May 2013 Outline Joint research with Antonio Parisi (Roma Tor Vergata) 1. Inferential

More information

Bayesian and empirical Bayesian approach to weighted averaging of ECG signal

Bayesian and empirical Bayesian approach to weighted averaging of ECG signal BULLETIN OF THE POLISH ACADEMY OF SCIENCES TECHNICAL SCIENCES Vol. 55, No. 4, 7 Bayesian and empirical Bayesian approach to weighted averaging of ECG signal A. MOMOT, M. MOMOT, and J. ŁĘSKI,3 Silesian

More information

Default Priors and Effcient Posterior Computation in Bayesian

Default Priors and Effcient Posterior Computation in Bayesian Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature

More information

The Polya-Gamma Gibbs Sampler for Bayesian. Logistic Regression is Uniformly Ergodic

The Polya-Gamma Gibbs Sampler for Bayesian. Logistic Regression is Uniformly Ergodic he Polya-Gamma Gibbs Sampler for Bayesian Logistic Regression is Uniformly Ergodic Hee Min Choi and James P. Hobert Department of Statistics University of Florida August 013 Abstract One of the most widely

More information

To Estimate or Not to Estimate?

To Estimate or Not to Estimate? To Estimate or Not to Estimate? Benjamin Kedem and Shihua Wen In linear regression there are examples where some of the coefficients are known but are estimated anyway for various reasons not least of

More information

STATISTICAL ANALYSIS WITH MISSING DATA

STATISTICAL ANALYSIS WITH MISSING DATA STATISTICAL ANALYSIS WITH MISSING DATA SECOND EDITION Roderick J.A. Little & Donald B. Rubin WILEY SERIES IN PROBABILITY AND STATISTICS Statistical Analysis with Missing Data Second Edition WILEY SERIES

More information

NONLINEAR APPLICATIONS OF MARKOV CHAIN MONTE CARLO

NONLINEAR APPLICATIONS OF MARKOV CHAIN MONTE CARLO NONLINEAR APPLICATIONS OF MARKOV CHAIN MONTE CARLO by Gregois Lee, B.Sc.(ANU), B.Sc.Hons(UTas) Submitted in fulfilment of the requirements for the Degree of Doctor of Philosophy Department of Mathematics

More information

Bayesian Inference. Chapter 1. Introduction and basic concepts

Bayesian Inference. Chapter 1. Introduction and basic concepts Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master

More information

Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter

More information

Prediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother

Prediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother Prediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother Simo Särkkä, Aki Vehtari and Jouko Lampinen Helsinki University of Technology Department of Electrical and Communications

More information

Bayesian Mixture Modeling

Bayesian Mixture Modeling University of California, Merced July 21, 2014 Mplus Users Meeting, Utrecht Organization of the Talk Organization s modeling estimation framework Motivating examples duce the basic LCA model Illustrated

More information

Expectation Propagation in Dynamical Systems

Expectation Propagation in Dynamical Systems Expectation Propagation in Dynamical Systems Marc Peter Deisenroth Joint Work with Shakir Mohamed (UBC) August 10, 2012 Marc Deisenroth (TU Darmstadt) EP in Dynamical Systems 1 Motivation Figure : Complex

More information

Multivariate Versus Multinomial Probit: When are Binary Decisions Made Separately also Jointly Optimal?

Multivariate Versus Multinomial Probit: When are Binary Decisions Made Separately also Jointly Optimal? Multivariate Versus Multinomial Probit: When are Binary Decisions Made Separately also Jointly Optimal? Dale J. Poirier and Deven Kapadia University of California, Irvine March 10, 2012 Abstract We provide

More information

RESEARCH ARTICLE. Online quantization in nonlinear filtering

RESEARCH ARTICLE. Online quantization in nonlinear filtering Journal of Statistical Computation & Simulation Vol. 00, No. 00, Month 200x, 3 RESEARCH ARTICLE Online quantization in nonlinear filtering A. Feuer and G. C. Goodwin Received 00 Month 200x; in final form

More information

Markov chain Monte Carlo methods for visual tracking

Markov chain Monte Carlo methods for visual tracking Markov chain Monte Carlo methods for visual tracking Ray Luo rluo@cory.eecs.berkeley.edu Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720

More information

Inferences on missing information under multiple imputation and two-stage multiple imputation

Inferences on missing information under multiple imputation and two-stage multiple imputation p. 1/4 Inferences on missing information under multiple imputation and two-stage multiple imputation Ofer Harel Department of Statistics University of Connecticut Prepared for the Missing Data Approaches

More information

Student-t Process as Alternative to Gaussian Processes Discussion

Student-t Process as Alternative to Gaussian Processes Discussion Student-t Process as Alternative to Gaussian Processes Discussion A. Shah, A. G. Wilson and Z. Gharamani Discussion by: R. Henao Duke University June 20, 2014 Contributions The paper is concerned about

More information

Heriot-Watt University

Heriot-Watt University Heriot-Watt University Heriot-Watt University Research Gateway Prediction of settlement delay in critical illness insurance claims by using the generalized beta of the second kind distribution Dodd, Erengul;

More information

MEASUREMENT UNCERTAINTY AND SUMMARISING MONTE CARLO SAMPLES

MEASUREMENT UNCERTAINTY AND SUMMARISING MONTE CARLO SAMPLES XX IMEKO World Congress Metrology for Green Growth September 9 14, 212, Busan, Republic of Korea MEASUREMENT UNCERTAINTY AND SUMMARISING MONTE CARLO SAMPLES A B Forbes National Physical Laboratory, Teddington,

More information

Dynamic System Identification using HDMR-Bayesian Technique

Dynamic System Identification using HDMR-Bayesian Technique Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in

More information

Finite Population Sampling and Inference

Finite Population Sampling and Inference Finite Population Sampling and Inference A Prediction Approach RICHARD VALLIANT ALAN H. DORFMAN RICHARD M. ROYALL A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane

More information

A Single Series from the Gibbs Sampler Provides a False Sense of Security

A Single Series from the Gibbs Sampler Provides a False Sense of Security A Single Series from the Gibbs Sampler Provides a False Sense of Security Andrew Gelman Department of Statistics University of California Berkeley, CA 9472 Donald B. Rubin Department of Statistics Harvard

More information

Will Penny. SPM short course for M/EEG, London 2013

Will Penny. SPM short course for M/EEG, London 2013 SPM short course for M/EEG, London 2013 Ten Simple Rules Stephan et al. Neuroimage, 2010 Model Structure Bayes rule for models A prior distribution over model space p(m) (or hypothesis space ) can be updated

More information

Hierarchical Bayesian Modeling

Hierarchical Bayesian Modeling Hierarchical Bayesian Modeling Making scientific inferences about a population based on many individuals Angie Wolfgang NSF Postdoctoral Fellow, Penn State Astronomical Populations Once we discover an

More information

arxiv: v1 [math.pr] 10 Oct 2017

arxiv: v1 [math.pr] 10 Oct 2017 Yet another skew-elliptical family but of a different kind: return to Lemma 1 arxiv:1710.03494v1 [math.p] 10 Oct 017 Adelchi Azzalini Dipartimento di Scienze Statistiche Università di Padova Italia Giuliana

More information

G. S. Maddala Kajal Lahiri. WILEY A John Wiley and Sons, Ltd., Publication

G. S. Maddala Kajal Lahiri. WILEY A John Wiley and Sons, Ltd., Publication G. S. Maddala Kajal Lahiri WILEY A John Wiley and Sons, Ltd., Publication TEMT Foreword Preface to the Fourth Edition xvii xix Part I Introduction and the Linear Regression Model 1 CHAPTER 1 What is Econometrics?

More information

Machine Learning Techniques for Computer Vision

Machine Learning Techniques for Computer Vision Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM

More information

Monitoring Random Start Forward Searches for Multivariate Data

Monitoring Random Start Forward Searches for Multivariate Data Monitoring Random Start Forward Searches for Multivariate Data Anthony C. Atkinson 1, Marco Riani 2, and Andrea Cerioli 2 1 Department of Statistics, London School of Economics London WC2A 2AE, UK, a.c.atkinson@lse.ac.uk

More information

Research Article A Nonparametric Two-Sample Wald Test of Equality of Variances

Research Article A Nonparametric Two-Sample Wald Test of Equality of Variances Advances in Decision Sciences Volume 211, Article ID 74858, 8 pages doi:1.1155/211/74858 Research Article A Nonparametric Two-Sample Wald Test of Equality of Variances David Allingham 1 andj.c.w.rayner

More information

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) X-prize: first $10k human genome technology

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

Statistical techniques for data analysis in Cosmology

Statistical techniques for data analysis in Cosmology Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction

More information

A student's t filter for heavy tailed process and measurement noise

A student's t filter for heavy tailed process and measurement noise A student's t filter for heavy tailed process and measurement noise Michael Roth, Emre Ozan and Fredri Gustafsson Linöping University Post Print N.B.: When citing this wor, cite the original article. Original

More information

Reduction of Random Variables in Structural Reliability Analysis

Reduction of Random Variables in Structural Reliability Analysis Reduction of Random Variables in Structural Reliability Analysis S. Adhikari and R. S. Langley Department of Engineering University of Cambridge Trumpington Street Cambridge CB2 1PZ (U.K.) February 21,

More information

Reconstruction of individual patient data for meta analysis via Bayesian approach

Reconstruction of individual patient data for meta analysis via Bayesian approach Reconstruction of individual patient data for meta analysis via Bayesian approach Yusuke Yamaguchi, Wataru Sakamoto and Shingo Shirahata Graduate School of Engineering Science, Osaka University Masashi

More information

ECE521 week 3: 23/26 January 2017

ECE521 week 3: 23/26 January 2017 ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear

More information

A Bayesian perspective on GMM and IV

A Bayesian perspective on GMM and IV A Bayesian perspective on GMM and IV Christopher A. Sims Princeton University sims@princeton.edu November 26, 2013 What is a Bayesian perspective? A Bayesian perspective on scientific reporting views all

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

More information

Deciding, Estimating, Computing, Checking

Deciding, Estimating, Computing, Checking Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:

More information

Deciding, Estimating, Computing, Checking. How are Bayesian posteriors used, computed and validated?

Deciding, Estimating, Computing, Checking. How are Bayesian posteriors used, computed and validated? Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:

More information

Directed acyclic graphs and the use of linear mixed models

Directed acyclic graphs and the use of linear mixed models Directed acyclic graphs and the use of linear mixed models Siem H. Heisterkamp 1,2 1 Groningen Bioinformatics Centre, University of Groningen 2 Biostatistics and Research Decision Sciences (BARDS), MSD,

More information

Introduction to Bayesian Data Analysis

Introduction to Bayesian Data Analysis Introduction to Bayesian Data Analysis Phil Gregory University of British Columbia March 2010 Hardback (ISBN-10: 052184150X ISBN-13: 9780521841504) Resources and solutions This title has free Mathematica

More information

EM Algorithm II. September 11, 2018

EM Algorithm II. September 11, 2018 EM Algorithm II September 11, 2018 Review EM 1/27 (Y obs, Y mis ) f (y obs, y mis θ), we observe Y obs but not Y mis Complete-data log likelihood: l C (θ Y obs, Y mis ) = log { f (Y obs, Y mis θ) Observed-data

More information

Linear Models A linear model is defined by the expression

Linear Models A linear model is defined by the expression Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose

More information

Shu Yang and Jae Kwang Kim. Harvard University and Iowa State University

Shu Yang and Jae Kwang Kim. Harvard University and Iowa State University Statistica Sinica 27 (2017), 000-000 doi:https://doi.org/10.5705/ss.202016.0155 DISCUSSION: DISSECTING MULTIPLE IMPUTATION FROM A MULTI-PHASE INFERENCE PERSPECTIVE: WHAT HAPPENS WHEN GOD S, IMPUTER S AND

More information

NON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES

NON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES 2013 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING NON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES Simo Särä Aalto University, 02150 Espoo, Finland Jouni Hartiainen

More information

Lecture 1 Bayesian inference

Lecture 1 Bayesian inference Lecture 1 Bayesian inference olivier.francois@imag.fr April 2011 Outline of Lecture 1 Principles of Bayesian inference Classical inference problems (frequency, mean, variance) Basic simulation algorithms

More information

Miscellanea Kernel density estimation and marginalization consistency

Miscellanea Kernel density estimation and marginalization consistency Biometrika (1991), 78, 2, pp. 421-5 Printed in Great Britain Miscellanea Kernel density estimation and marginalization consistency BY MIKE WEST Institute of Statistics and Decision Sciences, Duke University,

More information

ABHELSINKI UNIVERSITY OF TECHNOLOGY

ABHELSINKI UNIVERSITY OF TECHNOLOGY Cross-Validation, Information Criteria, Expected Utilities and the Effective Number of Parameters Aki Vehtari and Jouko Lampinen Laboratory of Computational Engineering Introduction Expected utility -

More information