On the Conditional Distribution of the Multivariate t Distribution
|
|
- Bertina Brown
- 6 years ago
- Views:
Transcription
1 On the Conditional Distribution of the Multivariate t Distribution arxiv: v [math.st] 2 Apr 206 Peng Ding Abstract As alternatives to the normal distributions, t distributions are widely applied in robust analysis for data with outliers or heavy tails. The properties of the multivariate t distribution are well documented in Kotz and Nadarajah s book, which, however, states a wrong conclusion about the conditional distribution of the multivariate t distribution. Previous literature has recognized that the conditional distribution of the multivariate t distribution also follows the multivariate t distribution. We provide an intuitive proof without directly manipulating the complicated density function of the multivariate t distribution. Key Words: Bayes Theorem; Data augmentation; Mahalanobis distance; Normal mixture; Representation. Introduction The conventional version of the multivariate t MVT distribution X t p µ,σ,ν, with location µ, scale matrix Σ, and degrees of freedom ν, has the probability density function fx= Γν+p/2} ν+p/2. Γν/2νπ p/2 Σ /2 +ν x µ Σ x µ} We can see from the above that the tail probability of the MVT distribution decays at a polynomial rate, resulting in heavier tails than the multivariate normal distribution. Because of this property, the MVT distribution is widely applied in robust data analysis including linear and nonlinear regressions Lange et al. 988; Liu 994; Liu 2004, linear mixed effects models Pinheiro et al. 200, and sample selection models Marchenko and Genton 202; Ding 204. Peng Ding is Assistant Professor pengdingpku@gmail.com, Department of Statistics, University of California, Berkeley, 425 Evans Hall, Berkeley, CA 94720, USA. The author thanks the Editor, Associate Editor and three reviewers for their helpful comments.
2 In the following discussion, we use W χ 2 b /c to denote the scaled χ2 distribution, with density proportional to w b/2 e cw/2. We exploit the following representation of the MVT distribution: X =µ+σ /2 Z/ q, 2 where Z follows a p dimensional standard normal distribution, q χ 2 ν/ν, and Z is independent of q Kotz and Nadarajah 2004; Nadarajah and Kotz It differs from the multivariate normal distribution N p µ,σ only by the random scaling factor q. The above representation 2 implies X q N p µ,σ/q, i.e., X follows a multivariate normal distribution given the latent variable q. If we partition X into two parts, X andx 2, with dimensions p and p 2, we obtain the following normal mixture representation conditional on q: X = X X 2 µ Σ Σ q N p +p 2, 2 µ 2 Σ 2 Σ 22 } /q, 3 where the location and scale parameters are partitioned corresponding to the partition of X. Marginally, we have X q N p µ,σ /q, and therefore X t p µ,σ,ν, which follows a p dimensional MVT distribution with degrees of freedom ν. Although we can obtain the marginal distribution in an obvious way, the conditional distribution ofx 2 givenx is less transparent. In fact, the conditional distribution of the MVT distribution is also a MVT distribution, with degrees of freedom different from the original distribution. 2 Conditional Distribution Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 claimed that the conditional distribution of the MVT distribution is not a MVT distribution except for some extreme cases. For the case with µ = 0, define x 2 = x Σ 2 Σ x, and define Σ 22 = Σ 22 Σ 2 Σ Σ 2 as the Schur complement of the block Σ in matrix Σ. They derived the conditional density of X 2 given 2
3 X by calculating fx/ f x using the probability density function in. Ignoring the normalizing constant in formula 5 of Nadarajah and Kotz 2005, we present only the key term: f 2 x 2 x +ν+p x ν+x Σ x 2 Σ ν+p 22 x 2 ν+p +p 2 /2. 4 Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 obtained the correct conditional density function, but due to the complex form of their formula 5, they did not recognize that the key term in 4 is essentially the unnormalized probability density function of a p 2 dimensional MVT distribution with location Σ 2 Σ x, scale matrix ν+x Σ x /ν+p Σ 22, and degrees of freedom ν+p. In the literature, some authors stated the right conclusion about the conditional distribution without providing a proof e.g., Liu 994, page 5, DeGroot 2005, page 6, and some other authors re-derived the correct conditional density function in 4 and pointed out that it is another MVT distribution e.g., Roth 203b. Instead of directly calculating the conditional density or using the general theory of elliptically contoured distributions Cambanis et al. 98; Fang et al. 990, Theorems 2.8 and 3.8; Kibria and Joarder 2006, we provide an elementary and thus more transparent proof of the conditional distribution based on the normal mixture representations in 3. 3 Proof via Representation Our proof proceeds in two steps: we first condition on q, and then average over q. First, we condition on both X and q. According to the property of the multivariate normal distribution, X 2 follows a multivariate normal distribution conditional on X,q, i.e., X 2 X,q N p2 µ2 +Σ 2 Σ X µ,σ 22 Σ 2 Σ Σ 2/q } = N p2 µ2,σ 22 /q, where µ 2 =µ 2 +Σ 2 Σ X µ is the linear regression ofx 2 on X. Second, we obtain the conditional distribution of q given X. According to Bayes Theorem, the 3
4 conditional probability density function of q given X is f q q X f q X qπq Σ /q /2 exp q } 2 X µ Σ X µ q ν/2 e νq/2 q ν+p/2 exp q } 2 ν+ d, where d = X µ Σ X µ is the squared Mahalanobis distance of X from µ with scale matrix Σ. Therefore, the conditional distribution of q given X is q X χ 2 ν+p /ν+ d, according to the density of the scaled χ 2 distribution. Finally, we can represent the conditional distribution ofx 2 given X as X 2 X µ 2 +Σ /2 22 Z 2/ q X / µ 2 +Σ /2 22 Z χν+p 2 2 ν+ d ν+ d / µ 2 + Σ /2 χν+p ν+p 22 Z 2 2, 5 ν+p where Z 2 is a p 2 dimensional standard multivariate normal vector, independent of χ 2 ν+p /ν + p. From the normal mixture representation of the MVT distribution in 2, we can obtain the following conditional distribution. Conclusion One: The conditional distribution of X 2 given X is X 2 X t p2 µ 2, ν+ d Σ ν+p 22,ν+p. The conditional distribution of the multivariate t distribution is very similar to that of the multivariate normal distribution. The conditional location parameter is the linear regression of X 2 on X. The conditional scale matrix is Σ 22 inflated or deflated by the factor ν+ d /ν + p. Because d p Fp,ν with mean being p ν/ν 2 or infinity according to ν > 2 or ν 2 Roth 203a, page 83, on average the factor ν+ d /ν + p is larger than one. With more extreme values of X, the conditional distributions ofx 2 are more disperse. More interestingly, the conditional degrees 4
5 of freedom increase to ν + p. The more dimensions we condition on, the less heavy-tailedness we have. When ν =, the MVT distribution is also called the multivariate Cauchy distribution. The marginal distributions are still Cauchy, but the conditional distributions are MVT instead of Cauchy. Although the marginal means of the multivariate Cauchy distribution do not exist for any dimensions, all the conditional means exist because the conditional distributions of the multivariate Cauchy distribution follow MVT distributions with degrees of freedom at least as large as two. As a byproduct, we can easily obtain from the representation 5 that ν+p ν+ d X2 µ 2 X t p2 0,Σ22,ν+p, which does not depend on X. The above result further implies the following conclusion. Conclusion Two: X and ν+p ν+d X2 µ 2 are independent. It is straightforward to show that X and X 2 µ 2 are uncorrelated for both multivariate normal and MVT distributions. If X follows the multivariate normal distribution, this also implies independence between X and X 2 µ 2. If X follows the MVT distribution, we need only to adjust the linear regression residual X 2 µ 2 by the factor ν+p /ν+ d and the independence result still holds. 4 Discussion Kotz and Nadarajah 2004, page 7 and Nadarajah and Kotz 2005 failed to recognize that the conditional distribution of the MVT distribution is also a MVT distribution due to the complexity of the conditional density function. Our proof, based on the normal mixture representation of the MVT distribution, offers a more direct and transparent way to revealing the property of the conditional distribution of the MVT distribution. Our representation in Section 3 implicitly exploits the data augmentation formula f 2 x 2 x = f 2,q x 2 x,q f q q x dq Tanner and Wong 987. To make our proof more intuitive, we used formula 5 to avoid the integral, which is essentially a Monte 5
6 Carlo version of the data augmentation formula. Conclusion One could also follow from the general theory of elliptically contoured distributions, including the MVT distribution as a special case Cambanis et al. 980; Fang et al Conditional distributions of elliptically contoured distributions are also elliptically contoured distributions. But this does not immediately guarantee that conditional distributions of the MVT distributions are also MVT distributions without some further algebra. The MVT distribution has the property of having MVT marginal and conditional distributions. It will be interesting to find other sub-classes of elliptically contoured distributions that have the same property. References Cambanis, S., Huang, S. and Simons, G. 98. On the theory of elliptically contoured distributions. Journal of Multivariate Analysis,, Ding, P Bayesian robust inference of sample selection using selection-t models. Journal of Multivariate Analysis, 24, DeGroot, M. H Optimal Statistical Decisions. New York: John Wiley & Sons. Fang, K. T., Kotz, S., and Ng, K. W Symmetric Multivariate and Related Distributions. London: Chapman & Hall. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D., Vehtari, A., and Rubin, D. B Bayesian Data Analysis, 2nd edition. London: Chapman & Hall/CRC. Kibria, B. G., and Joarder, A. H A short review of multivariate t-distribution. Journal of Statistical Research, 40, Kotz, S. and Nadarajah, S Multivariate t Distributions and Their Applications. New York: Cambridge University Press. Lange, K. L., Little, R. J., and Taylor, J. M Robust statistical modeling using the t distribution. 6
7 Journal of the American Statistical Association, 84, Liu, C Statistical Analysis Using the Multivariate t Distribution. Doctoral dissertation, Harvard University. Liu, C Robit regression: a simple robust alternative to logistic and probit regression. In Gelman, A. and Meng, X.-L. Eds. Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives pp New York: John Wiley & Sons. Marchenko, Y. V., and Genton, M. G A Heckman selection-t model. Journal of the American Statistical Association, 07, Nadarajah, S. and Kotz, S Mathematical properties of the multivariate t distribution. Acta Applicandae Mathematica, 89, Pinheiro, J. C., Liu, C., and Wu, Y. N Efficient algorithms for robust estimation in linear mixed-effects models using the multivariate t distribution. Journal of Computational and Graphical Statistics, 0, Roth, M. 203a. Kalman Filters for Nonlinear Systems and Heavy-Tailed Noise, Licentiate of Engineering Thesis, Linköping University, Linköping. Roth, M. 203b. On the multivariate t distribution. Technical report from Automatic Control at Linköpings Universitet. Report no.: LiTH-ISY-R Tanner, M. A., and Wong, W. H The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82,
Bayesian Inference for the Multivariate Normal
Bayesian Inference for the Multivariate Normal Will Penny Wellcome Trust Centre for Neuroimaging, University College, London WC1N 3BG, UK. November 28, 2014 Abstract Bayesian inference for the multivariate
More informationRECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T DISTRIBUTION
1 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 3 6, 1, SANTANDER, SPAIN RECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T
More informationOnline tests of Kalman filter consistency
Tampere University of Technology Online tests of Kalman filter consistency Citation Piché, R. (216). Online tests of Kalman filter consistency. International Journal of Adaptive Control and Signal Processing,
More informationTools for Parameter Estimation and Propagation of Uncertainty
Tools for Parameter Estimation and Propagation of Uncertainty Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu Outline Models, parameters, parameter estimation,
More informationOn Bayesian Inference with Conjugate Priors for Scale Mixtures of Normal Distributions
Journal of Applied Probability & Statistics Vol. 5, No. 1, xxx xxx c 2010 Dixie W Publishing Corporation, U. S. A. On Bayesian Inference with Conjugate Priors for Scale Mixtures of Normal Distributions
More informationBayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
More informationSupplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements
Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Jeffrey N. Rouder Francis Tuerlinckx Paul L. Speckman Jun Lu & Pablo Gomez May 4 008 1 The Weibull regression model
More informationDiscussion on Fygenson (2007, Statistica Sinica): a DS Perspective
1 Discussion on Fygenson (2007, Statistica Sinica): a DS Perspective Chuanhai Liu Purdue University 1. Introduction In statistical analysis, it is important to discuss both uncertainty due to model choice
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationSTA 216, GLM, Lecture 16. October 29, 2007
STA 216, GLM, Lecture 16 October 29, 2007 Efficient Posterior Computation in Factor Models Underlying Normal Models Generalized Latent Trait Models Formulation Genetic Epidemiology Illustration Structural
More informationeqr094: Hierarchical MCMC for Bayesian System Reliability
eqr094: Hierarchical MCMC for Bayesian System Reliability Alyson G. Wilson Statistical Sciences Group, Los Alamos National Laboratory P.O. Box 1663, MS F600 Los Alamos, NM 87545 USA Phone: 505-667-9167
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationTesting Restrictions and Comparing Models
Econ. 513, Time Series Econometrics Fall 00 Chris Sims Testing Restrictions and Comparing Models 1. THE PROBLEM We consider here the problem of comparing two parametric models for the data X, defined by
More informationResearch Article The Laplace Likelihood Ratio Test for Heteroscedasticity
International Mathematics and Mathematical Sciences Volume 2011, Article ID 249564, 7 pages doi:10.1155/2011/249564 Research Article The Laplace Likelihood Ratio Test for Heteroscedasticity J. Martin van
More informationSTATISTICAL INFERENCE WITH DATA AUGMENTATION AND PARAMETER EXPANSION
STATISTICAL INFERENCE WITH arxiv:1512.00847v1 [math.st] 2 Dec 2015 DATA AUGMENTATION AND PARAMETER EXPANSION Yannis G. Yatracos Faculty of Communication and Media Studies Cyprus University of Technology
More informationBagging During Markov Chain Monte Carlo for Smoother Predictions
Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationUniform Correlation Mixture of Bivariate Normal Distributions and. Hypercubically-contoured Densities That Are Marginally Normal
Uniform Correlation Mixture of Bivariate Normal Distributions and Hypercubically-contoured Densities That Are Marginally Normal Kai Zhang Department of Statistics and Operations Research University of
More informationBayesian data analysis in practice: Three simple examples
Bayesian data analysis in practice: Three simple examples Martin P. Tingley Introduction These notes cover three examples I presented at Climatea on 5 October 0. Matlab code is available by request to
More informationQuantile POD for Hit-Miss Data
Quantile POD for Hit-Miss Data Yew-Meng Koh a and William Q. Meeker a a Center for Nondestructive Evaluation, Department of Statistics, Iowa State niversity, Ames, Iowa 50010 Abstract. Probability of detection
More informationChris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
More informationBayesian Inference. p(y)
Bayesian Inference There are different ways to interpret a probability statement in a real world setting. Frequentist interpretations of probability apply to situations that can be repeated many times,
More informationA Note on Bayesian Inference After Multiple Imputation
A Note on Bayesian Inference After Multiple Imputation Xiang Zhou and Jerome P. Reiter Abstract This article is aimed at practitioners who plan to use Bayesian inference on multiplyimputed datasets in
More informationDiagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo
Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo Michael Betancourt arxiv:1604.00695v1 [stat.me] 3 Apr 2016 Abstract. When properly tuned, Hamiltonian Monte Carlo scales to some
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationSlice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method
Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Madeleine B. Thompson Radford M. Neal Abstract The shrinking rank method is a variation of slice sampling that is efficient at
More informationThe Goodness-of-fit Test for Gumbel Distribution: A Comparative Study
MATEMATIKA, 2012, Volume 28, Number 1, 35 48 c Department of Mathematics, UTM. The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study 1 Nahdiya Zainal Abidin, 2 Mohd Bakri Adam and 3 Habshah
More informationSome Approximations of the Logistic Distribution with Application to the Covariance Matrix of Logistic Regression
Working Paper 2013:9 Department of Statistics Some Approximations of the Logistic Distribution with Application to the Covariance Matrix of Logistic Regression Ronnie Pingel Working Paper 2013:9 June
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationarxiv: v1 [math.st] 7 Jan 2014
Three Occurrences of the Hyperbolic-Secant Distribution Peng Ding Department of Statistics, Harvard University, One Oxford Street, Cambridge 02138 MA Email: pengding@fas.harvard.edu arxiv:1401.1267v1 [math.st]
More informationOn Reparametrization and the Gibbs Sampler
On Reparametrization and the Gibbs Sampler Jorge Carlos Román Department of Mathematics Vanderbilt University James P. Hobert Department of Statistics University of Florida March 2014 Brett Presnell Department
More informationMcGill University. Department of Epidemiology and Biostatistics. Bayesian Analysis for the Health Sciences. Course EPIB-682.
McGill University Department of Epidemiology and Biostatistics Bayesian Analysis for the Health Sciences Course EPIB-682 Lawrence Joseph Intro to Bayesian Analysis for the Health Sciences EPIB-682 2 credits
More informationThe Variational Gaussian Approximation Revisited
The Variational Gaussian Approximation Revisited Manfred Opper Cédric Archambeau March 16, 2009 Abstract The variational approximation of posterior distributions by multivariate Gaussians has been much
More informationComputation of Signal to Noise Ratios
MATCH Communications in Mathematical in Computer Chemistry MATCH Commun. Math. Comput. Chem. 57 7) 15-11 ISS 34-653 Computation of Signal to oise Ratios Saralees adarajah 1 & Samuel Kotz Received May,
More informationIdentifiability problems in some non-gaussian spatial random fields
Chilean Journal of Statistics Vol. 3, No. 2, September 2012, 171 179 Probabilistic and Inferential Aspects of Skew-Symmetric Models Special Issue: IV International Workshop in honour of Adelchi Azzalini
More informationA Bayesian Treatment of Linear Gaussian Regression
A Bayesian Treatment of Linear Gaussian Regression Frank Wood December 3, 2009 Bayesian Approach to Classical Linear Regression In classical linear regression we have the following model y β, σ 2, X N(Xβ,
More informationItem Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions
R U T C O R R E S E A R C H R E P O R T Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions Douglas H. Jones a Mikhail Nediak b RRR 7-2, February, 2! " ##$%#&
More informationSampling Methods (11/30/04)
CS281A/Stat241A: Statistical Learning Theory Sampling Methods (11/30/04) Lecturer: Michael I. Jordan Scribe: Jaspal S. Sandhu 1 Gibbs Sampling Figure 1: Undirected and directed graphs, respectively, with
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationA Semi-parametric Bayesian Framework for Performance Analysis of Call Centers
Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session STS065) p.2345 A Semi-parametric Bayesian Framework for Performance Analysis of Call Centers Bangxian Wu and Xiaowei
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationPrior Distributions for Variance Parameters in Hierarchical Models. Andrew Gelman. EERI Research Paper Series No 6/2004
EERI Economics and Econometrics Research Institute Prior Distributions for Variance Parameters in Hierarchical Models Andrew Gelman EERI Research Paper Series No 6/2004 Copyright 2004 by Andrew Gelman
More informationRonald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California
Texts in Statistical Science Bayesian Ideas and Data Analysis An Introduction for Scientists and Statisticians Ronald Christensen University of New Mexico Albuquerque, New Mexico Wesley Johnson University
More informationStatistical Practice
Statistical Practice A Note on Bayesian Inference After Multiple Imputation Xiang ZHOU and Jerome P. REITER This article is aimed at practitioners who plan to use Bayesian inference on multiply-imputed
More informationBiostat 2065 Analysis of Incomplete Data
Biostat 2065 Analysis of Incomplete Data Gong Tang Dept of Biostatistics University of Pittsburgh October 20, 2005 1. Large-sample inference based on ML Let θ is the MLE, then the large-sample theory implies
More informationChoosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation
Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation COMPSTAT 2010 Revised version; August 13, 2010 Michael G.B. Blum 1 Laboratoire TIMC-IMAG, CNRS, UJF Grenoble
More informationThe Mixture Approach for Simulating New Families of Bivariate Distributions with Specified Correlations
The Mixture Approach for Simulating New Families of Bivariate Distributions with Specified Correlations John R. Michael, Significance, Inc. and William R. Schucany, Southern Methodist University The mixture
More informationLearning Energy-Based Models of High-Dimensional Data
Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal
More informationBayesian inference for multivariate skew-normal and skew-t distributions
Bayesian inference for multivariate skew-normal and skew-t distributions Brunero Liseo Sapienza Università di Roma Banff, May 2013 Outline Joint research with Antonio Parisi (Roma Tor Vergata) 1. Inferential
More informationBayesian and empirical Bayesian approach to weighted averaging of ECG signal
BULLETIN OF THE POLISH ACADEMY OF SCIENCES TECHNICAL SCIENCES Vol. 55, No. 4, 7 Bayesian and empirical Bayesian approach to weighted averaging of ECG signal A. MOMOT, M. MOMOT, and J. ŁĘSKI,3 Silesian
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationThe Polya-Gamma Gibbs Sampler for Bayesian. Logistic Regression is Uniformly Ergodic
he Polya-Gamma Gibbs Sampler for Bayesian Logistic Regression is Uniformly Ergodic Hee Min Choi and James P. Hobert Department of Statistics University of Florida August 013 Abstract One of the most widely
More informationTo Estimate or Not to Estimate?
To Estimate or Not to Estimate? Benjamin Kedem and Shihua Wen In linear regression there are examples where some of the coefficients are known but are estimated anyway for various reasons not least of
More informationSTATISTICAL ANALYSIS WITH MISSING DATA
STATISTICAL ANALYSIS WITH MISSING DATA SECOND EDITION Roderick J.A. Little & Donald B. Rubin WILEY SERIES IN PROBABILITY AND STATISTICS Statistical Analysis with Missing Data Second Edition WILEY SERIES
More informationNONLINEAR APPLICATIONS OF MARKOV CHAIN MONTE CARLO
NONLINEAR APPLICATIONS OF MARKOV CHAIN MONTE CARLO by Gregois Lee, B.Sc.(ANU), B.Sc.Hons(UTas) Submitted in fulfilment of the requirements for the Degree of Doctor of Philosophy Department of Mathematics
More informationBayesian Inference. Chapter 1. Introduction and basic concepts
Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter
More informationPrediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother
Prediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother Simo Särkkä, Aki Vehtari and Jouko Lampinen Helsinki University of Technology Department of Electrical and Communications
More informationBayesian Mixture Modeling
University of California, Merced July 21, 2014 Mplus Users Meeting, Utrecht Organization of the Talk Organization s modeling estimation framework Motivating examples duce the basic LCA model Illustrated
More informationExpectation Propagation in Dynamical Systems
Expectation Propagation in Dynamical Systems Marc Peter Deisenroth Joint Work with Shakir Mohamed (UBC) August 10, 2012 Marc Deisenroth (TU Darmstadt) EP in Dynamical Systems 1 Motivation Figure : Complex
More informationMultivariate Versus Multinomial Probit: When are Binary Decisions Made Separately also Jointly Optimal?
Multivariate Versus Multinomial Probit: When are Binary Decisions Made Separately also Jointly Optimal? Dale J. Poirier and Deven Kapadia University of California, Irvine March 10, 2012 Abstract We provide
More informationRESEARCH ARTICLE. Online quantization in nonlinear filtering
Journal of Statistical Computation & Simulation Vol. 00, No. 00, Month 200x, 3 RESEARCH ARTICLE Online quantization in nonlinear filtering A. Feuer and G. C. Goodwin Received 00 Month 200x; in final form
More informationMarkov chain Monte Carlo methods for visual tracking
Markov chain Monte Carlo methods for visual tracking Ray Luo rluo@cory.eecs.berkeley.edu Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720
More informationInferences on missing information under multiple imputation and two-stage multiple imputation
p. 1/4 Inferences on missing information under multiple imputation and two-stage multiple imputation Ofer Harel Department of Statistics University of Connecticut Prepared for the Missing Data Approaches
More informationStudent-t Process as Alternative to Gaussian Processes Discussion
Student-t Process as Alternative to Gaussian Processes Discussion A. Shah, A. G. Wilson and Z. Gharamani Discussion by: R. Henao Duke University June 20, 2014 Contributions The paper is concerned about
More informationHeriot-Watt University
Heriot-Watt University Heriot-Watt University Research Gateway Prediction of settlement delay in critical illness insurance claims by using the generalized beta of the second kind distribution Dodd, Erengul;
More informationMEASUREMENT UNCERTAINTY AND SUMMARISING MONTE CARLO SAMPLES
XX IMEKO World Congress Metrology for Green Growth September 9 14, 212, Busan, Republic of Korea MEASUREMENT UNCERTAINTY AND SUMMARISING MONTE CARLO SAMPLES A B Forbes National Physical Laboratory, Teddington,
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationFinite Population Sampling and Inference
Finite Population Sampling and Inference A Prediction Approach RICHARD VALLIANT ALAN H. DORFMAN RICHARD M. ROYALL A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane
More informationA Single Series from the Gibbs Sampler Provides a False Sense of Security
A Single Series from the Gibbs Sampler Provides a False Sense of Security Andrew Gelman Department of Statistics University of California Berkeley, CA 9472 Donald B. Rubin Department of Statistics Harvard
More informationWill Penny. SPM short course for M/EEG, London 2013
SPM short course for M/EEG, London 2013 Ten Simple Rules Stephan et al. Neuroimage, 2010 Model Structure Bayes rule for models A prior distribution over model space p(m) (or hypothesis space ) can be updated
More informationHierarchical Bayesian Modeling
Hierarchical Bayesian Modeling Making scientific inferences about a population based on many individuals Angie Wolfgang NSF Postdoctoral Fellow, Penn State Astronomical Populations Once we discover an
More informationarxiv: v1 [math.pr] 10 Oct 2017
Yet another skew-elliptical family but of a different kind: return to Lemma 1 arxiv:1710.03494v1 [math.p] 10 Oct 017 Adelchi Azzalini Dipartimento di Scienze Statistiche Università di Padova Italia Giuliana
More informationG. S. Maddala Kajal Lahiri. WILEY A John Wiley and Sons, Ltd., Publication
G. S. Maddala Kajal Lahiri WILEY A John Wiley and Sons, Ltd., Publication TEMT Foreword Preface to the Fourth Edition xvii xix Part I Introduction and the Linear Regression Model 1 CHAPTER 1 What is Econometrics?
More informationMachine Learning Techniques for Computer Vision
Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM
More informationMonitoring Random Start Forward Searches for Multivariate Data
Monitoring Random Start Forward Searches for Multivariate Data Anthony C. Atkinson 1, Marco Riani 2, and Andrea Cerioli 2 1 Department of Statistics, London School of Economics London WC2A 2AE, UK, a.c.atkinson@lse.ac.uk
More informationResearch Article A Nonparametric Two-Sample Wald Test of Equality of Variances
Advances in Decision Sciences Volume 211, Article ID 74858, 8 pages doi:1.1155/211/74858 Research Article A Nonparametric Two-Sample Wald Test of Equality of Variances David Allingham 1 andj.c.w.rayner
More informationINTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP
INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) X-prize: first $10k human genome technology
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationStatistical techniques for data analysis in Cosmology
Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction
More informationA student's t filter for heavy tailed process and measurement noise
A student's t filter for heavy tailed process and measurement noise Michael Roth, Emre Ozan and Fredri Gustafsson Linöping University Post Print N.B.: When citing this wor, cite the original article. Original
More informationReduction of Random Variables in Structural Reliability Analysis
Reduction of Random Variables in Structural Reliability Analysis S. Adhikari and R. S. Langley Department of Engineering University of Cambridge Trumpington Street Cambridge CB2 1PZ (U.K.) February 21,
More informationReconstruction of individual patient data for meta analysis via Bayesian approach
Reconstruction of individual patient data for meta analysis via Bayesian approach Yusuke Yamaguchi, Wataru Sakamoto and Shingo Shirahata Graduate School of Engineering Science, Osaka University Masashi
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More informationA Bayesian perspective on GMM and IV
A Bayesian perspective on GMM and IV Christopher A. Sims Princeton University sims@princeton.edu November 26, 2013 What is a Bayesian perspective? A Bayesian perspective on scientific reporting views all
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
More informationDeciding, Estimating, Computing, Checking
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationDeciding, Estimating, Computing, Checking. How are Bayesian posteriors used, computed and validated?
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationDirected acyclic graphs and the use of linear mixed models
Directed acyclic graphs and the use of linear mixed models Siem H. Heisterkamp 1,2 1 Groningen Bioinformatics Centre, University of Groningen 2 Biostatistics and Research Decision Sciences (BARDS), MSD,
More informationIntroduction to Bayesian Data Analysis
Introduction to Bayesian Data Analysis Phil Gregory University of British Columbia March 2010 Hardback (ISBN-10: 052184150X ISBN-13: 9780521841504) Resources and solutions This title has free Mathematica
More informationEM Algorithm II. September 11, 2018
EM Algorithm II September 11, 2018 Review EM 1/27 (Y obs, Y mis ) f (y obs, y mis θ), we observe Y obs but not Y mis Complete-data log likelihood: l C (θ Y obs, Y mis ) = log { f (Y obs, Y mis θ) Observed-data
More informationLinear Models A linear model is defined by the expression
Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose
More informationShu Yang and Jae Kwang Kim. Harvard University and Iowa State University
Statistica Sinica 27 (2017), 000-000 doi:https://doi.org/10.5705/ss.202016.0155 DISCUSSION: DISSECTING MULTIPLE IMPUTATION FROM A MULTI-PHASE INFERENCE PERSPECTIVE: WHAT HAPPENS WHEN GOD S, IMPUTER S AND
More informationNON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES
2013 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING NON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES Simo Särä Aalto University, 02150 Espoo, Finland Jouni Hartiainen
More informationLecture 1 Bayesian inference
Lecture 1 Bayesian inference olivier.francois@imag.fr April 2011 Outline of Lecture 1 Principles of Bayesian inference Classical inference problems (frequency, mean, variance) Basic simulation algorithms
More informationMiscellanea Kernel density estimation and marginalization consistency
Biometrika (1991), 78, 2, pp. 421-5 Printed in Great Britain Miscellanea Kernel density estimation and marginalization consistency BY MIKE WEST Institute of Statistics and Decision Sciences, Duke University,
More informationABHELSINKI UNIVERSITY OF TECHNOLOGY
Cross-Validation, Information Criteria, Expected Utilities and the Effective Number of Parameters Aki Vehtari and Jouko Lampinen Laboratory of Computational Engineering Introduction Expected utility -
More information