Bayesian Modeling of Conditional Distributions
|
|
- Mary Davidson
- 5 years ago
- Views:
Transcription
1 Bayesian Modeling of Conditional Distributions John Geweke University of Iowa Indiana University Department of Economics February 27, 2007
2 Outline Motivation Model description Methods of inference Earnings example Asset returns example Conclusion
3 Motivation The general case and two examples The general case and an example x t, y t n1 i.i.d. p (y t j x t ) =? Example: Earnings y t x 1t x 2t Earnings or wage of individual t Experience or age of individual t Education of individual t
4 Motivation The general case and two examples A second example: Asset returns y t = Return on asset in period t y t j (y 1,..., y t 1 ) s? The covariates are x 1t Return on asset in period t 1, x 1t = y t 1 x 2t Recent volatility, g x 2,t 1 + (1 g) ja t 1 j κ = s=0 g s jy t 2 s j κ (g = 0.95, κ = 1)
5 Model description Mixture modeling The common structure of normal mixture models y t x t n1 Variable of interest Vector of covariates es t Latent state, es t 2 f1,..., mg y t j (x t,es t = j) s N β 0 j x t, σ 2 j 0 y t j (x t,es t = j) s N β + α j xt, (h h j ) 1
6 Model description Mixture modeling A simple special case: The normal mixture model with no covariates x t = 1 11 es t i.i.d., P (es t = j) = p j y t j (x t = 1,es t = j) s N β j, σ 2 j y t j (x t = 1,es t = j) s N hβ + α j, (h h j ) 1i
7 Model description Mixture modeling
8 Model description Mixture modeling What is new in this work Begin with the same normal mixture model y t j (x t,es t = j) s N β 0 j x t, σ 2 j Determination of latent states es t : iid ew t = Γ x t + ζ t ; ζ t s N (0, Im ) m1 n1 es t = j i ew tj ew ti 8 i = 1,..., m () es t = arg max ( ew ti ) i =1,...,m
9 Model description Literature context Literature context Bayesian nonparametric regression (e.g. Wahba, Shiller, Smith & Kohn) Non-Bayesian nonparametric regression (e.g. Härdle & Tsybakov) Quantile regression (e.g. Koenker & Basset, Yu & Jones) Dirichlet process priors (e.g. Gri n & Steel, Dunson & Pilai) Mixture of Experts Models (e.g. Jacobs & Jordan, Jian and Tanner)
10 Model description Identi cation issues Identi cation issues: Multinomial probit 0 ew t = Γx t + ζ t, j = arg max ( ew ti ), y t s N β + α j xt, (h h j ) 1 i=1,...,m iid Because ζ t s N (0, Im ) translation but not scaling issues arise in ew t = Γ x t + ζ t m1 n1 Impose ι 0 mγ= 0 through Γ = P mq " 0 0 q Γ (m 1)q # ; P = mm ι m m 1/2 P 2 m(m 1), P 0 P = I m
11 Model description Identi cation issues Identi cation issues: Labeling 0 ew t = Γx t + ζ t, j = arg max ( ew ti ), y t s N β + α j xt, (h h j ) 1 i=1,...,m States have no substantive interpretation and are exchangeable implications for prior distribution m! state permutations =) m! re ections of the posterior distribution Not to worry: p (y t j x t ) invariant to permutations of the states Geweke (2006) CSDA forthcoming
12 Model description Prior distributions Conditionally conjugate prior distributions 0 ew t = Γx t + ζ t, j = arg max ( ew ti ), y t s N β + α j xt, (h h j ) 1 i=1,...,m Distribution type Parameters Hyperparameters Gaussian: β, Γ µ, τ 2 β, τ2 γ Gaussian conditional on h: α j (j = 1,..., m) τ 2 α Inverse gamma: h; h j (j = 1,..., m) s 2, ν, ν
13 Model description Some theory Some theory: Conditions 1 Distribution of x 1 x 2 Ω R n, Ω compact 2 p (x) > 0 is continuous w.r.t. Lebesgue measure. 2 Conditional distribution of y 2A 1 For some function h (x), p (yj x) is an exponential distribution, p (yj x) = exp fa [h (x)] y + b [h (x)] + c(yg 8x 2 Ω, y 2 A; a () and b () are analytic, a 0 () 6 0, b 0 () h is in a ball of nite values in a Sobolev space with sup-norm and second-order di erentiability: 1 sup x2ω jh (x)j = c 0 < 2 sup i=1,...,n (sup x2ω j h (x) / x i j) = c 1 < 3 sup i=1,...,n supj=1,...,n sup x2ω 2 h (x) / x i x j = c2 <
14 Model description Some theory Some theory: A de nition De nition. For any two pdf s f () and g () w.r.t. Lebesgue measure de ned on Z, the Kullback-Leibler directed distance from f to g is Z f (z) KL (f, g) = f (z) log dz. g (z) Remark. In a model Z p (y t j θ A, A) (t = 1,...., T ) let T!. The posterior density of θ A will collapse about θ A = θ A, where θa = arg min KL [p (y), p (y j θ A, A)] θ A (Geweke, 2005, Theorem 3.4.2).
15 Model description Some theory Some theory: A result Theorem. For all x 2Ω, sup p(y jx) inf KL [p (y j x), p (y j x,a m)] < p(y jx,a m ) c m 4/n where c depends on p (x), c 0, c 1 and c 2, but does not depend on m or n. Remarks The result is a consequence of Jiang and Tanner (1999) Theorem 2 for the class of gating functions de ned by the multinomial probit model with scalar variance matrix. Condition 1 of Jiang and Tanner (1999) for gating functions is important but not given here. They conjecture that it holds for multinomial logit gating functions. The proof for multinomial probit gating functions is not di cult.
16 Methods of inference Blocking for Gibbs sampling Blocking for Gibbs sampling 0 ew t = Γx t + ζ t, j = arg max ( ew ti ), y t s N β + α j xt, (h h j ) 1 i=1,...,m h, h 1,..., h m Separately conditionally independent gamma β, α 1,..., α m Jointly conditionally Gaussian vec (Γ ) ew t Conditionally Gaussian Gaussian times orthant-speci c likelihood factors Conditional posteriors and code have been tested (Geweke 2004).
17 Methods of inference Quantile functions of interest Quantile functions of interest Conditional CDFs: P (y t c j x t ) Quantiles: c (q) = fc : P (y t c j x t ) = qg Note P (es t = j j Γ, x t ) = P ew tj ew ti (i = 1,..., m) j Γ, x t = R p ew tj = w j Γ, x t P [ ew ti w (i = 1,..., m) j Γ, x t ] dw = R φ w γ 0 j z t Φ y γi 0z t dw. i 6=j Given M Markov chain Monte Carlo replications of a mixture model with m components, the posterior distribution is a mixture of normals with M m components.
18 Earnings example Data Earnings example Data 2698 men from PSID, interviewed in 1994, data pertain to Age 65 Not black Earnings exceed $1,000
19 Earnings example Prior distribution Prior distribution hyperparameters Gaussian priors: Inverse gamma priors: β: µ = 10, τ 2 β = 1 α: µ = 0, τ 2 α = 4 Γ µ = 0, τ 2 γ = 16 2h s χ 2 (2) 2h j s χ 2 (2) Information contribution from prior relative to data is minute.
20 Earnings example Model speci cation comparison Comparison of alternative model speci cations: Modi ed cross-validated log scores Randomly permute all 2698 observations (just to be safe; works for time series too) Select rst T 1 = 2153 for inference (full MCMC, gives θ (m) m = 1,..., M = 10 4 Modi ed cross-validated log scoring rule (Draper and Krnjajic (2005)): T log [p (y t j x t, Y T1, SMR)], t=t 1 +1 p (y t j x t, Y T1, SMR) u M 1 M p y t j θ (m), x t, SMR m=1
21 Earnings example Model speci cation comparison
22 Earnings example Earnings quantiles: Model with m = 8 mixture components
23 Earnings example Median of conditional posterior earnings distribution
24 Earnings example Median of conditional posterior earnings distribution
25 Earnings example Median of conditional posterior earnings distribution
26 Earnings example Earnings quantiles: Model with m = 8 mixture components
27 Earnings example Earnings quantiles: Model with m = 8 mixture components
28 Earnings example Earnings quantiles: Model with m = 8 mixture components
29 Asset Returns Example The data Asset Returns Example The data Standard and Poors 500 daily open (O t ) and close (C t ), (C t 1 = O t ) Dependent variable: Percent log return, y t = 100 log (C t /O t ) Covariates: x 1t = y t 1 x 2t = g x 2,t 1 + (1 g) ja t 1 j κ = s=0 g s jy t 2 s j κ The evidence favors g = 0.95 and κ = 1.
30 Asset Returns Example The model The model: Same as in Geweke and Keane (2006) 0 ew t = Γx t + ζ t, j = arg max ( ew ti ), y t s N β + α j xt, (h h j ) 1 i=1,...,m In Geweke and Keane (2006) linear functions of x t are replaced by polynomial functions of x t. Based on the evidence (Modi ed cross-validated log scores) Γx t become second order polynomials in x t β + α j 0 xt become zero-order polynomials in x t. Mixture of m = 3 components Work with linear functions of x t is currently proceeding.
31 Asset Returns Example Comparison with other models Model comparison: Predictive likelihoods (Bayesian) and Recursive maximum likelihood (non-bayesian) Sample: Prediction: Model Log Predictive likelihood Log Recursive ML Normal iid GARCH(1,1) EGARCH(1,1) t-garch(1,1) Stochastic volatility SMR Dynamic predictive properties of t-garch and stochastic volatility are similar. Dynamic predictive properties of SMR are quite di erent from t-garch and stochastic volatility.
32 Asset Returns Example Posterior means of population moments
33 Asset Returns Example Posterior means of population moments
34 Asset Returns Example Posterior means of population moments
35 Asset Returns Example Posterior means of population moments
36 Asset Returns Example Posterior quantiles
37 Asset Returns Example Posterior quantiles
38 Asset Returns Example Posterior quantiles
39 Asset Returns Example Posterior quantiles
40 Asset Returns Example Posterior quantiles
41 Conclusions Summary Summary The smoothly mixing regressions model is easy to apply is competitive with other models produces interesting results provides fully Bayesian answer to questions of interest
42 Conclusions Current research Current research Convergence of SMR Calibration properties of SMR Contrast with parsimoniously parameterized models in time series applications Extensions into di cult territory More covariates: x t (n 1) Multivariate models p y t `1 j x t n1
Modeling conditional distributions with mixture models: Theory and Inference
Modeling conditional distributions with mixture models: Theory and Inference John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università di Venezia Italia June 2, 2005
More informationModeling conditional distributions with mixture models: Applications in finance and financial decision-making
Modeling conditional distributions with mixture models: Applications in finance and financial decision-making John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università
More informationSmoothly Mixing Regressions
Smoothly Mixing Regressions John Geweke and Michael Keane Departments of Economics and Statistics, University of Iowa john-geweke@uiowa.edu Department of Economics, Yale University michael.keane@yale.edu
More informationOutline. Introduction to Bayesian nonparametrics. Notation and discrete example. Books on Bayesian nonparametrics
Outline Introduction to Bayesian nonparametrics Andriy Norets Department of Economics, Princeton University September, 200 Dirichlet Process (DP) - prior on the space of probability measures. Some applications
More informationSession 5B: A worked example EGARCH model
Session 5B: A worked example EGARCH model John Geweke Bayesian Econometrics and its Applications August 7, worked example EGARCH model August 7, / 6 EGARCH Exponential generalized autoregressive conditional
More informationWageningen Summer School in Econometrics. The Bayesian Approach in Theory and Practice
Wageningen Summer School in Econometrics The Bayesian Approach in Theory and Practice September 2008 Slides for Lecture on Qualitative and Limited Dependent Variable Models Gary Koop, University of Strathclyde
More informationGaussian Mixture Model
Case Study : Document Retrieval MAP EM, Latent Dirichlet Allocation, Gibbs Sampling Machine Learning/Statistics for Big Data CSE599C/STAT59, University of Washington Emily Fox 0 Emily Fox February 5 th,
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationBayesian Semiparametric GARCH Models
Bayesian Semiparametric GARCH Models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics xibin.zhang@monash.edu Quantitative Methods
More informationBayesian Semiparametric GARCH Models
Bayesian Semiparametric GARCH Models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics xibin.zhang@monash.edu Quantitative Methods
More informationJournal of Econometrics
Journal of Econometrics 168 (2012) 332 346 Contents lists available at SciVerse ScienceDirect Journal of Econometrics ournal homepage: www.elsevier.com/locate/econom Bayesian modeling of oint and conditional
More informationSession 2B: Some basic simulation methods
Session 2B: Some basic simulation methods John Geweke Bayesian Econometrics and its Applications August 14, 2012 ohn Geweke Bayesian Econometrics and its Applications Session 2B: Some () basic simulation
More informationOnline appendix to On the stability of the excess sensitivity of aggregate consumption growth in the US
Online appendix to On the stability of the excess sensitivity of aggregate consumption growth in the US Gerdie Everaert 1, Lorenzo Pozzi 2, and Ruben Schoonackers 3 1 Ghent University & SHERPPA 2 Erasmus
More informationChapter 4 Dynamic Bayesian Networks Fall Jin Gu, Michael Zhang
Chapter 4 Dynamic Bayesian Networks 2016 Fall Jin Gu, Michael Zhang Reviews: BN Representation Basic steps for BN representations Define variables Define the preliminary relations between variables Check
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
More informationOnline Appendix to: Marijuana on Main Street? Estimating Demand in Markets with Limited Access
Online Appendix to: Marijuana on Main Street? Estating Demand in Markets with Lited Access By Liana Jacobi and Michelle Sovinsky This appendix provides details on the estation methodology for various speci
More informationUnsupervised Learning
Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College
More informationKatsuhiro Sugita Faculty of Law and Letters, University of the Ryukyus. Abstract
Bayesian analysis of a vector autoregressive model with multiple structural breaks Katsuhiro Sugita Faculty of Law and Letters, University of the Ryukyus Abstract This paper develops a Bayesian approach
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More information13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationBayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
More informationFoundations of Nonparametric Bayesian Methods
1 / 27 Foundations of Nonparametric Bayesian Methods Part II: Models on the Simplex Peter Orbanz http://mlg.eng.cam.ac.uk/porbanz/npb-tutorial.html 2 / 27 Tutorial Overview Part I: Basics Part II: Models
More informationStatistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling
1 / 27 Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling Melih Kandemir Özyeğin University, İstanbul, Turkey 2 / 27 Monte Carlo Integration The big question : Evaluate E p(z) [f(z)]
More informationE cient Importance Sampling
E cient David N. DeJong University of Pittsburgh Spring 2008 Our goal is to calculate integrals of the form Z G (Y ) = ϕ (θ; Y ) dθ. Special case (e.g., posterior moment): Z G (Y ) = Θ Θ φ (θ; Y ) p (θjy
More informationBayesian estimation of bandwidths for a nonparametric regression model with a flexible error density
ISSN 1440-771X Australia Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Bayesian estimation of bandwidths for a nonparametric regression model
More informationOn the Power of Tests for Regime Switching
On the Power of Tests for Regime Switching joint work with Drew Carter and Ben Hansen Douglas G. Steigerwald UC Santa Barbara May 2015 D. Steigerwald (UCSB) Regime Switching May 2015 1 / 42 Motivating
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationGaussian kernel GARCH models
Gaussian kernel GARCH models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics 7 June 2013 Motivation A regression model is often
More informationMotivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University
Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined
More informationLecture Notes based on Koop (2003) Bayesian Econometrics
Lecture Notes based on Koop (2003) Bayesian Econometrics A.Colin Cameron University of California - Davis November 15, 2005 1. CH.1: Introduction The concepts below are the essential concepts used throughout
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationINFERENCE APPROACHES FOR INSTRUMENTAL VARIABLE QUANTILE REGRESSION. 1. Introduction
INFERENCE APPROACHES FOR INSTRUMENTAL VARIABLE QUANTILE REGRESSION VICTOR CHERNOZHUKOV CHRISTIAN HANSEN MICHAEL JANSSON Abstract. We consider asymptotic and finite-sample confidence bounds in instrumental
More informationVariational inference
Simon Leglaive Télécom ParisTech, CNRS LTCI, Université Paris Saclay November 18, 2016, Télécom ParisTech, Paris, France. Outline Introduction Probabilistic model Problem Log-likelihood decomposition EM
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationSIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER. Donald W. K. Andrews. August 2011
SIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER By Donald W. K. Andrews August 2011 COWLES FOUNDATION DISCUSSION PAPER NO. 1815 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS
More informationOn Bayesian Computation
On Bayesian Computation Michael I. Jordan with Elaine Angelino, Maxim Rabinovich, Martin Wainwright and Yun Yang Previous Work: Information Constraints on Inference Minimize the minimax risk under constraints
More informationIntroduction to Applied Bayesian Modeling. ICPSR Day 4
Introduction to Applied Bayesian Modeling ICPSR Day 4 Simple Priors Remember Bayes Law: Where P(A) is the prior probability of A Simple prior Recall the test for disease example where we specified the
More informationBayesian spatial quantile regression
Brian J. Reich and Montserrat Fuentes North Carolina State University and David B. Dunson Duke University E-mail:reich@stat.ncsu.edu Tropospheric ozone Tropospheric ozone has been linked with several adverse
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationMarkov Chain Monte Carlo Methods
Markov Chain Monte Carlo Methods John Geweke University of Iowa, USA 2005 Institute on Computational Economics University of Chicago - Argonne National Laboaratories July 22, 2005 The problem p (θ, ω I)
More informationCompound Random Measures
Compound Random Measures Jim Griffin (joint work with Fabrizio Leisen) University of Kent Introduction: Two clinical studies 3 CALGB8881 3 CALGB916 2 2 β 1 1 β 1 1 1 5 5 β 1 5 5 β Infinite mixture models
More informationGibbs Sampling in Linear Models #2
Gibbs Sampling in Linear Models #2 Econ 690 Purdue University Outline 1 Linear Regression Model with a Changepoint Example with Temperature Data 2 The Seemingly Unrelated Regressions Model 3 Gibbs sampling
More information2014 Preliminary Examination
014 reliminary Examination 1) Standard error consistency and test statistic asymptotic normality in linear models Consider the model for the observable data y t ; x T t n Y = X + U; (1) where is a k 1
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More informationVariational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
More informationECO 513 Fall 2009 C. Sims HIDDEN MARKOV CHAIN MODELS
ECO 513 Fall 2009 C. Sims HIDDEN MARKOV CHAIN MODELS 1. THE CLASS OF MODELS y t {y s, s < t} p(y t θ t, {y s, s < t}) θ t = θ(s t ) P[S t = i S t 1 = j] = h ij. 2. WHAT S HANDY ABOUT IT Evaluating the
More informationSpatially Adaptive Smoothing Splines
Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia speckman@statmissouriedu September 11, 23 Banff 9/7/3 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, =
More informationOutline. Clustering. Capturing Unobserved Heterogeneity in the Austrian Labor Market Using Finite Mixtures of Markov Chain Models
Capturing Unobserved Heterogeneity in the Austrian Labor Market Using Finite Mixtures of Markov Chain Models Collaboration with Rudolf Winter-Ebmer, Department of Economics, Johannes Kepler University
More informationPatterns of Scalable Bayesian Inference Background (Session 1)
Patterns of Scalable Bayesian Inference Background (Session 1) Jerónimo Arenas-García Universidad Carlos III de Madrid jeronimo.arenas@gmail.com June 14, 2017 1 / 15 Motivation. Bayesian Learning principles
More informationPart 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
More informationBayesian Aggregation for Extraordinarily Large Dataset
Bayesian Aggregation for Extraordinarily Large Dataset Guang Cheng 1 Department of Statistics Purdue University www.science.purdue.edu/bigdata Department Seminar Statistics@LSE May 19, 2017 1 A Joint Work
More informationCS839: Probabilistic Graphical Models. Lecture 7: Learning Fully Observed BNs. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 7: Learning Fully Observed BNs Theo Rekatsinas 1 Exponential family: a basic building block For a numeric random variable X p(x ) =h(x)exp T T (x) A( ) = 1
More informationIntegrated Non-Factorized Variational Inference
Integrated Non-Factorized Variational Inference Shaobo Han, Xuejun Liao and Lawrence Carin Duke University February 27, 2014 S. Han et al. Integrated Non-Factorized Variational Inference February 27, 2014
More informationBayesian Regression with Heteroscedastic Error Density and Parametric Mean Function
Bayesian Regression with Heteroscedastic Error Density and Parametric Mean Function Justinas Pelenis pelenis@ihs.ac.at Institute for Advanced Studies, Vienna May 8, 2013 Abstract This paper considers a
More informationBayesian inference for multivariate skew-normal and skew-t distributions
Bayesian inference for multivariate skew-normal and skew-t distributions Brunero Liseo Sapienza Università di Roma Banff, May 2013 Outline Joint research with Antonio Parisi (Roma Tor Vergata) 1. Inferential
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationMultivariate Asset Return Prediction with Mixture Models
Multivariate Asset Return Prediction with Mixture Models Swiss Banking Institute, University of Zürich Introduction The leptokurtic nature of asset returns has spawned an enormous amount of research into
More informationPractical Bayesian Optimization of Machine Learning. Learning Algorithms
Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that
More informationIndex. Pagenumbersfollowedbyf indicate figures; pagenumbersfollowedbyt indicate tables.
Index Pagenumbersfollowedbyf indicate figures; pagenumbersfollowedbyt indicate tables. Adaptive rejection metropolis sampling (ARMS), 98 Adaptive shrinkage, 132 Advanced Photo System (APS), 255 Aggregation
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationBayesian Inference. Chapter 4: Regression and Hierarchical Models
Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Advanced Statistics and Data Mining Summer School
More informationAsymptotics for posterior hazards
Asymptotics for posterior hazards Pierpaolo De Blasi University of Turin 10th August 2007, BNR Workshop, Isaac Newton Intitute, Cambridge, UK Joint work with Giovanni Peccati (Université Paris VI) and
More informationExercises Chapter 4 Statistical Hypothesis Testing
Exercises Chapter 4 Statistical Hypothesis Testing Advanced Econometrics - HEC Lausanne Christophe Hurlin University of Orléans December 5, 013 Christophe Hurlin (University of Orléans) Advanced Econometrics
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationBAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA
BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci
More informationDensity Estimation: ML, MAP, Bayesian estimation
Density Estimation: ML, MAP, Bayesian estimation CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Maximum-Likelihood Estimation Maximum
More informationGAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
More informationCombining Macroeconomic Models for Prediction
Combining Macroeconomic Models for Prediction John Geweke University of Technology Sydney 15th Australasian Macro Workshop April 8, 2010 Outline 1 Optimal prediction pools 2 Models and data 3 Optimal pools
More informationHypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33
Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationEcon Some Bayesian Econometrics
Econ 8208- Some Bayesian Econometrics Patrick Bajari Patrick Bajari () Econ 8208- Some Bayesian Econometrics 1 / 72 Motivation Next we shall begin to discuss Gibbs sampling and Markov Chain Monte Carlo
More informationPractical Bayesian Quantile Regression. Keming Yu University of Plymouth, UK
Practical Bayesian Quantile Regression Keming Yu University of Plymouth, UK (kyu@plymouth.ac.uk) A brief summary of some recent work of us (Keming Yu, Rana Moyeed and Julian Stander). Summary We develops
More informationHierarchical models. Dr. Jarad Niemi. August 31, Iowa State University. Jarad Niemi (Iowa State) Hierarchical models August 31, / 31
Hierarchical models Dr. Jarad Niemi Iowa State University August 31, 2017 Jarad Niemi (Iowa State) Hierarchical models August 31, 2017 1 / 31 Normal hierarchical model Let Y ig N(θ g, σ 2 ) for i = 1,...,
More informationVCMC: Variational Consensus Monte Carlo
VCMC: Variational Consensus Monte Carlo Maxim Rabinovich, Elaine Angelino, Michael I. Jordan Berkeley Vision and Learning Center September 22, 2015 probabilistic models! sky fog bridge water grass object
More informationMinimum Message Length Analysis of the Behrens Fisher Problem
Analysis of the Behrens Fisher Problem Enes Makalic and Daniel F Schmidt Centre for MEGA Epidemiology The University of Melbourne Solomonoff 85th Memorial Conference, 2011 Outline Introduction 1 Introduction
More informationBayesian spatial hierarchical modeling for temperature extremes
Bayesian spatial hierarchical modeling for temperature extremes Indriati Bisono Dr. Andrew Robinson Dr. Aloke Phatak Mathematics and Statistics Department The University of Melbourne Maths, Informatics
More informationIntroduction: structural econometrics. Jean-Marc Robin
Introduction: structural econometrics Jean-Marc Robin Abstract 1. Descriptive vs structural models 2. Correlation is not causality a. Simultaneity b. Heterogeneity c. Selectivity Descriptive models Consider
More informationLecture 5: Spatial probit models. James P. LeSage University of Toledo Department of Economics Toledo, OH
Lecture 5: Spatial probit models James P. LeSage University of Toledo Department of Economics Toledo, OH 43606 jlesage@spatial-econometrics.com March 2004 1 A Bayesian spatial probit model with individual
More informationF denotes cumulative density. denotes probability density function; (.)
BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models
More informationA Fully Nonparametric Modeling Approach to. BNP Binary Regression
A Fully Nonparametric Modeling Approach to Binary Regression Maria Department of Applied Mathematics and Statistics University of California, Santa Cruz SBIES, April 27-28, 2012 Outline 1 2 3 Simulation
More informationSIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER. Donald W. K. Andrews. August 2011 Revised March 2012
SIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER By Donald W. K. Andrews August 2011 Revised March 2012 COWLES FOUNDATION DISCUSSION PAPER NO. 1815R COWLES FOUNDATION FOR
More informationA test of the conditional independence assumption in sample selection models
A test of the conditional independence assumption in sample selection models Martin Huber, Blaise Melly First draft: December 2006, Last changes: September 2012 Abstract: Identi cation in most sample selection
More informationBayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
More informationGMM-based inference in the AR(1) panel data model for parameter values where local identi cation fails
GMM-based inference in the AR() panel data model for parameter values where local identi cation fails Edith Madsen entre for Applied Microeconometrics (AM) Department of Economics, University of openhagen,
More informationAges of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008
Ages of stellar populations from color-magnitude diagrams Paul Baines Department of Statistics Harvard University September 30, 2008 Context & Example Welcome! Today we will look at using hierarchical
More informationBayesian Inference. Chapter 4: Regression and Hierarchical Models
Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative
More informationWeb Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.
Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we
More informationNonparametric Bayes tensor factorizations for big data
Nonparametric Bayes tensor factorizations for big data David Dunson Department of Statistical Science, Duke University Funded from NIH R01-ES017240, R01-ES017436 & DARPA N66001-09-C-2082 Motivation Conditional
More informationSimple Estimators for Monotone Index Models
Simple Estimators for Monotone Index Models Hyungtaik Ahn Dongguk University, Hidehiko Ichimura University College London, James L. Powell University of California, Berkeley (powell@econ.berkeley.edu)
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationBayesian (conditionally) conjugate inference for discrete data models. Jon Forster (University of Southampton)
Bayesian (conditionally) conjugate inference for discrete data models Jon Forster (University of Southampton) with Mark Grigsby (Procter and Gamble?) Emily Webb (Institute of Cancer Research) Table 1:
More informationBayes methods for categorical data. April 25, 2017
Bayes methods for categorical data April 25, 2017 Motivation for joint probability models Increasing interest in high-dimensional data in broad applications Focus may be on prediction, variable selection,
More informationLarge-scale Ordinal Collaborative Filtering
Large-scale Ordinal Collaborative Filtering Ulrich Paquet, Blaise Thomson, and Ole Winther Microsoft Research Cambridge, University of Cambridge, Technical University of Denmark ulripa@microsoft.com,brmt2@cam.ac.uk,owi@imm.dtu.dk
More information