Approximating high-dimensional posteriors with nuisance parameters via integrated rotated Gaussian approximation (IRGA)

Similar documents
Integrated Non-Factorized Variational Inference

Expectation Propagation for Approximate Bayesian Inference

Partial factor modeling: predictor-dependent shrinkage for linear regression

13: Variational inference II

Part 1: Expectation Propagation

Black-box α-divergence Minimization

STA 4273H: Statistical Machine Learning

Expectation Propagation Algorithm

Foundations of Nonparametric Bayesian Methods

Geometric ergodicity of the Bayesian lasso

Sparse Linear Models (10/7/13)

BAGUS: Bayesian Regularization for Graphical Models with Unequal Shrinkage

Lecture 3. Linear Regression II Bastian Leibe RWTH Aachen

Variational Inference. Sargur Srihari

Bayesian Sparse Linear Regression with Unknown Symmetric Error

Machine Learning Basics: Maximum Likelihood Estimation

Statistical Inference

Lecture 13 : Variational Inference: Mean Field Approximation

Bayesian Methods for Machine Learning

ICES REPORT Model Misspecification and Plausibility

PATTERN RECOGNITION AND MACHINE LEARNING

an introduction to bayesian inference

Bayesian Linear Regression

Probabilistic Machine Learning

The Variational Gaussian Approximation Revisited

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu

Probabilistic Graphical Models for Image Analysis - Lecture 1

Consistent high-dimensional Bayesian variable selection via penalized credible regions

STA 4273H: Statistical Machine Learning

Learning with Noisy Labels. Kate Niehaus Reading group 11-Feb-2014

Part 2: Multivariate fmri analysis using a sparsifying spatio-temporal prior

Bayesian Machine Learning

Bayesian linear regression

STA414/2104 Statistical Methods for Machine Learning II

Bayesian Inference and MCMC

Machine Learning using Bayesian Approaches

A Bayesian Treatment of Linear Gaussian Regression

Approximate Message Passing

Machine Learning Techniques for Computer Vision

SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati

Multiple regression. CM226: Machine Learning for Bioinformatics. Fall Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar

Expectation Propagation in Dynamical Systems

Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems

An Algorithm for Bayesian Variable Selection in High-dimensional Generalized Linear Models

Model Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes

Or How to select variables Using Bayesian LASSO

Bayesian Model Averaging

Chapter 17: Undirected Graphical Models

Probabilistic Graphical Networks: Definitions and Basic Results

Linear Dynamical Systems

Large-scale Ordinal Collaborative Filtering

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

Nonparametric Bayes Density Estimation and Regression with High Dimensional Data

GWAS V: Gaussian processes

CPSC 540: Machine Learning

Probabilistic Graphical Models

Nonparametric Bayesian Methods (Gaussian Processes)

Bayesian Approaches Data Mining Selected Technique

L 0 methods. H.J. Kappen Donders Institute for Neuroscience Radboud University, Nijmegen, the Netherlands. December 5, 2011.

Bayesian Machine Learning

Based on slides by Richard Zemel

arxiv: v1 [stat.me] 6 Jul 2017

20: Gaussian Processes

10708 Graphical Models: Homework 2

Bayesian Hidden Markov Models and Extensions

Probabilistic machine learning group, Aalto University Bayesian theory and methods, approximative integration, model

A Survey of L 1. Regression. Céline Cunen, 20/10/2014. Vidaurre, Bielza and Larranaga (2013)

Efficient Variational Inference in Large-Scale Bayesian Compressed Sensing

Gaussian Process Approximations of Stochastic Differential Equations

Estimating Sparse High Dimensional Linear Models using Global-Local Shrinkage

Discussion of Predictive Density Combinations with Dynamic Learning for Large Data Sets in Economics and Finance

Nonparametric Bayes tensor factorizations for big data

Machine Learning for Economists: Part 4 Shrinkage and Sparsity

Variational Learning : From exponential families to multilinear systems

Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference

Probabilistic Reasoning in Deep Learning

Least Absolute Shrinkage is Equivalent to Quadratic Penalization

Advances and Applications in Perfect Sampling

STA 4273H: Statistical Machine Learning

The Minimum Message Length Principle for Inductive Inference

The Behaviour of the Akaike Information Criterion when Applied to Non-nested Sequences of Models

Sampling Methods (11/30/04)

Variational Message Passing. By John Winn, Christopher M. Bishop Presented by Andy Miller

Bayesian Nonparametric Regression for Diabetes Deaths

CS242: Probabilistic Graphical Models Lecture 4A: MAP Estimation & Graph Structure Learning

Regularized Regression A Bayesian point of view

Distributed Bayesian Learning with Stochastic Natural-gradient EP and the Posterior Server

Variational Inference (11/04/13)

Model Selection for Gaussian Processes

Accounting for Complex Sample Designs via Mixture Models

Gaussian Mixture Model

Bayesian shrinkage approach in variable selection for mixed

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING. Non-linear regression techniques Part - II

Linear Regression and Discrimination

Lecture 2: Statistical Decision Theory (Part I)

a b = a T b = a i b i (1) i=1 (Geometric definition) The dot product of two Euclidean vectors a and b is defined by a b = a b cos(θ a,b ) (2)

Latent factor density regression models

g-priors for Linear Regression

Virtual Sensors and Large-Scale Gaussian Processes

Transcription:

Approximating high-dimensional posteriors with nuisance parameters via integrated rotated Gaussian approximation (IRGA) Willem van den Boom Department of Statistics and Applied Probability National University of Singapore Bayesian Computation for High-Dimensional Statistical Models Institute of Mathematical Sciences National University of Singapore September 10, 2018

Collaborators David B. Dunson Galen Reeves Department of Statistical Science Duke University 2

Outline IRGA with Bayesian variable selection as an example Approximation accuracy IRGA in a more general setup 3

IRGA with Bayesian variable selection as an example 4

Standard linear model n x p matrix y = X + n observations p unknown parameters Gaussian errors N(0, 2 I) The entries of β are conditionally independent given hyperparameters θ: py p( ) = p( j ) j=1 5

Example: Bayesian Variable Selection Entries of β conditionally independent given hyperparameters θ py p( ) = p( j ) j=1 Mixed discrete-continuous distribution for marginal prior probability equal to zero distribution of nonzero p( j ) =(1 ) ( j )+ N ( j 0, ), =(, ) 6

Inference Goal: compute posterior marginal distribution of first entry Z p( 1 y) = p( y)d p 2 e.g. for P(β 1 0 y) where p( y) = p( )p(y ) R p( )p(y )d Treat β p 2 as nuisance parameters Number of possible subsets with number of variables p {j β j 0} grows exponentially Posterior is a mixture of 2 p Gaussians Intractable for large p. Therefore, approximations are used for scalable inference. 7

Approximation methods Most popular approximation methods fit into two groups: Sampling based: MCMC, Gibbs sampling, Stochastic Search Variable Selection (George & McCulloch, 1993), or Bayesian Adaptive Sampling (Clyde et al., 2011). Exact solution asymptotically Hard to establish convergence due to exponential and discrete nature of the posterior Deterministic: Variational Bayes (Carbonetto & Stephens, 2012, Ormerod et al., 2017), Expectation propagation (Hernández-Lobato et al., 2015), or EM variable selection (Ročková & George, 2014). Often unclear what quality of approximation is achieved IRGA provides interpretable approximations to posterior marginals and can overcome some of these issues. 8

Overview of IRGA Goal: Marginal posterior, such as p(β 1 y) = π(β y) dβ p 2 Achieved by approximately integrating outβ p 2 Based on a data rotation Transparent approximation allows for theoretical analysis 9

Overview of IRGA 1. Rotate the data to isolate the parameter of interest Introduce an auxiliary variable which summarizes the influence of the nuisance parameters on β p 2 β 1 2. Use any means possible to compute/estimate the posterior mean and posterior variance of the auxiliary variable 3. Apply a Gaussian approximation to the auxiliary variable and solve the one-dimensional integration problem to obtain the posterior approximation for β 1 β 1 10

2 6 1: Rotate Apply rotation matrix 4 Q to the data which 5 zeros out all but the first entry in the first column of the data ỹ = Qy and X = QX Only the first observation depends on 1: px ỹ 1 = x 1,1 1 + x 1,j j + 1 j=2 3 7 2 3 x 1,1 x 1,2 x 1,p 0 x 2,2 x 2,p ỹ = 6 7 4... 5 + 0 x n,2 x n,p N(0, 2 I) ζ auxiliary variable captures influence of nuisance parameters 11

1: Rotate p unknown parameters n observations (the data) 1 y 1 2 y 2 3 y 3 4 y n p 12

1: Rotate y 1 2 y 2 3 y 3 1 4 p y n 13

1: Rotate y 1 y 2 2 y 3 3 1 4 y n p 14

1: Rotate ỹ 1 ỹ 2 2 ỹ 3 3 1 4 ỹ n p 15

1: Rotate ζ ỹ 1 ỹ 2 ỹ 3 2 3 4 ζ = p j=2 x 1,j β j auxiliary variable encapsulates influence of nuisance parameters 1 ỹ n p 16

2: Estimate / compute Compute the posterior mean and variance of the auxiliary variable: E[ζ ỹ n 2 ] Cov[ζ ỹn 2 ] Vector Approximate Message Passing (VAMP, Rangan et al., 2017) LASSO Variational Bayes [your favorite method] The quantities are independent of the target parameter β 1 17

3: Approximate Apply Gaussian approximation to auxiliary variable to compute posterior approximation prior distribution p(β 1 y) p(ỹ 1 ζ, β 1 ) p(β 1 ) p(ζ ỹ p 2 ) dζ Gaussian by assumption on noise replace with Gaussian using mean and variance from previous step Approximation p(ζ ỹ p ) can be accurate even if the prior 2 and posterior are highly non-gaussian: Most low-dimensional projections of high-dimensional distributions are close to Gaussian (projection pursuit, Diaconis & Freedman, 1984). p Additionally, CLT or Bernstein-von Mises: ζ = x 1,j β j j=2 18

Overview of IRGA 1. Rotate the data to isolate the parameter of interest Introduce an auxiliary variable ζ the nuisance parameters β p on β 2 1 which summarizes the influence of 2. Use any means possible to compute/estimate the posterior mean and posterior variance of p(ζ ỹ p ) 2 3. Apply a Gaussian approximation to p(ζ ỹ p ) 2 and solve the onedimensional integration problem to obtain the approximation for p(β 1 y) β 1 19

Approximation accuracy 20

21 Gaussian approximation accuracy 3 1 1 2 3 0 5 10 15 Spike and slab σ 2 = 1 Theoretical Quantiles Sample Quantiles 3 1 1 2 3 4 6 8 10 12 14 Spike and slab σ 2 = 0.5 Theoretical Quantiles Sample Quantiles 3 1 1 2 3 0 5 10 Horseshoe σ 2 = 1 Theoretical Quantiles Sample Quantiles 3 1 1 2 3 0 5 10 Horseshoe σ 2 = 0.5 Theoretical Quantiles Sample Quantiles Q-Q plots of from simulated data with X equal to 3,571 highly correlated SNPs from Friedman et al. (2010). p(ζ ỹ n 2 )

Quadratic Wasserstein distance W 2 {p 1 (x 1 ), p 2 (x 2 )} = inf E ( x 1 x 2 2 ) 1 2 where the infimum is over all joint distributions on such that x 1 p 1 ( ) and x 2 p 2 ( ). (x 1, x 2 ) 22

Gaussian approximation accuracy Theorem If p(β p is nearly isotropic and its distance from its mean 2 ỹp) 2 concentrates, then the Wasserstein distance W 2 {p(ζ ỹ n 2 ), p(ζ ỹn 2 ) } is small for most vectors ( x 1,j ) p. j=2 Isotropy is when independent draws from too correlated. p(β p 2 ỹp 2 ) are not Isotropic Non-isotropic 23

Kullback-Leibler divergence D {p 1 (x 1 ) p 2 (x 2 )} = p 1 (x) log p 1 (x) p 2 (x) dx 24

Posterior approximation accuracy Theorem E [ D {p(β 1 y) IRGA approximation Gaussian approximation p(β 1 y)} ỹ n 2] 1 2σ 2 W2 {p(ζ ỹ 2 n 2 ), p(ζ ỹn 2 ) } where y β N (Xβ, σ 2 I n ) with β p(β) 25

Posterior approximation accuracy Theorem E [ D {p(β 1 y) IRGA approximation Gaussian approximation p(β 1 y)} ỹ n 2] 1 2σ 2 W2 {p(ζ ỹ 2 n 2 ), p(ζ ỹn 2 ) } where y β N (Xβ, σ 2 I n ) with β p(β) 26

Application Error in posterior inclusion prob. 0.0 0.1 0.2 0.3 0.4 Difference from Gibbs estimate 0 10 20 30 40 50 60 Predictor x Integrated rotated Gaussian approximation Expectation propagation (Hernández-Lobato et al., 2015) Variational Bayes (Carbonetto & Stephens, 2012) Diabetes data (Efron et al., 2004) 27

IRGA in a more general setup 28

Standard linear model n x p matrix y = X + p unknown parameters 29

<latexit sha1_base64="4vfe4dnx9/wwclgiw3s9xapw2dy=">aaacbxicbvdlssnafj3uv42vqktddbahiprebn0irufcvrapbekztg/aozmhmxohhg7c+ctuxcji1n9w5984abpq1goxezjnxmbu8wloplksb6owsli0vfjcndfwnza3sts7trklgkkdrjwsby9i4cyehmkkqzswqakpq8sbxmv+6wgezff4p0yxuahph8xnlcgtduv7i3xhth0pfmhh+lpyf6sba7fkplplvtwaam8toydllkpelx05vygmayskcijlx7zi5azekey5je0nkratoir96ggakgckm06ugondrfswhwldocit9fdgsgipr4gnjwoibnlwy8t/ve6i/hm3zwgckajp9ce/4vhfoise95gaqvhie0if03/fdeaeouohz+oq7nmt50nzpgpbvfv2tfy7zomooj10gcrirmeohm5qhtuqry/ogb2in+pjedhejy/pamhid3brhxifp43clha=</latexit> <latexit sha1_base64="4vfe4dnx9/wwclgiw3s9xapw2dy=">aaacbxicbvdlssnafj3uv42vqktddbahiprebn0irufcvrapbekztg/aozmhmxohhg7c+ctuxcji1n9w5984abpq1goxezjnxmbu8wloplksb6owsli0vfjcndfwnza3sts7trklgkkdrjwsby9i4cyehmkkqzswqakpq8sbxmv+6wgezff4p0yxuahph8xnlcgtduv7i3xhth0pfmhh+lpyf6sba7fkplplvtwaam8toydllkpelx05vygmayskcijlx7zi5azekey5je0nkratoir96ggakgckm06ugondrfswhwldocit9fdgsgipr4gnjwoibnlwy8t/ve6i/hm3zwgckajp9ce/4vhfoise95gaqvhie0if03/fdeaeouohz+oq7nmt50nzpgpbvfv2tfy7zomooj10gcrirmeohm5qhtuqry/ogb2in+pjedhejy/pamhid3brhxifp43clha=</latexit> <latexit sha1_base64="4vfe4dnx9/wwclgiw3s9xapw2dy=">aaacbxicbvdlssnafj3uv42vqktddbahiprebn0irufcvrapbekztg/aozmhmxohhg7c+ctuxcji1n9w5984abpq1goxezjnxmbu8wloplksb6owsli0vfjcndfwnza3sts7trklgkkdrjwsby9i4cyehmkkqzswqakpq8sbxmv+6wgezff4p0yxuahph8xnlcgtduv7i3xhth0pfmhh+lpyf6sba7fkplplvtwaam8toydllkpelx05vygmayskcijlx7zi5azekey5je0nkratoir96ggakgckm06ugondrfswhwldocit9fdgsgipr4gnjwoibnlwy8t/ve6i/hm3zwgckajp9ce/4vhfoise95gaqvhie0if03/fdeaeouohz+oq7nmt50nzpgpbvfv2tfy7zomooj10gcrirmeohm5qhtuqry/ogb2in+pjedhejy/pamhid3brhxifp43clha=</latexit> <latexit sha1_base64="4vfe4dnx9/wwclgiw3s9xapw2dy=">aaacbxicbvdlssnafj3uv42vqktddbahiprebn0irufcvrapbekztg/aozmhmxohhg7c+ctuxcji1n9w5984abpq1goxezjnxmbu8wloplksb6owsli0vfjcndfwnza3sts7trklgkkdrjwsby9i4cyehmkkqzswqakpq8sbxmv+6wgezff4p0yxuahph8xnlcgtduv7i3xhth0pfmhh+lpyf6sba7fkplplvtwaam8toydllkpelx05vygmayskcijlx7zi5azekey5je0nkratoir96ggakgckm06ugondrfswhwldocit9fdgsgipr4gnjwoibnlwy8t/ve6i/hm3zwgckajp9ce/4vhfoise95gaqvhie0if03/fdeaeouohz+oq7nmt50nzpgpbvfv2tfy7zomooj10gcrirmeohm5qhtuqry/ogb2in+pjedhejy/pamhid3brhxifp43clha=</latexit> <latexit sha1_base64="f5smxrusgpqnj32wfaywghhhei4=">aaacenicbvc7sgnbfl3rm66vqkxnybdeiuza+aaxkihlfncesjhmtibjknnzdwzwcevat7dxv2wsvcxsroz8gyepiizeuha451zupteiovpacx6sqemz2bn5zik9uls8sppdw79rusij9ujei1koskkccepppjktx5limoc0fltpenrpnkrfingtozgthrgpwimrra1vy+6ehye/xlovbolv9zyvvmyhveiui3wdjurgnhpytr/qjhchihfyar8/aecxlv326xfjqio04vipiuveuppiqrnhtgv7iaixjm3cpbudbtalq2k/uxdtg6aogpe0lttqs6mtkq6v6osbcfzuvonaj/xpqys6cvbnmygttquzlgokhjm0vqehopouan4xabpjzk2itlderjs32uyj7njksedt5q/z7qwbk5zcodkwcvuway7sqweuoageehiez3ifn+vjerhery+bdcoazmzan7k+fgg9np/t</latexit> <latexit sha1_base64="9hskttzqesa0izxf6o1+tbaq3dm=">aaacenicbvdlssnafj3uv42vqks3g0uqfyvx4wpeoiauqxhbagkztkft0mkkzkyeevipbvwc/8gncxuxbly5ef/gsdtfbb1w4xdoudx7rh8xkpvl/ri5qemz2bn8vlmwuls8ulhdu5zhldbxcmhcuforjixy4iiqgklfgqdaz6tqd08zvxphhkqhv1k9ihgbanpaohgpttuko2eh0a2q6vh+cpnejnxvncas3qbqvegopm1fq2t1c04cewikxx/muft4bvyahs+3gei4ifxhhqss21akvaqjrtejqengkkqid1gb1dxksc/2kn6mfg5ppglboddnfeyzoxmjcqtsbb52zjfkcs0j/9pqswrtewnluawix4nfrzhbntz7egxsqbbipq0qfltfcnehcysvfqopn2cpr54ezm7pogrf2mxycrhuhmyatbanblahyuacviadmlght+afvbopxrpxzrwprdljolmo/ptx+qutuafh</latexit> <latexit sha1_base64="9hskttzqesa0izxf6o1+tbaq3dm=">aaacenicbvdlssnafj3uv42vqks3g0uqfyvx4wpeoiauqxhbagkztkft0mkkzkyeevipbvwc/8gncxuxbly5ef/gsdtfbb1w4xdoudx7rh8xkpvl/ri5qemz2bn8vlmwuls8ulhdu5zhldbxcmhcuforjixy4iiqgklfgqdaz6tqd08zvxphhkqhv1k9ihgbanpaohgpttuko2eh0a2q6vh+cpnejnxvncas3qbqvegopm1fq2t1c04cewikxx/muft4bvyahs+3gei4ifxhhqss21akvaqjrtejqengkkqid1gb1dxksc/2kn6mfg5ppglboddnfeyzoxmjcqtsbb52zjfkcs0j/9pqswrtewnluawix4nfrzhbntz7egxsqbbipq0qfltfcnehcysvfqopn2cpr54ezm7pogrf2mxycrhuhmyatbanblahyuacviadmlght+afvbopxrpxzrwprdljolmo/ptx+qutuafh</latexit> <latexit sha1_base64="6bcbuzr6v+q9ehf2nba8s4thee4=">aaacenicbvbns8naen34wetx1aoxxskih5j48enufmrjfwmltsyb7bzdutne3ylqqv6df/+kfw8qxj1589+4axuorq8ghu/nmdmviaxxyns/1tz8wulscmglulq2vrfz2tq+01gikhnpjclvcihmgkvmagfbgrfijaweqwf9i9yvpzklesrvyrazpyrdytucejbsq3r4eya9keavcnkb7d6vhvcqafyqyq+iscs0l+2kpqsejc6ylneytvbp22thnamzbcqi1k3hjsfpiqjobcukxqjztgifdfntuenmyj8d/pthfao0csdspitgoto5kzjq60eymm78rj3t5ej/xjobzomfchknwcqdleokaptv84bwmytgqqwmivrxcyumpaiibrnj0ytgtl88s9yjymnfuxbk1fnxggw0i/bqaxlqmaqik1rdlqlocb2gn/rupvuv1of1owqds8yzo+gprk9fudoebg==</latexit> Linear model with nuisance parameter n x p matrix n x q matrix y = X + F (Z)+ p < n unknown parameters nuisance parameter F : R n q! R n 30

<latexit sha1_base64="dynxuojsdpvdu8vb72qkenvwvqa=">aaacuxiclzfnb9qweiadukbdvpzy5gj1bsqirjkquqneoqkpai9vov26yp2uhk+tdws7kt1bwln5kftgv8hj5tapopbkll49m+pxjnnscgtr9dsih6w8fpr4da335omz5y/6l9e/26iyji9yiqsztqnlumg+aggsj0vdquolp0+vvjtx85/cwfhom1iupfe01yitjijh0/4v962+ommltjsyyzjyofsh+c2xqnleyc6odmc1ktwd4nrya1kxbnnb8l5db5s/3m2rfvn6xnglbxw0lt0grurzidumtdid/nfl03/f7/shsr7ry9ofrmoofb5v4s4muketaf+azapwka6bswrtji5ksbw1ijjkdy9ulpeuxdgct7zvvhgbuhb1nx7jyqxnhffha27pzqphlbullfrmzjr7n9bav8umfwqfeyd0wqhxbnkoqysgajf/igfccazy4q1lrvi3yjanhjlwv93zs4jvjnzfjlahe8p4685g/3o3jvx0gm2gtrsjxbspdtejgiew7azjkav5+clmw3l4uuwng67mfbql0p4bktznfa==</latexit> <latexit sha1_base64="dynxuojsdpvdu8vb72qkenvwvqa=">aaacuxiclzfnb9qweiadukbdvpzy5gj1bsqirjkquqneoqkpai9vov26yp2uhk+tdws7kt1bwln5kftgv8hj5tapopbkll49m+pxjnnscgtr9dsih6w8fpr4da335omz5y/6l9e/26iyji9yiqsztqnlumg+aggsj0vdquolp0+vvjtx85/cwfhom1iupfe01yitjijh0/4v962+ommltjsyyzjyofsh+c2xqnleyc6odmc1ktwd4nrya1kxbnnb8l5db5s/3m2rfvn6xnglbxw0lt0grurzidumtdid/nfl03/f7/shsr7ry9ofrmoofb5v4s4muketaf+azapwka6bswrtji5ksbw1ijjkdy9ulpeuxdgct7zvvhgbuhb1nx7jyqxnhffha27pzqphlbullfrmzjr7n9bav8umfwqfeyd0wqhxbnkoqysgajf/igfccazy4q1lrvi3yjanhjlwv93zs4jvjnzfjlahe8p4685g/3o3jvx0gm2gtrsjxbspdtejgiew7azjkav5+clmw3l4uuwng67mfbql0p4bktznfa==</latexit> <latexit sha1_base64="dynxuojsdpvdu8vb72qkenvwvqa=">aaacuxiclzfnb9qweiadukbdvpzy5gj1bsqirjkquqneoqkpai9vov26yp2uhk+tdws7kt1bwln5kftgv8hj5tapopbkll49m+pxjnnscgtr9dsih6w8fpr4da335omz5y/6l9e/26iyji9yiqsztqnlumg+aggsj0vdquolp0+vvjtx85/cwfhom1iupfe01yitjijh0/4v962+ommltjsyyzjyofsh+c2xqnleyc6odmc1ktwd4nrya1kxbnnb8l5db5s/3m2rfvn6xnglbxw0lt0grurzidumtdid/nfl03/f7/shsr7ry9ofrmoofb5v4s4muketaf+azapwka6bswrtji5ksbw1ijjkdy9ulpeuxdgct7zvvhgbuhb1nx7jyqxnhffha27pzqphlbullfrmzjr7n9bav8umfwqfeyd0wqhxbnkoqysgajf/igfccazy4q1lrvi3yjanhjlwv93zs4jvjnzfjlahe8p4685g/3o3jvx0gm2gtrsjxbspdtejgiew7azjkav5+clmw3l4uuwng67mfbql0p4bktznfa==</latexit> <latexit sha1_base64="dynxuojsdpvdu8vb72qkenvwvqa=">aaacuxiclzfnb9qweiadukbdvpzy5gj1bsqirjkquqneoqkpai9vov26yp2uhk+tdws7kt1bwln5kftgv8hj5tapopbkll49m+pxjnnscgtr9dsih6w8fpr4da335omz5y/6l9e/26iyji9yiqsztqnlumg+aggsj0vdquolp0+vvjtx85/cwfhom1iupfe01yitjijh0/4v962+ommltjsyyzjyofsh+c2xqnleyc6odmc1ktwd4nrya1kxbnnb8l5db5s/3m2rfvn6xnglbxw0lt0grurzidumtdid/nfl03/f7/shsr7ry9ofrmoofb5v4s4muketaf+azapwka6bswrtji5ksbw1ijjkdy9ulpeuxdgct7zvvhgbuhb1nx7jyqxnhffha27pzqphlbullfrmzjr7n9bav8umfwqfeyd0wqhxbnkoqysgajf/igfccazy4q1lrvi3yjanhjlwv93zs4jvjnzfjlahe8p4685g/3o3jvx0gm2gtrsjxbspdtejgiew7azjkav5+clmw3l4uuwng67mfbql0p4bktznfa==</latexit> <latexit sha1_base64="vgixoijal4spmy24zpirblyelfk=">aaab63icbva9swnbej2lxzf+rs0voqycvbizubugjwucngkkr9jb7cvl9vao3tkhhimtbsxubp0p+r12/gb/hjupqhmfddzem2fmxpairtfxvqzc0vlk6lp+vbcxubw9u9zdu9nxqijzacxi1qiizojl5ifhwrqjyiqkbksh/euxx79nsvny3uigyx5eupkhnbi0ktd6yejaxzjtdiawf4k7i6xk4aj2/xg0qraln61otnoisascan10nqt9jcjkvlbhozvqlhdaj13wnfssigk/mxw7te+m0rhdwjmsae/u3xmzibqerihpjaj29lw3fv/zmimgf37gzziik3s6keyfjbe9/tzucmuoioehhcpubrvpjyhc0ertmcg48y8veu+sffl2ayamk5gidwdwdkfgwjlu4aaq4aefdk/waq+wtj6tn+t92pqzzjp78afwxw8+nji3</latexit> sha1_base64="xmnlatczj/0h8crjzuzzz+slrua=">aaab63icbvbnt8jaej36ififevtssew8kdalein68yijfrjoyhazwobtttmdmidhn3jxomarf8ib/8yfeldwjzo8vdetmxlrjouhz/t2vlbx1jc2s1vl7z3dvf3kwegdsxpnmecpthurygalubiqiimttcnlionnahgz9zupqi1i1t2nmgwt1lcifpyrlyloexlrvqpezzvbxsz+qapqongtfhv6kc8tvmqlm6btexmfy6zjcimtcic3mde+zh1sw6pygiycz46dukdw6blxqm0pcmfq74kxs4wzjzhttbgnzki3ff/z2jnfl+fyqcwnvhy+km6ls6k7/dztcy2c5mgsxrwwt7p8wdtjzpmp2xd8xzexsxbeu6r5d161fl2kuyjjoiez8oec6naldqiag4bneiu3rzkvzrvzmw9dcyqzi/gd5/mhvaioja==</latexit> <latexit sha1_base64="s1pfrennkgbghukbb0henxrw2t4=">aaab63icbzc7sgnbfibpxlumt6ilfonbsaq7nmoxtlfmwdwbzamzk9lkyozsmnnwceuewczcxdz38dns7hwuj5dce38y+pj/c5hztphkydb1v5zcyura+kzxs7s1vbo7v94/uddjphn3wsit3qqp4vio7qnayvup5jqojw+gw5tj3nzg2ohe3eeo5ufm+0peglg0lt8jodjuuejw3animnhzqnsopxrfafdvlj87vyrlmvfijdwm7bkpbjnvkjjk41inmzylbej7vg1r0zibij8ooyan1umrknh2ksrt93dhtmnjrnfok2oka7oytcz/snag0wwqc5vmybwbfrrlkmbcjputntccorxzoewloytha6opq3ufkj2ct7jymvjn1auq1/aqtwuyqqhhcajn4mef1oaw6uadawgp8awvjnkenffnbvzacoy9h/bhzvsppx2qug==</latexit> <latexit sha1_base64="86la2fyceuu8lzvatfuadn2atbs=">aaab63icbzc7sgnbfibpeo3xfrvuzdaivmhxru2cnpyjucaqlgf2mkmgzm4um2efsks0trfqsfud8hx2pomv4ersaoipax//fw5zzgktkqy67peztlyyurae28hvbm3v7bb29u9nngrgfrblwnddarguivsoupj6ojmnqslryf9mnnceudyivnc4shgq0a4shceowstvhhxpq1b0s+5ezbg8grtlr6pq9+pxqniqfdbbmusjrpbjakzdcxmmmqprmmmh+wzqeejzn3z5w6kietdbnhl2se6t0yadwnunkezc3x0zjywzrkgtjcj2zhw2nv/lgil2lonmqcrfrtj0o04qcczkvdlpc80zyoefyrswsxlwo5oytpfj2yn48ysvgn9euip5va9yvoapcnaij3aghlxagw6haj4wepael/dqkofzexpep6vlzqznap7i+fgbglysia==</latexit> <latexit sha1_base64="86la2fyceuu8lzvatfuadn2atbs=">aaab63icbzc7sgnbfibpeo3xfrvuzdaivmhxru2cnpyjucaqlgf2mkmgzm4um2efsks0trfqsfud8hx2pomv4ersaoipax//fw5zzgktkqy67peztlyyurae28hvbm3v7bb29u9nngrgfrblwnddarguivsoupj6ojmnqslryf9mnnceudyivnc4shgq0a4shceowstvhhxpq1b0s+5ezbg8grtlr6pq9+pxqniqfdbbmusjrpbjakzdcxmmmqprmmmh+wzqeejzn3z5w6kietdbnhl2se6t0yadwnunkezc3x0zjywzrkgtjcj2zhw2nv/lgil2lonmqcrfrtj0o04qcczkvdlpc80zyoefyrswsxlwo5oytpfj2yn48ysvgn9euip5va9yvoapcnaij3aghlxagw6haj4wepael/dqkofzexpep6vlzqznap7i+fgbglysia==</latexit> <latexit sha1_base64="jpanwujdoqhzfi77wyjqcntcajk=">aaab63icbvbnt8jaej3if+ix6tflizhxrfov6o3oxsmmvkigidtlchu222z3akiiv8glbzve/upe/dcu0iocl5nk5b2zzmylmikmed63u1pb39jckm9xdnb39g+qh0epjs01x4cnmtxtibmuqmfagis2m40siss2othtzg89otyivq80zjbm2ecjwhbgvgq6erlrvwte3zvdxsv+qwpqonmrfnx7kc8tvmqlm6bjexmfe6zjcintsjc3mde+ygpswkpygiaczi+dumdw6btxqm0pcufq74kjs4wzj5httbgnzbi3e//zojnfv+feqcwnvhyxkm6ls6k7+9ztc42c5ngsxrwwt7p8ydtjzpop2bd85zdxsxbrv677936tcvokuyytoivz8oesgnahtqiag4bneiu3rzkvzrvzswgtocxmmfyb8/kdl8iodq==</latexit> <latexit sha1_base64="s1pfrennkgbghukbb0henxrw2t4=">aaab63icbzc7sgnbfibpxlumt6ilfonbsaq7nmoxtlfmwdwbzamzk9lkyozsmnnwceuewczcxdz38dns7hwuj5dce38y+pj/c5hztphkydb1v5zcyura+kzxs7s1vbo7v94/uddjphn3wsit3qqp4vio7qnayvup5jqojw+gw5tj3nzg2ohe3eeo5ufm+0peglg0lt8jodjuuejw3animnhzqnsopxrfafdvlj87vyrlmvfijdwm7bkpbjnvkjjk41inmzylbej7vg1r0zibij8ooyan1umrknh2ksrt93dhtmnjrnfok2oka7oytcz/snag0wwqc5vmybwbfrrlkmbcjputntccorxzoewloytha6opq3ufkj2ct7jymvjn1auq1/aqtwuyqqhhcajn4mef1oaw6uadawgp8awvjnkenffnbvzacoy9h/bhzvsppx2qug==</latexit> <latexit sha1_base64="86la2fyceuu8lzvatfuadn2atbs=">aaab63icbzc7sgnbfibpeo3xfrvuzdaivmhxru2cnpyjucaqlgf2mkmgzm4um2efsks0trfqsfud8hx2pomv4ersaoipax//fw5zzgktkqy67peztlyyurae28hvbm3v7bb29u9nngrgfrblwnddarguivsoupj6ojmnqslryf9mnnceudyivnc4shgq0a4shceowstvhhxpq1b0s+5ezbg8grtlr6pq9+pxqniqfdbbmusjrpbjakzdcxmmmqprmmmh+wzqeejzn3z5w6kietdbnhl2se6t0yadwnunkezc3x0zjywzrkgtjcj2zhw2nv/lgil2lonmqcrfrtj0o04qcczkvdlpc80zyoefyrswsxlwo5oytpfj2yn48ysvgn9euip5va9yvoapcnaij3aghlxagw6haj4wepael/dqkofzexpep6vlzqznap7i+fgbglysia==</latexit> <latexit sha1_base64="86la2fyceuu8lzvatfuadn2atbs=">aaab63icbzc7sgnbfibpeo3xfrvuzdaivmhxru2cnpyjucaqlgf2mkmgzm4um2efsks0trfqsfud8hx2pomv4ersaoipax//fw5zzgktkqy67peztlyyurae28hvbm3v7bb29u9nngrgfrblwnddarguivsoupj6ojmnqslryf9mnnceudyivnc4shgq0a4shceowstvhhxpq1b0s+5ezbg8grtlr6pq9+pxqniqfdbbmusjrpbjakzdcxmmmqprmmmh+wzqeejzn3z5w6kietdbnhl2se6t0yadwnunkezc3x0zjywzrkgtjcj2zhw2nv/lgil2lonmqcrfrtj0o04qcczkvdlpc80zyoefyrswsxlwo5oytpfj2yn48ysvgn9euip5va9yvoapcnaij3aghlxagw6haj4wepael/dqkofzexpep6vlzqznap7i+fgbglysia==</latexit> <latexit sha1_base64="jpanwujdoqhzfi77wyjqcntcajk=">aaab63icbvbnt8jaej3if+ix6tflizhxrfov6o3oxsmmvkigidtlchu222z3akiiv8glbzve/upe/dcu0iocl5nk5b2zzmylmikmed63u1pb39jckm9xdnb39g+qh0epjs01x4cnmtxtibmuqmfagis2m40siss2othtzg89otyivq80zjbm2ecjwhbgvgq6erlrvwte3zvdxsv+qwpqonmrfnx7kc8tvmqlm6bjexmfe6zjcintsjc3mde+ygpswkpygiaczi+dumdw6btxqm0pcufq74kjs4wzj5httbgnzbi3e//zojnfv+feqcwnvhyxkm6ls6k7+9ztc42c5ngsxrwwt7p8ydtjzpop2bd85zdxsxbrv677936tcvokuyytoivz8oesgnahtqiag4bneiu3rzkvzrvzswgtocxmmfyb8/kdl8iodq==</latexit> <latexit sha1_base64="39wqq+cbicqvo/g1cciy5gfbvo4=">aaab63icbvc7tsnaefyhvwivacuujyikqsimaboigspewgqpsalz5zyccj5bd2ukyoubacga0fipfacdhz/c5vfawkgrjwz2tbstpliydn0vp7c0vlk6vlwvbwxube+ud/dutzjpxn2wyetfhdrwkrt3uadkd6nmna4lb4adq7hfvofaietd4ddlqux7skscubss337gsdvlilt1jyclxjursu3wo/enapvo+bpdtvgwc4vmumnanptikfongkk+kruzw1pkbrthw5yqgnmt5jnjr+tykl0sjdqwqjjrf0/kndzmgie2m6byn/pewpzpa2uynqe5ugmgxlhpoiitbbmy/px0heym5dasyrswtxlwp5oytpmubaje/mulxd+txls9hg3jeqyowgecwql4cay1uiy6+mbawcm8w4ujncfn1xmbthac2cw+/ihz/gnhvzdr</latexit> <latexit sha1_base64="vgixoijal4spmy24zpirblyelfk=">aaab63icbva9swnbej2lxzf+rs0voqycvbizubugjwucngkkr9jb7cvl9vao3tkhhimtbsxubp0p+r12/gb/hjupqhmfddzem2fmxpairtfxvqzc0vlk6lp+vbcxubw9u9zdu9nxqijzacxi1qiizojl5ifhwrqjyiqkbksh/euxx79nsvny3uigyx5eupkhnbi0ktd6yejaxzjtdiawf4k7i6xk4aj2/xg0qraln61otnoisascan10nqt9jcjkvlbhozvqlhdaj13wnfssigk/mxw7te+m0rhdwjmsae/u3xmzibqerihpjaj29lw3fv/zmimgf37gzziik3s6keyfjbe9/tzucmuoioehhcpubrvpjyhc0ertmcg48y8veu+sffl2ayamk5gidwdwdkfgwjlu4aaq4aefdk/waq+wtj6tn+t92pqzzjp78afwxw8+nji3</latexit> sha1_base64="xmnlatczj/0h8crjzuzzz+slrua=">aaab63icbvbnt8jaej36ififevtssew8kdalein68yijfrjoyhazwobtttmdmidhn3jxomarf8ib/8yfeldwjzo8vdetmxlrjouhz/t2vlbx1jc2s1vl7z3dvf3kwegdsxpnmecpthurygalubiqiimttcnlionnahgz9zupqi1i1t2nmgwt1lcifpyrlyloexlrvqpezzvbxsz+qapqongtfhv6kc8tvmqlm6btexmfy6zjcimtcic3mde+zh1sw6pygiycz46dukdw6blxqm0pcmfq74kxs4wzjzhttbgnzki3ff/z2jnfl+fyqcwnvhy+km6ls6k7/dztcy2c5mgsxrwwt7p8wdtjzpmp2xd8xzexsxbeu6r5d161fl2kuyjjoiez8oec6naldqiag4bneiu3rzkvzrvzmw9dcyqzi/gd5/mhvaioja==</latexit> Overview of IRGA Apply rotation matrix Q = (R, S) where R (n x p) is an orthonormal basis for the column space of X and S (n x (n-p)) for the orthogonal complement. Then, the likelihood for Q T y factorizes as R T y,f N R T X + R T F (Z), 2 I p S T y,f N S T F (Z), 2 I n p Only the first distribution involves Auxiliary variable <latexit sha1_base64="39wqq+cbicqvo/g1cciy5gfbvo4=">aaab63icbvc7tsnaefyhvwivacuujyikqsimaboigspewgqpsalz5zyccj5bd2ukyoubacga0fipfacdhz/c5vfawkgrjwz2tbstpliydn0vp7c0vlk6vlwvbwxube+ud/dutzjpxn2wyetfhdrwkrt3uadkd6nmna4lb4adq7hfvofaietd4ddlqux7skscubss337gsdvlilt1jyclxjursu3wo/enapvo+bpdtvgwc4vmumnanptikfongkk+kruzw1pkbrthw5yqgnmt5jnjr+tykl0sjdqwqjjrf0/kndzmgie2m6byn/pewpzpa2uynqe5ugmgxlhpoiitbbmy/px0heym5dasyrswtxlwp5oytpmubaje/mulxd+txls9hg3jeqyowgecwql4cay1uiy6+mbawcm8w4ujncfn1xmbthac2cw+/ihz/gnhvzdr</latexit> captures influence of F on Approximate p(ζ S T y) by a Gaussian Compute p(β y) p(r T y β, ζ) p(β) p(ζ S T y) dζ 31

JBOZXCXMwAFdFRz6ECX5EvK8=">AAACoXicbVHbjtMwEHXCbSm3Ao9IaESFtCuhKomQ4HEFiwQPoALb7UpxWjmu01prJ8GeIIWQ/+I7eONvcNMg6JaRRj4+x55jz6SlkhaD4JfnX7l67fqNg5uDW7fv3L03vP/gzBaV4WLKC1WY85RZoWQupihRifPSCKZTJWbpxeuNPvsqjJVFfop1KRLNVrnMJGfoqMXwxxuqRIYxnHQrbcpDmgpkVMsl1EcU6HeXa4ZN2e4qRq7WSFvo9s3ndn4KNWzZZOCKfQGaGcabsG0iauVKs3nUwmwRzSPovcCZfXMld2ocPfvrty/CH9/FcBSMgy5gH4Q9GJE+JovhT7oseKVFjlwxa+MwKDFpmEHJlWgHtLKiZPyCrUTsYM60sEnTdbiFp45ZQlYYlzlCx/57o2Ha2lqn7qRmuLaXtQ35Py2uMHuZNDIvKxQ53xpllQIsYDMuWEojOKraAcaNdG8Fvmaur+iGOnBNCC9/eR+cReMwGIcfn4+OX/XtOCCPyBNySELyghyTt2RCpoR7j70T7733wR/57/yJ/2l71Pf6Ow/JTvjxb/+CzEQ=</latexit> JBOZXCXMwAFdFRz6ECX5EvK8=">AAACoXicbVHbjtMwEHXCbSm3Ao9IaESFtCuhKomQ4HEFiwQPoALb7UpxWjmu01prJ8GeIIWQ/+I7eONvcNMg6JaRRj4+x55jz6SlkhaD4JfnX7l67fqNg5uDW7fv3L03vP/gzBaV4WLKC1WY85RZoWQupihRifPSCKZTJWbpxeuNPvsqjJVFfop1KRLNVrnMJGfoqMXwxxuqRIYxnHQrbcpDmgpkVMsl1EcU6HeXa4ZN2e4qRq7WSFvo9s3ndn4KNWzZZOCKfQGaGcabsG0iauVKs3nUwmwRzSPovcCZfXMld2ocPfvrty/CH9/FcBSMgy5gH4Q9GJE+JovhT7oseKVFjlwxa+MwKDFpmEHJlWgHtLKiZPyCrUTsYM60sEnTdbiFp45ZQlYYlzlCx/57o2Ha2lqn7qRmuLaXtQ35Py2uMHuZNDIvKxQ53xpllQIsYDMuWEojOKraAcaNdG8Fvmaur+iGOnBNCC9/eR+cReMwGIcfn4+OX/XtOCCPyBNySELyghyTt2RCpoR7j70T7733wR/57/yJ/2l71Pf6Ow/JTvjxb/+CzEQ=</latexit> JBOZXCXMwAFdFRz6ECX5EvK8=">AAACoXicbVHbjtMwEHXCbSm3Ao9IaESFtCuhKomQ4HEFiwQPoALb7UpxWjmu01prJ8GeIIWQ/+I7eONvcNMg6JaRRj4+x55jz6SlkhaD4JfnX7l67fqNg5uDW7fv3L03vP/gzBaV4WLKC1WY85RZoWQupihRifPSCKZTJWbpxeuNPvsqjJVFfop1KRLNVrnMJGfoqMXwxxuqRIYxnHQrbcpDmgpkVMsl1EcU6HeXa4ZN2e4qRq7WSFvo9s3ndn4KNWzZZOCKfQGaGcabsG0iauVKs3nUwmwRzSPovcCZfXMld2ocPfvrty/CH9/FcBSMgy5gH4Q9GJE+JovhT7oseKVFjlwxa+MwKDFpmEHJlWgHtLKiZPyCrUTsYM60sEnTdbiFp45ZQlYYlzlCx/57o2Ha2lqn7qRmuLaXtQ35Py2uMHuZNDIvKxQ53xpllQIsYDMuWEojOKraAcaNdG8Fvmaur+iGOnBNCC9/eR+cReMwGIcfn4+OX/XtOCCPyBNySELyghyTt2RCpoR7j70T7733wR/57/yJ/2l71Pf6Ow/JTvjxb/+CzEQ=</latexit> JBOZXCXMwAFdFRz6ECX5EvK8=">AAACoXicbVHbjtMwEHXCbSm3Ao9IaESFtCuhKomQ4HEFiwQPoALb7UpxWjmu01prJ8GeIIWQ/+I7eONvcNMg6JaRRj4+x55jz6SlkhaD4JfnX7l67fqNg5uDW7fv3L03vP/gzBaV4WLKC1WY85RZoWQupihRifPSCKZTJWbpxeuNPvsqjJVFfop1KRLNVrnMJGfoqMXwxxuqRIYxnHQrbcpDmgpkVMsl1EcU6HeXa4ZN2e4qRq7WSFvo9s3ndn4KNWzZZOCKfQGaGcabsG0iauVKs3nUwmwRzSPovcCZfXMld2ocPfvrty/CH9/FcBSMgy5gH4Q9GJE+JovhT7oseKVFjlwxa+MwKDFpmEHJlWgHtLKiZPyCrUTsYM60sEnTdbiFp45ZQlYYlzlCx/57o2Ha2lqn7qRmuLaXtQ35Py2uMHuZNDIvKxQ53xpllQIsYDMuWEojOKraAcaNdG8Fvmaur+iGOnBNCC9/eR+cReMwGIcfn4+OX/XtOCCPyBNySELyghyTt2RCpoR7j70T7733wR/57/yJ/2l71Pf6Ow/JTvjxb/+CzEQ=</latexit> <latexit sha1_base64="n5zgcu6hmy5da0/qqa9eay68ez0=">aaacmnicbvbdsxtbfj21h+pq29g+9mvoecyvsbsk6pu0ig0fikjtqznpudu5mwzozc4zd4ww5d/54h/pq6h0wyqv/rfoyh6q9sda4zx7uxnovmrlkul+rusphj56vlyygq+tp3n6rlhx/ksvkiexiwtdug4ghrwy2cffgrulqzczxups5p3mpz5f51vhv9ckxl6bkvw5kkbbgjq+tbgwashfhgtbb8irewsgsqrdf54kjtmjou7off6gh2x9e73nw9jiwpc2/ziwsxbqncyxhtsassuzg98n6yi02qkhg8ypmsxkzdcs1ob9l01k6tfgsemn01huhkuqjzdcxqawdpp+pc885ztbgfk8cofz4np1340ajpctk4xjwrx/15uj//n6few7/vrzsik08uzqxmlobz8vyifkosq9cqsku+gvxi7bgarqcxxkso9gvk867dzekz1629x/t2hjhb1kr9gws9ko22cf2chrmmno2e92wf5e59hv6dk6uhldihy7l9gtrh+vadmaqm0=</latexit> <latexit sha1_base64="n5zgcu6hmy5da0/qqa9eay68ez0=">aaacmnicbvbdsxtbfj21h+pq29g+9mvoecyvsbsk6pu0ig0fikjtqznpudu5mwzozc4zd4ww5d/54h/pq6h0wyqv/rfoyh6q9sda4zx7uxnovmrlkul+rusphj56vlyygq+tp3n6rlhx/ksvkiexiwtdug4ghrwy2cffgrulqzczxups5p3mpz5f51vhv9ckxl6bkvw5kkbbgjq+tbgwashfhgtbb8irewsgsqrdf54kjtmjou7off6gh2x9e73nw9jiwpc2/ziwsxbqncyxhtsassuzg98n6yi02qkhg8ypmsxkzdcs1ob9l01k6tfgsemn01huhkuqjzdcxqawdpp+pc885ztbgfk8cofz4np1340ajpctk4xjwrx/15uj//n6few7/vrzsik08uzqxmlobz8vyifkosq9cqsku+gvxi7bgarqcxxkso9gvk867dzekz1629x/t2hjhb1kr9gws9ko22cf2chrmmno2e92wf5e59hv6dk6uhldihy7l9gtrh+vadmaqm0=</latexit> <latexit sha1_base64="n5zgcu6hmy5da0/qqa9eay68ez0=">aaacmnicbvbdsxtbfj21h+pq29g+9mvoecyvsbsk6pu0ig0fikjtqznpudu5mwzozc4zd4ww5d/54h/pq6h0wyqv/rfoyh6q9sda4zx7uxnovmrlkul+rusphj56vlyygq+tp3n6rlhx/ksvkiexiwtdug4ghrwy2cffgrulqzczxups5p3mpz5f51vhv9ckxl6bkvw5kkbbgjq+tbgwashfhgtbb8irewsgsqrdf54kjtmjou7off6gh2x9e73nw9jiwpc2/ziwsxbqncyxhtsassuzg98n6yi02qkhg8ypmsxkzdcs1ob9l01k6tfgsemn01huhkuqjzdcxqawdpp+pc885ztbgfk8cofz4np1340ajpctk4xjwrx/15uj//n6few7/vrzsik08uzqxmlobz8vyifkosq9cqsku+gvxi7bgarqcxxkso9gvk867dzekz1629x/t2hjhb1kr9gws9ko22cf2chrmmno2e92wf5e59hv6dk6uhldihy7l9gtrh+vadmaqm0=</latexit> <latexit sha1_base64="n5zgcu6hmy5da0/qqa9eay68ez0=">aaacmnicbvbdsxtbfj21h+pq29g+9mvoecyvsbsk6pu0ig0fikjtqznpudu5mwzozc4zd4ww5d/54h/pq6h0wyqv/rfoyh6q9sda4zx7uxnovmrlkul+rusphj56vlyygq+tp3n6rlhx/ksvkiexiwtdug4ghrwy2cffgrulqzczxups5p3mpz5f51vhv9ckxl6bkvw5kkbbgjq+tbgwashfhgtbb8irewsgsqrdf54kjtmjou7off6gh2x9e73nw9jiwpc2/ziwsxbqncyxhtsassuzg98n6yi02qkhg8ypmsxkzdcs1ob9l01k6tfgsemn01huhkuqjzdcxqawdpp+pc885ztbgfk8cofz4np1340ajpctk4xjwrx/15uj//n6few7/vrzsik08uzqxmlobz8vyifkosq9cqsku+gvxi7bgarqcxxkso9gvk867dzekz1629x/t2hjhb1kr9gws9ko22cf2chrmmno2e92wf5e59hv6dk6uhldihy7l9gtrh+vadmaqm0=</latexit> <latexit sha1_base64="acp/iepfd0w5i/saqobwiiubvow=">aaacbhicbvdlsgmxfm3uv62vuzfdbivqgpqzexrzfirlcvybnafk0kwbmmrckhfk6cknv+lghsju/qh3/o1powttpxdh5jx7yb0nkoxq43nftm5tfwnzk79d2nnd2z9wd49aokkvjk2cser1iqqjo4i0dtwmdkqiieemtkprzcxvpxclasluzviskkobodhfyfip5xblquqmoqtxykaph3lxrshyvdjzs17vmwouej8jjzch0xo/gn6cu06ewqxp3fu9acijuozirqafinveijxca9k1vcboddizhzgfp1bpwzhrtosbc/x3xarxrcc8sp0cmafe9mbif143nffvokfcpoyivpgothk0czwlavtuewzy2bkefbw7qjxecmfjcyvyepzlk1dj67zqe1x/7qjuu87iyimioafl4inluao3oagaainh8axewzvz5lw4787hojxnzdph4a+czx+mbzyz</latexit> <latexit sha1_base64="acp/iepfd0w5i/saqobwiiubvow=">aaacbhicbvdlsgmxfm3uv62vuzfdbivqgpqzexrzfirlcvybnafk0kwbmmrckhfk6cknv+lghsju/qh3/o1powttpxdh5jx7yb0nkoxq43nftm5tfwnzk79d2nnd2z9wd49aokkvjk2cser1iqqjo4i0dtwmdkqiieemtkprzcxvpxclasluzviskkobodhfyfip5xblquqmoqtxykaph3lxrshyvdjzs17vmwouej8jjzch0xo/gn6cu06ewqxp3fu9acijuozirqafinveijxca9k1vcboddizhzgfp1bpwzhrtosbc/x3xarxrcc8sp0cmafe9mbif143nffvokfcpoyivpgothk0czwlavtuewzy2bkefbw7qjxecmfjcyvyepzlk1dj67zqe1x/7qjuu87iyimioafl4inluao3oagaainh8axewzvz5lw4787hojxnzdph4a+czx+mbzyz</latexit> <latexit sha1_base64="acp/iepfd0w5i/saqobwiiubvow=">aaacbhicbvdlsgmxfm3uv62vuzfdbivqgpqzexrzfirlcvybnafk0kwbmmrckhfk6cknv+lghsju/qh3/o1powttpxdh5jx7yb0nkoxq43nftm5tfwnzk79d2nnd2z9wd49aokkvjk2cser1iqqjo4i0dtwmdkqiieemtkprzcxvpxclasluzviskkobodhfyfip5xblquqmoqtxykaph3lxrshyvdjzs17vmwouej8jjzch0xo/gn6cu06ewqxp3fu9acijuozirqafinveijxca9k1vcboddizhzgfp1bpwzhrtosbc/x3xarxrcc8sp0cmafe9mbif143nffvokfcpoyivpgothk0czwlavtuewzy2bkefbw7qjxecmfjcyvyepzlk1dj67zqe1x/7qjuu87iyimioafl4inluao3oagaainh8axewzvz5lw4787hojxnzdph4a+czx+mbzyz</latexit> <latexit sha1_base64="acp/iepfd0w5i/saqobwiiubvow=">aaacbhicbvdlsgmxfm3uv62vuzfdbivqgpqzexrzfirlcvybnafk0kwbmmrckhfk6cknv+lghsju/qh3/o1powttpxdh5jx7yb0nkoxq43nftm5tfwnzk79d2nnd2z9wd49aokkvjk2cser1iqqjo4i0dtwmdkqiieemtkprzcxvpxclasluzviskkobodhfyfip5xblquqmoqtxykaph3lxrshyvdjzs17vmwouej8jjzch0xo/gn6cu06ewqxp3fu9acijuozirqafinveijxca9k1vcboddizhzgfp1bpwzhrtosbc/x3xarxrcc8sp0cmafe9mbif143nffvokfcpoyivpgothk0czwlavtuewzy2bkefbw7qjxecmfjcyvyepzlk1dj67zqe1x/7qjuu87iyimioafl4inluao3oagaainh8axewzvz5lw4787hojxnzdph4a+czx+mbzyz</latexit> Posterior approximation accuracy Theorem IRGA approximation Gaussian approximation E D {p( y) k ˆp( y)} S T y apple 1 2 2 W 2 2 p( S T y), ˆp( S T y) where y,f N X + F (Z), 2 I n with (,F) p( )p(f ) 32

Advantages of IRGA Approximation employed is transparent and allows for analysis yielding theoretical guarantees Can leverage your favorite method (e.g. lasso, approximate message passing, etc.) to produce accurate approximations of marginal posteriors 33

Related papers van den Boom, W., Reeves, G., and Dunson, D. B. (2015). Scalable approximations of marginal posteriors in variable selection. arxiv:1506.06629. van den Boom, W., and Dunson, D. B., and Reeves, G. (2015). Quantifying uncertainty in variable selection with arbitrary matrices. In IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing. van den Boom, W. (2018). Tailored Scalable Dimensionality Reduction. PhD thesis, Department of Statistical Science, Duke University. 34

References Carbonetto, P. & Stephens, M. (2012). Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies. Bayesian Analysis, 7: 73 108. Clyde, M. A., Ghosh, J. & Littman, M. L. (2011). Bayesian adaptive sampling for variable selection and model averaging. Journal of Computational and Graphical Statistics, 20: 80 101. Diaconis, P. & Freedman, D. (1984). Asymptotics of graphical projection pursuit. The Annals of Statistics, 12: 793 815. Friedman, J., Hastie, T. & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1): 1 22. George, E. I. & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 85: 398 409. Hernández-Lobato, J. M., Hernández-Lobato, D. & Suárez, A. (2015). Expectation propagation in linear regression models with spike-and-slab priors. Machine Learning, 99: 437 487. Ormerod, J. T., You, C. & Müller S. (2017). A variational Bayes approach to variable selection. Electronic Journal of Statistics, 11: 3549 7524. Ročková, V. & George, E. I. (2014). EMVS: The EM approach to Bayesian variable selection. Journal of the American Statistical Association, 109: 828 846. 35

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback