The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects
|
|
- Camron Allen
- 5 years ago
- Views:
Transcription
1 The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects Thai T. Pham Stanford Graduate School of Business May, 2016
2 Introduction Observations Many sequential treatment settings: patients make adjustments in medications in multiple periods; students decide whether to follow an educational honors program over multiple years; in labor market, the unemployed might participate in a set of programs (job search, subsidized job, training) sequentially. Heterogeneity in treatment sequence reactions: medication effects can be heterogeneous across patients and across time; the same for educational program and labor market. Hard to set up sequential randomized experiments in reality. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 2 / 27
3 Introduction Contributions Develop a nonparametric framework using supervised learning to estimate heterogeneous treatment regime effects from observational (or experimental) data. Treatment Regime: a set of functions of characteristics and intermediate outcomes. Propose using supervised learning approach (deep learning), which gives good estimation accuracy and which is robust to model misspecification. Propose matching based testing method for the estimation of heterogeneous treatment regime effects. Propose matching based kernel estimator for variance of heterogeneous treatment regime effects (time allows). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 3 / 27
4 Contributions (cont d) Introduction In this paper, we Focus on dynamic setting with multiple treatments applied sequentially (in contrast to a single treatment). Focus on the heterogeneous (in contrast to average) effect of a sequence of treatments, i.e. a treatment regime. Focus on observational data (in contrast to experimental data). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 4 / 27
5 An Illustrative Model Setup - Motivational Dataset The Setup The North Carolina Honors Program Dataset. There are 24, 112 observations in total. X 0 = [Y 0, d 1, d 2, d 3 ], where Y 0 is the Math test score at the end of 8th grade and d 1, d 2, d 3 are census-data dummy variables. W 0, W 1 {0, 1} are treatment variables. Y 1 : end of 9th grade Math test score. Y 2 : end of 10th grade Math test score (object of interest). Y 0, Y 1, Y 2 are pre-scaled to have zero mean and unit variance. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 5 / 27
6 Setup - Model An Illustrative Model The Setup End of eighth grade: Students initial information X 0, which includes Math test score Y 0 and other personal information (d 1, d 2, d 3 ), is observed. Decide to follow honors (W 0 = 1) or standard (W 0 = 0) program. End of ninth grade: X 0, W 0, and Math test score Y 1 are observed. Decide to switch or stay in current program (W 1 = 1 or 0). End of tenth grade: X 0, W 0, Y 1, W 1, and Math test score Y 2 are observed. Object of interest: Y 2 (It could be any functions of X 0, Y 1, Y 2 ). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 6 / 27
7 An Illustrative Model The Setup Potential Outcome (PO) Framework Treatment regime d = (d 0, d 1 ) has d 0 : X 0 W 0 {0, 1} and d 1 : X 0 W 0 Y 1 W 1 {0, 1}. Potential Outcome Y 1 (W 0 ) = Y 1. Also, the observed outcome Y 1 = W 0 Y 1 (1) + (1 W 0 ) Y 1 (0). Similarly, Y d 2 = Y 2 if the subject follows regime d. We also write Y 2 = Y d 2 = Y 2(W 0, W 1 ) when d 0 maps to W 0 and d 1 maps to W 1. We have Y 2 = W 0 W 1 Y 2 (1, 1) + W 0 (1 W 1 ) Y 2 (1, 0)+ (1 W 0 )W 1 Y 2 (0, 1) + (1 W 0 )(1 W 1 ) Y 2 (0, 0). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 7 / 27
8 An Illustrative Model Types of Treatment Regime The Setup Static Treatment Regime: subjects specify (or are specified) the whole treatment plan based only on the initial covariates (X 0 ). So d : X 0 (W 0, W 1 ) {0, 1} 2. Dynamic Treatment Regime: subjects choose (or are assigned) the initial treatment based on the initial covariates (X 0 ); then subsequently choose (or are assigned) the next treatment based on the initial covariates (X 0 ), the first period treatment (W 0 ), and the intermediate outcome (Y 1 ); and so on. This is our original setup. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 8 / 27
9 An Illustrative Model The Setup Potential Outcome (PO) Framework (Cont d) [ ] Objective: Estimate E Y2 d Y d 2 for individuals (or average), and derive heterogeneous optimal regime Difficulties: d (C) = arg max E d [ Y d 2 C ] for individual covariates C. Fundamental Problem of Causal Inference: for each subject, we never observe both Y d 2 and Y d 2. Selection Bias: students following d may fundamentally be different from those following d (e.g., students with good test scores choose the honors program in each period). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 9 / 27
10 An Illustrative Model Identification Results Identification Result - Static Treatment Regime Theorem (Identification Result - STR) Let d 0 = d 0 (X 0 ) and d 1 = d 1 (X 0 ). Then (with Assumptions) [ ] Y2 1{W 0 = d 0 } 1{W 1 = d 1 } [ ] E P(W 0 = d 0 X 0 ) P(W 1 = d 1 X 0 ) X 0 = E Y2 d X 0. Corollary: observed/estimable, Transformed Outcome [ { [ }}{ W0 W 1 E Y 2 (1 W ] ] 0)(1 W 1 ) X 0 = E [ unobserved, PO {}}{ Y 2 (1, 1) Y 2 (0, 0) ] X 0. e 0 e 1 (1 e 0 )(1 e 1 ) Here, e 0 = P(W 0 = 1 X 0 ) and e 1 = P(W 1 = 1 X 0 ). Matching Based Testing Method STR Estimation Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 10 / 27
11 An Illustrative Model Identification Results Identification Result - Dynamic Treatment Regime Theorem (Identification Result - DTR) Let d 0 = d 0 (X 0 ) and d 1 = d 1 (X 0, X 1, Y 1, W 0 ). Then (with Assumptions) In period T = 1: [ ] 1{W 1 = d 1 } E Y 2 P(W 1 = d 1 X 0, X 1, Y 1, W 0 ) X 0, X 1, Y 1, W 0 }{{} observed/estimable = E [ Y d ] 1 2 X 0, X 1, Y 1, W 0. }{{} In period T = 0: PO [ ] Y 2 1{W 1 = d 1 } 1{W 0 = d 0 } E P(W 1 = d 1 X 0, X 1, Y 1, W 0 ) P(W 0 = d 0 X 0 ) X 0 = E [ Y2 d ] X 0. DTR Estimation Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 11 / 27
12 Model Estimation Challenges In Traditional Approach Goal: Specify a relation b/w transformed outcome T and covariates C. Econometric approaches assume T = h(c; β) + ɛ for a fixed (linear) function h( ) and E[ɛ C] = 0, and estimate β by minimizing T h(c; β) 2. Problem: Linear models need not give good estimates in general. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 12 / 27
13 Model Estimation Machine Learning Approach Machine learning methods generally give much more accurate estimates than traditional econometric models. Empirical comparisons of different machine learning methods with linear regressions: Caruana and Niculescu-Mizil (2006) 1 Morton, Marzban, Giannoulis, Patel, Aparasu, and Kakadiaris (2014) 2 1 Caruana, R. and A. Niculescu-Mizil, (2006), An Empirical Comparison of Supervised Learning Algorithms, Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA. 2 Morton, A., E. Marzban, G. Giannoulis, A. Patel, R. Aparasu, and I. A. Kakadiaris, (2014), A Comparison of Supervised Machine Learning Techniques for Predicting Short-Term In-Hospital Length of Stay Among Diabetic Patients, 13th International Conference on Machine Learning and Applications. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 13 / 27
14 Model Estimation Machine Learning Approach (Cont d) Goal: Specify a relation b/w transformed outcome T and covariates C. Machine learning (ML) methods allow h( ) to vary in terms of complexity, and estimate β by minimizing T h(c; β) 2 + λg(β) where g penalizes complex models. Data set = (Training, Validation, Test). Use Training set (with CV) to choose the optimal h( ) in terms of complexity, validation set to choose the optimal hyperparameter λ, and test set to evaluate the performance. RMSE is the comparison criterion. Hence, ML approach is flexible and performance oriented. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 14 / 27
15 Model Estimation Estimating Model Propensity Score Estimation: Use logistic regression or other ML techniques such as Random Forest, Gradient Boosting, etc. Full Model Estimation: (Though many ML techniques would work) We use a deep learning method in machine learning literature called Multilayer Perceptron. It possesses the universal approximation property: it can approximate any continuous function on any compact subset of R n. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 15 / 27
16 Model Estimation Multilayer Perceptron (MLP) Assume we want to estimate T = h(c; β) + ɛ. MLP (with one hidden layer) considers K ( ) h(c; β) = α j σ(γj T C + θ j ) and β = K, (α j, γ j, θ j ) K j=1, j=1 where σ is a sigmoid function such as σ(x) = 1/(1 + exp( x)). Empirically, MLP (and deep learning in general) is shown to work very well. (Lecun et al. 3, Mnih et al. 4 ) 3 LeCun, Y., Y. Bengio, and G. Hinton, (2015), Deep Learning, Nature 521, (28 May). 4 Mnih, V., K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, (2015), Human-level control through deep reinforcement learning, Nature 518, (26 February). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 16 / 27
17 Model Estimation Testing Method Matching Based Testing Method The identification results relate unobserved difference of potential outcomes Z to observed (or estimable) transformed outcome T : E[T C] = E[Z C]. For example in STR : Z = Y d 2 Y d 2, C = X 0. Randomly draw M units with treatment regime d. Denote by x d,m 0 s and y d,m 2 s the covariates and corresponding outcomes. For each m, determine x d,m 0 = arg min x i 0 regime = d x i 0 x d,m 0 2. Let τ m = y d,m 2 y d,m 2. Here, τ is a proxy for the unobserved Z. Let τ be the estimator which fits x 0 to T. Define τ m = 1 d,m 2 ( τ(x 0 ) + τ(x d,m 0 )). Define the matching loss M : 1 M M m=1 ( τ m τ m ) 2. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 17 / 27
18 Simulation Setup Simulations Setup Test the ability of our method in adapting to heterogeneity in the treatment regime effect. 50, 000 obs for training; 5, 000 obs for validation; 5, 000 for testing. X 0 U([0, 1] 10 ); W 0 {0, 1}; Y 1 R with standard normal noise; W 1 {0, 1}; Y 2 R with standard normal noise. e 0 (X 0 ) = e 1 (X 0 ) = e 1 (X 0, W 0, Y 1 ) = 0.5. [ τ 1 (X 0 ) = E Y W 0=1 1 Y W 0=0 1 τ 2 (X 0, W 0, Y 1 ) = E where ξ(x) = ] X 0 = ξ(x 0 [1])ξ(X 0 [2]); and ] X 0, W 0, Y 1 [ Y W 1=1 2 Y W 1=0 2 = ρ(y 1 )ρ(w 0 )ξ(x 0 [1]) e 12(x 1/2) ; ρ(x) = e 20(x 1/3). Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 18 / 27
19 Simulations Results Simulation Results Table: Performance In Terms of Root Mean Squared Error (RMSE) Method Linear Regression (LR) Multilayer Perceptron (MLP) STR DTR: T = DTR: T = sdv (TO: T = 0) = 2.41; sdv (true effect: T = 0) = sdv (TO: T = 1) = 3.21; sdv (true effect: T = 1) = Comments: MLP returns really good results, and it outperforms LR. Static setting does not fit here as the RMSEs on STR are bad. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 19 / 27
20 Simulations Results Simulation Results (cont d) 5 MLP: T = 0 True Effect: T = 0 LR: T = 0 MLP: T = 1 True Effect: T = 1 LR: T = 1 Figure: Heterogeneous Treatment Regime Effect Using Validation and Test Data. The first row corresponds to period T = 0 and the second row corresponds to period T = 1. In each period: the middle picture visualizes the true treatment effect; the left one is the estimated effect by using Multilayer Perceptron; and the right one is the estimated effect by using Linear Regression. 5 We thank Wager and Athey (2015) for sharing their visualization code. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 20 / 27
21 Empirical Application Propensity Score Estimation Estimation of Propensity Scores Use the North Carolina Honors Program data (in illustrative model). Estimate and P(W 0 = 1 X 0 ), P(W 1 = 1 X 0 ) P(W 1 = 1 X 0, Y 1, W 0 ). Use Random Forest as a probabilistic classification problem. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 21 / 27
22 Empirical Application Static Treatment Regime Estimation Model Estimation - Static Treatment Regime Use three methods: Linear Regression, Gradient Boosting, and Multilayer Perceptron. Method Validation Matching Loss Test Matching Loss Linear Regression Gradient Boosting Multilayer Perceptron Comments: MLP outperforms other methods. All results are bad, which signals the dynamic nature of the data. STR Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 22 / 27
23 Empirical Application Dynamic Treatment Regime Estimation Model Estimation - Dynamic Treatment Regime Period T = 1 - Method Validation Matching Loss Test Matching Loss Linear Regression Gradient Boosting Multilayer Perceptron *sdv(to: val) = 4.06; sdv(est. true effect: val) = *sdv(to: test) = 4.03; sdv(est. true effect: test) = Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 23 / 27
24 Empirical Application Dynamic Treatment Regime Estimation Model Estimation - Dynamic Treatment Regime (Cont d) Period T = 0 - Method Validation Matching Loss Test Matching Loss Linear Regression Gradient Boosting Multilayer Perceptron DTR - Use only students who follow the optimal treatment in T = 1. *sdv(to: val) = 6.94; sdv(est. true effect: val) = *sdv(to: test) = 7.45; sdv(est. true effect: test) = Remark: The results are worse than that in simulations due to unobserved heterogeneity. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 24 / 27
25 Empirical Application Heterogeneous Optimal Regime Estimation Heterogeneous Optimal Regime Static Treatment Regime Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 25 / 27
26 Empirical Application Heterogeneous Optimal Regime Estimation Heterogeneous Optimal Regime (cont d) STR: Estimated gain per student from heterogeneous optimal regime over homogeneous optimal ( regime (0, 0): 1 [Ŷ2 (1, 1) 0)] Ŷ2(0, + (1,1) opt (0,0) not opt [Ŷ2 (1, 0) 0)] Ŷ2(0, + 1 ] [Ŷ2 ) (0, 1) Ŷ2(0, 0) = (1,0) opt (0,1) opt DTR: Estimated gain per student in T = 0 from heterogeneous optimal W 0 over homogeneous optimal treatment W 0 = 0: ] W W 0 =1 optimal [Ŷ 0 =1 2 Ŷ W 0=0 2 #obs used in T = 0 s.t. W 0 = 1 opt = mean(y 2 ) = 0; min(y 2 ) = 4.06; max(y 2 ) = Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 26 / 27
27 Conclusion Conclusion We developed a nonparametric framework using supervised learning to estimate heterogeneous causal regime effects. Our model addresses the dynamic treatment setting, the population heterogeneity, and the difficulty in setting up sequential randomized experiments in reality. We introduced machine learning approach, in particular deep learning, which demonstrates its estimation power and which is robust to model misspecification. We also introduced matching based testing method for the estimation of heterogeneous treatment regime effects. A matching based kernel estimator for variance of these effects is introduced in Appendix. Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 27 / 27
28 Appendix Variance Estimation - Matching Kernel Approach Matching M matching pairs [ ] (x d,m 0, y d,m 2 ); (x d,m 0, y d,m 2 ), m = 1,..., M. Fix x new 0. To estimate σ 2 (x new Let x mean,m 0 = x d,m 0 +x d,m 0 0 ) = Var(Y2 d Y d 2 ɛ m = τ m τ m. 2. An estimator for σ 2 (x0 new ) is x new 0 ), we define M σ 2 (x0 new m=1 ) = K (H 1 [x mean,m 0 x0 new ]) ɛ 2 m M m=1 K (H 1 [x mean,m. 0 x0 new ]) Thai T. Pham Estimation of Heterogeneous Causal Regime Effects 27 / 27
The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects
The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects Thai T. Pham August, 2016 Abstract We develop a nonparametric framework using supervised learning to estimate heterogeneous
Deep reinforcement learning. Dialogue Systems Group, Cambridge University Engineering Department
Deep reinforcement learning Milica Gašić Dialogue Systems Group, Cambridge University Engineering Department 1 / 25 In this lecture... Introduction to deep reinforcement learning Value-based Deep RL Deep
Driving a car with low dimensional input features
December 2016 Driving a car with low dimensional input features Jean-Claude Manoli jcma@stanford.edu Abstract The aim of this project is to train a Deep Q-Network to drive a car on a race track using only
Deep Q-Learning with Recurrent Neural Networks
Deep Q-Learning with Recurrent Neural Networks Clare Chen cchen9@stanford.edu Vincent Ying vincenthying@stanford.edu Dillon Laird dalaird@cs.stanford.edu Abstract Deep reinforcement learning models have
Selection on Observables: Propensity Score Matching.
Selection on Observables: Propensity Score Matching. Department of Economics and Management Irene Brunetti ireneb@ec.unipi.it 24/10/2017 I. Brunetti Labour Economics in an European Perspective 24/10/2017
Causal Inference Basics
Causal Inference Basics Sam Lendle October 09, 2013 Observed data, question, counterfactuals Observed data: n i.i.d copies of baseline covariates W, treatment A {0, 1}, and outcome Y. O i = (W i, A i,
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies Kosuke Imai Department of Politics Princeton University November 13, 2013 So far, we have essentially assumed
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University Joint
Introduction to Machine Learning and Cross-Validation
Introduction to Machine Learning and Cross-Validation Jonathan Hersh 1 February 27, 2019 J.Hersh (Chapman ) Intro & CV February 27, 2019 1 / 29 Plan 1 Introduction 2 Preliminary Terminology 3 Bias-Variance
Deep Learning Architecture for Univariate Time Series Forecasting
CS229,Technical Report, 2014 Deep Learning Architecture for Univariate Time Series Forecasting Dmitry Vengertsev 1 Abstract This paper studies the problem of applying machine learning with deep architecture
Optimal Treatment Regimes for Survival Endpoints from a Classification Perspective. Anastasios (Butch) Tsiatis and Xiaofei Bai
Optimal Treatment Regimes for Survival Endpoints from a Classification Perspective Anastasios (Butch) Tsiatis and Xiaofei Bai Department of Statistics North Carolina State University 1/35 Optimal Treatment
Big Data, Machine Learning, and Causal Inference
Big Data, Machine Learning, and Causal Inference I C T 4 E v a l I n t e r n a t i o n a l C o n f e r e n c e I F A D H e a d q u a r t e r s, R o m e, I t a l y P a u l J a s p e r p a u l. j a s p e
Linear Models for Regression CS534
Linear Models for Regression CS534 Prediction Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict the
Classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Topics Discriminant functions Logistic regression Perceptron Generative models Generative vs. discriminative
Quantitative Economics for the Evaluation of the European Policy
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti Davide Fiaschi Angela Parenti 1 25th of September, 2017 1 ireneb@ec.unipi.it, davide.fiaschi@unipi.it,
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University
Reinforcement Learning. Odelia Schwartz 2016
Reinforcement Learning Odelia Schwartz 2016 Forms of learning? Forms of learning Unsupervised learning Supervised learning Reinforcement learning Forms of learning Unsupervised learning Supervised learning
Measuring Social Influence Without Bias
Measuring Social Influence Without Bias Annie Franco Bobbie NJ Macdonald December 9, 2015 The Problem CS224W: Final Paper How well can statistical models disentangle the effects of social influence from
AAEC/ECON 5126 FINAL EXAM: SOLUTIONS
AAEC/ECON 5126 FINAL EXAM: SOLUTIONS SPRING 2013 / INSTRUCTOR: KLAUS MOELTNER This exam is open-book, open-notes, but please work strictly on your own. Please make sure your name is on every sheet you
Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity
Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity P. Richard Hahn, Jared Murray, and Carlos Carvalho June 22, 2017 The problem setting We want to estimate
Linear Regression. Volker Tresp 2018
Linear Regression Volker Tresp 2018 1 Learning Machine: The Linear Model / ADALINE As with the Perceptron we start with an activation functions that is a linearly weighted sum of the inputs h = M j=0 w
Least Squares Estimation of a Panel Data Model with Multifactor Error Structure and Endogenous Covariates
Least Squares Estimation of a Panel Data Model with Multifactor Error Structure and Endogenous Covariates Matthew Harding and Carlos Lamarche January 12, 2011 Abstract We propose a method for estimating
Learning Gaussian Process Models from Uncertain Data
Learning Gaussian Process Models from Uncertain Data Patrick Dallaire, Camille Besse, and Brahim Chaib-draa DAMAS Laboratory, Computer Science & Software Engineering Department, Laval University, Canada
Be able to define the following terms and answer basic questions about them:
CS440/ECE448 Section Q Fall 2017 Final Review Be able to define the following terms and answer basic questions about them: Probability o Random variables, axioms of probability o Joint, marginal, conditional
Reinforcement. Function Approximation. Learning with KATJA HOFMANN. Researcher, MSR Cambridge
Reinforcement Learning with Function Approximation KATJA HOFMANN Researcher, MSR Cambridge Representation and Generalization in RL Focus on training stability Learning generalizable value functions Navigating
Comments. Assignment 3 code released. Thought questions 3 due this week. Mini-project: hopefully you have started. implement classification algorithms
Neural networks Comments Assignment 3 code released implement classification algorithms use kernels for census dataset Thought questions 3 due this week Mini-project: hopefully you have started 2 Example:
Predicting the Treatment Status
Predicting the Treatment Status Nikolay Doudchenko 1 Introduction Many studies in social sciences deal with treatment effect models. 1 Usually there is a treatment variable which determines whether a particular
Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies
Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies Kosuke Imai Princeton University February 23, 2012 Joint work with L. Keele (Penn State)
Introduction to Gaussian Process
Introduction to Gaussian Process CS 778 Chris Tensmeyer CS 478 INTRODUCTION 1 What Topic? Machine Learning Regression Bayesian ML Bayesian Regression Bayesian Non-parametric Gaussian Process (GP) GP Regression
Association studies and regression
Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration
IFT Lecture 7 Elements of statistical learning theory
IFT 6085 - Lecture 7 Elements of statistical learning theory This version of the notes has not yet been thoroughly checked. Please report any bugs to the scribes or instructor. Scribe(s): Brady Neal and
Linear Models for Regression CS534
Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict
Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
Linear Models for Regression CS534
Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict
Covariate Balancing Propensity Score for General Treatment Regimes
Covariate Balancing Propensity Score for General Treatment Regimes Kosuke Imai Princeton University October 14, 2014 Talk at the Department of Psychiatry, Columbia University Joint work with Christian
Label-Free Supervision of Neural Networks with Physics and Domain Knowledge
Label-Free Supervision of Neural Networks with Physics and Domain Knowledge Russell Stewart, Stefano Ermon Department of Computer Science, Stanford University {stewartr, ermon}@cs.stanford.edu where the
Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels
Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)
causal inference at hulu
causal inference at hulu Allen Tran July 17, 2016 Hulu Introduction Most interesting business questions at Hulu are causal Business: what would happen if we did x instead of y? dropped prices for risky
Flexible Estimation of Treatment Effect Parameters
Flexible Estimation of Treatment Effect Parameters Thomas MaCurdy a and Xiaohong Chen b and Han Hong c Introduction Many empirical studies of program evaluations are complicated by the presence of both
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters
Machine Learning. Lecture 9: Learning Theory. Feng Li.
Machine Learning Lecture 9: Learning Theory Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Why Learning Theory How can we tell
The exam is closed book, closed notes except your one-page (two sides) or two-page (one side) crib sheet.
CS 189 Spring 013 Introduction to Machine Learning Final You have 3 hours for the exam. The exam is closed book, closed notes except your one-page (two sides) or two-page (one side) crib sheet. Please
CS Machine Learning Qualifying Exam
CS Machine Learning Qualifying Exam Georgia Institute of Technology March 30, 2017 The exam is divided into four areas: Core, Statistical Methods and Models, Learning Theory, and Decision Processes. There
Estimation of Optimal Treatment Regimes Via Machine Learning. Marie Davidian
Estimation of Optimal Treatment Regimes Via Machine Learning Marie Davidian Department of Statistics North Carolina State University Triangle Machine Learning Day April 3, 2018 1/28 Optimal DTRs Via ML
arxiv: v1 [cs.lg] 28 Dec 2016
THE PREDICTRON: END-TO-END LEARNING AND PLANNING David Silver*, Hado van Hasselt*, Matteo Hessel*, Tom Schaul*, Arthur Guez*, Tim Harley, Gabriel Dulac-Arnold, David Reichert, Neil Rabinowitz, Andre Barreto,
Statistical Machine Learning
Statistical Machine Learning Lecture 9 Numerical optimization and deep learning Niklas Wahlström Division of Systems and Control Department of Information Technology Uppsala University niklas.wahlstrom@it.uu.se
Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies
Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies Kosuke Imai Princeton University January 23, 2012 Joint work with L. Keele (Penn State)
Dynamics in Social Networks and Causality
Web Science & Technologies University of Koblenz Landau, Germany Dynamics in Social Networks and Causality JProf. Dr. University Koblenz Landau GESIS Leibniz Institute for the Social Sciences Last Time:
Estimation Considerations in Contextual Bandits
Estimation Considerations in Contextual Bandits Maria Dimakopoulou Zhengyuan Zhou Susan Athey Guido Imbens arxiv:1711.07077v4 [stat.ml] 16 Dec 2018 Abstract Contextual bandit algorithms are sensitive to
Apprentissage, réseaux de neurones et modèles graphiques (RCP209) Neural Networks and Deep Learning
Apprentissage, réseaux de neurones et modèles graphiques (RCP209) Neural Networks and Deep Learning Nicolas Thome Prenom.Nom@cnam.fr http://cedric.cnam.fr/vertigo/cours/ml2/ Département Informatique Conservatoire
Probabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016
Probabilistic classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Probabilistic approach Bayes decision theory Generative models Gaussian Bayes classifier
Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels
Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)
Probabilistic & Bayesian deep learning. Andreas Damianou
Probabilistic & Bayesian deep learning Andreas Damianou Amazon Research Cambridge, UK Talk at University of Sheffield, 19 March 2019 In this talk Not in this talk: CRFs, Boltzmann machines,... In this
A Measure of Robustness to Misspecification
A Measure of Robustness to Misspecification Susan Athey Guido W. Imbens December 2014 Graduate School of Business, Stanford University, and NBER. Electronic correspondence: athey@stanford.edu. Graduate
An Introduction to Statistical Machine Learning - Theoretical Aspects -
An Introduction to Statistical Machine Learning - Theoretical Aspects - Samy Bengio bengio@idiap.ch Dalle Molle Institute for Perceptual Artificial Intelligence (IDIAP) CP 592, rue du Simplon 4 1920 Martigny,
Mark your answers ON THE EXAM ITSELF. If you are not sure of your answer you may wish to provide a brief explanation.
CS 189 Spring 2015 Introduction to Machine Learning Midterm You have 80 minutes for the exam. The exam is closed book, closed notes except your one-page crib sheet. No calculators or electronic items.
ECO Class 6 Nonparametric Econometrics
ECO 523 - Class 6 Nonparametric Econometrics Carolina Caetano Contents 1 Nonparametric instrumental variable regression 1 2 Nonparametric Estimation of Average Treatment Effects 3 2.1 Asymptotic results................................
Weighted Double Q-learning
Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-7) Weighted Double Q-learning Zongzhang Zhang Soochow University Suzhou, Jiangsu 56 China zzzhang@suda.edu.cn
Random Forests. These notes rely heavily on Biau and Scornet (2016) as well as the other references at the end of the notes.
Random Forests One of the best known classifiers is the random forest. It is very simple and effective but there is still a large gap between theory and practice. Basically, a random forest is an average
4 Bias-Variance for Ridge Regression (24 points)
Implement Ridge Regression with λ = 0.00001. Plot the Squared Euclidean test error for the following values of k (the dimensions you reduce to): k = {0, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500,
Deep Reinforcement Learning SISL. Jeremy Morton (jmorton2) November 7, Stanford Intelligent Systems Laboratory
Deep Reinforcement Learning Jeremy Morton (jmorton2) November 7, 2016 SISL Stanford Intelligent Systems Laboratory Overview 2 1 Motivation 2 Neural Networks 3 Deep Reinforcement Learning 4 Deep Learning
Online Learning and Sequential Decision Making
Online Learning and Sequential Decision Making Emilie Kaufmann CNRS & CRIStAL, Inria SequeL, emilie.kaufmann@univ-lille.fr Research School, ENS Lyon, Novembre 12-13th 2018 Emilie Kaufmann Online Learning
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013 Exam policy: This exam allows two one-page, two-sided cheat sheets; No other materials. Time: 2 hours. Be sure to write your name and
Relevance Vector Machines
LUT February 21, 2011 Support Vector Machines Model / Regression Marginal Likelihood Regression Relevance vector machines Exercise Support Vector Machines The relevance vector machine (RVM) is a bayesian
ECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: June 9, 2018, 09.00 14.00 RESPONSIBLE TEACHER: Andreas Svensson NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical
Optmization Methods for Machine Learning Beyond Perceptron Feed Forward neural networks (FFN)
Optmization Methods for Machine Learning Beyond Perceptron Feed Forward neural networks (FFN) Laura Palagi http://www.dis.uniroma1.it/ palagi Dipartimento di Ingegneria informatica automatica e gestionale
Statistical Machine Learning Hilary Term 2018
Statistical Machine Learning Hilary Term 2018 Pier Francesco Palamara Department of Statistics University of Oxford Slide credits and other course material can be found at: http://www.stats.ox.ac.uk/~palamara/sml18.html
Stochastic Video Prediction with Deep Conditional Generative Models
Stochastic Video Prediction with Deep Conditional Generative Models Rui Shu Stanford University ruishu@stanford.edu Abstract Frame-to-frame stochasticity remains a big challenge for video prediction. The
Linear Models for Regression
Linear Models for Regression CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 The Regression Problem Training data: A set of input-output
Gradient Boosting (Continued)
Gradient Boosting (Continued) David Rosenberg New York University April 4, 2016 David Rosenberg (New York University) DS-GA 1003 April 4, 2016 1 / 31 Boosting Fits an Additive Model Boosting Fits an Additive
CSE 417T: Introduction to Machine Learning. Lecture 11: Review. Henry Chai 10/02/18
CSE 417T: Introduction to Machine Learning Lecture 11: Review Henry Chai 10/02/18 Unknown Target Function!: # % Training data Formal Setup & = ( ), + ),, ( -, + - Learning Algorithm 2 Hypothesis Set H
ESTIMATION OF TREATMENT EFFECTS VIA MATCHING
ESTIMATION OF TREATMENT EFFECTS VIA MATCHING AAEC 56 INSTRUCTOR: KLAUS MOELTNER Textbooks: R scripts: Wooldridge (00), Ch.; Greene (0), Ch.9; Angrist and Pischke (00), Ch. 3 mod5s3 General Approach The
A summary of Deep Learning without Poor Local Minima
A summary of Deep Learning without Poor Local Minima by Kenji Kawaguchi MIT oral presentation at NIPS 2016 Learning Supervised (or Predictive) learning Learn a mapping from inputs x to outputs y, given
Neural Networks and Deep Learning
Neural Networks and Deep Learning Professor Ameet Talwalkar November 12, 2015 Professor Ameet Talwalkar Neural Networks and Deep Learning November 12, 2015 1 / 16 Outline 1 Review of last lecture AdaBoost
Multilayer Perceptron
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Single Perceptron 3 Boolean Function Learning 4
Causal Sensitivity Analysis for Decision Trees
Causal Sensitivity Analysis for Decision Trees by Chengbo Li A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Master of Mathematics in Computer
Linear Models for Classification
Linear Models for Classification Oliver Schulte - CMPT 726 Bishop PRML Ch. 4 Classification: Hand-written Digit Recognition CHINE INTELLIGENCE, VOL. 24, NO. 24, APRIL 2002 x i = t i = (0, 0, 0, 1, 0, 0,
Fundamentals of Machine Learning
Fundamentals of Machine Learning Mohammad Emtiyaz Khan AIP (RIKEN), Tokyo http://icapeople.epfl.ch/mekhan/ emtiyaz@gmail.com Jan 2, 27 Mohammad Emtiyaz Khan 27 Goals Understand (some) fundamentals of Machine
Variational Autoencoder
Variational Autoencoder Göker Erdo gan August 8, 2017 The variational autoencoder (VA) [1] is a nonlinear latent variable model with an efficient gradient-based training procedure based on variational
Finite Population Sampling and Inference
Finite Population Sampling and Inference A Prediction Approach RICHARD VALLIANT ALAN H. DORFMAN RICHARD M. ROYALL A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane
Week 5: Logistic Regression & Neural Networks
Week 5: Logistic Regression & Neural Networks Instructor: Sergey Levine 1 Summary: Logistic Regression In the previous lecture, we covered logistic regression. To recap, logistic regression models and
Stochastic optimization in Hilbert spaces
Stochastic optimization in Hilbert spaces Aymeric Dieuleveut Aymeric Dieuleveut Stochastic optimization Hilbert spaces 1 / 48 Outline Learning vs Statistics Aymeric Dieuleveut Stochastic optimization Hilbert
σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: August 30, 2018, 14.00 19.00 RESPONSIBLE TEACHER: Niklas Wahlström NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical
A Significance Test for the Lasso
A Significance Test for the Lasso Lockhart R, Taylor J, Tibshirani R, and Tibshirani R Ashley Petersen May 14, 2013 1 Last time Problem: Many clinical covariates which are important to a certain medical
Importance Reweighting Using Adversarial-Collaborative Training
Importance Reweighting Using Adversarial-Collaborative Training Yifan Wu yw4@andrew.cmu.edu Tianshu Ren tren@andrew.cmu.edu Lidan Mu lmu@andrew.cmu.edu Abstract We consider the problem of reweighting a
Empirical approaches in public economics
Empirical approaches in public economics ECON4624 Empirical Public Economics Fall 2016 Gaute Torsvik Outline for today The canonical problem Basic concepts of causal inference Randomized experiments Non-experimental
Evaluation. Andrea Passerini Machine Learning. Evaluation
Andrea Passerini passerini@disi.unitn.it Machine Learning Basic concepts requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain
Guanglei Hong. University of Chicago. Presentation at the 2012 Atlantic Causal Inference Conference
Guanglei Hong University of Chicago Presentation at the 2012 Atlantic Causal Inference Conference 2 Philosophical Question: To be or not to be if treated ; or if untreated Statistics appreciates uncertainties
Introduction to Deep Neural Networks
Introduction to Deep Neural Networks Presenter: Chunyuan Li Pattern Classification and Recognition (ECE 681.01) Duke University April, 2016 Outline 1 Background and Preliminaries Why DNNs? Model: Logistic
From perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning
Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function
Deep Learning Lab Course 2017 (Deep Learning Practical)
Deep Learning Lab Course 207 (Deep Learning Practical) Labs: (Computer Vision) Thomas Brox, (Robotics) Wolfram Burgard, (Machine Learning) Frank Hutter, (Neurorobotics) Joschka Boedecker University of
arxiv: v1 [cs.ne] 4 Dec 2015
Q-Networks for Binary Vector Actions arxiv:1512.01332v1 [cs.ne] 4 Dec 2015 Naoto Yoshida Tohoku University Aramaki Aza Aoba 6-6-01 Sendai 980-8579, Miyagi, Japan naotoyoshida@pfsl.mech.tohoku.ac.jp Abstract
A Sampling of IMPACT Research:
A Sampling of IMPACT Research: Methods for Analysis with Dropout and Identifying Optimal Treatment Regimes Marie Davidian Department of Statistics North Carolina State University http://www.stat.ncsu.edu/
Neural Networks and the Back-propagation Algorithm
Neural Networks and the Back-propagation Algorithm Francisco S. Melo In these notes, we provide a brief overview of the main concepts concerning neural networks and the back-propagation algorithm. We closely
Scalable Model Selection for Belief Networks
Scalable Model Selection for Belief Networks Zhao Song, Yusuke Muraoka, Ryohei Fujimaki, Lawrence Carin Department of ECE, Duke University Durham, NC 27708, USA {zhao.song, lcarin}@duke.edu NEC Data Science
NONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function
Deep Feedforward Networks
Deep Feedforward Networks Liu Yang March 30, 2017 Liu Yang Short title March 30, 2017 1 / 24 Overview 1 Background A general introduction Example 2 Gradient based learning Cost functions Output Units 3