Efficient Identification of Heterogeneous Treatment Effects via Anomalous Pattern Detection
|
|
|
- Ashlyn Powell
- 5 years ago
- Views:
Transcription
1 Efficient Identification of Heterogeneous Treatment Effects via Anomalous Pattern Detection Edward McFowland III Joint work with (Sriram Somanchi & Daniel B. Neill) 1
2 Treatment Effects 2
3 Treatment Effects 3
4 Treatment Effect Heterogeneity Control Group Treatment Group Age = Young Age = Mid Age = Old 4
5 Treatment Effect Heterogeneity Age = Young Control Group Treatment Group ( Positive) Effect Positive and negative effects can cancel Age = Mid ( Negative) Effect Age = Old ( No ) Effect ( ) No Effect 5
6 Treatment Effect Heterogeneity Age = Young Control Group Treatment Group ( Positive) Effect Positive and negative effects can cancel Age = Mid ( ) No Effect True effect can be masked Age = Old ( No ) Effect ( No ) Effect 6
7 Treatment Effect Heterogeneity Age = Young Control Group Treatment Group ( Very ) Positive Effect Positive and negative effects can cancel Age = Mid ( No ) Effect True effect can be masked ( No ) Age = Old Effect ( Slight ) Effect Effects could really be driven by a subpopulation 7
8 Treatment Effect Heterogeneity Age = Young Control Group Treatment Group ( Very ) Positive Effect Positive and negative effects can cancel Age = Mid ( No ) Effect True effect can be masked Ex: FDA Approved BiDil Drug ( No ) Age = Old Effect ( Slight ) Effect Effects could really be driven by a subpopulation Ex: Perry Preschool 8
9 Treatment Effect Heterogeneity (Big Data) Control Group Treatment Group.. 9
10 Machine Learning s Contributions Regression Methods OLS and Regularized Regression (e.g., LASSO)* Imai and Ratkovic (2013) Single Tree Methods Su et al (2009) Imai and Strauss (2011) Athey and Imbens (2015)* Ensemble Methods Green and Kern (2012) Wager and Athey (2012)* * provide asym confidence intervals, for inference of effects significance 10
11 Limitations Regression Methods Pre-specification of the model Single Tree Methods Greedy and unstable Ensemble Methods Fairly uninterpretable/no natural subpopulations General Limitations The mean and only the mean Other moments can be effected Simpsons Paradox Risk minimization not effect maximization Small number of subpopulations considered No guarantee on their interestingness No discovery, only model inspection 11
12 Anomaly Detection Paradigm Identifying when a system deviates away from its expected behavior. 12
13 Anomalous Pattern Detection Procedure Normal Activity (M 0 ) A1 A2 A3 A4 A6 A5 Detect Anomalous Pattern Given M 0 Novel Pattern Test Data 13
14 Control Group HTE Pattern Detection Detect Anomalous Pattern Given M 0 Novel Pattern Treatment Group 14
15 Control Group HTE Pattern Detection Detect Anomalous Subpopulation Given M 0 Novel Pattern Treatment Group 15
16 Control Group HTE Pattern Detection Detect Anomalous Subpopulation Given M 0 Subpopulation Treatment Group 16
17 Male Female The Goal Black White Hispanic Asian Detect a subpopulation (subsets of attribute values), which correspond to anomalous outcomes for subjects in the treatment group Native American Other 17
18 Male Female The Goal Black White Hispanic Asian Native American Detect a subpopulation (subsets of attribute values), which correspond to anomalous outcomes for subjects in the treatment group The Optimization s 1 {a 1 a t },, s M {a 1 a p } Other 18
19 Male Female The Goal Black White Hispanic Asian Native American Other Detect a subpopulation (subsets of attribute values), which correspond to anomalous outcomes for subjects in the treatment group The Optimization s 1 {a 1 a t },, s M {a 1 a p } S = s 1 s M 19
20 Male Female The Goal Black White Hispanic Asian Native American Other Detect a subpopulation (subsets of attribute values), which correspond to anomalous outcomes for subjects in the treatment group The Optimization s 1 {a 1 a t },, s M {a 1 a p } S = s 1 s M S* = argmax S F(S) 20
21 Treatment Effects Subset Scan (TESS) Male Female Black White Y BM Y WM Y BF Y WF I. Compute the statistical anomalousness of each treatment group subject Hispanic Y HM Y HF Asian Native American Y AM Y NM Y AF Y NF II. Detect subpopulation that is collectively the most anomalous Other Y OM Y OF 21
22 Treatment Effects Subset Scan (TESS) Male Female Black White P BM P WM P BF P WF I. Compute the statistical anomalousness of each treatment group subject -- This measurement will be a p-value Hispanic P HM P HF Asian Native American P AM P NM P AF P NF II. Detect subpopulation that is collectively the most anomalous -- Many subjects with significant p-values Other P OM P OF 22
23 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Y BM Y WM Y HM Y BF Y WF Y HF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 Asian Y AM Y AF Native American Y NM Y NF Other Y OM Y OF Control Group 23
24 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Y BM Y WM Y HM Y BF Y WF Y HF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values Asian Y AM Y AF Native American Y NM Y NF Other Y OM Y OF Treatment Group 24
25 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Y BM Y WM Y HM Y BF Y WF Y HF y i BF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values Asian Y AM Y AF Native American Y NM Y NF Other Y OM Y OF Treatment Group 25
26 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic P BM P WM P HM P BF P WF P HF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values Asian P AM P AF Native American P NM P NF Other P OM P OF Treatment Group 26
27 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Asian Native American P BM P WM P HM P AM P NM P BF P WF P HF P AF P NF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values i. Maps each bin s distribution to the same interval Other P OM P OF Treatment Group 27
28 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Asian Native American P BM P WM P HM P AM P NM P BF P WF P HF P AF P NF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values i. Maps each bin s distribution to the same interval ii. P ij ~ Uniform[0,1] under H 0 Other P OM P OF Treatment Group 28
29 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM P BF P WF P HF P AF P NF P OF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values i. Maps each bin s distribution to the same interval ii. P ij ~ Uniform[0,1] under H 0 iii. For any N p-values, we expect N*α to be significant at level α Treatment Group 29
30 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM P BF P WF P HF P AF P NF P OF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values i. Maps each bin s distribution to the same interval ii. P ij ~ Uniform[0,1] under H 0 iii. For any N p-values, we expect N*α to be significant at level α Treatment Group Higher Criticism: 30
31 Treatment Effects Subset Scan (TESS) Male Female Black White Hispanic P BM P WM P HM P BF P WF P HF I. Compute the statistical anomalousness of each treatment group subject II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of Asian Native American P AM P NM P AF P NF s 1 s M Naïve search is infeasible O(2 A i ) Other P OM P OF Treatment Group 31
32 Treatment Effects Subset Scan (TESS) Nonparametric Scan Statistic (NPSS) Have: S {A 1 A M } = {s 1 s M } Select: F(S) Want: max S F(S) I. Compute the statistical anomalousness of each treatment group subject II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M Naïve search is infeasible O(2 A i ) 32
33 Treatment Effects Subset Scan (TESS) Nonparametric Scan Statistic (NPSS) Have: S {A 1 A M } = {s 1 s M } Select: F(S) Want: max S F(S) There Exist: G(a i ) I. Compute the statistical anomalousness of each treatment group subject II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M Naïve search is infeasible O(2 A i ) 33
34 Treatment Effects Subset Scan (TESS) Nonparametric Scan Statistic (NPSS) Have: S {A 1 A M } = {s 1 s M } Select: F(S) Want: max S F(S) There Exist: G(a i ) Such That: I. Compute the statistical anomalousness of each treatment group subject II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M Naïve search is infeasible O(2 A i ) Only Consider: {Black} {Black, Hispanic} {Black, Hispanic, Asian}. {Black, Hispanic, Asian,, White } 34
35 Treatment Effects Subset Scan (TESS) Nonparametric Scan Statistic (NPSS) Have: S {A 1 A M } = {s 1 s M } Select: F(S) Want: max S F(S) There Exist: G(a i ) I. Compute the statistical anomalousness of each treatment group subject II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) Such That: Only Consider: {Black} {Black, Hispanic} {Black, Hispanic, Asian}. {Black, Hispanic, Asian,, White } 35
36 TESS Search Procedure Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM P BF P WF P HF P AF P NF P OF Treatment Group I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) 36
37 TESS Search Procedure Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM (Score = 7.5) P BF P WF P HF P AF P NF P OF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) 37
38 TESS Search Procedure Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM (Score = 8.1) P BF P WF P HF P AF P NF P OF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) 38
39 TESS Search Procedure Male Female Black White Hispanic Asian Native American Other P BM P WM P HM P AM P NM P OM (Score = 9.3) P BF P WF P HF P AF P NF P OF I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) 39
40 TESS Search Procedure Male Female Black White Hispanic Asian Native American Other P BM P BF P WM P WF P HM P HF P AM P AF P NM P NF P OM P OF (Score = 9.3) I. Compute the statistical anomalousness of each treatment group subject 1. Estimate Conditional Distribution Under H 0 2. Compute empirical p-values II. Discover subsets of attribute values that define the most anomalous outcomes 1. Maximize F(S) over all subsets of s 1 s M NPSS over an attribute in O(t log t) Significance of our subpopulation Compare subpopulation score to maximum scores of simulated datasets under H 0 40
41 Tennessee Star Analysis (1985) Effect of classrooms size on achievement (test scores) 4 year panel (kindergarten to 3 rd grade) 6,500 students, 330 classrooms, 80 schools Total of over 11,000 records Treatment Conditions (randomized within school) Regular Size Class (20-25 students) Regular Size + Aide Class (20-25 students) Small Size Class (13-17 students) 42
42 Tennessee Star Analysis (1985) 43
43 Tennessee Star Analysis Treatment (1) (2.547) (2) (2.277) Sample All 2 nd Grade All 3 rd Grade R-squared Observations Notes: All estimates are from OLS models. Standard errors are in parentheses. *** p<0.01, ** p<0.05, * p<0.1 44
44 Tennessee Star Analysis (1985) Detected Subpopulation grade: 2nd or 3rd school: inner-city or urban experience: [10, infinity) other features are considered irrelevant 45
45 Tennessee Star Analysis 46
46 Tennessee Star Analysis 47
47 Tennessee Star Analysis (1) (2) (3) Treatment *** (2.547) (8.727) (2.639) Sample All 2 nd Grade Detected Subpopulation Undetected Subpopulation R-squared 0.001< < Observations Notes: All estimates are from OLS models. Standard errors are in parentheses. *** p<0.01, ** p<0.05, * p<0.1 48
48 Tennessee Star Analysis (1) (2) (3) Treatment ** (2.277) (9.154) (2.318) Sample All 3 rd Grade Detected Group (3 rd Grade) Undetected Group (3 rd Grade) R-squared 0.001< < Observations Notes: All estimates are from OLS models. Standard errors are in parentheses. *** p<0.01, ** p<0.05, * p<0.1 49
49 Conclusion Discovering subpopulations with significant treatment effects can be paramount Machine Learning can flexibly estimate effects but it limited when goal is to identify subpopulations with large effects Anomalous Pattern Detection paradigm offers overcome some abilities to overcome these limitations Maintain high power to detect by searching over and combining signal across various subpopulations 50
50 References Athey, S., & Imbens, G. W. (2015). Machine Learning Methods for Estimating Heterogeneous Causal Effects. arxiv.org. Green, D. P., & Kern, H. L. (2012). Modeling Heterogeneous Treatment Effects in Survey Experiments with Bayesian Additive Regression Trees. Public Opinion Quarterly, 76(3), Imai, K., & Ratkovic, M. (2013). Estimating treatment effect heterogeneity in randomized program evaluation. The Annals of Applied Statistics, 7(1), Imai, K., & Strauss, A. (2011). Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign. Political Analysis, 19(1), McFowland, E., III, Speakman, S. D., & Neill, D. B. (2013). Fast generalized subset scan for anomalous pattern detection. The Journal of Machine Learning Research, 14(1), Mehlig, Kirsten et al. CETP TaqIB genotype modifies the association between alcohol and coronary heart disease: The INTERGENE case-control study. Alcohol, 48(7), Su, X., Tsai, C.-L., Wang, H., Nickerson, D. M., & Li, B. (2009). Subgroup Analysis via Recursive Partitioning. The Journal of Machine Learning Research, 10, Wager, S., & Athey, S. (2015). Estimation and Inference of Heterogeneous Treatment Effects using Random Forests. arxiv.org. 51
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests Stefan Wager Department of Statistics Stanford University swager@stanford.edu Susan Athey Graduate School of Business Stanford
Automated Discovery of Novel Anomalous Patterns
Automated Discovery of Novel Anomalous Patterns Edward McFowland III Machine Learning Department School of Computer Science Carnegie Mellon University mcfowland@cmu.edu DAP Committee: Daniel B. Neill Jeff
Selective Inference for Effect Modification
Inference for Modification (Joint work with Dylan Small and Ashkan Ertefaie) Department of Statistics, University of Pennsylvania May 24, ACIC 2017 Manuscript and slides are available at http://www-stat.wharton.upenn.edu/~qyzhao/.
Bayesian Causal Forests
Bayesian Causal Forests for estimating personalized treatment effects P. Richard Hahn, Jared Murray, and Carlos Carvalho December 8, 2016 Overview We consider observational data, assuming conditional unconfoundedness,
Machine Learning Linear Regression. Prof. Matteo Matteucci
Machine Learning Linear Regression Prof. Matteo Matteucci Outline 2 o Simple Linear Regression Model Least Squares Fit Measures of Fit Inference in Regression o Multi Variate Regession Model Least Squares
Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity
Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity P. Richard Hahn, Jared Murray, and Carlos Carvalho June 22, 2017 The problem setting We want to estimate
Why experimenters should not randomize, and what they should do instead
Why experimenters should not randomize, and what they should do instead Maximilian Kasy Department of Economics, Harvard University Maximilian Kasy (Harvard) Experimental design 1 / 42 project STAR Introduction
Variable selection and machine learning methods in causal inference
Variable selection and machine learning methods in causal inference Debashis Ghosh Department of Biostatistics and Informatics Colorado School of Public Health Joint work with Yeying Zhu, University of
Ensemble learning 11/19/13. The wisdom of the crowds. Chapter 11. Ensemble methods. Ensemble methods
The wisdom of the crowds Ensemble learning Sir Francis Galton discovered in the early 1900s that a collection of educated guesses can add up to very accurate predictions! Chapter 11 The paper in which
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University Joint
The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects
The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects Thai T. Pham Stanford Graduate School of Business thaipham@stanford.edu May, 2016 Introduction Observations Many sequential
Selection on Observables: Propensity Score Matching.
Selection on Observables: Propensity Score Matching. Department of Economics and Management Irene Brunetti ireneb@ec.unipi.it 24/10/2017 I. Brunetti Labour Economics in an European Perspective 24/10/2017
Covariate Balancing Propensity Score for General Treatment Regimes
Covariate Balancing Propensity Score for General Treatment Regimes Kosuke Imai Princeton University October 14, 2014 Talk at the Department of Psychiatry, Columbia University Joint work with Christian
Fast Multidimensional Subset Scan for Outbreak Detection and Characterization
Fast Multidimensional Subset Scan for Outbreak Detection and Characterization Daniel B. Neill* and Tarun Kumar Event and Pattern Detection Laboratory Carnegie Mellon University neill@cs.cmu.edu This project
Learning Representations for Counterfactual Inference. Fredrik Johansson 1, Uri Shalit 2, David Sontag 2
Learning Representations for Counterfactual Inference Fredrik Johansson 1, Uri Shalit 2, David Sontag 2 1 2 Counterfactual inference Patient Anna comes in with hypertension. She is 50 years old, Asian
Results and Findings. I. Theory of linear time subset scanning
Results and Findings I. Theory of linear time subset scanning In Neill (2012), we present formal proofs of the linear time subset scanning property for a large class of score functions, using the proof
Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
Bayesian Nonparametric Regression for Diabetes Deaths
Bayesian Nonparametric Regression for Diabetes Deaths Brian M. Hartman PhD Student, 2010 Texas A&M University College Station, TX, USA David B. Dahl Assistant Professor Texas A&M University College Station,
Big Data, Machine Learning, and Causal Inference
Big Data, Machine Learning, and Causal Inference I C T 4 E v a l I n t e r n a t i o n a l C o n f e r e n c e I F A D H e a d q u a r t e r s, R o m e, I t a l y P a u l J a s p e r p a u l. j a s p e
Supplemental Appendix to "Alternative Assumptions to Identify LATE in Fuzzy Regression Discontinuity Designs"
Supplemental Appendix to "Alternative Assumptions to Identify LATE in Fuzzy Regression Discontinuity Designs" Yingying Dong University of California Irvine February 2018 Abstract This document provides
DO NOT CITE WITHOUT AUTHOR S PERMISSION:
Study Context & Purpose Prior to the 2013-14 school year a Tennessee policy went into effect such that a teacher s level of effectiveness (LOE) and certification status determined their minimum number
High Dimensional Propensity Score Estimation via Covariate Balancing
High Dimensional Propensity Score Estimation via Covariate Balancing Kosuke Imai Princeton University Talk at Columbia University May 13, 2017 Joint work with Yang Ning and Sida Peng Kosuke Imai (Princeton)
Causal Effect Evaluation and Causal Network Learning
and Peking University, China June 25, 2014 and Outline 1 Yule-Simpson paradox Causal effects Surrogate and surrogate paradox 2 and Outline Yule-Simpson paradox Causal effects Surrogate and surrogate paradox
An Experimental Evaluation of High-Dimensional Multi-Armed Bandits
An Experimental Evaluation of High-Dimensional Multi-Armed Bandits Naoki Egami Romain Ferrali Kosuke Imai Princeton University Talk at Political Data Science Conference Washington University, St. Louis
Causal Inference with Big Data Sets
Causal Inference with Big Data Sets Marcelo Coca Perraillon University of Colorado AMC November 2016 1 / 1 Outlone Outline Big data Causal inference in economics and statistics Regression discontinuity
From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu
From statistics to data science BAE 815 (Fall 2017) Dr. Zifei Liu Zifeiliu@ksu.edu Why? How? What? How much? How many? Individual facts (quantities, characters, or symbols) The Data-Information-Knowledge-Wisdom
MS&E 226: Small Data
MS&E 226: Small Data Lecture 9: Logistic regression (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 28 Regression methods for binary outcomes 2 / 28 Binary outcomes For the duration of this lecture suppose
Individualized Treatment Effects with Censored Data via Nonparametric Accelerated Failure Time Models
Individualized Treatment Effects with Censored Data via Nonparametric Accelerated Failure Time Models Nicholas C. Henderson Thomas A. Louis Gary Rosner Ravi Varadhan Johns Hopkins University July 31, 2018
Binary Models with Endogenous Explanatory Variables
Binary Models with Endogenous Explanatory Variables Class otes Manuel Arellano ovember 7, 2007 Revised: January 21, 2008 1 Introduction In Part I we considered linear and non-linear models with additive
Econometrics Problem Set 6
Econometrics Problem Set 6 WISE, Xiamen University Spring 2016-17 Conceptual Questions 1. This question refers to the estimated regressions shown in Table 1 computed using data for 1988 from the CPS. The
From Sentiment Analysis to Preference Aggregation
From Sentiment Analysis to Preference Aggregation Umberto Grandi Department of Mathematics University of Padova 22 November 2013 [Joint work with Andrea Loreggia, Francesca Rossi and Vijay Saraswat] What
Random Forests. These notes rely heavily on Biau and Scornet (2016) as well as the other references at the end of the notes.
Random Forests One of the best known classifiers is the random forest. It is very simple and effective but there is still a large gap between theory and practice. Basically, a random forest is an average
MS&E 226: Small Data
MS&E 226: Small Data Lecture 12: Logistic regression (v1) Ramesh Johari ramesh.johari@stanford.edu Fall 2015 1 / 30 Regression methods for binary outcomes 2 / 30 Binary outcomes For the duration of this
Association studies and regression
Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration
Data analysis strategies for high dimensional social science data M3 Conference May 2013
Data analysis strategies for high dimensional social science data M3 Conference May 2013 W. Holmes Finch, Maria Hernández Finch, David E. McIntosh, & Lauren E. Moss Ball State University High dimensional
Meelis Kull Autumn Meelis Kull - Autumn MTAT Data Mining - Lecture 05
Meelis Kull meelis.kull@ut.ee Autumn 2017 1 Sample vs population Example task with red and black cards Statistical terminology Permutation test and hypergeometric test Histogram on a sample vs population
Flexible Estimation of Treatment Effect Parameters
Flexible Estimation of Treatment Effect Parameters Thomas MaCurdy a and Xiaohong Chen b and Han Hong c Introduction Many empirical studies of program evaluations are complicated by the presence of both
Gov 2002: 4. Observational Studies and Confounding
Gov 2002: 4. Observational Studies and Confounding Matthew Blackwell September 10, 2015 Where are we? Where are we going? Last two weeks: randomized experiments. From here on: observational studies. What
Review of Panel Data Model Types Next Steps. Panel GLMs. Department of Political Science and Government Aarhus University.
Panel GLMs Department of Political Science and Government Aarhus University May 12, 2015 1 Review of Panel Data 2 Model Types 3 Review and Looking Forward 1 Review of Panel Data 2 Model Types 3 Review
Regression Discontinuity Designs
Regression Discontinuity Designs Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 1 / 1 Observational
Interaction Effects and Causal Heterogeneity
Interaction Effects and Causal Heterogeneity Kosuke Imai Princeton University Winter Conference in Statistics Borgafjäll, Sweden March 11, 2015 Joint work with Naoki Egami and Marc Ratkovic Kosuke Imai
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University
Final Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
Causal Inference with General Treatment Regimes: Generalizing the Propensity Score
Causal Inference with General Treatment Regimes: Generalizing the Propensity Score David van Dyk Department of Statistics, University of California, Irvine vandyk@stat.harvard.edu Joint work with Kosuke
causal inference at hulu
causal inference at hulu Allen Tran July 17, 2016 Hulu Introduction Most interesting business questions at Hulu are causal Business: what would happen if we did x instead of y? dropped prices for risky
Randomized Decision Trees
Randomized Decision Trees compiled by Alvin Wan from Professor Jitendra Malik s lecture Discrete Variables First, let us consider some terminology. We have primarily been dealing with real-valued data,
Model Selection. Frank Wood. December 10, 2009
Model Selection Frank Wood December 10, 2009 Standard Linear Regression Recipe Identify the explanatory variables Decide the functional forms in which the explanatory variables can enter the model Decide
Correlation Paradox. istation Whitepaper Rev A. September 2009
Correlation Paradox istation Whitepaper 2009 01 Rev A September 2009 istation.com 800 E Campbell Rd, Ste 224 Richardson, TX 75081 (866) 883 READ info@istation.com Summary In research, when correlations
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies Kosuke Imai Department of Politics Princeton University November 13, 2013 So far, we have essentially assumed
3 Comparison with Other Dummy Variable Methods
Stats 300C: Theory of Statistics Spring 2018 Lecture 11 April 25, 2018 Prof. Emmanuel Candès Scribe: Emmanuel Candès, Michael Celentano, Zijun Gao, Shuangning Li 1 Outline Agenda: Knockoffs 1. Introduction
15-381: Artificial Intelligence. Decision trees
15-381: Artificial Intelligence Decision trees Bayes classifiers find the label that maximizes: Naïve Bayes models assume independence of the features given the label leading to the following over documents
arxiv: v3 [stat.ml] 30 Dec 2015
![arxiv: v3 [stat.ml] 30 Dec 2015](/thumbs/86/93312693.jpg "arxiv: v3 [stat.ml] 30 Dec 2015")
Recursive Partitioning for Heterogeneous Causal Effects Susan Athey Guido W. Imbens arxiv:1504.01132v3 [stat.ml] 30 Dec 2015 First Draft: October 2013 This Draft: December 2015 Abstract In this paper we
Online Supplementary Material. MetaLP: A Nonparametric Distributed Learning Framework for Small and Big Data
Online Supplementary Material MetaLP: A Nonparametric Distributed Learning Framework for Small and Big Data PI : Subhadeep Mukhopadhyay Department of Statistics, Temple University Philadelphia, Pennsylvania,
Expectation Propagation Algorithm
Expectation Propagation Algorithm 1 Shuang Wang School of Electrical and Computer Engineering University of Oklahoma, Tulsa, OK, 74135 Email: {shuangwang}@ou.edu This note contains three parts. First,
Empirical Methods in Applied Microeconomics
Empirical Methods in Applied Microeconomics Jörn-Ste en Pischke LSE November 2007 1 Nonlinearity and Heterogeneity We have so far concentrated on the estimation of treatment e ects when the treatment e
Local Structure Discovery in Bayesian Networks
1 / 45 HELSINGIN YLIOPISTO HELSINGFORS UNIVERSITET UNIVERSITY OF HELSINKI Local Structure Discovery in Bayesian Networks Teppo Niinimäki, Pekka Parviainen August 18, 2012 University of Helsinki Department
Linear Model Selection and Regularization
Linear Model Selection and Regularization Recall the linear model Y = β 0 + β 1 X 1 + + β p X p + ɛ. In the lectures that follow, we consider some approaches for extending the linear model framework. In
Undirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
Decision Tree And Random Forest
Decision Tree And Random Forest Dr. Ammar Mohammed Associate Professor of Computer Science ISSR, Cairo University PhD of CS ( Uni. Koblenz-Landau, Germany) Spring 2019 Contact: mailto: Ammar@cu.edu.eg
Econometrics of causal inference. Throughout, we consider the simplest case of a linear outcome equation, and homogeneous
Econometrics of causal inference Throughout, we consider the simplest case of a linear outcome equation, and homogeneous effects: y = βx + ɛ (1) where y is some outcome, x is an explanatory variable, and
Causal Interaction in High Dimension
Causal Interaction in High Dimension Naoki Egami Kosuke Imai Princeton University Annual Summer Meeting The Society for Political Methodology University of Rochester July 23, 2015 Egami and Imai (Princeton)
Sparse Linear Models (10/7/13)
STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine
Review of the Normal Distribution
Sampling and s Normal Distribution Aims of Sampling Basic Principles of Probability Types of Random Samples s of the Mean Standard Error of the Mean The Central Limit Theorem Review of the Normal Distribution
Introduction to Machine Learning and Cross-Validation
Introduction to Machine Learning and Cross-Validation Jonathan Hersh 1 February 27, 2019 J.Hersh (Chapman ) Intro & CV February 27, 2019 1 / 29 Plan 1 Introduction 2 Preliminary Terminology 3 Bias-Variance
Using Decision Trees to Evaluate the Impact of Title 1 Programs in the Windham School District
Using Decision Trees to Evaluate the Impact of Programs in the Windham School District Box 41071 Lubbock, Texas 79409-1071 Phone: (806) 742-1958 www.depts.ttu.edu/immap/ Stats Camp offers week long sessions
Overview. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Overview DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing
CPSC 340: Machine Learning and Data Mining. Feature Selection Fall 2018
CPSC 340: Machine Learning and Data Mining Feature Selection Fall 2018 Admin Assignment 3 is due tonight. You can use a late day to submit up to 48 hours late. Solutions will be posted Monday. Midterm
Constructing Prediction Intervals for Random Forests
Senior Thesis in Mathematics Constructing Prediction Intervals for Random Forests Author: Benjamin Lu Advisor: Dr. Jo Hardin Submitted to Pomona College in Partial Fulfillment of the Degree of Bachelor
Problem Set #5. Due: 1pm on Friday, Nov 16th
1 Chris Piech CS 109 Problem Set #5 Due: 1pm on Friday, Nov 16th Problem Set #5 Nov 7 th, 2018 With problems by Mehran Sahami and Chris Piech For each problem, briefly explain/justify how you obtained
Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
An Introduction to Causal Analysis on Observational Data using Propensity Scores
An Introduction to Causal Analysis on Observational Data using Propensity Scores Margie Rosenberg*, PhD, FSA Brian Hartman**, PhD, ASA Shannon Lane* *University of Wisconsin Madison **University of Connecticut
Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning Sören R. Künzel a, Jasjeet S. Sekhon a,b, Peter J. Bickel a, and Bin Yu a,c arxiv:1706.03461v4 math.st 16 Mar 2018 a
Mostly Dangerous Econometrics: How to do Model Selection with Inference in Mind
Outline Introduction Analysis in Low Dimensional Settings Analysis in High-Dimensional Settings Bonus Track: Genaralizations Econometrics: How to do Model Selection with Inference in Mind June 25, 2015,
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University August 30, 2017 Today: Decision trees Overfitting The Big Picture Coming soon Probabilistic learning MLE,
CHAPTER-17. Decision Tree Induction
CHAPTER-17 Decision Tree Induction 17.1 Introduction 17.2 Attribute selection measure 17.3 Tree Pruning 17.4 Extracting Classification Rules from Decision Trees 17.5 Bayesian Classification 17.6 Bayes
Bayesian Nonparametric Accelerated Failure Time Models for Analyzing Heterogeneous Treatment Effects
Bayesian Nonparametric Accelerated Failure Time Models for Analyzing Heterogeneous Treatment Effects Nicholas C. Henderson Thomas A. Louis Gary Rosner Ravi Varadhan Johns Hopkins University September 28,
Classification and Pattern Recognition
Classification and Pattern Recognition Léon Bottou NEC Labs America COS 424 2/23/2010 The machine learning mix and match Goals Representation Capacity Control Operational Considerations Computational Considerations
Predictive analysis on Multivariate, Time Series datasets using Shapelets
1 Predictive analysis on Multivariate, Time Series datasets using Shapelets Hemal Thakkar Department of Computer Science, Stanford University hemal@stanford.edu hemal.tt@gmail.com Abstract Multivariate,
Lecture 7 Decision Tree Classifier
Machine Learning Dr.Ammar Mohammed Lecture 7 Decision Tree Classifier Decision Tree A decision tree is a simple classifier in the form of a hierarchical tree structure, which performs supervised classification
Likelihood and Fairness in Multidimensional Item Response Theory
Likelihood and Fairness in Multidimensional Item Response Theory or What I Thought About On My Holidays Giles Hooker and Matthew Finkelman Cornell University, February 27, 2008 Item Response Theory Educational
Part 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis
Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis Weilan Yang wyang@stat.wisc.edu May. 2015 1 / 20 Background CATE i = E(Y i (Z 1 ) Y i (Z 0 ) X i ) 2 / 20 Background
Bayesian Learning in Undirected Graphical Models
Bayesian Learning in Undirected Graphical Models Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK http://www.gatsby.ucl.ac.uk/ Work with: Iain Murray and Hyun-Chul
Education Production Functions. April 7, 2009
Education Production Functions April 7, 2009 Outline I Production Functions for Education Hanushek Paper Card and Krueger Tennesee Star Experiment Maimonides Rule What do I mean by Production Function?
arxiv: v1 [stat.ml] 1 Jul 2017
![arxiv: v1 [stat.ml] 1 Jul 2017](/thumbs/72/67910236.jpg "arxiv: v1 [stat.ml] 1 Jul 2017")
Some methods for heterogeneous treatment effect estimation in high-dimensions arxiv:1707.00102v1 [stat.ml] 1 Jul 2017 Scott Powers, Junyang Qian, Kenneth Jung, Alejandro Schuler, Nigam H. Shah, Trevor
Paradoxical Results in Multidimensional Item Response Theory
UNC, December 6, 2010 Paradoxical Results in Multidimensional Item Response Theory Giles Hooker and Matthew Finkelman UNC, December 6, 2010 1 / 49 Item Response Theory Educational Testing Traditional model
Probability: Why do we care? Lecture 2: Probability and Distributions. Classical Definition. What is Probability?
Probability: Why do we care? Lecture 2: Probability and Distributions Sandy Eckel seckel@jhsph.edu 22 April 2008 Probability helps us by: Allowing us to translate scientific questions into mathematical
STAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
Discussion of Papers on the Extensions of Propensity Score
Discussion of Papers on the Extensions of Propensity Score Kosuke Imai Princeton University August 3, 2010 Kosuke Imai (Princeton) Generalized Propensity Score 2010 JSM (Vancouver) 1 / 11 The Theme and
Probabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016
Probabilistic classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Probabilistic approach Bayes decision theory Generative models Gaussian Bayes classifier
Data Mining Classification: Basic Concepts and Techniques. Lecture Notes for Chapter 3. Introduction to Data Mining, 2nd Edition
Data Mining Classification: Basic Concepts and Techniques Lecture Notes for Chapter 3 by Tan, Steinbach, Karpatne, Kumar 1 Classification: Definition Given a collection of records (training set ) Each
Cross-Validation with Confidence
Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University WHOA-PSI Workshop, St Louis, 2017 Quotes from Day 1 and Day 2 Good model or pure model? Occam s razor We really
Bayesian variable selection via. Penalized credible regions. Brian Reich, NCSU. Joint work with. Howard Bondell and Ander Wilson
Bayesian variable selection via penalized credible regions Brian Reich, NC State Joint work with Howard Bondell and Ander Wilson Brian Reich, NCSU Penalized credible regions 1 Motivation big p, small n
Instrumental Variables
Instrumental Variables Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Noncompliance in Experiments Stat186/Gov2002 Fall 2018 1 / 18 Instrumental Variables
Structure Learning: the good, the bad, the ugly
Readings: K&F: 15.1, 15.2, 15.3, 15.4, 15.5 Structure Learning: the good, the bad, the ugly Graphical Models 10708 Carlos Guestrin Carnegie Mellon University September 29 th, 2006 1 Understanding the uniform
The Union of Intersections (UoI) Method for Interpretable Data Driven Discovery and Prediction Kristofer E. Bouchard 1,2,3*, Alejandro Bujan 3,
The Union of Intersections (UoI) Method for Interpretable Data Driven Discovery and Prediction Kristofer E. Bouchard 1,2,3*, Alejandro Bujan 3, Farbod Roosta 4, Prabhat 5, Antoine Snijdes 1, J-H Mao 1,
Habilitationsvortrag: Machine learning, shrinkage estimation, and economic theory
Habilitationsvortrag: Machine learning, shrinkage estimation, and economic theory Maximilian Kasy May 25, 218 1 / 27 Introduction Recent years saw a boom of machine learning methods. Impressive advances
Part I. Linear regression & LASSO. Linear Regression. Linear Regression. Week 10 Based in part on slides from textbook, slides of Susan Holmes
Week 10 Based in part on slides from textbook, slides of Susan Holmes Part I Linear regression & December 5, 2012 1 / 1 2 / 1 We ve talked mostly about classification, where the outcome categorical. If
Introduction to Econometrics
Introduction to Econometrics STAT-S-301 Experiments and Quasi-Experiments (2016/2017) Lecturer: Yves Dominicy Teaching Assistant: Elise Petit 1 Why study experiments? Ideal randomized controlled experiments
Hidden Markov models
Hidden Markov models Charles Elkan November 26, 2012 Important: These lecture notes are based on notes written by Lawrence Saul. Also, these typeset notes lack illustrations. See the classroom lectures