Regression. Jordan Boyd-Graber. University of Colorado Boulder LECTURE 11. Jordan Boyd-Graber Boulder Regression 1 of 19
|
|
|
- Candice Patrick
- 5 years ago
- Views:
Transcription
1 Regression Jordan Boyd-Graber University of Colorado Boulder LECTURE 11 Jordan Boyd-Graber Boulder Regression 1 of 19
2 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
3 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
4 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
5 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
6 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
7 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
8 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
9 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
10 Content Questions Jordan Boyd-Graber Boulder Regression 2 of 19
11 Content Questions. Jordan Boyd-Graber Boulder Regression 2 of 19
12 Administrivia Learnability HW due Friday SVM HW next week I m out of town next week Jordan Boyd-Graber Boulder Regression 3 of 19
13 Project Default project or choose your own I ll meet with your group early in the week of Oct 12 Project proposal First deliverable due Nov 6: data / baseline Jordan Boyd-Graber Boulder Regression 4 of 19
14 Basics Plan Basics Regularization Sklearn Jordan Boyd-Graber Boulder Regression 5 of 19
15 Basics Predictions dimension weight b 1 w w σ x 1 = {0.0, 0.0}; y 1 = 2. x 2 = {1.0, 1.0}; y 2 = 3. x 3 = {.5, 2}; y 3 = Jordan Boyd-Graber Boulder Regression 6 of 19
16 Basics Predictions dimension weight b 1 w w σ x 1 = {0.0, 0.0}; y 1 = x 2 = {1.0, 1.0}; y 2 = 3. x 3 = {.5, 2}; y 3 = Jordan Boyd-Graber Boulder Regression 6 of 19
17 Basics Predictions dimension weight b 1 w w σ x 1 = {0.0, 0.0}; y 1 = x 2 = {1.0, 1.0}; y 2 = x 3 = {.5, 2}; y 3 = Jordan Boyd-Graber Boulder Regression 6 of 19
18 Basics Predictions dimension weight b 1 w w σ x 1 = {0.0, 0.0}; y 1 = x 2 = {1.0, 1.0}; y 2 = x 3 = {.5, 2}; y 3 =0.0 Jordan Boyd-Graber Boulder Regression 6 of 19
19 Basics Probabilities dimension weight w 0 1 w w σ 1.0 p(y x) = y N b + exp p(y x) = { (y ŷ)2 2 2π p w j x j, σ 2 j=1 } 1. p(y 1 = 1 x 1 = {0.0, 0.0}) = 2. p(y 2 = 3 x 2 = {1.0, 1.0}) = 3. p(y 3 = 1 x 3 = {.5, 2}) = Jordan Boyd-Graber Boulder Regression 7 of 19
20 Basics Probabilities dimension weight w 0 1 w w σ 1.0 p(y x) = y N b + exp p(y x) = { (y ŷ)2 2 2π p w j x j, σ 2 j=1 } 1. p(y 1 = 1 x 1 = {0.0, 0.0}) = p(y 2 = 3 x 2 = {1.0, 1.0}) = 3. p(y 3 = 1 x 3 = {.5, 2}) = Jordan Boyd-Graber Boulder Regression 7 of 19
21 Basics Probabilities dimension weight w 0 1 w w σ 1.0 p(y x) = y N b + exp p(y x) = { (y ŷ)2 2 2π p w j x j, σ 2 j=1 } 1. p(y 1 = 1 x 1 = {0.0, 0.0}) = p(y 2 = 3 x 2 = {1.0, 1.0}) = p(y 3 = 1 x 3 = {.5, 2}) = Jordan Boyd-Graber Boulder Regression 7 of 19
22 Basics Probabilities dimension weight w 0 1 w w σ 1.0 p(y x) = y N b + exp p(y x) = { (y ŷ)2 2 2π p w j x j, σ 2 j=1 } 1. p(y 1 = 1 x 1 = {0.0, 0.0}) = p(y 2 = 3 x 2 = {1.0, 1.0}) = p(y 3 = 1 x 3 = {.5, 2}) = Jordan Boyd-Graber Boulder Regression 7 of 19
23 Regularization Plan Basics Regularization Sklearn Jordan Boyd-Graber Boulder Regression 8 of 19
24 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 Jordan Boyd-Graber Boulder Regression 9 of 19
25 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 Which is the better OLS solution? Jordan Boyd-Graber Boulder Regression 9 of 19
26 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 Blue! It has lower RSS. Jordan Boyd-Graber Boulder Regression 9 of 19
27 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 What is the RSS of the better solution? Jordan Boyd-Graber Boulder Regression 9 of 19
28 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 1 2 i r i 2 = 1 ( 2 (1 1) 2 + (2.5 2) 2 + (2.5 3) 2) = 1 4 Jordan Boyd-Graber Boulder Regression 9 of 19
29 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 What is the RSS of the red line? Jordan Boyd-Graber Boulder Regression 9 of 19
30 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 1 2 i r i 2 = 1 ( 2 (1 1) 2 + ( ) 2 + ( ) 2) = 3 8 Jordan Boyd-Graber Boulder Regression 9 of 19
31 Regularization Consider these points and data w=.75,b=.25 w=1,b=0 For what λ does the blue line have a better regularized solution with L 2 and L 1? Jordan Boyd-Graber Boulder Regression 9 of 19
32 Regularization When Regularization Wins L 2 L 1 Jordan Boyd-Graber Boulder Regression 10 of 19
33 Regularization When Regularization Wins L 2 RSS(x, y, w) + λ d w 2 d > RSS(x, y, w) + λ d w 2 d L 1 Jordan Boyd-Graber Boulder Regression 10 of 19
34 Regularization When Regularization Wins L 2 RSS(x, y, w) + λ d w 2 d > RSS(x, y, w) + λ d w 2 d λ1 > λ 9 16 L 1 Jordan Boyd-Graber Boulder Regression 10 of 19
35 Regularization When Regularization Wins L λ1 > λ λ > 1 8 L 1 Jordan Boyd-Graber Boulder Regression 10 of 19
36 Regularization When Regularization Wins L λ > 1 8 λ > 2 7 L 1 Jordan Boyd-Graber Boulder Regression 10 of 19
37 Regularization When Regularization Wins L 2 λ > 2 7 L 1 RSS(x, y, w) + λ d w d > RSS(x, y, w) + λ d w d Jordan Boyd-Graber Boulder Regression 10 of 19
38 Regularization When Regularization Wins L 2 λ > 2 7 L 1 RSS(x, y, w) + λ d w d > RSS(x, y, w) + λ d w d λ1 > λ3 4 Jordan Boyd-Graber Boulder Regression 10 of 19
39 Regularization When Regularization Wins L 2 λ > 2 7 L λ1 > λ3 4 Jordan Boyd-Graber Boulder Regression 10 of 19
40 Regularization When Regularization Wins L 2 λ > 2 7 L λ > 1 8 Jordan Boyd-Graber Boulder Regression 10 of 19
41 Regularization When Regularization Wins L 2 λ > 2 7 L λ > 1 8 λ > 1 2 Jordan Boyd-Graber Boulder Regression 10 of 19
42 Regularization When Regularization Wins L 2 λ > 2 7 L 1 λ > 1 2 Bigger λ: preference for lower weights w Jordan Boyd-Graber Boulder Regression 10 of 19
43 Sklearn Plan Basics Regularization Sklearn Jordan Boyd-Graber Boulder Regression 11 of 19
44 Sklearn MPG Dataset Predict mpg from features of a car 1. Number of cylinders 2. Displacement 3. Horsepower 4. Weight 5. Acceleration 6. Year 7. Country (ignore this) Jordan Boyd-Graber Boulder Regression 12 of 19
45 Sklearn Simple Regression If w = 0, what s the intercept? Jordan Boyd-Graber Boulder Regression 13 of 19
46 Sklearn Simple Regression If w = 0, what s the intercept? 23.4 Jordan Boyd-Graber Boulder Regression 13 of 19
47 Sklearn Simple Linear Regression What are the coefficients for OLS? Jordan Boyd-Graber Boulder Regression 14 of 19
48 Sklearn Simple Linear Regression What are the coefficients for OLS? c y l d i s hp wgt a c l y r Jordan Boyd-Graber Boulder Regression 14 of 19
49 Sklearn Simple Linear Regression What are the coefficients for OLS? c y l d i s hp wgt a c l y r Intercept: Jordan Boyd-Graber Boulder Regression 14 of 19
50 Sklearn Simple Linear Regression from s k l e a r n import l i n e a r m o d e l l i n e a r m o d e l. L i n e a r R e g r e s s i o n ( ) f i t = model. f i t ( x, y ) Jordan Boyd-Graber Boulder Regression 15 of 19
51 Sklearn Lasso As you increase the weight of alpha, what feature dominates? What happens to the other features? Jordan Boyd-Graber Boulder Regression 16 of 19
52 Sklearn Weight is Everything Jordan Boyd-Graber Boulder mpg = Weight Regression 17 of 19
53 Sklearn How is ridge different? Jordan Boyd-Graber Boulder Regression 18 of 19
54 Sklearn Regression isn t special Feature engineering Regularization Overfitting Development / Test Data Jordan Boyd-Graber Boulder Regression 19 of 19
Solving Regression. Jordan Boyd-Graber. University of Colorado Boulder LECTURE 12. Slides adapted from Matt Nedrich and Trevor Hastie
Solving Regression Jordan Boyd-Graber University of Colorado Boulder LECTURE 12 Slides adapted from Matt Nedrich and Trevor Hastie Jordan Boyd-Graber Boulder Solving Regression 1 of 17 Roadmap We talked
Classification: Logistic Regression from Data
Classification: Logistic Regression from Data Machine Learning: Jordan Boyd-Graber University of Colorado Boulder LECTURE 3 Slides adapted from Emily Fox Machine Learning: Jordan Boyd-Graber Boulder Classification:
Logistic Regression. Introduction to Data Science Algorithms Jordan Boyd-Graber and Michael Paul SLIDES ADAPTED FROM HINRICH SCHÜTZE
Logistic Regression Introduction to Data Science Algorithms Jordan Boyd-Graber and Michael Paul SLIDES ADAPTED FROM HINRICH SCHÜTZE Introduction to Data Science Algorithms Boyd-Graber and Paul Logistic
COMS 4771 Introduction to Machine Learning. James McInerney Adapted from slides by Nakul Verma
COMS 4771 Introduction to Machine Learning James McInerney Adapted from slides by Nakul Verma Announcements HW1: Please submit as a group Watch out for zero variance features (Q5) HW2 will be released
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 5
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 5 Slides adapted from Jordan Boyd-Graber, Tom Mitchell, Ziv Bar-Joseph Machine Learning: Chenhao Tan Boulder 1 of 27 Quiz question For
Classification: The PAC Learning Framework
Classification: The PAC Learning Framework Machine Learning: Jordan Boyd-Graber University of Colorado Boulder LECTURE 5 Slides adapted from Eli Upfal Machine Learning: Jordan Boyd-Graber Boulder Classification:
II. Linear Models (pp.47-70)
Notation: Means pencil-and-paper QUIZ Means coding QUIZ Agree or disagree: Regression can be always reduced to classification. Explain, either way! A certain classifier scores 98% on the training set,
Introduction to Machine Learning
Introduction to Machine Learning Machine Learning: Jordan Boyd-Graber University of Maryland LOGISTIC REGRESSION FROM TEXT Slides adapted from Emily Fox Machine Learning: Jordan Boyd-Graber UMD Introduction
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 6
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 6 Slides adapted from Jordan Boyd-Graber, Chris Ketelsen Machine Learning: Chenhao Tan Boulder 1 of 39 HW1 turned in HW2 released Office
Logistic Regression. INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM HINRICH SCHÜTZE
Logistic Regression INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM HINRICH SCHÜTZE INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Logistic Regression
Discrete Probability Distributions
Discrete Probability Distributions Data Science: Jordan Boyd-Graber University of Maryland JANUARY 18, 2018 Data Science: Jordan Boyd-Graber UMD Discrete Probability Distributions 1 / 1 Refresher: Random
Lecture 14: Shrinkage
Lecture 14: Shrinkage Reading: Section 6.2 STATS 202: Data mining and analysis October 27, 2017 1 / 19 Shrinkage methods The idea is to perform a linear regression, while regularizing or shrinking the
COMS 4771 Regression. Nakul Verma
COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the
Machine Learning (CSE 446): Learning as Minimizing Loss; Least Squares
Machine Learning (CSE 446): Learning as Minimizing Loss; Least Squares Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu 1 / 13 Review 1 / 13 Alternate View of PCA: Minimizing
Online Learning. Jordan Boyd-Graber. University of Colorado Boulder LECTURE 21. Slides adapted from Mohri
Online Learning Jordan Boyd-Graber University of Colorado Boulder LECTURE 21 Slides adapted from Mohri Jordan Boyd-Graber Boulder Online Learning 1 of 31 Motivation PAC learning: distribution fixed over
Inexact Search is Good Enough
Inexact Search is Good Enough Advanced Machine Learning for NLP Jordan Boyd-Graber MATHEMATICAL TREATMENT Advanced Machine Learning for NLP Boyd-Graber Inexact Search is Good Enough 1 of 1 Preliminaries:
LINEAR REGRESSION, RIDGE, LASSO, SVR
LINEAR REGRESSION, RIDGE, LASSO, SVR Supervised Learning Katerina Tzompanaki Linear regression one feature* Price (y) What is the estimated price of a new house of area 30 m 2? 30 Area (x) *Also called
Machine Learning & Data Mining Caltech CS/CNS/EE 155 Set 2 January 12 th, 2018
Policies Due 9 PM, January 9 th, via Moodle. You are free to collaborate on all of the problems, subject to the collaboration policy stated in the syllabus. You should submit all code used in the homework.
Self-Assessment Weeks 6 and 7: Multiple Regression with a Qualitative Predictor; Multiple Comparisons
Self-Assessment Weeks 6 and 7: Multiple Regression with a Qualitative Predictor; Multiple Comparisons 1. Suppose we wish to assess the impact of five treatments on an outcome Y. How would these five treatments
Announcements. J. Parman (UC-Davis) Analysis of Economic Data, Winter 2011 February 8, / 45
Announcements Solutions to Problem Set 3 are posted Problem Set 4 is posted, It will be graded and is due a week from Friday You already know everything you need to work on Problem Set 4 Professor Miller
COMP 551 Applied Machine Learning Lecture 3: Linear regression (cont d)
COMP 551 Applied Machine Learning Lecture 3: Linear regression (cont d) Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless
Lecture 6: Linear Regression (continued)
Lecture 6: Linear Regression (continued) Reading: Sections 3.1-3.3 STATS 202: Data mining and analysis October 6, 2017 1 / 23 Multiple linear regression Y = β 0 + β 1 X 1 + + β p X p + ε Y ε N (0, σ) i.i.d.
Lecture 6: Linear Regression
Lecture 6: Linear Regression Reading: Sections 3.1-3 STATS 202: Data mining and analysis Jonathan Taylor, 10/5 Slide credits: Sergio Bacallado 1 / 30 Simple linear regression Model: y i = β 0 + β 1 x i
Applied Machine Learning Lecture 5: Linear classifiers, continued. Richard Johansson
Applied Machine Learning Lecture 5: Linear classifiers, continued Richard Johansson overview preliminaries logistic regression training a logistic regression classifier side note: multiclass linear classifiers
Support Vector Machines
Support Vector Machines INFO-4604, Applied Machine Learning University of Colorado Boulder September 28, 2017 Prof. Michael Paul Today Two important concepts: Margins Kernels Large Margin Classification
Warm up: risk prediction with logistic regression
Warm up: risk prediction with logistic regression Boss gives you a bunch of data on loans defaulting or not: {(x i,y i )} n i= x i 2 R d, y i 2 {, } You model the data as: P (Y = y x, w) = + exp( yw T
Introduction to Machine Learning
Introduction to Machine Learning Slides adapted from Eli Upfal Machine Learning: Jordan Boyd-Graber University of Maryland FEATURE ENGINEERING Machine Learning: Jordan Boyd-Graber UMD Introduction to Machine
EE 511 Online Learning Perceptron
Slides adapted from Ali Farhadi, Mari Ostendorf, Pedro Domingos, Carlos Guestrin, and Luke Zettelmoyer, Kevin Jamison EE 511 Online Learning Perceptron Instructor: Hanna Hajishirzi hannaneh@washington.edu
LECTURE 04: LINEAR REGRESSION PT. 2. September 20, 2017 SDS 293: Machine Learning
LECTURE 04: LINEAR REGRESSION PT. 2 September 20, 2017 SDS 293: Machine Learning Announcements Stats TA hours start Monday (sorry for the confusion) Looking for some refreshers on mathematical concepts?
Support Vector Machines
Support Vector Machines Jordan Boyd-Graber University of Colorado Boulder LECTURE 7 Slides adapted from Tom Mitchell, Eric Xing, and Lauren Hannah Jordan Boyd-Graber Boulder Support Vector Machines 1 of
Modeling Data with Linear Combinations of Basis Functions. Read Chapter 3 in the text by Bishop
Modeling Data with Linear Combinations of Basis Functions Read Chapter 3 in the text by Bishop A Type of Supervised Learning Problem We want to model data (x 1, t 1 ),..., (x N, t N ), where x i is a vector
Machine learning - HT Basis Expansion, Regularization, Validation
Machine learning - HT 016 4. Basis Expansion, Regularization, Validation Varun Kanade University of Oxford Feburary 03, 016 Outline Introduce basis function to go beyond linear regression Understanding
Lecture #11: Classification & Logistic Regression
Lecture #11: Classification & Logistic Regression CS 109A, STAT 121A, AC 209A: Data Science Weiwei Pan, Pavlos Protopapas, Kevin Rader Fall 2016 Harvard University 1 Announcements Midterm: will be graded
You can use numeric categorical predictors. A categorical predictor is one that takes values from a fixed set of possibilities.
CONTENTS Linear Regression Prepare Data To begin fitting a regression, put your data into a form that fitting functions expect. All regression techniques begin with input data in an array X and response
Week 3: Linear Regression
Week 3: Linear Regression Instructor: Sergey Levine Recap In the previous lecture we saw how linear regression can solve the following problem: given a dataset D = {(x, y ),..., (x N, y N )}, learn to
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 16
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 16 Slides adapted from Jordan Boyd-Graber, Justin Johnson, Andrej Karpathy, Chris Ketelsen, Fei-Fei Li, Mike Mozer, Michael Nielson
SVM. Introduction to Data Science Algorithms Jordan Boyd-Graber and Michael Paul SLIDES ADAPTED FROM HINRICH SCHÜTZE
SVM Introduction to Data Science Algorithms Jordan Boyd-Graber and Michael Paul SLIDES ADAPTED FROM HINRICH SCHÜTZE Introduction to Data Science Algorithms Boyd-Graber and Paul SVM 1 of 11 Find the maximum
Analytics 512: Homework # 2 Tim Ahn February 9, 2016
Analytics 512: Homework # 2 Tim Ahn February 9, 2016 Chapter 3 Problem 1 (# 3) Suppose we have a data set with five predictors, X 1 = GP A, X 2 = IQ, X 3 = Gender (1 for Female and 0 for Male), X 4 = Interaction
Chapter 1 Problem 28: Agenda. Quantities in Motion. Displacement Isn t Distance. Velocity. Speed 1/23/14
Agenda We need a note-taker! If you re interested, see me after class. Today: HW Quiz #1, 1D Motion Lecture for this week: Chapter 2 (finish reading Chapter 2 by Thursday) Homework #2: continue to check
MS&E 226: Small Data
MS&E 226: Small Data Lecture 3: More on linear regression (v3) Ramesh Johari ramesh.johari@stanford.edu 1 / 59 Recap: Linear regression 2 / 59 The linear regression model Given: n outcomes Y i, i = 1,...,
Self-Assessment Weeks 8: Multiple Regression with Qualitative Predictors; Multiple Comparisons
Self-Assessment Weeks 8: Multiple Regression with Qualitative Predictors; Multiple Comparisons 1. Suppose we wish to assess the impact of five treatments while blocking for study participant race (Black,
Linear Regression (continued)
Linear Regression (continued) Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 6, 2017 1 / 39 Outline 1 Administration 2 Review of last lecture 3 Linear regression
Data Mining - SVM. Dr. Jean-Michel RICHER Dr. Jean-Michel RICHER Data Mining - SVM 1 / 55
Data Mining - SVM Dr. Jean-Michel RICHER 2018 jean-michel.richer@univ-angers.fr Dr. Jean-Michel RICHER Data Mining - SVM 1 / 55 Outline 1. Introduction 2. Linear regression 3. Support Vector Machine 4.
INTRODUCTION TO DATA SCIENCE
INTRODUCTION TO DATA SCIENCE JOHN P DICKERSON Lecture #13 3/9/2017 CMSC320 Tuesdays & Thursdays 3:30pm 4:45pm ANNOUNCEMENTS Mini-Project #1 is due Saturday night (3/11): Seems like people are able to do
Association studies and regression
Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration
Machine Learning Linear Classification. Prof. Matteo Matteucci
Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)
LECTURE 10: LINEAR MODEL SELECTION PT. 1. October 16, 2017 SDS 293: Machine Learning
LECTURE 10: LINEAR MODEL SELECTION PT. 1 October 16, 2017 SDS 293: Machine Learning Outline Model selection: alternatives to least-squares Subset selection - Best subset - Stepwise selection (forward and
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 9
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 9 Slides adapted from Jordan Boyd-Graber Machine Learning: Chenhao Tan Boulder 1 of 39 Recap Supervised learning Previously: KNN, naïve
CS489/698: Intro to ML
CS489/698: Intro to ML Lecture 04: Logistic Regression 1 Outline Announcements Baseline Learning Machine Learning Pyramid Regression or Classification (that s it!) History of Classification History of
MS&E 226. In-Class Midterm Examination Solutions Small Data October 20, 2015
MS&E 226 In-Class Midterm Examination Solutions Small Data October 20, 2015 PROBLEM 1. Alice uses ordinary least squares to fit a linear regression model on a dataset containing outcome data Y and covariates
Generative Clustering, Topic Modeling, & Bayesian Inference
Generative Clustering, Topic Modeling, & Bayesian Inference INFO-4604, Applied Machine Learning University of Colorado Boulder December 12-14, 2017 Prof. Michael Paul Unsupervised Naïve Bayes Last week
Linear regression. Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1,X 2,...X p is linear.
Linear regression Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1,X 2,...X p is linear. 1/48 Linear regression Linear regression is a simple approach
CPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining MLE and MAP Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due tonight. Assignment 5: Will be released
Classification: Logistic Regression from Data
Classification: Logistic Regression from Data Machine Learning: Alvin Grissom II University of Colorado Boulder Slides adapted from Emily Fox Machine Learning: Alvin Grissom II Boulder Classification:
Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul
Generative Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 29, 2018 Prof. Michael Paul Generative vs Discriminative The classification algorithms we have seen so far
A Decision Stump. Decision Trees, cont. Boosting. Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University. October 1 st, 2007
Decision Trees, cont. Boosting Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 1 st, 2007 1 A Decision Stump 2 1 The final tree 3 Basic Decision Tree Building Summarized
COS 402 Machine Learning and Artificial Intelligence Fall Lecture 3: Learning Theory
COS 402 Machine Learning and Artificial Intelligence Fall 2016 Lecture 3: Learning Theory Sanjeev Arora Elad Hazan Admin Exercise 1 due next Tue, in class Enrolment Recap We have seen: AI by introspection
Final Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
Generative v. Discriminative classifiers Intuition
Logistic Regression (Continued) Generative v. Discriminative Decision rees Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University January 31 st, 2007 2005-2007 Carlos Guestrin 1 Generative
CSC 411: Lecture 02: Linear Regression
CSC 411: Lecture 02: Linear Regression Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto (Most plots in this lecture are from Bishop s book) Zemel, Urtasun, Fidler (UofT) CSC 411: 02-Regression
Maximum Entropy Klassifikator; Klassifikation mit Scikit-Learn
Maximum Entropy Klassifikator; Klassifikation mit Scikit-Learn Benjamin Roth Centrum für Informations- und Sprachverarbeitung Ludwig-Maximilian-Universität München beroth@cis.uni-muenchen.de Benjamin Roth
Simple Linear Regression for the MPG Data
Simple Linear Regression for the MPG Data 2000 2500 3000 3500 15 20 25 30 35 40 45 Wgt MPG What do we do with the data? y i = MPG of i th car x i = Weight of i th car i =1,...,n n = Sample Size Exploratory
Data Analysis and Machine Learning Lecture 12: Multicollinearity, Bias-Variance Trade-off, Cross-validation and Shrinkage Methods.
TheThalesians Itiseasyforphilosopherstoberichiftheychoose Data Analysis and Machine Learning Lecture 12: Multicollinearity, Bias-Variance Trade-off, Cross-validation and Shrinkage Methods Ivan Zhdankin
5. Let W follow a normal distribution with mean of μ and the variance of 1. Then, the pdf of W is
Practice Final Exam Last Name:, First Name:. Please write LEGIBLY. Answer all questions on this exam in the space provided (you may use the back of any page if you need more space). Show all work but do
Case Study 1: Estimating Click Probabilities. Kakade Announcements: Project Proposals: due this Friday!
Case Study 1: Estimating Click Probabilities Intro Logistic Regression Gradient Descent + SGD Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade April 4, 017 1 Announcements:
LECTURE 15: SIMPLE LINEAR REGRESSION I
David Youngberg BSAD 20 Montgomery College LECTURE 5: SIMPLE LINEAR REGRESSION I I. From Correlation to Regression a. Recall last class when we discussed two basic types of correlation (positive and negative).
ESS2222. Lecture 4 Linear model
ESS2222 Lecture 4 Linear model Hosein Shahnas University of Toronto, Department of Earth Sciences, 1 Outline Logistic Regression Predicting Continuous Target Variables Support Vector Machine (Some Details)
CPSC 340: Machine Learning and Data Mining. MLE and MAP Fall 2017
CPSC 340: Machine Learning and Data Mining MLE and MAP Fall 2017 Assignment 3: Admin 1 late day to hand in tonight, 2 late days for Wednesday. Assignment 4: Due Friday of next week. Last Time: Multi-Class
Recall that a measure of fit is the sum of squared residuals: where. The F-test statistic may be written as:
1 Joint hypotheses The null and alternative hypotheses can usually be interpreted as a restricted model ( ) and an model ( ). In our example: Note that if the model fits significantly better than the restricted
Introduction to Machine Learning
Introduction to Machine Learning Linear Regression Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB CSE 474/574 1
9/26/17. Ridge regression. What our model needs to do. Ridge Regression: L2 penalty. Ridge coefficients. Ridge coefficients
What our model needs to do regression Usually, we are not just trying to explain observed data We want to uncover meaningful trends And predict future observations Our questions then are Is β" a good estimate
Assignment 3. Introduction to Machine Learning Prof. B. Ravindran
Assignment 3 Introduction to Machine Learning Prof. B. Ravindran 1. In building a linear regression model for a particular data set, you observe the coefficient of one of the features having a relatively
Classification Logistic Regression
Announcements: Classification Logistic Regression Machine Learning CSE546 Sham Kakade University of Washington HW due on Friday. Today: Review: sub-gradients,lasso Logistic Regression October 3, 26 Sham
Expectations and Entropy
Expectations and Entropy Data Science: Jordan Boyd-Graber University of Maryland SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH Data Science: Jordan Boyd-Graber UMD Expectations and Entropy 1 / 9 Expectation
BNAD 276 Lecture 10 Simple Linear Regression Model
1 / 27 BNAD 276 Lecture 10 Simple Linear Regression Model Phuong Ho May 30, 2017 2 / 27 Outline 1 Introduction 2 3 / 27 Outline 1 Introduction 2 4 / 27 Simple Linear Regression Model Managerial decisions
Random Variables and Events
Random Variables and Events Data Science: Jordan Boyd-Graber University of Maryland SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH Data Science: Jordan Boyd-Graber UMD Random Variables and Events 1 /
Overfitting, Bias / Variance Analysis
Overfitting, Bias / Variance Analysis Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 8, 207 / 40 Outline Administration 2 Review of last lecture 3 Basic
Classification. Jordan Boyd-Graber University of Maryland WEIGHTED MAJORITY. Slides adapted from Mohri. Jordan Boyd-Graber UMD Classification 1 / 13
Classification Jordan Boyd-Graber University of Maryland WEIGHTED MAJORITY Slides adapted from Mohri Jordan Boyd-Graber UMD Classification 1 / 13 Beyond Binary Classification Before we ve talked about
Linear Regression 9/23/17. Simple linear regression. Advertising sales: Variance changes based on # of TVs. Advertising sales: Normal error?
Simple linear regression Linear Regression Nicole Beckage y " = β % + β ' x " + ε so y* " = β+ % + β+ ' x " Method to assess and evaluate the correlation between two (continuous) variables. The slope of
Online Videos FERPA. Sign waiver or sit on the sides or in the back. Off camera question time before and after lecture. Questions?
Online Videos FERPA Sign waiver or sit on the sides or in the back Off camera question time before and after lecture Questions? Lecture 1, Slide 1 CS224d Deep NLP Lecture 4: Word Window Classification
Machine Learning Linear Regression. Prof. Matteo Matteucci
Machine Learning Linear Regression Prof. Matteo Matteucci Outline 2 o Simple Linear Regression Model Least Squares Fit Measures of Fit Inference in Regression o Multi Variate Regession Model Least Squares
Decision Trees. Machine Learning CSEP546 Carlos Guestrin University of Washington. February 3, 2014
Decision Trees Machine Learning CSEP546 Carlos Guestrin University of Washington February 3, 2014 17 Linear separability n A dataset is linearly separable iff there exists a separating hyperplane: Exists
... SPARROW. SPARse approximation Weighted regression. Pardis Noorzad. Department of Computer Engineering and IT Amirkabir University of Technology
..... SPARROW SPARse approximation Weighted regression Pardis Noorzad Department of Computer Engineering and IT Amirkabir University of Technology Université de Montréal March 12, 2012 SPARROW 1/47 .....
Midterm. You may use a calculator, but not any device that can access the Internet or store large amounts of data.
INST 737 April 1, 2013 Midterm Name: }{{} by writing my name I swear by the honor code Read all of the following information before starting the exam: For free response questions, show all work, clearly
CPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining Linear Classifiers: predictions Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due Friday of next
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 10
COS53: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 0 MELISSA CARROLL, LINJIE LUO. BIAS-VARIANCE TRADE-OFF (CONTINUED FROM LAST LECTURE) If V = (X n, Y n )} are observed data, the linear regression problem
Week 5: Logistic Regression & Neural Networks
Week 5: Logistic Regression & Neural Networks Instructor: Sergey Levine 1 Summary: Logistic Regression In the previous lecture, we covered logistic regression. To recap, logistic regression models and
Lecture 7: Kernels for Classification and Regression
Lecture 7: Kernels for Classification and Regression CS 194-10, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 15, 2011 Outline Outline A linear regression problem Linear auto-regressive
Shrinkage Methods: Ridge and Lasso
Shrinkage Methods: Ridge and Lasso Jonathan Hersh 1 Chapman University, Argyros School of Business hersh@chapman.edu February 27, 2019 J.Hersh (Chapman) Ridge & Lasso February 27, 2019 1 / 43 1 Intro and
A Short Introduction to the Lasso Methodology
A Short Introduction to the Lasso Methodology Michael Gutmann sites.google.com/site/michaelgutmann University of Helsinki Aalto University Helsinki Institute for Information Technology March 9, 2016 Michael
Lecture 3. Linear Regression
Lecture 3. Linear Regression COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Andrey Kan Copyright: University of Melbourne Weeks 2 to 8 inclusive Lecturer: Andrey Kan MS: Moscow, PhD:
1 Effects of Regularization For this problem, you are required to implement everything by yourself and submit code.
This set is due pm, January 9 th, via Moodle. You are free to collaborate on all of the problems, subject to the collaboration policy stated in the syllabus. Please include any code ith your submission.
Polynomial Regression and Regularization
Polynomial Regression and Regularization Administrivia o If you still haven t enrolled in Moodle yet Enrollment key on Piazza If joined course recently, email me to get added to Piazza o Homework 1 posted
Support Vector Machine I
Support Vector Machine I Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Please use piazza. No emails. HW 0 grades are back. Re-grade request for one week. HW 1 due soon. HW
Multiple Regression. Midterm results: AVG = 26.5 (88%) A = 27+ B = C =
Economics 130 Lecture 6 Midterm Review Next Steps for the Class Multiple Regression Review & Issues Model Specification Issues Launching the Projects!!!!! Midterm results: AVG = 26.5 (88%) A = 27+ B =
EC4051 Project and Introductory Econometrics
EC4051 Project and Introductory Econometrics Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Intro to Econometrics 1 / 23 Project Guidelines Each student is required to undertake
Lecture 18: Kernels Risk and Loss Support Vector Regression. Aykut Erdem December 2016 Hacettepe University
Lecture 18: Kernels Risk and Loss Support Vector Regression Aykut Erdem December 2016 Hacettepe University Administrative We will have a make-up lecture on next Saturday December 24, 2016 Presentations
Lecture 3: Statistical Decision Theory (Part II)
Lecture 3: Statistical Decision Theory (Part II) Hao Helen Zhang Hao Helen Zhang Lecture 3: Statistical Decision Theory (Part II) 1 / 27 Outline of This Note Part I: Statistics Decision Theory (Classical
Lecture 17: Small-Sample Inferences for Normal Populations. Confidence intervals for µ when σ is unknown
Lecture 17: Small-Sample Inferences for Normal Populations Confidence intervals for µ when σ is unknown If the population distribution is normal, then X µ σ/ n has a standard normal distribution. If σ
Machine Learning and Computational Statistics, Spring 2017 Homework 2: Lasso Regression
Machine Learning and Computational Statistics, Spring 2017 Homework 2: Lasso Regression Due: Monday, February 13, 2017, at 10pm (Submit via Gradescope) Instructions: Your answers to the questions below,