Learning From Data Lecture 14 Three Learning Principles
|
|
|
- Jayson Blankenship
- 6 years ago
- Views:
Transcription
1 Learning From Data Lecture 14 Three Learning Principles Occam s Razor Sampling Bias Data Snooping M. Magdon-Ismail CSCI 4100/6100
2 recap: Validation and Cross Validation Validation Cross Validation D (N) D 1 D 2 D N D train (N K) D g 1 g 2 g N g D val (K) (x 1,y 1 ) (x 2,y 2 ) (x N,y N ) e 1 } e 2 {{ e N } take average g E val (g ) g E cv Model Selection H 1 H 2 H 3 H M g 1 g 2 g 3 g M c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 2 /36 Occam, bias, snooping
3 We Will Discuss... Occam s Razor: pick a model carefully Sampling Bias: generate the data carefuly Data Snooping: handle the data carefully c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 3 /36 Occam
4 Occam s Razor use a razor to trim down an explanation of the data to make it as simple as possible but no simpler. attributed to William of Occam (14th Century) and often mistakenly to Einstein c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 4 /36 Simpler is Better
5 Simpler is Better The simplest model that fits the data is also the most plausible....or, beware of using complex models to fit data c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 5 /36 What is Simpler?
6 What is Simpler? simple hypothesis h Ω(h) simple hypothesis set H Ω(H) low order polynomial H with small d vc hypothesis with small weights small number of hypotheses easily described hypothesis low entropy set.. The equivalence: A hypothesis set with simple hypotheses must be small We had a glimpse of this: soft order constraint (smaller H) λ minimize E aug (favors simpler h). c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 6 /36 Why is Simpler Better
7 Why is Simpler Better Mathematically: simple curtails ability to fit noise, VC-dimension is small, and blah and blah... simpler is better because you will be more surprised when you fit the data. If something unlikely happens, it is very significant when it happens.. Detective Gregory: Is there any other point to which you would wish to draw my attention? Sherlock Holmes: To the curious incident of the dog in the night-time. Detective Gregory: The dog did nothing in the night-time. Sherlock Holmes: That was the curious incident.. Silver Blaze, Sir Arthur Conan Doyle c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 7 /36 Scientific Experiment
8 A Scientific Experiment Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence for the hypothesis ρ is linear in T? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 8 /36 Scientist 2 vs. 3
9 Scientist 2 Versus Scientist 3 Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 9 /36 Scientist 1 vs. 3
10 Scientist 1 versus Scientist 3 Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 10 /36 Non-Falsifiability
11 Axiom of Non-Falsifiability Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 11 /36 Falsification and m H (N)
12 Falsification and m H (N) If H shatters x 1,,x N,...don t be surprised if you fit the data. If H doesn t shatter x 1,,x N, and the target values are uniformly distributed, P[falsification] 1 m H(N) 2 N. A good fit is surprising with simpler H (small m H (N)) and hence more significant. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 12 /36 Beyond Occam
13 Learning Goes Beyond Occam s Razor We may opt for a simpler fit than possible, namely an imperfect fit of the data using a simple model over a perfect fit using a more complex one. The reason is that the price we pay for a perfect fit in terms of the penalty for model complexity may be too much in comparison to the benefit of the better fit. Learning From Data, Abu-Mostafa, Magdon-Ismail, Lin c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 13 /36 Puzzle 1: football oracle
14 A Puzzle The Football Oracle Saturday, Oct 13, 2012 Home team will win the Monday Night Footbal Game. This happens for 5 weeks in a row. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 14 /36 Got it right
15 A Puzzle The Football Oracle Saturday, Oct 13, 2012 Home team will win the Monday Night Footbal Game. This happens for 5 weeks in a row. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 15 /36 5 weeks in a row
16 A Puzzle The Football Oracle Saturday, Oct 13, 2012 Home team will the Monday Night Footbal Game. This happens for 5 weeks in a row. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 16 /36 Pay for more predictions
17 A Puzzle The Football Oracle...on the 6th week Saturday, Nov 17, 2012 Call for winner; $50 charge applied E in = 0! Meaningless without knowing the complexity of the process leading to that! c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 17 /36 Oracle is a single predictor
18 What did the Oracle Really Do? you day day day day day Single hypothesis that worked? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 18 /36 Oracle is every hypothesis
19 What did the Oracle Really Do? you day day day day day Every possible hypothesis one of which worked? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 19 /36 Pay for more predictions
20 A Puzzle The Football Oracle...on the 6th week Saturday, Nov 17, 2012 Call for winner; $50 charge applied E in = 0! Meaningless without the complexity of the process leading to that! c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 20 /36 Sampling bias
21 We Will Discuss... Occam s Razor: pick a model carefully Sampling Bias: generate the data carefuly Data Snooping: handle the data carefully c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 21 /36 Dewey Defeats Truman
22 November 3rd 1948, Dewey Defeats Truman Tribune wanted to show off its latest technology could go earlier to press. Telephone poll on how people voted statisticians had done their thing and were confident. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 22 /36 Truman defeats Dewey
23 Imagine Their Surprise When... c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 23 /36 Sampling Bias in Learning
24 Sampling Bias in Learning If the data is sampled in a biased way, learning will produce a similarly biased outcome....or, make sure the training and test distributions are the same. You cannot draw a sample from one bin and make claims about another bin c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 24 /36 Extrapolation
25 Extrapolation 100 # Copies Sold Amazon Ranking c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 25 /36 Extrapolation is Hard
26 Extrapolation is Hard 100 # Copies Sold Amazon Ranking c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 26 /36 Dealing with Mismatch
27 Dealing with the Training-Test Mismatch 100 Think more carefully about what f should look like Need some additional help outside the data, by choosing a good H In our ranking example, account for the fat tail hyperbola # Copies Sold Amazon Ranking (hyperbola fit) 10 3 Account for the training-test mismatch during learning There are methods that reweight/resample data can help If test data have zero representation in training, you are in trouble Think carefully about f Amazon Ranking Probability (test versus training distributions) c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 27 /36 Puzzle credit analysis
28 Puzzle - Credit Analysis Determine credit given salary, debt, years in residence,... Banks have lots of data customer information: salary, debt, etc. whether or not they defaulted on their credit. age 32 years gender male salary 40,000 debt 26,000 years in job 1 year years at home 3 years Approve for credit? where is the sampling bias? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 28 /36 Bias in approvals
29 Puzzle - Credit Analysis Determine credit given salary, debt, years in residence,... Banks have lots of data customer information: salary, debt, etc. whether or not who? defaulted on their credit. age 32 years gender male salary 40,000 debt 26,000 years in job 1 year years at home 3 years Approve for credit? only data on approved customers c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 29 /36 Data Snooping
30 We Will Discuss... Occam s Razor: pick a model carefully Sampling Bias: generate the data carefuly Data Snooping: handle the data carefully c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 30 /36 Data snooping definition
31 Data Snooping If a data set has affected any step in the learning process, it cannot be fully trusted in assessing the outcome.... or, estimate performance with a completely uncontaminated test set...and, choose H before looking at the data c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 31 /36 Puzzle buy and hold on S&P
32 Puzzle: The Buy and Hold Strategy on S&P 500 Stocks 16.2% return Sampling Bias: didn t buy and hold a random sample of stocks. Snooping: Choose which stocks to hold by snooping into the test set (the future). c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 32 /36 Actual S&P
33 Puzzle: The Buy and Hold Strategy on S&P 500 Stocks 16.2% return snooping/sampling bias actual S&P 8.3% return Sampling Bias: didn t buy and hold a random sample of stocks. Snooping: Choose which stocks to hold by snooping into the test set (the future). c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 33 /36 Data snooping is subtle
34 Data Snooping is a Subtle Happy Hell The data looks linear, so I will use a linear model, and it worked. If the data were different and didn t look linear, would you do something different? Try linear, it fails; try circles it works. If you torture the data enough, it will confess. Try linear, it works; so I don t need to try circles. Would you have tried circles if the data were different? Read papers, see what others did on the data. Modify and improve on that. If the data were different, would that modify what others did and hence what you did? the data snooping can happen all at once or sequentially by different people Input normalization: normalize the data, now set aside the test set. Since the test set was involved in the normalization, wouldn t your g change if the test set changed? c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 34 /36 Account for data snooping
35 Account for Data Snooping Ask yourself: If the data were different, could/would I have done something different? if yes, then there is data snooping.? h H? Data? g D your choices g You must account for every choice influenced by D. We know how to account for the choice of g from H. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 35 /36 Account for all data snooping
36 Account for Data Snooping Ask yourself: If the data were different, could/would I have done something different? if yes, then there is data snooping.? h H? Data? g D your choices g You must account for every choice influenced by D. We know how to account for the choice of g from H. c AM L Creator: Malik Magdon-Ismail Three Learning Principles: 36 /36
Learning From Data Lecture 2 The Perceptron
Learning From Data Lecture 2 The Perceptron The Learning Setup A Simple Learning Algorithm: PLA Other Views of Learning Is Learning Feasible: A Puzzle M. Magdon-Ismail CSCI 4100/6100 recap: The Plan 1.
Learning From Data Lecture 13 Validation and Model Selection
Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection Cross Validation M. Magdon-Ismail CSCI 4100/6100 recap: Regularization Regularization combats the effects
Learning From Data Lecture 10 Nonlinear Transforms
Learning From Data Lecture 0 Nonlinear Transforms The Z-space Polynomial transforms Be careful M. Magdon-Ismail CSCI 400/600 recap: The Linear Model linear in w: makes the algorithms work linear in x:
Learning From Data Lecture 8 Linear Classification and Regression
Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis E out E
Learning From Data Lecture 3 Is Learning Feasible?
Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue Learning vs. Verification Selection Bias - A Cartoon M. Magdon-Ismail CSCI 4100/6100 recap: The Perceptron
Learning From Data Lecture 12 Regularization
Learning From Data Lecture 12 Regularization Constraining the Model Weight Decay Augmented Error M. Magdon-Ismail CSCI 4100/6100 recap: Overfitting Fitting the data more than is warranted Data Target Fit
Learning From Data Lecture 5 Training Versus Testing
Learning From Data Lecture 5 Training Versus Testing The Two Questions of Learning Theory of Generalization (E in E out ) An Effective Number of Hypotheses A Combinatorial Puzzle M. Magdon-Ismail CSCI
Learning From Data Lecture 15 Reflecting on Our Path - Epilogue to Part I
Learning From Data Lecture 15 Reflecting on Our Path - Epilogue to Part I What We Did The Machine Learning Zoo Moving Forward M Magdon-Ismail CSCI 4100/6100 recap: Three Learning Principles Scientist 2
Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16
600.463 Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16 25.1 Introduction Today we re going to talk about machine learning, but from an
Statistics for IT Managers
Statistics for IT Managers 95-796, Fall 2012 Module 2: Hypothesis Testing and Statistical Inference (5 lectures) Reading: Statistics for Business and Economics, Ch. 5-7 Confidence intervals Given the sample
Learning From Data Lecture 7 Approximation Versus Generalization
Learning From Data Lecture 7 Approimation Versus Generalization The VC Dimension Approimation Versus Generalization Bias and Variance The Learning Curve M. Magdon-Ismail CSCI 4100/6100 recap: The Vapnik-Chervonenkis
Hypothesis testing I. - In particular, we are talking about statistical hypotheses. [get everyone s finger length!] n =
![Hypothesis testing I. - In particular, we are talking about statistical hypotheses. [get everyone s finger length!] n =](/thumbs/86/94764601.jpg "Hypothesis testing I. - In particular, we are talking about statistical hypotheses. [get everyone s finger length!] n =")
Hypothesis testing I I. What is hypothesis testing? [Note we re temporarily bouncing around in the book a lot! Things will settle down again in a week or so] - Exactly what it says. We develop a hypothesis,
CS 2336 Discrete Mathematics
CS 2336 Discrete Mathematics Lecture 3 Logic: Rules of Inference 1 Outline Mathematical Argument Rules of Inference 2 Argument In mathematics, an argument is a sequence of propositions (called premises)
Do students sleep the recommended 8 hours a night on average?
BIEB100. Professor Rifkin. Notes on Section 2.2, lecture of 27 January 2014. Do students sleep the recommended 8 hours a night on average? We first set up our null and alternative hypotheses: H0: μ= 8
Last few slides from last time
Last few slides from last time Example 3: What is the probability that p will fall in a certain range, given p? Flip a coin 50 times. If the coin is fair (p=0.5), what is the probability of getting an
Today s lecture. Scientific method Hypotheses, models, theories... Occam' razor Examples Diet coke and menthos
Announcements The first homework is available on ICON. It is due one minute before midnight on Tuesday, August 30. Labs start this week. All lab sections will be in room 665 VAN. Kaaret has office hours
Why maximize entropy?
Why maximize entropy? TSILB Version 1.0, 19 May 1982 It is commonly accepted that if one is asked to select a distribution satisfying a bunch of constraints, and if these constraints do not determine a
The PROMYS Math Circle Problem of the Week #3 February 3, 2017
The PROMYS Math Circle Problem of the Week #3 February 3, 2017 You can use rods of positive integer lengths to build trains that all have a common length. For instance, a train of length 12 is a row of
Hypothesis testing. Data to decisions
Hypothesis testing Data to decisions The idea Null hypothesis: H 0 : the DGP/population has property P Under the null, a sample statistic has a known distribution If, under that that distribution, the
CS 124 Math Review Section January 29, 2018
CS 124 Math Review Section CS 124 is more math intensive than most of the introductory courses in the department. You re going to need to be able to do two things: 1. Perform some clever calculations to
Guide to Proofs on Sets
CS103 Winter 2019 Guide to Proofs on Sets Cynthia Lee Keith Schwarz I would argue that if you have a single guiding principle for how to mathematically reason about sets, it would be this one: All sets
An analogy from Calculus: limits
COMP 250 Fall 2018 35 - big O Nov. 30, 2018 We have seen several algorithms in the course, and we have loosely characterized their runtimes in terms of the size n of the input. We say that the algorithm
Solutions to In-Class Problems Week 14, Mon.
Massachusetts Institute of Technology 6.042J/18.062J, Spring 10: Mathematics for Computer Science May 10 Prof. Albert R. Meyer revised May 10, 2010, 677 minutes Solutions to In-Class Problems Week 14,
Uncertainty. Michael Peters December 27, 2013
Uncertainty Michael Peters December 27, 20 Lotteries In many problems in economics, people are forced to make decisions without knowing exactly what the consequences will be. For example, when you buy
S.R.S Varadhan by Professor Tom Louis Lindstrøm
S.R.S Varadhan by Professor Tom Louis Lindstrøm Srinivasa S. R. Varadhan was born in Madras (Chennai), India in 1940. He got his B. Sc. from Presidency College in 1959 and his Ph.D. from the Indian Statistical
Simple Regression Model. January 24, 2011
Simple Regression Model January 24, 2011 Outline Descriptive Analysis Causal Estimation Forecasting Regression Model We are actually going to derive the linear regression model in 3 very different ways
SELECTIVE APPROACH TO SOLVING SYLLOGISM
SELECTIVE APPROACH TO SOLVING SYLLOGISM While solving Syllogism questions, we encounter many weird Statements: Some Cows are Ugly, Some Lions are Vegetarian, Some Cats are not Dogs, Some Girls are Boys,
Announcements. Lecture 5: Probability. Dangling threads from last week: Mean vs. median. Dangling threads from last week: Sampling bias
Recap Announcements Lecture 5: Statistics 101 Mine Çetinkaya-Rundel September 13, 2011 HW1 due TA hours Thursday - Sunday 4pm - 9pm at Old Chem 211A If you added the class last week please make sure to
2.57 when the critical value is 1.96, what decision should be made?
Math 1342 Ch. 9-10 Review Name SHORT ANSWER. Write the word or phrase that best completes each statement or answers the question. 9.1 1) If the test value for the difference between the means of two large
The dark energ ASTR509 - y 2 puzzl e 2. Probability ASTR509 Jasper Wal Fal term
The ASTR509 dark energy - 2 puzzle 2. Probability ASTR509 Jasper Wall Fall term 2013 1 The Review dark of energy lecture puzzle 1 Science is decision - we must work out how to decide Decision is by comparing
Machine Learning Foundations
Machine Learning Foundations ( 機器學習基石 ) Lecture 8: Noise and Error Hsuan-Tien Lin ( 林軒田 ) htlin@csie.ntu.edu.tw Department of Computer Science & Information Engineering National Taiwan University ( 國立台灣大學資訊工程系
Hypothesis Testing and Confidence Intervals (Part 2): Cohen s d, Logic of Testing, and Confidence Intervals
Hypothesis Testing and Confidence Intervals (Part 2): Cohen s d, Logic of Testing, and Confidence Intervals Lecture 9 Justin Kern April 9, 2018 Measuring Effect Size: Cohen s d Simply finding whether a
Introduction and Models
CSE522, Winter 2011, Learning Theory Lecture 1 and 2-01/04/2011, 01/06/2011 Lecturer: Ofer Dekel Introduction and Models Scribe: Jessica Chang Machine learning algorithms have emerged as the dominant and
Probability Exercises. Problem 1.
Probability Exercises. Ma 162 Spring 2010 Ma 162 Spring 2010 April 21, 2010 Problem 1. ˆ Conditional Probability: It is known that a student who does his online homework on a regular basis has a chance
MATH 19-02: HW 5 TUFTS UNIVERSITY DEPARTMENT OF MATHEMATICS SPRING 2018
MATH 19-02: HW 5 TUFTS UNIVERSITY DEPARTMENT OF MATHEMATICS SPRING 2018 As we ve discussed, a move favorable to X is one in which some voters change their preferences so that X is raised, while the relative
Lesson 8: Graphs of Simple Non Linear Functions
Student Outcomes Students examine the average rate of change for non linear functions and learn that, unlike linear functions, non linear functions do not have a constant rate of change. Students determine
Machine Learning Foundations
Machine Learning Foundations ( 機器學習基石 ) Lecture 4: Feasibility of Learning Hsuan-Tien Lin ( 林軒田 ) htlin@csie.ntu.edu.tw Department of Computer Science & Information Engineering National Taiwan University
Please bring the task to your first physics lesson and hand it to the teacher.
Pre-enrolment task for 2014 entry Physics Why do I need to complete a pre-enrolment task? This bridging pack serves a number of purposes. It gives you practice in some of the important skills you will
Correlation. We don't consider one variable independent and the other dependent. Does x go up as y goes up? Does x go down as y goes up?
Comment: notes are adapted from BIOL 214/312. I. Correlation. Correlation A) Correlation is used when we want to examine the relationship of two continuous variables. We are not interested in prediction.
Regression, part II. I. What does it all mean? A) Notice that so far all we ve done is math.
Regression, part II I. What does it all mean? A) Notice that so far all we ve done is math. 1) One can calculate the Least Squares Regression Line for anything, regardless of any assumptions. 2) But, if
CIS 2033 Lecture 5, Fall
CIS 2033 Lecture 5, Fall 2016 1 Instructor: David Dobor September 13, 2016 1 Supplemental reading from Dekking s textbook: Chapter2, 3. We mentioned at the beginning of this class that calculus was a prerequisite
Chapter 26: Comparing Counts (Chi Square)
Chapter 6: Comparing Counts (Chi Square) We ve seen that you can turn a qualitative variable into a quantitative one (by counting the number of successes and failures), but that s a compromise it forces
Introduction to Design of Experiments
Introduction to Design of Experiments Jean-Marc Vincent and Arnaud Legrand Laboratory ID-IMAG MESCAL Project Universities of Grenoble {Jean-Marc.Vincent,Arnaud.Legrand}@imag.fr November 20, 2011 J.-M.
Critical Reading Answer Key 1. D 2. B 3. D 4. A 5. A 6. B 7. E 8. C 9. D 10. C 11. D
Critical Reading Answer Key 1. D 3. D 4. A 5. A 6. B 7. E 8. C 9. D 10. C 11. D Sentence Completion Answer Key 1. D Key- Find the key word. You must train yourself to pick out key words and phrases that
Name Solutions Linear Algebra; Test 3. Throughout the test simplify all answers except where stated otherwise.
Name Solutions Linear Algebra; Test 3 Throughout the test simplify all answers except where stated otherwise. 1) Find the following: (10 points) ( ) Or note that so the rows are linearly independent, so
Section 2: Estimation, Confidence Intervals and Testing Hypothesis
Section 2: Estimation, Confidence Intervals and Testing Hypothesis Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/
Solving Equations by Adding and Subtracting
SECTION 2.1 Solving Equations by Adding and Subtracting 2.1 OBJECTIVES 1. Determine whether a given number is a solution for an equation 2. Use the addition property to solve equations 3. Determine whether
Bayesian Learning. Artificial Intelligence Programming. 15-0: Learning vs. Deduction
15-0: Learning vs. Deduction Artificial Intelligence Programming Bayesian Learning Chris Brooks Department of Computer Science University of San Francisco So far, we ve seen two types of reasoning: Deductive
Computational Learning Theory: Shattering and VC Dimensions. Machine Learning. Spring The slides are mainly from Vivek Srikumar
Computational Learning Theory: Shattering and VC Dimensions Machine Learning Spring 2018 The slides are mainly from Vivek Srikumar 1 This lecture: Computational Learning Theory The Theory of Generalization
Regression Analysis IV... More MLR and Model Building
Regression Analysis IV... More MLR and Model Building This session finishes up presenting the formal methods of inference based on the MLR model and then begins discussion of "model building" (use of regression
CSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
Student Book SERIES. Time and Money. Name
Student Book Name ontents Series Topic Time (pp. 24) l months of the year l calendars and dates l seasons l ordering events l duration and language of time l hours, minutes and seconds l o clock l half
1 Recap: Interactive Proofs
Theoretical Foundations of Cryptography Lecture 16 Georgia Tech, Spring 2010 Zero-Knowledge Proofs 1 Recap: Interactive Proofs Instructor: Chris Peikert Scribe: Alessio Guerrieri Definition 1.1. An interactive
1 Impact Evaluation: Randomized Controlled Trial (RCT)
Introductory Applied Econometrics EEP/IAS 118 Fall 2013 Daley Kutzman Section #12 11-20-13 Warm-Up Consider the two panel data regressions below, where i indexes individuals and t indexes time in months:
ORIE 4741: Learning with Big Messy Data. Generalization
ORIE 4741: Learning with Big Messy Data Generalization Professor Udell Operations Research and Information Engineering Cornell September 23, 2017 1 / 21 Announcements midterm 10/5 makeup exam 10/2, by
Decision Tree Learning. Dr. Xiaowei Huang
Decision Tree Learning Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ After two weeks, Remainders Should work harder to enjoy the learning procedure slides Read slides before coming, think them
An Algorithms-based Intro to Machine Learning
CMU 15451 lecture 12/08/11 An Algorithmsbased Intro to Machine Learning Plan for today Machine Learning intro: models and basic issues An interesting algorithm for combining expert advice Avrim Blum [Based
CONSTRUCTION OF THE REAL NUMBERS.
CONSTRUCTION OF THE REAL NUMBERS. IAN KIMING 1. Motivation. It will not come as a big surprise to anyone when I say that we need the real numbers in mathematics. More to the point, we need to be able to
Indicative conditionals
Indicative conditionals PHIL 43916 November 14, 2012 1. Three types of conditionals... 1 2. Material conditionals... 1 3. Indicatives and possible worlds... 4 4. Conditionals and adverbs of quantification...
Discrete Distributions
Discrete Distributions STA 281 Fall 2011 1 Introduction Previously we defined a random variable to be an experiment with numerical outcomes. Often different random variables are related in that they have
Lesson 7: Classification of Solutions
Student Outcomes Students know the conditions for which a linear equation will have a unique solution, no solution, or infinitely many solutions. Lesson Notes Part of the discussion on the second page
5.2 Tests of Significance
5.2 Tests of Significance Example 5.7. Diet colas use artificial sweeteners to avoid sugar. Colas with artificial sweeteners gradually lose their sweetness over time. Manufacturers therefore test new colas
Last week: Sample, population and sampling distributions finished with estimation & confidence intervals
Past weeks: Measures of central tendency (mean, mode, median) Measures of dispersion (standard deviation, variance, range, etc). Working with the normal curve Last week: Sample, population and sampling
( ) P A B : Probability of A given B. Probability that A happens
A B A or B One or the other or both occurs At least one of A or B occurs Probability Review A B A and B Both A and B occur ( ) P A B : Probability of A given B. Probability that A happens given that B
DIFFERENTIAL EQUATIONS
DIFFERENTIAL EQUATIONS Basic Concepts Paul Dawkins Table of Contents Preface... Basic Concepts... 1 Introduction... 1 Definitions... Direction Fields... 8 Final Thoughts...19 007 Paul Dawkins i http://tutorial.math.lamar.edu/terms.aspx
LECTURE 15: SIMPLE LINEAR REGRESSION I
David Youngberg BSAD 20 Montgomery College LECTURE 5: SIMPLE LINEAR REGRESSION I I. From Correlation to Regression a. Recall last class when we discussed two basic types of correlation (positive and negative).
Grades 7 & 8, Math Circles 24/25/26 October, Probability
Faculty of Mathematics Waterloo, Ontario NL 3G1 Centre for Education in Mathematics and Computing Grades 7 & 8, Math Circles 4/5/6 October, 017 Probability Introduction Probability is a measure of how
Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 10
EECS 70 Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 10 Introduction to Basic Discrete Probability In the last note we considered the probabilistic experiment where we flipped
Achilles: Now I know how powerful computers are going to become!
A Sigmoid Dialogue By Anders Sandberg Achilles: Now I know how powerful computers are going to become! Tortoise: How? Achilles: I did curve fitting to Moore s law. I know you are going to object that technological
Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten
download instant at
Answers to Odd-Numbered Exercises Chapter One: An Overview of Regression Analysis 1-3. (a) Positive, (b) negative, (c) positive, (d) negative, (e) ambiguous, (f) negative. 1-5. (a) The coefficients in
*Karle Laska s Sections: There is no class tomorrow and Friday! Have a good weekend! Scores will be posted in Compass early Friday morning
STATISTICS 100 EXAM 3 Spring 2016 PRINT NAME (Last name) (First name) *NETID CIRCLE SECTION: Laska MWF L1 Laska Tues/Thurs L2 Robin Tu Write answers in appropriate blanks. When no blanks are provided CIRCLE
Chapter 18. Sampling Distribution Models. Copyright 2010, 2007, 2004 Pearson Education, Inc.
Chapter 18 Sampling Distribution Models Copyright 2010, 2007, 2004 Pearson Education, Inc. Normal Model When we talk about one data value and the Normal model we used the notation: N(μ, σ) Copyright 2010,
Econ 325: Introduction to Empirical Economics
Econ 325: Introduction to Empirical Economics Chapter 9 Hypothesis Testing: Single Population Ch. 9-1 9.1 What is a Hypothesis? A hypothesis is a claim (assumption) about a population parameter: population
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 14, 2015 Today: The Big Picture Overfitting Review: probability Readings: Decision trees, overfiting
Correlation and regression
NST 1B Experimental Psychology Statistics practical 1 Correlation and regression Rudolf Cardinal & Mike Aitken 11 / 12 November 2003 Department of Experimental Psychology University of Cambridge Handouts:
Final Exam Comments. UVa - cs302: Theory of Computation Spring < Total
UVa - cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95-102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,
Discrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 7
EECS 70 Discrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 7 Polynomials Polynomials constitute a rich class of functions which are both easy to describe and widely applicable in topics
Elementary Statistics Triola, Elementary Statistics 11/e Unit 17 The Basics of Hypotheses Testing
(Section 8-2) Hypotheses testing is not all that different from confidence intervals, so let s do a quick review of the theory behind the latter. If it s our goal to estimate the mean of a population,
Last two weeks: Sample, population and sampling distributions finished with estimation & confidence intervals
Past weeks: Measures of central tendency (mean, mode, median) Measures of dispersion (standard deviation, variance, range, etc). Working with the normal curve Last two weeks: Sample, population and sampling
Is the Universe Random and Meaningless?
Is the Universe Random and Meaningless? By Claude LeBlanc, M.A., Magis Center, 2016 Opening Prayer Lord, we live in a universe so immense we can t imagine all it contains, and so intricate we can t fully
Weighted Majority and the Online Learning Approach
Statistical Techniques in Robotics (16-81, F12) Lecture#9 (Wednesday September 26) Weighted Majority and the Online Learning Approach Lecturer: Drew Bagnell Scribe:Narek Melik-Barkhudarov 1 Figure 1: Drew
Solving Quadratic & Higher Degree Equations
Chapter 9 Solving Quadratic & Higher Degree Equations Sec 1. Zero Product Property Back in the third grade students were taught when they multiplied a number by zero, the product would be zero. In algebra,
CONTINUOUS RANDOM VARIABLES
the Further Mathematics network www.fmnetwork.org.uk V 07 REVISION SHEET STATISTICS (AQA) CONTINUOUS RANDOM VARIABLES The main ideas are: Properties of Continuous Random Variables Mean, Median and Mode
Parametric Equations, Function Composition and the Chain Rule: A Worksheet
Parametric Equations, Function Composition and the Chain Rule: A Worksheet Prof.Rebecca Goldin Oct. 8, 003 1 Parametric Equations We have seen that the graph of a function f(x) of one variable consists
Example. How to Guess What to Prove
How to Guess What to Prove Example Sometimes formulating P (n) is straightforward; sometimes it s not. This is what to do: Compute the result in some specific cases Conjecture a generalization based on
Overview. Overview. Overview. Specific Examples. General Examples. Bivariate Regression & Correlation
Bivariate Regression & Correlation Overview The Scatter Diagram Two Examples: Education & Prestige Correlation Coefficient Bivariate Linear Regression Line SPSS Output Interpretation Covariance ou already
Algebra Exam. Solutions and Grading Guide
Algebra Exam Solutions and Grading Guide You should use this grading guide to carefully grade your own exam, trying to be as objective as possible about what score the TAs would give your responses. Full
Statistics and Data Analysis
Statistics and Data Analysis Professor William Greene Phone: 212.998.0876 Office: KMC 7-90 Home page: http://people.stern.nyu.edu/wgreene Email: wgreene@stern.nyu.edu Course web page: http://people.stern.nyu.edu/wgreene/statistics/outline.htm
Primary KS1 1 VotesForSchools2018
Primary KS1 1 Do aliens exist? This photo of Earth was taken by an astronaut on the moon! Would you like to stand on the moon? What is an alien? You probably drew some kind of big eyed, blue-skinned,
WINGS scientific whitepaper, version 0.8
WINGS scientific whitepaper, version 0.8 Serguei Popov March 11, 2017 Abstract We describe our system, a forecasting method enabling decentralized decision taking via predictions on particular proposals,
STATS216v Introduction to Statistical Learning Stanford University, Summer Midterm Exam (Solutions) Duration: 1 hours
Instructions: STATS216v Introduction to Statistical Learning Stanford University, Summer 2017 Remember the university honor code. Midterm Exam (Solutions) Duration: 1 hours Write your name and SUNet ID
Lecture 2: Linear regression
Lecture 2: Linear regression Roger Grosse 1 Introduction Let s ump right in and look at our first machine learning algorithm, linear regression. In regression, we are interested in predicting a scalar-valued
MATH 10 INTRODUCTORY STATISTICS
MATH 10 INTRODUCTORY STATISTICS Tommy Khoo Your friendly neighbourhood graduate student. It is Time for Homework! ( ω `) First homework + data will be posted on the website, under the homework tab. And
Sampling Distribution Models. Central Limit Theorem
Sampling Distribution Models Central Limit Theorem Thought Questions 1. 40% of large population disagree with new law. In parts a and b, think about role of sample size. a. If randomly sample 10 people,
Menu. Lecture 2: Orders of Growth. Predicting Running Time. Order Notation. Predicting Program Properties
CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Lecture 2: Orders of Growth Menu Predicting program properties Orders of Growth: O, Ω Course Survey
Probability Theory and Simulation Methods
Feb 28th, 2018 Lecture 10: Random variables Countdown to midterm (March 21st): 28 days Week 1 Chapter 1: Axioms of probability Week 2 Chapter 3: Conditional probability and independence Week 4 Chapters
CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes
CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes Roger Grosse Roger Grosse CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes 1 / 55 Adminis-Trivia Did everyone get my e-mail
Inference in Regression Model
Inference in Regression Model Christopher Taber Department of Economics University of Wisconsin-Madison March 25, 2009 Outline 1 Final Step of Classical Linear Regression Model 2 Confidence Intervals 3
Recap Social Choice Fun Game Voting Paradoxes Properties. Social Choice. Lecture 11. Social Choice Lecture 11, Slide 1
Social Choice Lecture 11 Social Choice Lecture 11, Slide 1 Lecture Overview 1 Recap 2 Social Choice 3 Fun Game 4 Voting Paradoxes 5 Properties Social Choice Lecture 11, Slide 2 Formal Definition Definition