COURSE INTRODUCTION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
|
|
|
- Egbert Hutchinson
- 5 years ago
- Views:
Transcription
1 COURSE INTRODUCTION
2 COMPUTATIONAL MODELING OF VISUAL PERCEPTION 2 The goal of this course is to provide a framework and computational tools for modeling visual inference, motivated by interesting examples from the recent literature. Models may be realized as algorithms to solve computer vision problems, or may constitute theories of visual processing in biological systems. The foundation of the course is a treatment of visual processing as a problem of statistical estimation and inference, grounded in the ecological statistics of the visual world.
3 Topics 3 Bayesian decision theory Principal components and factor analysis Graphical Models Markov Random Fields Conditional Random Fields Belief Propagation Clustering Mean Shift Expectation Maximization Spectral Methods (Graph Cuts) Sampling Gibbs Sampling Markov Chain Monte Carlo Classifiers Support Vector Machines Neural Networks
4 Course Format 4 Each week will consist of two 1.5 hour meetings: Meeting 1. A lecture by the instructor on a specific computational tool or approach Meeting 2. A discussion, led by a specified student, of a selected computational vision paper in which this approach is applied to a specific problem.
5 Evaluation 5 In addition to student presentations of short computational vision papers, two short MATLAB assignments will be collected and graded. The final project will involve application and possibly extension of a technique studied in the class to a problem chosen by the student. Class Participation 10% Paper Presentation 20% Assignment 1 20% Assignment 2 20% Final Project 30%
6 Main Texts 6 C.M. Bishop Pattern Recognition and Machine Learning. New York: Springer, S.J.D. Prince Computer Vision Models. Available in draft form at
7 7 ek Date Topic Required Readings Additional Readings Application Paper M Sept 13 W Sept 15 M Sept 20 W Sept 22 M Sept 27 W Sept 29 M Oct 4 W Oct 6 M Oct 11 W Oct 13 M Oct 18 W Oct 20 M Oct 25 W Oct 27 M Nov 1 W Nov 3 M Nov 8 W Nov 10 M Nov 15 W Nov M Nov 22 W Nov M Nov 29 W Dec 1 12 M Dec 6 W Dec 8 Probability & Bayesian Inference Probability Distributions & Parametric Modeling Bishop Ch (29 pages) Bishop Ch (skip 2.3.5) (43 pages) Pearl Ch , 2 Howson & Urbach 1991 Prince Ch 1-4 Duda Ch Probability Distributions & Parametric Modeling (cntd.) Non-Parametric Modeling Bishop Ch 2.5 (7 pages) Duda Ch Expectation Maximization Linear Subspace Models Reading Week Prince Ch 5 (11 pages) Prince Ch , 6.8 (24 pages) Prince Ch , 6.9 (12 pages) Bishop Ch 12 (40 pages) Bishop Ch 9 Duda Ch Linear Regression Bishop Ch 3 (36 pages) Prince Ch Linear Classifiers Bishop Ch (34 pages) Duda Non-Linear Regression & Classification Sparse Kernel Machines Graphical Models: Introduction Graphical Models: Inference Graphical Models: Applications Sampling Methods Bishop Ch 6 (29 pages) Prince Ch Bishop 7.1 (20 pages) Bishop Ch (34 pages) Bishop Ch 8.4 (25 pages) Prince Ch (56 pages) Bishop Ch 11 (32 pages) Comaniciu & Meer 2002 (Mean Shift)
8 Approximate Schedule 8 ek Date Topic Required Readings Additional Readings Application Paper M Sept 13 W Sept 15 M Sept 20 W Sept 22 M Sept 27 W Sept 29 M Oct 4 W Oct 6 M Oct 11 W Oct 13 M Oct 18 W Oct 20 M Oct 25 W Oct 27 M Nov 1 W Nov 3 M Nov 8 W Nov 10 M Nov 15 W Nov M Nov 22 W Nov M Nov 29 W Dec 1 12 M Dec 6 W Dec 8 Probability & Bayesian Inference Probability Distributions & Parametric Modeling Bishop Ch (29 pages) Bishop Ch (skip 2.3.5) (43 pages) Pearl Ch , 2 Howson & Urbach 1991 Prince Ch 1-4 Duda Ch Probability Distributions & Parametric Modeling (cntd.) Non-Parametric Modeling Bishop Ch 2.5 (7 pages) Duda Ch Expectation Maximization Linear Subspace Models Reading Week Prince Ch 5 (11 pages) Prince Ch , 6.8 (24 pages) Prince Ch , 6.9 (12 pages) Bishop Ch 12 (40 pages) Bishop Ch 9 Duda Ch Linear Regression Bishop Ch 3 (36 pages) Prince Ch Linear Classifiers Bishop Ch (34 pages) Duda Non-Linear Regression & Classification Sparse Kernel Machines Graphical Models: Introduction Graphical Models: Inference Graphical Models: Applications Sampling Methods Bishop Ch 6 (29 pages) Prince Ch Bishop 7.1 (20 pages) Bishop Ch (34 pages) Bishop Ch 8.4 (25 pages) Prince Ch (56 pages) Bishop Ch 11 (32 pages) Comaniciu & Meer 2002 (Mean Shift)
9 PROBABILITY AND BAYESIAN INFERENCE
10 Credits 10 Some of these slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Simon Prince, UCL
11 INTRODUCTION: VISION AS BAYESIAN INFERENCE
12 Helmholtz 12 Recognized ambiguity of images. Knowledge of scene properties and image formation used to resolve ambiguity and infer object properties. Vision as Unconscious Inference Can be formalized as Bayesian Decision Theory. Hermann von Helmholtz
13 Helmholtz Likelihood Principle 13 Claim 1: The world is uncertain (to the observer) Claim 2: Vision is ill-posed Claim 3: Observers have evolved (are built) to perform valuable tasks well Conclusion: Vision is probabilistic inference
14 Vision is Ill-Posed 14 Noise surface noise atmospheric effects photon noise neural noise Dimensionality 1D 2D 2D 3D Composition e.g. Bilinear problem of colour (lightness) constancy:
15 Vision is Ill-Posed 2D 3D (N:1 Mapping) 15 Different Objects Similar Images From Kersten et al., 2004
16 Vision is ill-posed (bilinearity of image) 16 1:N Mapping N:1 Mapping From Kersten et al., 2004
17
18 Julian Beever 18
19 Julian Beever 19
20 Julian Beever 20
21 Liu Bolin 21
22 Liu Bolin 22
23 Liu Bolin 23
24 Bayes Rule 24 Scene Property To Be Inferred Image Observation p(s I) p(i S)p(S) Posterior Likelihood Prior
25 Generative Models and Ideal Observers 25 Generative Model: From Kersten et al., 2004
26 TOPIC 1. PROBABILITY & BAYESIAN INFERENCE
27 Random Variables 27 A random variable is a variable whose value is uncertain. For example, the height of a randomly selected person in this class is a random variable I won t know its value until the person is selected. Note that we are not completely uncertain about most random variables. For example, we know that height will probably be in the 5-6 range. In addition, 5 6 is more likely than 5 0 or 6 0. The function that describes the probability of each possible value of the random variable is called a probability distribution.
28 Probability Distributions 28 For a discrete distribution, the probabilities over all possible values of the random variable must sum to 1.
29 Probability Distributions 29 For a discrete distribution, we can talk about the probability of a particular score occurring, e.g., p(province = Ontario) = We can also talk about the probability of any one of a subset of scores occurring, e.g., p(province = Ontario or Quebec) = In general, we refer to these occurrences as events.
30 Probability Distributions 30 For a continuous distribution, the probabilities over all possible values of the random variable must integrate to 1 (i.e., the area under the curve must be 1). Note that the height of a continuous distribution can exceed 1! S h a d e d a r e a = S h a d e d a r e a = S h a d e d a r e a =
31 Continuous Distributions 31 For continuous distributions, it does not make sense to talk about the probability of an exact score. e.g., what is the probability that your height is exactly inches? Normal Approximation to probability distribution for height of Canadian females (parameters from General Social Survey, 1991) Probability p Height (in)?
32 Continuous Distributions 32 It does make sense to talk about the probability of observing a score that falls within a certain range e.g., what is the probability that you are between 5 3 and 5 7? e.g., what is the probability that you are less than 5 10? Valid events Normal Approximation to probability distribution for height of Canadian females (parameters from General Social Survey, 1991) Probability p Height (in)
33 Probability Densities 33 Cumulative distribution (CDF) Probability density (PDF)
34 Transformed Densities 34
35 Joint Distributions 35 Marginal Probability Joint Probability Conditional Probability
36 Joint Distributions 36 Sum Rule Product Rule
37 Joint Distributions: The Rules of Probability 37 Sum Rule Product Rule
38 END OF LECTURE 1 SEPT 13, 2010
39 Application Papers 39 ek Date Topic Required Readings Additional Readings Application Papers M Sept 13 W Sept 15 M Sept 20 W Sept 22 M Sept 27 W Sept 29 M Oct 4 W Oct 6 Probability & Bayesian Inference Probability Distributions & Parametric Modeling Bishop Ch (29 pages) Bishop Ch (skip 2.3.5) (43 pages) Pearl Ch , 2 Howson & Urbach 1991 Prince Ch 1-4 Duda Ch Probability Distributions & Parametric Modeling (cntd.) Non-Parametric Modeling Bishop Ch 2.5 (7 pages) Duda Ch Expectation Maximization Subspace Models Prince Ch 5 (11 pages) Prince Ch , 6.8 (24 pages) Prince Ch , 6.9 (12 pages) Bishop Ch 12 (40 pages) Comaniciu & Meer 2002 (Mean Shift) Bishop Ch 9 Stauffer & Grimson 1998 Weber & Perona 2000 Duda Ch Tenenbaum et al 2000 Roweis & Saul 2000 M Oct 11 W Oct 13 M Oct 18 W Oct 20 M Oct 25 W Oct 27 M Nov 1 W Nov 3 M Nov 8 W Nov 10 M Nov 15 W Nov M Nov 22 W Nov M Nov 29 W Dec 1 12 M Dec 6 W Dec 8 Reading Week Linear Regression Bishop Ch 3 (36 pages) Prince Ch Moghaddam 2002 Cremers 2003 Linear Classifiers Bishop Ch (34 pages) Duda Belhumeur et al 1997 Martin et al 2004 Kernel Methods Bishop Ch 6 (29 pages) Prince Ch Toyama & Blake 2001 Grochow et al 2004 Sparse Kernel Machines Combining Models Graphical Models: Introduction Graphical Models: Inference Graphical Models: Applications Bishop 7.1 (20 pages) Bishop Ch 14 (20 pages) Agarwal & Triggs 2006 Zhang et al 2007 Bishop Ch (34 pages) Freeman et al 2000 Shi & Malik 2000 Bishop Ch 8.4 (25 pages) Boykov & Funka-Lea 2006 He et al 2004 Prince Ch (56 pages) Frey & Jojic 2005 Szeliski et al 2008 Sampling Methods Bishop Ch 11 (32 pages) Zhu 1999 Yuille & Kersten 2006
40 Marginalization 40 We can recover probability distribution of any variable in a joint distribution by integrating (or summing) over the other variables
41 Conditional Probability 41 Conditional probability of X given that Y=y* is relative propensity of variable X to take different outcomes given that Y is fixed to be equal to y* Written as Pr(X Y=y*)
42 Conditional Probability 42 Conditional probability can be extracted from joint probability Extract appropriate slice and normalize
43 Conditional Probability 43 More usually written in compact form Can be re-arranged to give
44 Independence 44 If two variables X and Y are independent then variable X tells us nothing about variable Y (and vice-versa)
45 Independence 45 When variables are independent, the joint factorizes into a product of the marginals:
46 Bayes Rule 46 From before: Combining: Re-arranging:
47 Bayes Rule Terminology 47 Likelihood propensity for observing a certain value of X given a certain value of Y Prior what we know about y before seeing x Posterior what we know about y after seeing x Evidence a constant to ensure that the left hand side is a valid distribution
48 Expectations 48 Condi3onal Expecta3on (discrete) Approximate Expecta3on (discrete and con3nuous)
49 Variances and Covariances 49
50 The Gaussian Distribution 50
51 Gaussian Mean and Variance 51
52 The Multivariate Gaussian 52
53 Gaussian Parameter Estimation 53 Likelihood func3on
54 Maximum (Log) Likelihood 54
55 Maximum likelihood estimates of normal parameters 55
56 APPLYING PROBABILITY THEORY TO INFERENCE
57 Polynomial Curve Fitting 57
58 Sum-of-Squares Error Function 58
59 1 st Order Polynomial 59
60 3 rd Order Polynomial 60
61 9 th Order Polynomial 61
62 Over-fitting 62 Root- Mean- Square (RMS) Error:
63 Overfitting and Sample Size 63 9 th Order Polynomial
64 Overfitting and Sample Size 64 9 th Order Polynomial
65 Regularization 65 Penalize large coefficient values
66 Regularization 66 9 th Order Polynomial
67 Regularization 67 9 th Order Polynomial
68 Regularization 68 9 th Order Polynomial
69 Probabilistic View of Curve Fitting 69
70 Maximum Likelihood 70 Determine by minimizing sum- of- squares error,.
71 MAP: A Step towards Bayes 71 Determine by minimizing regularized sum- of- squares error,.
72 Some Key Ideas 72 Change of variables and transformed densities Derivation of sum and product rules of probability Maximum likelihood and bias Least-squares as optimal probabilistic modeling
COMPUTATIONAL MODELING OF VISUAL PERCEPTION
COMPUTATIONAL MODELING OF VISUAL PERCEPTION CSE6390D - PSYC6225A 3.0 (F) James Elder MW 13:00-14:30 Bethune 322 Enrolment limit: none Purpose: The goal of this course is to provide a framework and computational
BAYESIAN DECISION THEORY
Last updated: September 17, 2012 BAYESIAN DECISION THEORY Problems 2 The following problems from the textbook are relevant: 2.1 2.9, 2.11, 2.17 For this week, please at least solve Problem 2.3. We will
Computer vision: models, learning and inference
Computer vision: models, learning and inference Chapter 2 Introduction to probability Please send errata to s.prince@cs.ucl.ac.uk Random variables A random variable x denotes a quantity that is uncertain
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
Recent Advances in Bayesian Inference Techniques
Recent Advances in Bayesian Inference Techniques Christopher M. Bishop Microsoft Research, Cambridge, U.K. research.microsoft.com/~cmbishop SIAM Conference on Data Mining, April 2004 Abstract Bayesian
Machine Learning Techniques for Computer Vision
Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM
Mathematical Formulation of Our Example
Mathematical Formulation of Our Example We define two binary random variables: open and, where is light on or light off. Our question is: What is? Computer Vision 1 Combining Evidence Suppose our robot
Machine Learning, Fall 2009: Midterm
10-601 Machine Learning, Fall 009: Midterm Monday, November nd hours 1. Personal info: Name: Andrew account: E-mail address:. You are permitted two pages of notes and a calculator. Please turn off all
Nonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
INTRODUCTION TO PATTERN RECOGNITION
INTRODUCTION TO PATTERN RECOGNITION INSTRUCTOR: WEI DING 1 Pattern Recognition Automatic discovery of regularities in data through the use of computer algorithms With the use of these regularities to take
Human-Oriented Robotics. Probability Refresher. Kai Arras Social Robotics Lab, University of Freiburg Winter term 2014/2015
Probability Refresher Kai Arras, University of Freiburg Winter term 2014/2015 Probability Refresher Introduction to Probability Random variables Joint distribution Marginalization Conditional probability
STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics
STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations
STA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
Overview of Statistical Tools. Statistical Inference. Bayesian Framework. Modeling. Very simple case. Things are usually more complicated
Fall 3 Computer Vision Overview of Statistical Tools Statistical Inference Haibin Ling Observation inference Decision Prior knowledge http://www.dabi.temple.edu/~hbling/teaching/3f_5543/index.html Bayesian
Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart
Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart 1 Motivation and Problem In Lecture 1 we briefly saw how histograms
BAYESIAN DECISION THEORY. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
BAYESIAN DECISION THEORY Credits 2 Some of these slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Simon Prince, University College London Sergios Theodoridis, University of Athens
Probabilistic Graphical Models for Image Analysis - Lecture 1
Probabilistic Graphical Models for Image Analysis - Lecture 1 Alexey Gronskiy, Stefan Bauer 21 September 2018 Max Planck ETH Center for Learning Systems Overview 1. Motivation - Why Graphical Models 2.
Lecture : Probabilistic Machine Learning
Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning
Machine Learning Lecture 1
Many slides adapted from B. Schiele Machine Learning Lecture 1 Introduction 18.04.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer Prof.
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University August 30, 2017 Today: Decision trees Overfitting The Big Picture Coming soon Probabilistic learning MLE,
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Regressione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida
Review. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
Computer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
STAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
GWAS IV: Bayesian linear (variance component) models
GWAS IV: Bayesian linear (variance component) models Dr. Oliver Stegle Christoh Lippert Prof. Dr. Karsten Borgwardt Max-Planck-Institutes Tübingen, Germany Tübingen Summer 2011 Oliver Stegle GWAS IV: Bayesian
Unsupervised Learning with Permuted Data
Unsupervised Learning with Permuted Data Sergey Kirshner skirshne@ics.uci.edu Sridevi Parise sparise@ics.uci.edu Padhraic Smyth smyth@ics.uci.edu School of Information and Computer Science, University
Probability Theory for Machine Learning. Chris Cremer September 2015
Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares
Computational Genomics
Computational Genomics http://www.cs.cmu.edu/~02710 Introduction to probability, statistics and algorithms (brief) intro to probability Basic notations Random variable - referring to an element / event
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
Brief Introduction of Machine Learning Techniques for Content Analysis
1 Brief Introduction of Machine Learning Techniques for Content Analysis Wei-Ta Chu 2008/11/20 Outline 2 Overview Gaussian Mixture Model (GMM) Hidden Markov Model (HMM) Support Vector Machine (SVM) Overview
Lecture Machine Learning (past summer semester) If at least one English-speaking student is present.
Advanced Machine Learning Lecture 1 Organization Lecturer Prof. Bastian Leibe (leibe@vision.rwth-aachen.de) Introduction 20.10.2015 Bastian Leibe Visual Computing Institute RWTH Aachen University http://www.vision.rwth-aachen.de/
Probabilistic Graphical Models
Probabilistic Graphical Models Introduction. Basic Probability and Bayes Volkan Cevher, Matthias Seeger Ecole Polytechnique Fédérale de Lausanne 26/9/2011 (EPFL) Graphical Models 26/9/2011 1 / 28 Outline
Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
Introduction to Graphical Models
Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) Yung-Kyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic
Least Squares Regression
CIS 50: Machine Learning Spring 08: Lecture 4 Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may not cover all the
MIXTURE MODELS AND EM
Last updated: November 6, 212 MIXTURE MODELS AND EM Credits 2 Some of these slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Simon Prince, University College London Sergios Theodoridis,
Probabilistic Graphical Networks: Definitions and Basic Results
This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical
9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering
Types of learning Modeling data Supervised: we know input and targets Goal is to learn a model that, given input data, accurately predicts target data Unsupervised: we know the input only and want to make
Computer Vision Group Prof. Daniel Cremers. 2. Regression (cont.)
Prof. Daniel Cremers 2. Regression (cont.) Regression with MLE (Rep.) Assume that y is affected by Gaussian noise : t = f(x, w)+ where Thus, we have p(t x, w, )=N (t; f(x, w), 2 ) 2 Maximum A-Posteriori
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
Computer Vision Group Prof. Daniel Cremers. 3. Regression
Prof. Daniel Cremers 3. Regression Categories of Learning (Rep.) Learnin g Unsupervise d Learning Clustering, density estimation Supervised Learning learning from a training data set, inference on the
Intelligent Systems I
Intelligent Systems I 00 INTRODUCTION Stefan Harmeling & Philipp Hennig 24. October 2013 Max Planck Institute for Intelligent Systems Dptmt. of Empirical Inference Which Card? Opening Experiment Which
Machine Learning
Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 13, 2011 Today: The Big Picture Overfitting Review: probability Readings: Decision trees, overfiting
Lecture 13: Data Modelling and Distributions. Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1
Lecture 13: Data Modelling and Distributions Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1 Why data distributions? It is a well established fact that many naturally occurring
Day 5: Generative models, structured classification
Day 5: Generative models, structured classification Introduction to Machine Learning Summer School June 18, 2018 - June 29, 2018, Chicago Instructor: Suriya Gunasekar, TTI Chicago 22 June 2018 Linear regression
Parameter Estimation. Industrial AI Lab.
Parameter Estimation Industrial AI Lab. Generative Model X Y w y = ω T x + ε ε~n(0, σ 2 ) σ 2 2 Maximum Likelihood Estimation (MLE) Estimate parameters θ ω, σ 2 given a generative model Given observed
Machine Learning Lecture 2
Machine Perceptual Learning and Sensory Summer Augmented 15 Computing Many slides adapted from B. Schiele Machine Learning Lecture 2 Probability Density Estimation 16.04.2015 Bastian Leibe RWTH Aachen
Least Squares Regression
E0 70 Machine Learning Lecture 4 Jan 7, 03) Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are a brief summary of the topics covered in the lecture. They are not a substitute
DEPARTMENT OF COMPUTER SCIENCE Autumn Semester MACHINE LEARNING AND ADAPTIVE INTELLIGENCE
Data Provided: None DEPARTMENT OF COMPUTER SCIENCE Autumn Semester 203 204 MACHINE LEARNING AND ADAPTIVE INTELLIGENCE 2 hours Answer THREE of the four questions. All questions carry equal weight. Figures
ECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
Machine Learning using Bayesian Approaches
Machine Learning using Bayesian Approaches Sargur N. Srihari University at Buffalo, State University of New York 1 Outline 1. Progress in ML and PR 2. Fully Bayesian Approach 1. Probability theory Bayes
PROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
PROBABILITY DISTRIBUTIONS Credits 2 These slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Parametric Distributions 3 Basic building blocks: Need to determine given Representation:
ECE-271B. Nuno Vasconcelos ECE Department, UCSD
ECE-271B Statistical ti ti Learning II Nuno Vasconcelos ECE Department, UCSD The course the course is a graduate level course in statistical learning in SLI we covered the foundations of Bayesian or generative
Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io
Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem
Variational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
Algorithms other than SGD. CS6787 Lecture 10 Fall 2017
Algorithms other than SGD CS6787 Lecture 10 Fall 2017 Machine learning is not just SGD Once a model is trained, we need to use it to classify new examples This inference task is not computed with SGD There
Autonomous Navigation for Flying Robots
Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 6.2: Kalman Filter Jürgen Sturm Technische Universität München Motivation Bayes filter is a useful tool for state
Conditional Random Field
Introduction Linear-Chain General Specific Implementations Conclusions Corso di Elaborazione del Linguaggio Naturale Pisa, May, 2011 Introduction Linear-Chain General Specific Implementations Conclusions
CS 6375 Machine Learning
CS 6375 Machine Learning Nicholas Ruozzi University of Texas at Dallas Slides adapted from David Sontag and Vibhav Gogate Course Info. Instructor: Nicholas Ruozzi Office: ECSS 3.409 Office hours: Tues.
STA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
Introduction to Computer Vision
Introduction to Computer Vision Michael J. Black Oct 2009 PCA and applications Probability and classification Admin Assignment 1 grades back shortly. Assignment 2, parts 1 and 2 due October 22 at 11:00am
Density Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
Multivariate Bayesian Linear Regression MLAI Lecture 11
Multivariate Bayesian Linear Regression MLAI Lecture 11 Neil D. Lawrence Department of Computer Science Sheffield University 21st October 2012 Outline Univariate Bayesian Linear Regression Multivariate
Introduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
Human Pose Tracking I: Basics. David Fleet University of Toronto
Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
Machine Learning (CS 567) Lecture 2
Machine Learning (CS 567) Lecture 2 Time: T-Th 5:00pm - 6:20pm Location: GFS118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
A graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
Multivariate statistical methods and data mining in particle physics
Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general
Bayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September
Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
Introduction to Bayesian Learning
Course Information Introduction Introduction to Bayesian Learning Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Apprendimento Automatico: Fondamenti - A.A. 2016/2017 Outline
Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak
Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,
MTTTS16 Learning from Multiple Sources
MTTTS16 Learning from Multiple Sources 5 ECTS credits Autumn 2018, University of Tampere Lecturer: Jaakko Peltonen Lecture 6: Multitask learning with kernel methods and nonparametric models On this lecture:
Loss Functions, Decision Theory, and Linear Models
Loss Functions, Decision Theory, and Linear Models CMSC 678 UMBC January 31 st, 2018 Some slides adapted from Hamed Pirsiavash Logistics Recap Piazza (ask & answer questions): https://piazza.com/umbc/spring2018/cmsc678
Machine Learning Lecture 2
Machine Perceptual Learning and Sensory Summer Augmented 6 Computing Announcements Machine Learning Lecture 2 Course webpage http://www.vision.rwth-aachen.de/teaching/ Slides will be made available on
Overview. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Overview DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing
Lecture 3. Linear Regression II Bastian Leibe RWTH Aachen
Advanced Machine Learning Lecture 3 Linear Regression II 02.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de This Lecture: Advanced Machine Learning Regression
Bayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
Probabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 13: Learning in Gaussian Graphical Models, Non-Gaussian Inference, Monte Carlo Methods Some figures
Machine Learning 2007: Slides 1. Instructor: Tim van Erven Website: erven/teaching/0708/ml/
Machine 2007: Slides 1 Instructor: Tim van Erven (Tim.van.Erven@cwi.nl) Website: www.cwi.nl/ erven/teaching/0708/ml/ September 6, 2007, updated: September 13, 2007 1 / 37 Overview The Most Important Supervised
1 Bayesian Linear Regression (BLR)
Statistical Techniques in Robotics (STR, S15) Lecture#10 (Wednesday, February 11) Lecturer: Byron Boots Gaussian Properties, Bayesian Linear Regression 1 Bayesian Linear Regression (BLR) In linear regression,
A Tutorial on Learning with Bayesian Networks
A utorial on Learning with Bayesian Networks David Heckerman Presented by: Krishna V Chengavalli April 21 2003 Outline Introduction Different Approaches Bayesian Networks Learning Probabilities and Structure
Bayesian Learning. CSL603 - Fall 2017 Narayanan C Krishnan
Bayesian Learning CSL603 - Fall 2017 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Bayes Theorem MAP Learners Bayes optimal classifier Naïve Bayes classifier Example text classification Bayesian networks
Lecture 1: Bayesian Framework Basics
Lecture 1: Bayesian Framework Basics Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de April 21, 2014 What is this course about? Building Bayesian machine learning models Performing the inference of
Machine learning for pervasive systems Classification in high-dimensional spaces
Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version
Grundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Uncertainty & Probabilities & Bandits Daniel Hennes 16.11.2017 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Uncertainty Probability
UVA CS / Introduc8on to Machine Learning and Data Mining
UVA CS 4501-001 / 6501 007 Introduc8on to Machine Learning and Data Mining Lecture 13: Probability and Sta3s3cs Review (cont.) + Naïve Bayes Classifier Yanjun Qi / Jane, PhD University of Virginia Department
p L yi z n m x N n xi
y i z n x n N x i Overview Directed and undirected graphs Conditional independence Exact inference Latent variables and EM Variational inference Books statistical perspective Graphical Models, S. Lauritzen
Density Propagation for Continuous Temporal Chains Generative and Discriminative Models
$ Technical Report, University of Toronto, CSRG-501, October 2004 Density Propagation for Continuous Temporal Chains Generative and Discriminative Models Cristian Sminchisescu and Allan Jepson Department
Modeling Data with Linear Combinations of Basis Functions. Read Chapter 3 in the text by Bishop
Modeling Data with Linear Combinations of Basis Functions Read Chapter 3 in the text by Bishop A Type of Supervised Learning Problem We want to model data (x 1, t 1 ),..., (x N, t N ), where x i is a vector
Machine Learning and Related Disciplines
Machine Learning and Related Disciplines The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 8-12 (Mon.-Fri.) Yung-Kyun Noh Machine Learning Interdisciplinary
Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
Machine Learning: Probability Theory
Machine Learning: Probability Theory Prof. Dr. Martin Riedmiller Albert-Ludwigs-University Freiburg AG Maschinelles Lernen Theories p.1/28 Probabilities probabilistic statements subsume different effects
Probability Based Learning
Probability Based Learning Lecture 7, DD2431 Machine Learning J. Sullivan, A. Maki September 2013 Advantages of Probability Based Methods Work with sparse training data. More powerful than deterministic
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian
Probability Theory. Prof. Dr. Martin Riedmiller AG Maschinelles Lernen Albert-Ludwigs-Universität Freiburg.
Probability Theory Prof. Dr. Martin Riedmiller AG Maschinelles Lernen Albert-Ludwigs-Universität Freiburg riedmiller@informatik.uni-freiburg.de Prof. Dr. Martin Riedmiller Machine Learning Lab, University