Perception: objects in the environment
|
|
- Regina Owens
- 5 years ago
- Views:
Transcription
1 Zsolt Vizi, Ph.D. 2018
2 Self-driving cars
3 Sensor fusion: one categorization Type 1: low-level/raw data fusion combining several sources of raw data to produce new data that is expected to be more informative than the inputs. Type 2: intermediate-level/feature level fusion combining various features (e.g. positions) into a feature map, which can be used for higher level decisions Type 3: high-level/decision fusion combining decisions from several experts (e.g. voting, fuzzy-logic, statistical methods)
4 Perception: topics of this lecture Object tracking Object type classification
5 Tracking problems The basic tracking problem is to estimate the position and velocity of the target(s), using the available sensor data (from a sequence of scans). The multi-target tracking problem is not simply a tracking problem when there are more than one target problem of associating measurements with targets.
6 Tracking problems vs. typical estimation problems Strong temporal component is involved. Estimation of quantities, which are expected to change over time. Current state is interested. Current state is computed from previous states.
7 Bayesian inference Inference methods consist of estimating the current values for a set of parameters based on a set of observations or measurements. Bayesian estimation: the parameters are random variables that have a prior probability and the observations are noisy as well
8 Recursive Bayesian estimation Source:
9 Bayes theorem p(x z) = p(z x)p(x) p(z) X: target state (random vector variable); Z: observation (random vector variable); p(x) (probability density function of X): prior density; p(x z): posterior density; p(z x): likelihood function; p(z): normalization constant p(z) = p(z x)p(x) dx R nx Notation: p(x, z) = joint density
10 Point estimators In summary: we estimate densities at each steps... How do we use it in a specific tracking problem? Mapping the density into real world: point estimator ˆx Procedure 1. Define cost function L(x, ˆx), which defines a penalty for an erroneous estimate ˆx x. Typical choice: L(x, ˆx) = (x ˆx) T M(x ˆx). 2. Bayesian risk R: R = E (L(x, ˆx)) = L(x, ˆx)p(x)dx R nx Note: ˆx = ˆx(z). Optimal choice: ˆx(z) = argmin L(x (z), x)p(x, z) dx dz x (z) R nx+nz
11 Point estimators In summary: we estimate densities at each steps... How do we use it in a specific tracking problem? Mapping the density into real world: point estimator ˆx Procedure 3. Using first derivative, we get [homework] ˆx(z) = E(x z) 4. Uncertainity for this estimation: P xx = E((x ˆx)(x ˆx) T z)
12 Dynamic and sensor model We focus on first-order Markov processes (current state is dependent only on the previous state). 1. System dynamic model: x n = f n 1 (x n 1 ) + u n + v n 1 x n : state vector at time t n f n 1 : deterministic transition function u n : known deterministic control v n 1 : additive noise
13 Dynamic and sensor model We focus on first-order Markov processes (current state is dependent only on the previous state). 2. Sensor model: z n = h n (x n ) + w n z n : current observation vector h n : deterministic observation function w n : additive noise Simplifying assumption: f n and h n are adiabatic (changing very slowly in time) f n = f, h n = h
14 Recursive Bayesian filtering Notation: z 1:n = {z 1, z 2,..., z n } (observations up to nth step) Goal: estimating p(x n z 1:n ) applying Bayes theorem: p(x n z 1:n ) = p(z 1:n x n )p(x n ) p(z 1:n ) After some calculations, we obtain p(x n z 1:n ) = p (z n x n ) p (x n z 1:n 1 ) p (z n z 1:n 1 ) Using Chapman-Kolmogorov equation for p (x n z 1:n 1 ), we derive p (x n z 1:n 1 ) = p(x n x n 1 )p(x n 1 z 1:n 1 ) dx n 1 R nx
15 Recursive Bayesian filtering
16 Back to the point estimators: a notation State estimation: ˆx n p = E{x n z 1:p } Uncertainity estimation: P xx n p = E { (x n ˆx n p )(x n ˆx n p ) T z 1:p }
17 Back to the point estimators: prediction More algebraic manipulation (combining previous formulae, system dynamic model) give: ˆx n n 1 = {f n 1 (x n 1 ) + u n + v n 1 } p(x n 1 z 1:n 1 )dx n 1 R nx and P xx n n 1 = R nx { fn 1 (x n 1 ) + u n ˆx n n 1 } { fn 1 (x n 1 ) + u n ˆx n n 1 } T p(x n 1 z 1:n 1 )dx n 1 + Q, where Q is the covariance matrix of the system noise: Q = v n 1 vn 1p(x T n 1 z 1:n 1 )dx n 1 R nx
18 Back to the point estimators: update [by Kálmán] ˆx n n = Az obs n + b Assumption: E(x n ˆx n z 1:n 1 ) = 0 E((x n ˆx n )zn obs z 1:n 1 ) = 0 Goal #1: Determine A, b Some calculations give b = ˆx n n 1 Aẑ n n 1 and ( ) 1 A = K n = Pn n 1 xz Pn n 1 zz which implies ˆx n n = ˆx n n 1 + K n (z obs n ẑ n n 1 ) P xx n n = K np zz n n 1 KT n
19 Back to the point estimators: update [by Kálmán] Goal #2: Determine ẑ n n, Pn n 1 xz, P n n 1 zz ẑ n n = {h n (x n ) + w n }p(x n z 1:n 1 )dx n, R nx P xz n n 1 = R nx {x n ˆx n n 1 }{h n (x n ) ẑ n n 1 } T p(x n z 1:n 1 )dx n, P zz n n 1 = R nx {h n (x n ) ẑ n n 1 }{h n (x n ) ẑ n n 1 } T p(x n z 1:n 1 )dx n + R where covariance matrix of observation noise is R = w n wn T p(x n z 1:n 1 )dx n R nx
20 Recursive point estimation process
21 Gaussian densities: Kálmán filters Linear KF: f, h linear Extended KF: f, h nonlinear + Taylor approximation Finite Difference KF: f, h nonlinear + Stirling approximation Unscented KF: f, h nonlinear + sigma points: on hypersphere in all directions Spherical Simplex KF: f, h nonlinear + sigma points: on intersection of simplex & hypersphere Gaussian-Hermite KF: f, h, nonlinear + sigma points: vertices of hypercube Monte Carlo KF: f, h nonlinear + MC sampling
22 Kálmán Filters Source: html
23 LKF System dynamic model: Sensor model: x n = F x n 1 + v n 1 z n = Hx n + w n All densities are Gaussian: 1 N (x; m, C) = { (2π) n det(c) exp 1 } 2 (x m)c 1 (x m) T ) p(x n z 1:n ) = N (x n ; ˆx n n, P xx n n ) p(q n z 1:n 1 ) = N (q n ; ˆq n n, P qq n n 1 ) p(z n ) = N (z n ; ẑ n n, P zz n n 1 )
24 LKF Using these assumptions, we obtain: 1. Prediction: ˆx n n 1 = F ˆx n 1 n 1, P xx n n 1 = F P xx n 1 n 1 F T + Q 2. Observation prediction/likelihood: ẑ n n 1 = H ˆx n n 1, Pn n 1 zz xx = HPn n 1 HT + R, P xz n n 1 = P xx n n 1 HT
25 LKF Using these assumptions, we obtain: 3. Update: ( ) 1 K n = Pn n 1 xz Pn n 1 zz ( ) ˆx n n = ˆx n n 1 + K n zn obs ẑ n n 1, P xx n n = P xx n n 1 K np zz n n 1 KT n,
26 An example [finally...] State vector: Constant velocity model: System noise: x n = [ p n v n ] = [ position velocity ] F = [ ] 1 dt 0 1 v n 1 N (0, Q) Observation model (sensor output: noisy measurement for position): H = [ 1 0 ] Measurement noise: w n 1 N (0, σ 2 )
27 An example [finally...] (
28 Another example Simo Särkkä, Bayesian Filtering and Smoothing, Cambridge Unversity Press, 2013 Figure: Chapter 4.3, Example 4.3
29 Some words about multi-target tracking problem Nearest Neighbors Filter Probabilistic Data Association Filter Multihypothesis Filter More details: freiburg.de/teaching/ws10/robotics2/pdfs/rob2-15- dataassociation.pdf
30 Non-Gaussian densities: Particle Filters Explanation without equations: Tutorial with a bunch of equations: Doucet A., Johansen A.M., A Tutorial on Particle Filtering and Smoothing: Fifteen years later
31 References 1. Anton J. Haug, Bayesian Estimation and Tracking: A Practical Guide, Wiley, Sudha Challa et. al, Fundamentals of object tracking, Cambridge University Press, 2011
32 Classification problems The problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known.
33 Classification problems Supervised learning: learning where a training set of correctly identified observations is available (Machine Learning) Feature/Explanatory variable/independent variable: quantifiable property, which is used in the representation of the observation Category/Outcome/Dependent variable/target variable/class/response variable: e.g. spam/not-spam Classifier: classification algorithm Binary and multiclass classification: two and more classes are involved
34 Classification vs. estimation Many classifiers output simply the best class as an answer Probabilistic classifiers/estimators return a probability of the instance being a member of each of the possible classes best class is naturally the one with the highest probability Advantages of probabilistic methods: probability = confidence value efficient in large-scale problems, because error-propagation can be avoided
35 General Linear Models GLM Large class of models, where the response variable is assumed to follow an exponential family distribution and this variable is a (typically nonlinear) function of the linear combination of feature values.
36 General Linear Models Exponential family p(x) = h(x) exp { η T T (x) A(η) } More details: jordan/courses/260- spring10/other-readings/chapter8.pdf Examples for family members Bernoulli distribution: X {0, 1}; P (X = 1) = π Normal distribution p(x) = π x (1 π) 1 x, x {0, 1}
37 General Linear Models General equation where X: explanatory variables; E(Y ) = g 1 (β T X), Y : response variable from an exponentatial family distribution; β: parameter vector; g: link function
38 General Linear Models An example for GLM: linear regression Model: E(Y ) = β 0 + β 1 x β k x k Y is normally distributed Link function: g 1 (E(Y )) = E(Y ) [identity]
39 Logistic regression Response variable Y is Bernoulli distributed with parameter π, i.e. the observations can correspond to two classes (binary case) if Y Bernoulli(π), then E(Y ) = π. X = {X 1, X 2,..., X k } is the feature vector. Odds ratio: ratio of π and 1 π, which gives a measure for comparing class memberships Link function: logarithm of odds ratio ( ) π g(π) = log 1 π Model: ( ) π log = β 0 + β 1 X β k X k 1 π
40 Logistic regression Probably more familiar with mean function g 1 (β T X): π = exp( β T X) Source:
41 Logistic regression We use a training set { X (i), Y (i)} N i=1. Goodness of parameter vector = how correctly it works on the training set gap/error is small Cost function/error function/loss function: measures the error of the estimate, the goal is to minimize Classical choice: mean squared error MSE(β) = 1 N N (Y i Ŷi) 2 i=1
42 Logistic regression A more convenient cost function now: CE(β) = 1 N 1 N N i=1 Y (i) log (g ( 1 β T X (i))) N (1 Y (i) ) log (1 ( g 1 β T X (i))) i=1 Finding optimal parameter vector = minimizing cost function
43 Some notes on optimization Gradient descent Source:
44 Some notes on optimization Stochastic gradient descent Source: Andrew Ng, Machine Learning (online course, Coursera)
45 Multiclass problem Possible modifications: 1 vs. rest [K classifiers] 1 vs. 1 [K(K 1)/2 classifiers] multinominal extension: for logistic regression, introduce softmax function ) exp (β j T X π j = K i=1 exp ( j = 1, 2,..., K βi T X);
46 Evaluation Confusion matrix Source: Liu et. al, Learning accurate and interpretable models based on regularized random forests regression, doi: / s3-s5
47 Evaluation ROC curve Source:
48 Naive Bayes classifier Probabilistic model: the goal is to calculate the probability p(c j x), where C j is the correspodence to class j, x = (x 1, x 2,..., x k ) is an instance of the feature vector. Approach: Bayes theorem! p(c k x) = p(x C k)p(c k ) p(x) Condition of naivety : features are conditionally independent p(x i x i+1,..., x k, C j ) = p(x i C j )
49 Naive Bayes classifier Using definition of conditional PDFs, law of total probability and chain rule, we can derive p(c j x) = p(c j) k i=1 p(x i C j ) k i=1 p(c i)p(x C j ) How to classify: k ŷ = argmax j {1,2,...,K} p(c j ) p(x i C j ) i=1
50 Naive Bayes-Gauss classifier
51 More general topics (not covered now) Bias, variance Problem of underfitting/overfitting Regularization Curse of dimensionality Feature engineering Model selection (e.g. elimination techniques)
52 More basic classification algorithms Decision Tree, Random Forest K-nearest neighbors Support Vector Machine
53 Deep Learning Artificial Neural Networks Source:
54 Deep Learning Artificial Neural Networks Source:
55 Deep Learning Artificial Neural Networks Source:
56 References Online courses Kirill Eremenko, Machine Learning (udemy.com) Kirill Eremenko, Deep Learning (udemy.com) Andrew Ng, Machine Learning (coursera.org) Pennsylvania State University, Statistics online ( Books Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2011 Goodfellow et. al, Deep Learning, MIT Press, 2016 Useful links
57 Thank You For Your Attention!
Lecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationClassification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Topics Discriminant functions Logistic regression Perceptron Generative models Generative vs. discriminative
More informationMachine Learning Lecture 5
Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory
More informationProbabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016
Probabilistic classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Probabilistic approach Bayes decision theory Generative models Gaussian Bayes classifier
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationInstance-based Learning CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016
Instance-based Learning CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Outline Non-parametric approach Unsupervised: Non-parametric density estimation Parzen Windows Kn-Nearest
More informationChris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
More informationMachine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
More informationMachine Learning Lecture 7
Course Outline Machine Learning Lecture 7 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Statistical Learning Theory 23.05.2016 Discriminative Approaches (5 weeks) Linear Discriminant
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 305 Part VII
More informationCSCI-567: Machine Learning (Spring 2019)
CSCI-567: Machine Learning (Spring 2019) Prof. Victor Adamchik U of Southern California Mar. 19, 2019 March 19, 2019 1 / 43 Administration March 19, 2019 2 / 43 Administration TA3 is due this week March
More informationCh 4. Linear Models for Classification
Ch 4. Linear Models for Classification Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Department of Computer Science and Engineering Pohang University of Science and echnology 77 Cheongam-ro,
More informationLinear Dynamical Systems
Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations
More information9 Multi-Model State Estimation
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 9 Multi-Model State
More informationLinear and Logistic Regression. Dr. Xiaowei Huang
Linear and Logistic Regression Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Two Classical Machine Learning Algorithms Decision tree learning K-nearest neighbor Model Evaluation Metrics
More informationFrom Bayes to Extended Kalman Filter
From Bayes to Extended Kalman Filter Michal Reinštein Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception http://cmp.felk.cvut.cz/
More informationLecture 4 Discriminant Analysis, k-nearest Neighbors
Lecture 4 Discriminant Analysis, k-nearest Neighbors Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University. Email: fredrik.lindsten@it.uu.se fredrik.lindsten@it.uu.se
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters
More information4 Derivations of the Discrete-Time Kalman Filter
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof N Shimkin 4 Derivations of the Discrete-Time
More informationLinear Classification
Linear Classification Lili MOU moull12@sei.pku.edu.cn http://sei.pku.edu.cn/ moull12 23 April 2015 Outline Introduction Discriminant Functions Probabilistic Generative Models Probabilistic Discriminative
More informationKalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein
Kalman filtering and friends: Inference in time series models Herke van Hoof slides mostly by Michael Rubinstein Problem overview Goal Estimate most probable state at time k using measurement up to time
More informationNaïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
More informationParametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
More informationGaussian and Linear Discriminant Analysis; Multiclass Classification
Gaussian and Linear Discriminant Analysis; Multiclass Classification Professor Ameet Talwalkar Slide Credit: Professor Fei Sha Professor Ameet Talwalkar CS260 Machine Learning Algorithms October 13, 2015
More information> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 BASEL. Logistic Regression. Pattern Recognition 2016 Sandro Schönborn University of Basel
Logistic Regression Pattern Recognition 2016 Sandro Schönborn University of Basel Two Worlds: Probabilistic & Algorithmic We have seen two conceptual approaches to classification: data class density estimation
More informationGaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
More informationIntroduction to Machine Learning. Introduction to ML - TAU 2016/7 1
Introduction to Machine Learning Introduction to ML - TAU 2016/7 1 Course Administration Lecturers: Amir Globerson (gamir@post.tau.ac.il) Yishay Mansour (Mansour@tau.ac.il) Teaching Assistance: Regev Schweiger
More informationOnline Learning and Sequential Decision Making
Online Learning and Sequential Decision Making Emilie Kaufmann CNRS & CRIStAL, Inria SequeL, emilie.kaufmann@univ-lille.fr Research School, ENS Lyon, Novembre 12-13th 2018 Emilie Kaufmann Online Learning
More informationMidterm Review CS 6375: Machine Learning. Vibhav Gogate The University of Texas at Dallas
Midterm Review CS 6375: Machine Learning Vibhav Gogate The University of Texas at Dallas Machine Learning Supervised Learning Unsupervised Learning Reinforcement Learning Parametric Y Continuous Non-parametric
More informationAn Introduction to Statistical and Probabilistic Linear Models
An Introduction to Statistical and Probabilistic Linear Models Maximilian Mozes Proseminar Data Mining Fakultät für Informatik Technische Universität München June 07, 2017 Introduction In statistical learning
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationDEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER MACHINE LEARNING AND ADAPTIVE INTELLIGENCE
Data Provided: None DEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER 204 205 MACHINE LEARNING AND ADAPTIVE INTELLIGENCE hour Please note that the rubric of this paper is made different from many other papers.
More informationUNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013 Exam policy: This exam allows two one-page, two-sided cheat sheets; No other materials. Time: 2 hours. Be sure to write your name and
More informationIntroduction to Neural Networks
Introduction to Neural Networks Steve Renals Automatic Speech Recognition ASR Lecture 10 24 February 2014 ASR Lecture 10 Introduction to Neural Networks 1 Neural networks for speech recognition Introduction
More informationMachine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io
Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem
More informationIntroduction to Machine Learning
Introduction to Machine Learning Logistic Regression Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB CSE 474/574
More informationMathematical Formulation of Our Example
Mathematical Formulation of Our Example We define two binary random variables: open and, where is light on or light off. Our question is: What is? Computer Vision 1 Combining Evidence Suppose our robot
More informationPATTERN RECOGNITION AND MACHINE LEARNING
PATTERN RECOGNITION AND MACHINE LEARNING Chapter 1. Introduction Shuai Huang April 21, 2014 Outline 1 What is Machine Learning? 2 Curve Fitting 3 Probability Theory 4 Model Selection 5 The curse of dimensionality
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationReading Group on Deep Learning Session 1
Reading Group on Deep Learning Session 1 Stephane Lathuiliere & Pablo Mesejo 2 June 2016 1/31 Contents Introduction to Artificial Neural Networks to understand, and to be able to efficiently use, the popular
More informationOutline. Supervised Learning. Hong Chang. Institute of Computing Technology, Chinese Academy of Sciences. Machine Learning Methods (Fall 2012)
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Linear Models for Regression Linear Regression Probabilistic Interpretation
More informationMultivariate statistical methods and data mining in particle physics
Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general
More informationMachine Learning (CS 567) Lecture 5
Machine Learning (CS 567) Lecture 5 Time: T-Th 5:00pm - 6:20pm Location: GFS 118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
More informationNaïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability
Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture 12 Dynamical Models CS/CNS/EE 155 Andreas Krause Homework 3 out tonight Start early!! Announcements Project milestones due today Please email to TAs 2 Parameter learning
More informationBayesian Networks BY: MOHAMAD ALSABBAGH
Bayesian Networks BY: MOHAMAD ALSABBAGH Outlines Introduction Bayes Rule Bayesian Networks (BN) Representation Size of a Bayesian Network Inference via BN BN Learning Dynamic BN Introduction Conditional
More informationPart 1: Expectation Propagation
Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud
More informationMachine Learning for Signal Processing Bayes Classification and Regression
Machine Learning for Signal Processing Bayes Classification and Regression Instructor: Bhiksha Raj 11755/18797 1 Recap: KNN A very effective and simple way of performing classification Simple model: For
More informationThe Kalman Filter ImPr Talk
The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman
More informationMachine Learning. Lecture 3: Logistic Regression. Feng Li.
Machine Learning Lecture 3: Logistic Regression Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2016 Logistic Regression Classification
More informationLogistic Regression Review Fall 2012 Recitation. September 25, 2012 TA: Selen Uguroglu
Logistic Regression Review 10-601 Fall 2012 Recitation September 25, 2012 TA: Selen Uguroglu!1 Outline Decision Theory Logistic regression Goal Loss function Inference Gradient Descent!2 Training Data
More informationLogistic Regression. Seungjin Choi
Logistic Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationLogistic Regression. Sargur N. Srihari. University at Buffalo, State University of New York USA
Logistic Regression Sargur N. University at Buffalo, State University of New York USA Topics in Linear Classification using Probabilistic Discriminative Models Generative vs Discriminative 1. Fixed basis
More informationIntroduction to Machine Learning Midterm Exam Solutions
10-701 Introduction to Machine Learning Midterm Exam Solutions Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes,
More informationMachine Learning Lecture 3
Announcements Machine Learning Lecture 3 Eam dates We re in the process of fiing the first eam date Probability Density Estimation II 9.0.207 Eercises The first eercise sheet is available on L2P now First
More informationECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering
ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering Lecturer: Nikolay Atanasov: natanasov@ucsd.edu Teaching Assistants: Siwei Guo: s9guo@eng.ucsd.edu Anwesan Pal:
More informationLecture 3: Machine learning, classification, and generative models
EE E6820: Speech & Audio Processing & Recognition Lecture 3: Machine learning, classification, and generative models 1 Classification 2 Generative models 3 Gaussian models Michael Mandel
More informationMachine Learning. Regression-Based Classification & Gaussian Discriminant Analysis. Manfred Huber
Machine Learning Regression-Based Classification & Gaussian Discriminant Analysis Manfred Huber 2015 1 Logistic Regression Linear regression provides a nice representation and an efficient solution to
More informationIntroduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
More informationUndirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
More informationPILCO: A Model-Based and Data-Efficient Approach to Policy Search
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (M.P. Deisenroth and C.E. Rasmussen) CSC2541 November 4, 2016 PILCO Graphical Model PILCO Probabilistic Inference for Learning COntrol
More informationMachine Learning Lecture 2
Machine Perceptual Learning and Sensory Summer Augmented 15 Computing Many slides adapted from B. Schiele Machine Learning Lecture 2 Probability Density Estimation 16.04.2015 Bastian Leibe RWTH Aachen
More informationDeep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
More informationLecture 7: Optimal Smoothing
Department of Biomedical Engineering and Computational Science Aalto University March 17, 2011 Contents 1 What is Optimal Smoothing? 2 Bayesian Optimal Smoothing Equations 3 Rauch-Tung-Striebel Smoother
More informationLinear Models for Classification
Linear Models for Classification Oliver Schulte - CMPT 726 Bishop PRML Ch. 4 Classification: Hand-written Digit Recognition CHINE INTELLIGENCE, VOL. 24, NO. 24, APRIL 2002 x i = t i = (0, 0, 0, 1, 0, 0,
More informationBayesian Learning (II)
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP
More informationBayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
More informationEE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS
EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005 Instructor: Professor Jeff A. Bilmes Uncertainty & Bayesian Networks
More informationMidterm Review CS 7301: Advanced Machine Learning. Vibhav Gogate The University of Texas at Dallas
Midterm Review CS 7301: Advanced Machine Learning Vibhav Gogate The University of Texas at Dallas Supervised Learning Issues in supervised learning What makes learning hard Point Estimation: MLE vs Bayesian
More informationBANA 7046 Data Mining I Lecture 4. Logistic Regression and Classications 1
BANA 7046 Data Mining I Lecture 4. Logistic Regression and Classications 1 Shaobo Li University of Cincinnati 1 Partially based on Hastie, et al. (2009) ESL, and James, et al. (2013) ISLR Data Mining I
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 218 Outlines Overview Introduction Linear Algebra Probability Linear Regression 1
More informationFinal Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
More information13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
More informationIntroduction to Machine Learning Midterm Exam
10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but
More informationRecent Advances in Bayesian Inference Techniques
Recent Advances in Bayesian Inference Techniques Christopher M. Bishop Microsoft Research, Cambridge, U.K. research.microsoft.com/~cmbishop SIAM Conference on Data Mining, April 2004 Abstract Bayesian
More informationStatistical Data Mining and Machine Learning Hilary Term 2016
Statistical Data Mining and Machine Learning Hilary Term 2016 Dino Sejdinovic Department of Statistics Oxford Slides and other materials available at: http://www.stats.ox.ac.uk/~sejdinov/sdmml Naïve Bayes
More informationIntroduction to Machine Learning
Introduction to Machine Learning Thomas G. Dietterich tgd@eecs.oregonstate.edu 1 Outline What is Machine Learning? Introduction to Supervised Learning: Linear Methods Overfitting, Regularization, and the
More informationNumerical Learning Algorithms
Numerical Learning Algorithms Example SVM for Separable Examples.......................... Example SVM for Nonseparable Examples....................... 4 Example Gaussian Kernel SVM...............................
More informationDay 5: Generative models, structured classification
Day 5: Generative models, structured classification Introduction to Machine Learning Summer School June 18, 2018 - June 29, 2018, Chicago Instructor: Suriya Gunasekar, TTI Chicago 22 June 2018 Linear regression
More informationContents Lecture 4. Lecture 4 Linear Discriminant Analysis. Summary of Lecture 3 (II/II) Summary of Lecture 3 (I/II)
Contents Lecture Lecture Linear Discriminant Analysis Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University Email: fredriklindsten@ituuse Summary of lecture
More informationLecture 3: Pattern Classification
EE E6820: Speech & Audio Processing & Recognition Lecture 3: Pattern Classification 1 2 3 4 5 The problem of classification Linear and nonlinear classifiers Probabilistic classification Gaussians, mixtures
More informationLecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
More informationSequential Monte Carlo Methods for Bayesian Computation
Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter
More informationBayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 2: Bayesian Basics https://people.orie.cornell.edu/andrew/orie6741 Cornell University August 25, 2016 1 / 17 Canonical Machine Learning
More informationProbability Models for Bayesian Recognition
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIAG / osig Second Semester 06/07 Lesson 9 0 arch 07 Probability odels for Bayesian Recognition Notation... Supervised Learning for Bayesian
More informationStochastic gradient descent; Classification
Stochastic gradient descent; Classification Steve Renals Machine Learning Practical MLP Lecture 2 28 September 2016 MLP Lecture 2 Stochastic gradient descent; Classification 1 Single Layer Networks MLP
More informationBased on slides by Richard Zemel
CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we
More informationHuman Pose Tracking I: Basics. David Fleet University of Toronto
Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 13: Learning in Gaussian Graphical Models, Non-Gaussian Inference, Monte Carlo Methods Some figures
More informationMachine Learning Linear Classification. Prof. Matteo Matteucci
Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)
More informationNaïve Bayes Introduction to Machine Learning. Matt Gormley Lecture 18 Oct. 31, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Naïve Bayes Matt Gormley Lecture 18 Oct. 31, 2018 1 Reminders Homework 6: PAC Learning
More informationLinear Regression (9/11/13)
STA561: Probabilistic machine learning Linear Regression (9/11/13) Lecturer: Barbara Engelhardt Scribes: Zachary Abzug, Mike Gloudemans, Zhuosheng Gu, Zhao Song 1 Why use linear regression? Figure 1: Scatter
More informationArtificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011!
Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011! 1 Todayʼs lecture" How the brain works (!)! Artificial neural networks! Perceptrons! Multilayer feed-forward networks! Error
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning Kernel Density Estimation, Factor Analysis Mark Schmidt University of British Columbia Winter 2017 Admin Assignment 2: 2 late days to hand it in tonight. Assignment 3: Due Feburary
More informationNeural Network Training
Neural Network Training Sargur Srihari Topics in Network Training 0. Neural network parameters Probabilistic problem formulation Specifying the activation and error functions for Regression Binary classification
More information