Probabilistic Fundamentals in Robotics
|
|
- Oscar Charles
- 5 years ago
- Views:
Transcription
1 Probablstc Fundamentals n Robotcs Probablstc Models of Moble Robots Robotc mappng Baslo Bona DAUIN Poltecnco d Torno Course Outlne Basc mathematcal framework Probablstc models of moble robots Moble robot localzaton problem Robotc mappng Probablstc plannng and control Reference textbook Thrun, Burgard, Fox, Probablstc Robotcs, MIT Press, robotcs.org/ Baslo Bona 2 1
2 Robotc mappng Occupacy grd mappng SLAM GraphSLAM Sparse extended nformaton flter FastSLAM Baslo Bona 3 Mappng In many cases maps are already avalable and are used to perform robot localzaton: we have already dscussed the localzaton problem wth known maps However n many cases maps are unknown and must be bult by the robot tself as t moves n the envronment Snce odometry s nexact, the true pose of the robot s not known wth precson, therefore mappng s and localzaton must be performed at the same tme, post processng the robot measurements The concurrent process of localzaton and mappng s called SLAM (smultaneous localzaton and mappng) Baslo Bona 4 2
3 Example The robot has to acqure the map of the envronment whle localzng tself relatve to ths map What happens f we smply perform localzaton usng the flters learned so far and we do the mappng wth the (uncertan) pose estmate? post process Occupancy grd map Raw range data, poston ndexed by odometry Baslo Bona 5 Map types landmarks Dgtal elevaton map Landmark based representaton (stochastc map) Occupancy grd map Pont cloud representaton Baslo Bona 6 3
4 Mappng and world representatons Two partcular representatons are common n the applcatons of moble robotcs n ndoor scenaros Landmark based maps: stochastc maps that contan a probablstc descrpton (usually mean+covarance) of the poston of salent features of the scenaro Occupancy grd maps: hgh resoluton models of the envronment; t s a grd n whch each cell contans the probablty of the cell beng occuped Landmarks (doors, corners, objects) Free cell Unknow cell Occuped cell Baslo Bona 7 3D mappng Other types of maps, as 3D mappng are possble, but computatonally much more complex Baslo Bona 8 4
5 Mappng Assumng that we know the robot poses, mappng s the task of buldng a consstent t trepresentaton tt of the envronment Dependng on the scenaro t s convenent to use dfferent world representatons Occupancy grd maps are the best choce for maps, and the related famly of algorthms s called occupancy grd mappng Baslo Bona 9 The mappng process The smplest case s ntroduced frst: the robot pose s know but the measurements are nosy known pose MAPPING nosy measurements 2D maps are the most common; they can be used when robot moton takes place on a flat surface and sensors capture essentally a slce of the surroundng envronment Occupancy grd maps Baslo Bona 10 5
6 Occupacy grd map algorthm In prncple, the algorthm shall compute the posteror probablty of the map pm ( z, x ) 1: t 1: t The (occupancy grd) map s descrbed by a set of N cells m = { m m m },,, N 1 2 At each cell a probablty of occupancy (0 = free whte, 1 = occuped black) s attached p( m ) The mappng problem s decomposed nto N smpler problems pm ( z, x ) = p( m z, x ) 1: t 1: t 1: t 1: t Ths decomposton does not allow to represent dependences among neghborng cells, and we throw away much nformaton Baslo Bona 11 Bnary Bayes flter The prevous factorzaton transforms the problem nto a bnary estmaton problem wth statc state; a bnary Bayes flter s adopted The log odds representaton of occupancy s used; the advantage s that numercal nstabltes near 0 or 1 are avoded l t, = p( m z, x ) 1: t 1: t log 1 - p ( m z 1: t, x1: t ) 1 p( m z, x ) = 1-1: t 1: t 1 + exp( l ) t, Baslo Bona 12 6
7 Occupancy grd mappng Occupancy_grd_mappng( { l }, x, z ) t-1, t t 1: forall cells m 2: f m n sensor feld of z t then 3: p( m z, x ) t t l = l + log -l t t- 1 - p ( m zt, xt ) 4: else 5: l = l 6: 7: endf endfor return, 1, 0 t, t- 1, { } l t, Ths s the nverse sensor model n log form Baslo Bona 13 Inverse sensor model ( m, x, z ) Inverse_sensor_model t t 1: Let x, y be the c.o.m. of m x = (,, x y q) 2: ( ) ( ) 2 2 r = x - x + y -y ( y y x x ) 3: f = atan2 -, - -q 4: k = argmn f-q k ( max t ) j, sens j, sens t robot pose 5: f r > mn z, z + a or f- q > b 2 then 6: return l 0 k k 7: f z < z and r - z < a 2 max 8: return l k 9: f r zt 10 : return l 11 : endf t j occ free outsde range occuped cells free cells Baslo Bona 14 t 7
8 Parameters of the nverse sensor model range closed to the measured range k th beam xy, r f b a xy, q sensor cone Baslo Bona 15 Example usng ultra sound sensors Intal map Fnal map Baslo Bona 16 8
9 Another example unknown occuped free Baslo Bona 17 Mult sensor fuson When robots have dfferent types of onboard sensors, t s natural to try to ntegrate ther measurements Sensors may have dfferent characterstcs,.e., dfferent models of the perceved world; sometmes an obstacle detected by one sensor s not detected by a dfferent sensor To avod the occurrence of conflctng nformaton, a popular soluton bulds separate maps for each sensor type and then ntegrates them usng a sutable combnaton functon Baslo Bona 18 9
10 Mult sensor fuson Let the map bult by the k th sensor type be { m, m,, m 1 2 N} { m } m = = k k k k k If the sensors measurements are ndependent, we can combne them usng the De Morgan s Law NOT (P OR Q) = (NOT P) AND (NOT Q) NOT (P AND Q) = (NOT P) OR (NOT Q) k p ( m ) = 1 - (1 - p ( m )) Alternatvely one can compute the most pessmstc estmate comng from the sensors Baslo Bona 19 k k p( m ) = max p( m ) k Smultaneous localzaton and mappng (SLAM) SLAM s also known as Concurrent Mappng and Localzaton or CML The problem arses when the robot does not have access to the envronment map and does not know ts pose So, both the map and the pose must be estmated concurrently from measurements z 1:t and controls u 1:t SLAM s a very complex problem, sgnfcantly harder that smple localzaton or smple mappng wth known pose Many types and algorthms have been studed; some are ncremental, some are complete Baslo Bona 20 10
11 SLAM We need to estmate the jont dstrbuton of both robot pose and map representaton of the envronment bel p( x, m z, u ) = t t 1: t 1, t Present state only Ths s called onlne SLAM problem snce t dscards past measurements The full SLAM problem consders the entre path bel = p( x, m z, u ) t 1: t 1: t 1, t Entre past state hstory Baslo Bona 21 Onlne SLAM u u u t- 1 t t+ 1 xt-1 x x t t+ 1 zt-1 z zt t+ 1 Estmated varables m Baslo Bona 22 11
12 Full SLAM u u u t- 1 t t+ 1 xt-1 x x t t+ 1 zt-1 z zt t+ 1 Estmated varables m Baslo Bona 23 SLAM The relaton between the two approaches s smple to state px (, m z, u ) px (, m z, u )dx dx dx - = òò ò t 1: t 1, t 1: t 1: t 1, t 1 2 t 1 Moreover, ts nature s both contnuous and dscrete The objects on the map and the robot pose are contnuous The correspondence between a measurement and a feature n the map or between a feature and a prevously detected one s dscrete If the correspondence must be made explct, we wrte px (, mc, z, u ) t t 1: t 1, t or px mc z u 1: t t 1: t 1, t (,,, ) Baslo Bona 24 12
13 Problems arse from SLAM Hgh dmensonalty of the contnuous parameter space Large number of dscrete correspondence varables The algorthms employed depend manly on whch type of map needs to be estmated Landmark based maps EKF SLAM Sparse Extended Informaton Flters GraphSLAM Rao Blackwellzed Partcle Flters (FastSLAM) Occupancy grd maps Rao Blackwellzed Partcle Flters GraphSLAM Baslo Bona 25 Assumptons: EKF SLAM wth landmark based maps Feature (.e., landmark) based maps; the number of landmarks must be relatvely small (< 1,000) Works well f landmarks are unambguous; feature detectors must be optmzed It s based on Gaussan nose assumpton for robot moton and percepton Makes lnearzaton, so the amount of uncertanty of the posteror shall be lmted Can only process postve sghtngs of landmarks (no nformaton from absence of landmarks) Landmark correspondences are exactly known Baslo Bona 26 13
14 EKF SLAM wth landmark based maps EKF SLAM ncludes the poston of the landmarks n the state vector and performs estmaton over the augmented state: y æ ö x t = ç = ( xyqq ) 1, 1, 2, 2,,, çèm ø t x y x y N x N y Localzaton Mappng/landmarks Observng a partcular landmark mproves the poston estmate of all landmarks, not only ths one Ths mprovement s medated by the robot pose: landmark observaton mproves the robot pose, that n turn mproves the localzaton of prevous observed landmarks Baslo Bona 27 Covarance matrx Mathematcally, ths dependence s captured by the off dagonal elements of the covarance matrx x t æ æ ö ö æm ö x 1 1 r s s s s s s s xx xy xq xl xl xl xl r r r r r r r x r y r Nx r Ny m y s s s s s s s xy yy yq yl yl yl yl r r r r r r r r 1x r 1y r Nx r Ny m s s s s s q x q y q q q q l q l r s s q l q l r r r r r r r 1x r 1y rnx rny m l 1x ~ N s s s s s s s x l y l q l l l l l l l l l r 1x r 1x r 1x 1x 1x 1x 1y 1x Nx 1x Ny, m l s s s s s s s 1y x l y l q l l l l l l l l l r 1y r 1y r 1y 1x 1y 1y 1y 1y Nx 1y Ny m l Nx s s s s s s s x l y l q l l l l l l l l l rnx rnx rnx 1xNx 1yNx NxNx NxNy ç m çè l Ny ø s s s s s s s ç è ç è x l y l q l l l l l l l rny rny rny 1xNy 1yNy NxNy l Ny l Ny øø Baslo Bona 28 14
15 EKF SLAM We can smply extend the expresson of the dynamc system used for EKF Localzaton ìï p = f u w ( p ï,, t t- 1 t t ) í ï z = h p u t t t ïî (, map, ) PROCESS MODEL r,, ù ( 1: t t t ) é r p ù é f p u w t t t t 1 1 = N N ê ë ú û ê ú ë û MEASUREMENT MODEL r 1,... N ( p,,, ) z = h p u t t t t t So the estmaton procedure can proceed accordng to EKF, teratng predcton and update phase Baslo Bona 29 EKF SLAM: a typcal stuaton (1) Robot starts; frst measurement of feature A (2) Robot drves forward (uncertanty grows) State vector x t can grow as new landmarks are dscovered (3) Robot makes frst measurements of B and C (4) Robot drves back towards the start (uncertanty grows) (5) Robot re observes A (loop closure); uncertanty shrnks (6) Robot re observes B; also the uncertanty of C shrnks Baslo Bona 30 15
16 Another example: EKF appled to the onlne SLAM problem The robot s path s a dotted lne, and ts estmates of ts own poston are shaded ellpses. Eght dstngushable landmarks of unknown locaton are shown as small dots, and ther locaton estmates are shown as whte ellpses. In (a) (c) the robot s postonal uncertanty s ncreasng, as s ts uncertanty about the landmarks t encounters. In (d) the robot senses the frst landmark agan, and the uncertanty of all landmarks decreases, as does the uncertanty of ts current pose Baslo Bona 31 SLAM wth unknown landmark correspondence When landmark correspondence s unknown, the EKF SLAM algorthm s extended usng an ncremental maxmum lkelhood estmator to determne landmark correspondence A new landmark s created f the Mahalanobs dstance to all exstng landmarks n the map exceeds a value a Outlers detecton: how to detect spurous landmarks that are outsde the uncertanty range of landmarks already present EKF s fragle wth respect to landmark confuson, so one of the followng methods s used to avod t Spatal arrangement: choose landmarks that are far apart so that t s unlkely to confuse them > optmal tradeoff between to few and too many landmarks Sgnatures: chose landmarks wth dstngushable sgnatures (colors, dmensons, etc.) Baslo Bona 32 16
17 EKF: summary EKF landmark based SLAM s an effectve and easy to mplement soluton to many problems It has been successfully appled also n large scale envronments Onlne verson tes to determne the momentary robot pose Global problem tes to determne all poses Wth known correspondence the algorthm s ncremental Maps management wrt outlers Include a provsonal of landmarks that are not observed suffcently often Include a landmark evdence counter that computes the psoteror evdence of the exstence of a landmark Baslo Bona 33 EKF: summary Crtcal Issues Complexty s quadratc n the number of landmarks: O(n 2 ) Can dverge f nonlneartes are large Data assocaton: how do we decde f we are observng an already seen landmark? Baslo Bona 34 17
18 Rao Blackwellzed Partcle Flters (RBPF SLAM) Rao Blackwellzed Partcle Flters (RBPF), also known as FastSLAM, are a sample based technques for solvng SLAM They are partcularly sutable for estmatng occupancy grd maps of an ndoor envronment Rao Blackwellzed Partcle Flters are based on the followng factorzaton of the SLAM belef: Robot trajectory Map of the envronment Measurements Commands bel = prob( x, m z, u ) = t 1: t 1: t 1, t prob( m x, z ) prob( x z, u ) SLAM belef mappng 1: t 1: t 1: t 1: t 1, t localzaton Baslo Bona 35 Rao Blackwellzed Partcle Flters (RBPF SLAM) The prevous formula s known as Rao Blackwell factorzaton, whch gves the name to the correspondng soluton to SLAM The underlyng math translates n the followng smple concept: RBPF estmates potental trajectores of the robot and for each hypothess the SLAM reduces to mappng wth known poses The flter estmates potental trajectores and a map s assocated to each trajectory hypothess Baslo Bona 36 18
19 Summary RBPF SLAM Rao Blackwellzed Partcle Flters (RBPF) have been demonstrated to be an effectve soluton for the estmaton of occupancy grd maps No data assocaton (grd maps does not dstngush elements n the envronment) Issues Each partcle carres on a complete map hypothess: a normal computer s not able to manage more than few hundred partcles But to cope wth large uncertanty you need a large number of partcles Baslo Bona 37 Thank you. Any queston? Baslo Bona 38 19
Probabilistic Fundamentals in Robotics. DAUIN Politecnico di Torino July 2010
Probabilistic Fundamentals in Robotics Probabilistic Models of Mobile Robots Robotic mapping Basilio Bona DAUIN Politecnico di Torino July 2010 Course Outline Basic mathematical framework Probabilistic
More informationAppendix B: Resampling Algorithms
407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationLecture 12: Classification
Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna
More informationTracking with Kalman Filter
Trackng wth Kalman Flter Scott T. Acton Vrgna Image and Vdeo Analyss (VIVA), Charles L. Brown Department of Electrcal and Computer Engneerng Department of Bomedcal Engneerng Unversty of Vrgna, Charlottesvlle,
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationA Robust Method for Calculating the Correlation Coefficient
A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationConjugacy and the Exponential Family
CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the
More informationLecture 3: Shannon s Theorem
CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationOutline. Communication. Bellman Ford Algorithm. Bellman Ford Example. Bellman Ford Shortest Path [1]
DYNAMIC SHORTEST PATH SEARCH AND SYNCHRONIZED TASK SWITCHING Jay Wagenpfel, Adran Trachte 2 Outlne Shortest Communcaton Path Searchng Bellmann Ford algorthm Algorthm for dynamc case Modfcatons to our algorthm
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationThe Expectation-Maximization Algorithm
The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationTime-Varying Systems and Computations Lecture 6
Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy
More informationChat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980
MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and
More informationSpace of ML Problems. CSE 473: Artificial Intelligence. Parameter Estimation and Bayesian Networks. Learning Topics
/7/7 CSE 73: Artfcal Intellgence Bayesan - Learnng Deter Fox Sldes adapted from Dan Weld, Jack Breese, Dan Klen, Daphne Koller, Stuart Russell, Andrew Moore & Luke Zettlemoyer What s Beng Learned? Space
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationOutline. Multivariate Parametric Methods. Multivariate Data. Basic Multivariate Statistics. Steven J Zeil
Outlne Multvarate Parametrc Methods Steven J Zel Old Domnon Unv. Fall 2010 1 Multvarate Data 2 Multvarate ormal Dstrbuton 3 Multvarate Classfcaton Dscrmnants Tunng Complexty Dscrete Features 4 Multvarate
More informationTutorial 2. COMP4134 Biometrics Authentication. February 9, Jun Xu, Teaching Asistant
Tutoral 2 COMP434 ometrcs uthentcaton Jun Xu, Teachng sstant csjunxu@comp.polyu.edu.hk February 9, 207 Table of Contents Problems Problem : nswer the questons Problem 2: Power law functon Problem 3: Convoluton
More informationQuantifying Uncertainty
Partcle Flters Quantfyng Uncertanty Sa Ravela M. I. T Last Updated: Sprng 2013 1 Quantfyng Uncertanty Partcle Flters Partcle Flters Appled to Sequental flterng problems Can also be appled to smoothng problems
More informationUncertainty as the Overlap of Alternate Conditional Distributions
Uncertanty as the Overlap of Alternate Condtonal Dstrbutons Olena Babak and Clayton V. Deutsch Centre for Computatonal Geostatstcs Department of Cvl & Envronmental Engneerng Unversty of Alberta An mportant
More informationInternational Journal of Pure and Applied Sciences and Technology
Int. J. Pure Appl. Sc. Technol., 4() (03), pp. 5-30 Internatonal Journal of Pure and Appled Scences and Technology ISSN 9-607 Avalable onlne at www.jopaasat.n Research Paper Schrödnger State Space Matrx
More informationHidden Markov Models
Hdden Markov Models Namrata Vaswan, Iowa State Unversty Aprl 24, 204 Hdden Markov Model Defntons and Examples Defntons:. A hdden Markov model (HMM) refers to a set of hdden states X 0, X,..., X t,...,
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More informationχ x B E (c) Figure 2.1.1: (a) a material particle in a body, (b) a place in space, (c) a configuration of the body
Secton.. Moton.. The Materal Body and Moton hyscal materals n the real world are modeled usng an abstract mathematcal entty called a body. Ths body conssts of an nfnte number of materal partcles. Shown
More informationGaussian Mixture Models
Lab Gaussan Mxture Models Lab Objectve: Understand the formulaton of Gaussan Mxture Models (GMMs) and how to estmate GMM parameters. You ve already seen GMMs as the observaton dstrbuton n certan contnuous
More informationSimultaneous Topological Map Prediction and Moving Object Trajectory Prediction in Unknown Environments
Smultaneous Topologcal Map Predcton and Movng Obect Traectory Predcton n Unnown Envronments Shu Yun Chung, Han Pang Huang, Member, IEEE Abstract To acheve fully autonomous moble robot n unnown envronment,
More informationLearning from Data 1 Naive Bayes
Learnng from Data 1 Nave Bayes Davd Barber dbarber@anc.ed.ac.uk course page : http://anc.ed.ac.uk/ dbarber/lfd1/lfd1.html c Davd Barber 2001, 2002 1 Learnng from Data 1 : c Davd Barber 2001,2002 2 1 Why
More informationBoostrapaggregating (Bagging)
Boostrapaggregatng (Baggng) An ensemble meta-algorthm desgned to mprove the stablty and accuracy of machne learnng algorthms Can be used n both regresson and classfcaton Reduces varance and helps to avod
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationClassification as a Regression Problem
Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class
More information1 Binary Response Models
Bnary and Ordered Multnomal Response Models Dscrete qualtatve response models deal wth dscrete dependent varables. bnary: yes/no, partcpaton/non-partcpaton lnear probablty model LPM, probt or logt models
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationInternational Journal of Mathematical Archive-3(3), 2012, Page: Available online through ISSN
Internatonal Journal of Mathematcal Archve-3(3), 2012, Page: 1136-1140 Avalable onlne through www.ma.nfo ISSN 2229 5046 ARITHMETIC OPERATIONS OF FOCAL ELEMENTS AND THEIR CORRESPONDING BASIC PROBABILITY
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationDETERMINATION OF UNCERTAINTY ASSOCIATED WITH QUANTIZATION ERRORS USING THE BAYESIAN APPROACH
Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata Proceedngs, XVII IMEKO World Congress, June 7, 3, Dubrovn, Croata TC XVII IMEKO World Congress Metrology n the 3rd Mllennum June 7, 3,
More informationMean Field / Variational Approximations
Mean Feld / Varatonal Appromatons resented by Jose Nuñez 0/24/05 Outlne Introducton Mean Feld Appromaton Structured Mean Feld Weghted Mean Feld Varatonal Methods Introducton roblem: We have dstrbuton but
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More informationParametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton
More informationA New Scrambling Evaluation Scheme based on Spatial Distribution Entropy and Centroid Difference of Bit-plane
A New Scramblng Evaluaton Scheme based on Spatal Dstrbuton Entropy and Centrod Dfference of Bt-plane Lang Zhao *, Avshek Adhkar Kouch Sakura * * Graduate School of Informaton Scence and Electrcal Engneerng,
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More information8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore
8/5/17 Data Modelng Patrce Koehl Department of Bologcal Scences atonal Unversty of Sngapore http://www.cs.ucdavs.edu/~koehl/teachng/bl59 koehl@cs.ucdavs.edu Data Modelng Ø Data Modelng: least squares Ø
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense,
More informationLecture 5.8 Flux Vector Splitting
Lecture 5.8 Flux Vector Splttng 1 Flux Vector Splttng The vector E n (5.7.) can be rewrtten as E = AU (5.8.1) (wth A as gven n (5.7.4) or (5.7.6) ) whenever, the equaton of state s of the separable form
More informationHidden Markov Models
CM229S: Machne Learnng for Bonformatcs Lecture 12-05/05/2016 Hdden Markov Models Lecturer: Srram Sankararaman Scrbe: Akshay Dattatray Shnde Edted by: TBD 1 Introducton For a drected graph G we can wrte
More informationMotion Perception Under Uncertainty. Hongjing Lu Department of Psychology University of Hong Kong
Moton Percepton Under Uncertanty Hongjng Lu Department of Psychology Unversty of Hong Kong Outlne Uncertanty n moton stmulus Correspondence problem Qualtatve fttng usng deal observer models Based on sgnal
More informationMAXIMUM A POSTERIORI TRANSDUCTION
MAXIMUM A POSTERIORI TRANSDUCTION LI-WEI WANG, JU-FU FENG School of Mathematcal Scences, Peng Unversty, Bejng, 0087, Chna Center for Informaton Scences, Peng Unversty, Bejng, 0087, Chna E-MIAL: {wanglw,
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationNON-CENTRAL 7-POINT FORMULA IN THE METHOD OF LINES FOR PARABOLIC AND BURGERS' EQUATIONS
IJRRAS 8 (3 September 011 www.arpapress.com/volumes/vol8issue3/ijrras_8_3_08.pdf NON-CENTRAL 7-POINT FORMULA IN THE METHOD OF LINES FOR PARABOLIC AND BURGERS' EQUATIONS H.O. Bakodah Dept. of Mathematc
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationDigital Modems. Lecture 2
Dgtal Modems Lecture Revew We have shown that both Bayes and eyman/pearson crtera are based on the Lkelhood Rato Test (LRT) Λ ( r ) < > η Λ r s called observaton transformaton or suffcent statstc The crtera
More informationWorkshop: Approximating energies and wave functions Quantum aspects of physical chemistry
Workshop: Approxmatng energes and wave functons Quantum aspects of physcal chemstry http://quantum.bu.edu/pltl/6/6.pdf Last updated Thursday, November 7, 25 7:9:5-5: Copyrght 25 Dan Dll (dan@bu.edu) Department
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 12 10/21/2013. Martingale Concentration Inequalities and Applications
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 1 10/1/013 Martngale Concentraton Inequaltes and Applcatons Content. 1. Exponental concentraton for martngales wth bounded ncrements.
More informationInductance Calculation for Conductors of Arbitrary Shape
CRYO/02/028 Aprl 5, 2002 Inductance Calculaton for Conductors of Arbtrary Shape L. Bottura Dstrbuton: Internal Summary In ths note we descrbe a method for the numercal calculaton of nductances among conductors
More informationA Weighted Range Sensor Matching Algorithm for Mobile Robot Displacement Estimation
A Weghted Range Sensor Matchng Algorthm for Moble Robot Dsplacement Estmaton Authors Samuel T. Pfster, Krsto L. Krechbaum, Stergos I. Roumelots, Joel W. Burdc Dvson of Engneerng and Appled Scence, Calforna
More informationA Bayes Algorithm for the Multitask Pattern Recognition Problem Direct Approach
A Bayes Algorthm for the Multtask Pattern Recognton Problem Drect Approach Edward Puchala Wroclaw Unversty of Technology, Char of Systems and Computer etworks, Wybrzeze Wyspanskego 7, 50-370 Wroclaw, Poland
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationCS47300: Web Information Search and Management
CS47300: Web Informaton Search and Management Probablstc Retreval Models Prof. Chrs Clfton 7 September 2018 Materal adapted from course created by Dr. Luo S, now leadng Albaba research group 14 Why probabltes
More informationA Weighted Range Sensor Matching Algorithm for Mobile Robot Displacement Estimation
A Weghted Range Sensor Matchng Algorthm for Moble Robot Dsplacement Estmaton Authors Samuel T. Pfster, Krsto L. Krechbaum, Stergos I. Roumelots, Joel W. Burdc Dvson of Engneerng and Appled Scence, Calforna
More information/ n ) are compared. The logic is: if the two
STAT C141, Sprng 2005 Lecture 13 Two sample tests One sample tests: examples of goodness of ft tests, where we are testng whether our data supports predctons. Two sample tests: called as tests of ndependence
More informationBayesian decision theory. Nuno Vasconcelos ECE Department, UCSD
Bayesan decson theory Nuno Vasconcelos ECE Department UCSD Notaton the notaton n DHS s qute sloppy e.. show that error error z z dz really not clear what ths means we wll use the follown notaton subscrpts
More informationPsychology 282 Lecture #24 Outline Regression Diagnostics: Outliers
Psychology 282 Lecture #24 Outlne Regresson Dagnostcs: Outlers In an earler lecture we studed the statstcal assumptons underlyng the regresson model, ncludng the followng ponts: Formal statement of assumptons.
More informationEngineering Risk Benefit Analysis
Engneerng Rsk Beneft Analyss.55, 2.943, 3.577, 6.938, 0.86, 3.62, 6.862, 22.82, ESD.72, ESD.72 RPRA 2. Elements of Probablty Theory George E. Apostolaks Massachusetts Insttute of Technology Sprng 2007
More information8 : Learning in Fully Observed Markov Networks. 1 Why We Need to Learn Undirected Graphical Models. 2 Structural Learning for Completely Observed MRF
10-708: Probablstc Graphcal Models 10-708, Sprng 2014 8 : Learnng n Fully Observed Markov Networks Lecturer: Erc P. Xng Scrbes: Meng Song, L Zhou 1 Why We Need to Learn Undrected Graphcal Models In the
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationPrimer on High-Order Moment Estimators
Prmer on Hgh-Order Moment Estmators Ton M. Whted July 2007 The Errors-n-Varables Model We wll start wth the classcal EIV for one msmeasured regressor. The general case s n Erckson and Whted Econometrc
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationA particle in a state of uniform motion remain in that state of motion unless acted upon by external force.
The fundamental prncples of classcal mechancs were lad down by Galleo and Newton n the 16th and 17th centures. In 1686, Newton wrote the Prncpa where he gave us three laws of moton, one law of gravty,
More informationStatistics II Final Exam 26/6/18
Statstcs II Fnal Exam 26/6/18 Academc Year 2017/18 Solutons Exam duraton: 2 h 30 mn 1. (3 ponts) A town hall s conductng a study to determne the amount of leftover food produced by the restaurants n the
More informationProbability Theory (revisited)
Probablty Theory (revsted) Summary Probablty v.s. plausblty Random varables Smulaton of Random Experments Challenge The alarm of a shop rang. Soon afterwards, a man was seen runnng n the street, persecuted
More informationBayesian predictive Configural Frequency Analysis
Psychologcal Test and Assessment Modelng, Volume 54, 2012 (3), 285-292 Bayesan predctve Confgural Frequency Analyss Eduardo Gutérrez-Peña 1 Abstract Confgural Frequency Analyss s a method for cell-wse
More informationNote 10. Modeling and Simulation of Dynamic Systems
Lecture Notes of ME 475: Introducton to Mechatroncs Note 0 Modelng and Smulaton of Dynamc Systems Department of Mechancal Engneerng, Unversty Of Saskatchewan, 57 Campus Drve, Saskatoon, SK S7N 5A9, Canada
More informationAN IMPROVED PARTICLE FILTER ALGORITHM BASED ON NEURAL NETWORK FOR TARGET TRACKING
AN IMPROVED PARTICLE FILTER ALGORITHM BASED ON NEURAL NETWORK FOR TARGET TRACKING Qn Wen, Peng Qcong 40 Lab, Insttuton of Communcaton and Informaton Engneerng,Unversty of Electronc Scence and Technology
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationCommunication with AWGN Interference
Communcaton wth AWG Interference m {m } {p(m } Modulator s {s } r=s+n Recever ˆm AWG n m s a dscrete random varable(rv whch takes m wth probablty p(m. Modulator maps each m nto a waveform sgnal s m=m
More informationCS 468 Lecture 16: Isometry Invariance and Spectral Techniques
CS 468 Lecture 16: Isometry Invarance and Spectral Technques Justn Solomon Scrbe: Evan Gawlk Introducton. In geometry processng, t s often desrable to characterze the shape of an object n a manner that
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More information4DVAR, according to the name, is a four-dimensional variational method.
4D-Varatonal Data Assmlaton (4D-Var) 4DVAR, accordng to the name, s a four-dmensonal varatonal method. 4D-Var s actually a drect generalzaton of 3D-Var to handle observatons that are dstrbuted n tme. The
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More information