Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Transcription

1 rofessor William Hoff Dept of Electrical Engineering &Computer Science 1

2 Review of robability For additional review material, see 02 Reviews/Reviewrobability.pdf Note some of the figures in this lecture are from Computer vision: models, learning and inference, by Simon J.D. rince, Cambridge University ress 2012

3 robability robability Marginalization Bayes rule Expectation Fitting probability models Robust statistics 3

4 robability The probability of an event is denoted as is the outcome of a random experiment we call this a random variable If we do a large number of experiments, the relative frequency of occurring approaches is between 0 and 1 inclusive is the joint probability of both events and occurring We can also write this as, The conditional probability of event that event has occurred, is occurring, given 4

5 Example ull a card at random from a deck of cards What is the conditional probability that the card is the ace of clubs, given that it is a black card? Solution: A is the event that it is the ace of clubs B is the event that it is a black card 1/52 26/ we know that AB=A because if the card is the ace of clubs, it must be black 5

6 Marginalization Let,,,, be an exhaustive, mutually exclusive set of events We can find the probability of another event by summing over conditional probabilities this is called the marginal probability : 6

7 Example A company makes widgets from three machines Machine M1 makes 3000/hour, and 80% are good Machine M2 makes 4000/hour, and 90% are good Machine M3 makes 3000/hour, and 60% are good All widgets are mixed together. What is the probability that a widget drawn at random is good? 7

8 Solution Let A be the event that the widget is good We know 0.3, 0.4, , 0.9, 0.6 By marginalization, = =

9 Bayes Rule Conditional probability relationships: The second expression may be written as which is known as Bayes' theorem, so named after the 18th century mathematician Thomas Bayes. 9 B AB B A A A B B B A AB A B B A A B

10 Example A blood test can test for the presence of a disease. Joe's doctor draws some of Joe's blood, and performs the test on his blood. The results indicate that the disease is present in Joe. Here's the information that Joe's doctor knows about the disease and the diagnostic blood test: 1 in 100 people have the disease. That is, if D is the event that a randomly selected individual has the disease, then pd = If H is the event that a randomly selected individual is healthy, then ph = If a person has the disease, then the probability that the blood test comes back positive is That is, pt+ D = If a person is healthy, then the probability that the diagnostic test comes back negative is That is, pt H = What is the probability that Joe has the disease? 10

11 Solution 11

12 Continuous variables A random variable can be continuous, taking on any real number The probability density function pdf describes the probability of a continuous variable Instead of sums, we use integrals From Computer vision: models, learning and inference, by Simon J.D. rince, Cambridge University ress

13 Expectation The expected value, or expectation, of a random variable is just the average We can compute it by summing or integrating over probabilities We can also compute the expectation of a function of a random variable 13

14 Fitting robability Models Maximum Likelihood Maximum likelihood ML method: find the parameters under which the data are most likely for that model: Example: the normal Gaussian probability model 14

15 Least Squares Fitting Assumes that measurements follow a normal distribution To find the parameters with the maximum likelihood, we use the log of the probability called the log likelihood These are constants So we are just minimizing the squared error 15

16 Least Squares Fitting continued To maximize, differentiate the log likelihood with respect to the parameters and set to zero: We get And similarly for 16

17 Fitting robability Models Maximum a osteriori Maximum a posteriori MA method: find the parameters to maximize the probability of the parameters given the data: This method is good to use when we have some prior information about the parameters We can discard the denominator as it is constant with respect to the parameters and so does not affect the position of the maximum 17

18 Robust Statistics What if the noise is not Gaussian? Measurements can be contaminated with large errors; e.g., due to gross failures in the measurement process Since residual errors are squared, a single large error can have a large influence on the fit In such cases, you can use a penalty function for the residuals that has a slower growth instead of squaring them This formulation of the inference problem is called an Mestimator in the robust statistics literature See appendix B.3 in the Szeliski book 18

19 Iteratively Re weighted Least Squares We can take the derivative of with respect to p and set it to 0: where is the derivative of ρ and is called the influence function. If we introduce a weight function, equivalent to minimizing, solving the above is The Iteratively re weighted least squares IRLS algorithm alternates between computing the weight functions and solving the resulting weighted least squares problem with fixed w values You should start IRLS in the vicinity of a good solution using for example, RANSAC 19

20 From us/um/people/zhang/inria/ublis/tutorial Estim/node24.html 20

21 From us/um/people/zhang/inria/ublis/tutorial Estim/node24.html 21