Quantifying Uncertainty

Size: px

Start display at page:

Download "Quantifying Uncertainty"

Shannon Randall
5 years ago
Views:

1 Sai Ravela M. I. T Last Updated: Spring

2 Markov Chain Monte Carlo Monte Carlo sampling made for large scale problems via Markov Chains Monte Carlo Sampling Rejection Sampling Importance Sampling Metropolis Hastings Gibbs Useful for MAP and MLE problems 2

3 MONTE CARLO Example: Calculate 1 2 x /2 (x 2) 2 /2 P(x) 0.5 e + e 2π (x 2 + cosh(x))p(x)dx May be difficult! 1 Ss f (x)p(x)dx = f (x s ) x s P(x) S " 1 s=1 " 1 When this becomes intractable Monte Carlo Sampling may still be feasible 3

4 Properties of Estimator S 1 s S s=1 Î s = f (x s ), x s P(x) I = f (x)p(x)dx lim ˆI s = I S From Introduction Class. σ σî = S unbiased 4

5 What s good about this? The good * Quick and dirty estimate (sometimes, it s the only way out) * Sampling is useful per se What s not good? * Quick and Dirty! * Rao-Blackwell Sample based estimator generally worse 5

6 Methods Basics Via CDF (random and stratified) Intermediate Importance Sampling Rejection Sampling Objective Metropolis Metropolis-Hasting Gibbs 6

7 Sampling from a CDF -Random Sampling 7

8 Latin Hypercube Sampling Stratified Sampling -e.g. Latin hypercube, Orthogonal samplling. Latin hypercube sampling, motivated by latin squares, the hypercube is in N-D. Each row and column have unique selection A way to cover the square uniformly. 8

9 LS example example Photo Credit: Wikipedia 9

10 Orthogonal/Stratified Sampling Example 10

11 Rejection Sampling y αq(x) P(x) x x i Q(x), αq(x) P(x) y i U[0, αq(x i )] 11

12 If y i P(x i ) accept else reject + Generates Samples - Can be very wasteful - Needs to be upper bound How to avoid waste? 12

13 Importance Sampling P(x) f (x)p(x)dx = f (x) Q(x)dx Q(x) 1 Ss P(x s ) = f (x s ) x S Q(x) S Q(x s ), s=1 P(x s ). Importance of sample = ω s Q(x s ) Î S = 1 S s=1 Ss f (x s )ω s Unbiased 13

14 Works with Potentials P(x) I = f (x)p(x)dx = f (x) Q(x)dx Q(x) Let s write Z p = P(x)dx & Z q = Q(x)dx and define Ṕ(x) P(x) = Q(x) = Z p Q(x) Here P(x) is just un-normalized, i.e. a potential as opposed to a probability we have access to. Q is still a proposal distribution we constructed. 14 Z q

15 Contd. Then, I = Z q Z p = Z q Z p = Z q Z p 1 S = Z q Z p 1 S f (x) Ṕ(x) Q(x) Q(x)dx f (x)ώ(x)q(x)dx Ss f (x s )ώ s ; s=1 Ss f (x s )ώ s s=1 x s Q(x) we still don t know what to do with Z q /Z p! 15

16 A simple normalization works Turns out So, A weighted normalization. Biased Z q 1 Ss = ώ s Z p S s=1 f ˆ s (x s) I = f ω s s ώś 16

17 How to select Q? y Bad idea Bad idea x 17

18 More on Q 1. Must generally cover the distribution 2. Not lead to undue importance Q Bad! P 3. Uniform is OK when P(.) is bounded 18

19 What s different Importance Sampling Does not reject a sample, just reweights it May be problematic to carry around weights during uncertainty propagation Rejection Sampling Wastes time (computation) Produces samples 19

20 What s common - Neither technique scales to high dimension - Sampling (all Monte Carlo so far) is brute force! (Dumb) Markov chain Monte Carlo 20

21 Markov Chain Monte Carlo 1. A proposal distribution from local moves (not globally, as in RS/IS). 1.1 Local moves could be in some subspace of state space. 2. Move is conditioned on most recent sample 21

22 Primer P(x i x i 1 ) x 0 x 1 x 2 x n P(x 0 ) P (α 1 ) Forward Problem: Given Transition end up where? MCMC: Given target, how to transition? 22

23 Transitions, Invariance and Equilibrium Contruct a transition x t P T (x t 1 ) x t " 1 Markov chain such that the equilibrium distribution π of P T, defined as: is the invariant distribution, i.e. π P N T π 0 π = P T π Which implies Condition 1: General balance. s PT (x ' x)π (x ' ) = π (x) x, And, π is the target distribution to sample from. 23

24 Regularity and Ergodicity Condition #2 (The whole state space is reachable) ' P T N (x x) > o x : π (x) > 0 Ergodicity Condition 2 says that all states are reachable, i.e. the chain is irreducible. When the states are aperiodic, i.e. transitions don t deterministically return to state i in integer multiples of a period, then chain is ergodic. 24

25 Detailed Balance Condition #3: Detailed Balance ' P T (x x)π (x ' ) = P T (x x ' )π (x) s s ' P T (x x)π (x ' ) = π (x) P(x x ' ) (Invariance) x, x ", 1 =1 Detailed balance implies general balance but easier to check for. Detailed balance implies convergence to a stationary distribution If π is in detailed balance with P T, then irrespective of π 0, there is some N for which π 0 π N. Detailed balance implies reversibility. 25

26 Metropolis Hastings ' Draw x Q(x ' ; x), the proposal distribution P(x ' )Q(x; x ' ) a = min 1, P(x)Q(x ' ; x) Accept x with prob. a, else retain x. No need to have pmf in Q(x ' ; x) Satisfies detailed balance Equilibrium distribution is target distribution Note: P T (x x ' ) = aq(x ' ; x) 26

27 MH Satisfied detailed balance Proof is easy ( ' π (x ' )Q(x; x ' ) ) P T (x x ' )π (x) = Q(x ; x) min 1, π (x) π (x)q(x ' ; x) ' = min (π (x)q(x ; x), π (x ' )Q(x; x ' )) ( π (x)q(x ' ) ; x) = Q(x; x ' ) min, 1 π (x ' ) π (x ' )Q(x; x ' ) ' = P T (x x)π (x ' ) 27

28 Limitations of MH The transition distribution N(x, σ 2 ) A local kernel. There can be other scale-parameterized possibilities. L α How to select σ adaptively? Small σ Slow sampling large σ many rejects 28

29 On Transitions N P T (x n x γ ) = P T (x n x n 1 )P T (x n 1 x n 2 )... P(x 1 ) a b or P T (x n x n 1 )P T (x n 1 x n 2 )... Each transition can be different and individually not be ergodic But if P N leaves P invariant and is ergodic then OK T Allows adaptive transitions 29

30 Gibbs bs Sampler: a different a trans transition Let x = x 1,, x n (a huge dimensional space) and we want to sample Gibbs: P(x) =P(x 1 x n ) P(x) = P(x 1 )P(x 2 x 1 )P(x 3 x 2, x 1 )... P(x n x n 1... x 1 ) P(x 1 ) P(x 2 x 1 ) P(x 3 x 1, x 2 ) P(x n x n 1... x 1 ) P(x 1 x ii=1 ) P(x 2 x ii=2 )... 30

31 i Transitions are simple i i ) f P(x i, x j= P(x i x j= i i ) = P(x ' i, x j=i ) Generally only one dimensional! easy to calculate Amenable to direct sampling no need for acceptance x i, 31

32 i Satisfies Detailed Balance ' π (x)p T (x x ' ) = P(x j, x = i j )P(x j x = i j ) ' = P(x j, x i=j )P(x j x =j i ) ' = P(x j x = j )P(x =j )P(x j x i i =j ) ' = P(x j x = i j )P(x j, x i=j ) = π (x ' )P T (x x ' ) 32

33 MCMC caveats What about burn in? Stuck? Stuck in a well? MCMC typically started from multiple initial starting points, and information is exchanged between chains to better track the underlying probability surface. 33

34 Slice Sampler Gap is Ok (x, y) P(y x) = u[0, P(x)] x U[xmin, xmax] y P(y x) P(x y) L(x; y) = 1 P(x) y 0 otherwise Accept if L(x; y) = 1, reject otherwise 34

35 Slicing the Slice Sampler 1. No step size like M-H. L/σ iterations vs L 2 /σ 2 2. A kind of Gibbs sampler. 3. Bracketing and Rejction can be incorporated. 4. Needs just evaluations of P(x) 5. Scaling in high dimensions? L 35

36 MIT OpenCourseWare 12.S990 Fall 2012 For information about citing these materials or our Terms of Use, visit:

Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods

Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric