Monte Carlo Techniques Basic Concepts

Transcription

1 Monte Carlo Techniques Basic Concepts Image Synthesis Torsten Möller

2 Reading Chapter 12, 13, of Physically Based Rendering by Pharr&Humphreys Chapter 7 in Principles of Digital Image Synthesis, by A. Glassner

3 Las Vegas Vs Monte Carlo Two kinds: Las Vegas - always give same answer (but use elements of randomness on the way) Monte Carlo - give the right answer on average Very difficult to evaluate No closed forms

4 Monte Carlo Integration Pick a set of evaluation points Simplifies evaluation of difficult integrals Accuracy grows with O(N -0.5 ), i.e. in order to do twice as good we need 4 times as many samples Artifacts manifest themselves as noise Research - minimize error while minimizing the number of necessary rays

5 Basic Concepts X, Y - random variables A value chosen by a random process Continuous or discrete Apply function f to get Y from X: Y=f(X) Example - dice 6 Set of events X i = {1, 2, 3, 4, 5, 6} p j Probability p i = 1/6 j=1 x as a continuous, uniformly distributed i 1 random variable: =1 i p j < ξ < p j j=1 j=1

6 Basic Concepts Example - lighting Probability of sampling illumination based on power F i -- power of (light) source i: p i = Φ i Φ j j

7 Continuous Variable Canonical uniform random variable x Takes on all values in [0,1) with equal probability Easy to create in software (pseudo-random number generator) Can create general (complex) random distributions by starting with x

8 Basic Concepts Cumulative distribution function (CDF) P(x) of a random variable X: Dice example P(2) = 1/3 P(4) = 2/3 P(6) = 1 P x ( ) = Pr X x x { } = p s ( )ds

9 Probability Distribution Function Relative probability of a random variable taking on a particular value Derivative of CDF: p( x) = dp ( x ) dx Non-negative Always integrate to one Uniform random variable: ( ) = p x P x a,b [ ] p x # b a ( )dx ( ) = 1 x [ 0,1 ] $ P x ( ) = x % 0 otherwise

10 Cond. Probability, Independence We know that the outcome is in A What is the probability that it is in B? B AB A Pr(B A) = Pr(AB)/Pr(A) Event space Independence: knowing A does not help: Pr(B A) = Pr(B) ==> Pr(AB)=Pr(A)Pr(B)

11 Expected Value µ Average value of the function f over some distribution of values p(x) over its domain D E p D [ f ( x) ] = µ = f x ( )p( x)dx Example - cos over [0,p] where p is uniform p( x) =1 π cos x + E [ p cos( x) ] = π 0 π = 1 ( sinπ + sin0) = 0 π -

12 Variance s Expected deviation of f from its expected value µ Fundamental concept of quantifying the error in Monte Carlo (MC) methods [ ] = σ 2 = E ( f ( x) µ ) 2 V f x ( ) [ ]

13 Hence we can write: Properties [ ] = ae[ f ( x) ] E af x ( ) E[ f ( X )] = E f X i i i V af x ( ) i [ ( )] [ ] = a 2 V [ f ( x) ] [ ] = E ( f ( x) ) 2 V f x ( ) [ ] µ2 For independent random variables: V [ f ( X )] i = V [ f ( X )] i i i

14 Uniform MC Estimator Assume we want to compute the integral of f(x) over [a,b] Assuming uniformly distributed random variables X i in [a,b] (i.e. p(x) = 1/(b-a)) Our MC estimator F N : F N = b a N N i=1 f ( X ) i

15 Simple Integration f ( x)dx f ( x )Δx i 0 N i=1 = 1 N N i=1 & Error = O 1 ) ( + ' N * f ( x ) i

16 Trapezoidal Rule f ( x)dx f x i 0 w i = % Error = O 1 ( ' * & N ) N 1 i= 0 = 1 N ( ) Δx N i=1 ( ) + f ( x ) i+1 w i f ( x ) i " 0.5 i = 0,N # $ 1 0 < i < N 2

17 Uniform MC Estimator E[F N ]s equal to the correct integral: (random variable Y=f(X)) E F N $ [ ] = E& b a % = b a N = b a N = 1 N N N i=1 N i=1 N i=1 N i=1 b a ( ) f X i ' ) ( E[ f ( X i )] b = f ( x)dx a b a f ( x) p( x)dx f ( x)dx

18 General MC Estimator Can relax condition for general PDF Important for efficient evaluation of integral N (random variable f ( X ) i Y=f(X)/p(X)) And hence: [ ] = E% 1 N E F N = 1 N # $ N i=1 a N i=1 F N = 1 N b a b = f ( x)dx ( ) ( ( )' ( ) ( ) p x f X i p X i f x p x & i=1 ( )dx p( X ) i

19 Confidence Interval We know we should expect the correct result, but how likely are we going to see it? Strong law of large numbers (assuming that Y i are independent and identically distributed): % Pr lim 1 N N N Y ( & i = E[ Y] ) ' * =1 i=1

20 Confidence Interval Rate of convergence: Chebychev Inequality Setting We have { } V F Pr F E F [ ] k %' Pr& F E F (' δ = V [ F ] k 2 [ ] k 2 [ ] V [ F ] δ )' * + ' δ

21 MC Estimator How good is it? What s our error? Our error (root-mean square) is in the variance, hence # V [ FN ] = V 1 N f ( x ) i & % ( $ N p( x ) i ' i=1 = 1 N # V f x i % N 2 $ p x i i=1 = 1 N V Y [ ] ( ) ( ) & ( '

22 MC Estimator Hence our overall error: %' Pr F N E[ F ] N 1 V [ Y] )' & * (' N δ + ' = δ V[F] measures square of RMS error! This result is independent of our dimension

23 Distribution of the Average Central limit theorem assuming normal distribution ' Pr F N E[ Y] t σ * Y ( + ) N, = 1 t e x 2 2 dx 2π This can be re-arranged as lim N Pr{ F N I tσ } FN = 2 π t well known Bell curve e x 2 2 dx E[f(x)] N=160 N=40 N=10

24 Distribution of the Average This can be re-arranged as Pr F N I tσ FN Hence for t=3 we can conclude Pr F N I 3σ FN I.e. pretty much all results are within three standard deviations (probabilistic error bound confidence) { } = 2 π t { } = E[f(x)] e x 2 2 dx N=160 N=40 N=10

25 Choosing Samples How to sample random variables? Assume we can do uniform distribution How to do general distributions? Inversion Rejection Transformation

26 Inversion Method Idea - we want all the events to be distributed according to y-axis, not x-axis 1 PDF 1 CDF 0 0 x Uniform distribution is easy! 1 PDF 1 CDF x 0 x 0 x

27 Inversion Method Compute CDF (make sure it is normalized!) P x x ( ) = p s ( )ds Compute the inverse P -1 (y) 1 0 PDF x 1 0 CDF 1 CDF 1 P -1 x 0 Obtain a uniformly distributed random number x Compute X i = P -1 (x) x 0 x

28 Example - Power Distribution Used in BSDF s Make sure it is normalized Compute the CDF Invert the CDF P x p x ( ) = cx n 0 x 1 cx n dx =1 c = n +1 P 1 ( x) n +1 = x Now we can choose a uniform x distribution to get a power distribution! 1 0 x 0 ( ) = n +1 ( )s n ds = x n +1 X = n +1 ξ

29 Example - Exponential Distrib. E.g. Blinn s Fresnel Term Make sure it is normalized Compute the CDF P x 0 Invert the CDF P 1 ( x) = 1 a ln( 1 x) Now we can choose a uniform x distribution to get an exponential distribution! extend to any funcs by piecewise approx. p x ( ) = ce ax 0 x 0 ce ax dx =1 x ( ) = ae as ds X = 1 a ln 1 ξ ( ) = 1 a lnξ c = a =1 e ax

30 Rejection Method Sometimes We cannot integrate p(x) We cannot invert a function P(x) (we don t have the function description) Need to find q(x), such that p(x) < cq(x) Dart throwing Choose a pair of random variables (X, x) test whether x < p(x)/cq(x)

31 Rejection Method Essentially we pick a point (X, xcq(x)) If point lies beneath p(x) then we are ok Not all points do -> expensive method Example - sampling a Circle: p/4=78.5% good samples Sphere: p/6=52.3% good samples Gets worst in higher dimensions 1 p(x) 0 1

32 Transforming between Distrib. Inversion Method --> transform uniform random distribution to general distribution transform general X (given by PDF p x (x)) to general Y (with unknown PDF p y (y)) Y is typically given by Y=y(X) y(x) must be one-to-one, i.e. monotonic hence P y { ( )} = Pr X x ( y) = Pr Y y x ( ) { } = P x x

33 Transforming between Distrib. Hence we have for the PDF s: p y p y ( y)dy = P ( y y) = P ( x x) = p ( x x)dx " ( y) = dy % $ ' # dx & Example: p x (x) = 2x; Y = sinx p y 1 p x x ( ) y ( ) = cos x ( ) 1 p x x ( ) = 2x cos x = 2sin 1 y 1 y 2

34 Transforming between Distrib. y(x) usually not given However, if CDF s are the same, we use generalization of inversion method: ( ) = P 1 P ( x) y x y x ( )

35 Multiple Dimensions Easily generalized - using the Jacobian of Y=T(X) p y ( T( x) ) = J ( T x) 1 p ( x x) Example - polar coordinates J T ( x) = $ x & r & y & % r x ' θ ) y ) = ) θ ( x = rcosθ y = rsinθ $ cosθ rsinθ' & ) % sinθ rcosθ ( p r,θ ( ) = J T p x,y ( ) = rp x,y ( )

36 Multiple Dimensions Spherical coordinates: p r,θ,φ ( ) = r 2 sinθp x, y,z ( ) Now looking at spherical directions: We want the solid angle to be uniformly distributed dω = sinθdθdφ Hence the density in terms of f and q: p θ,φ ( )dθdφ = p ω ( )dω p θ,φ ( ) = sinθp ω ( )

37 Multidimensional Sampling Separable case - independently sample X from p x and Y from p y : p( x, y) = p ( x x) p y y Often times this is not possible - compute the marginal density function p(x) first: p( x) = p( x, y) Then compute conditional density function (p of y given x) ( ) p y x Use 1D sampling with p(x) and p(y x) dy ( ) = p x, y p( x) ( )

38 Sampling of Hemisphere Uniformly, i.e. p(w) = c Sampling q first: p θ 2π ( ) = p θ,φ Now sampling in f: 1= p( ω) c = 1 H 2 2π ( )dφ = 0 p φ θ 2π 0 ( ) = p ( θ,φ ) p( θ) sinθ 2π dφ = sinθ = 1 2π

39 Sampling of Hemisphere Now we use inversion technique in order to sample the PDF s: Inverting these: P θ θ 0 ( ) = sinαdα P φ θ ( ) = φ 0 θ = cos 1 ξ 1 φ = 2πξ 2 1 2π dα = φ 2π =1 cosθ

40 Sampling of Hemisphere Converting these to Cartesian coords: θ = cos 1 ξ 1 φ = 2πξ 2 x = sinθ cosφ = cos( 2πξ ) ξ 1 y = sinθ sinφ = sin( 2πξ ) ξ 1 z = cosθ = ξ 1 Similar derivation for a full sphere

41 Sampling a Disk Uniformly: p x, y ( ) = 1 π p r,θ ( ) = rp x, y ( ) = r π Sampling r first: Then sampling in q: Inverting the CDF: p r p θ r 2π 0 ( ) = p r,θ ( ) = p ( r,θ ) p( r) ( )dθ = 2r = 1 2π P r ( ) = r 2 P θ r ( ) = θ 2π r = ξ 1 θ = 2πξ 2

42 Sampling a Disk Given method distorts size of compartments Better method r = x θ = x y

43 Cosine Weighted Hemisphere Our scattering equations are cos-weighted!! Hence we would like a sampling distribution, that reflects that! Cos-distributed p(w) = c.cosq 1 = p ω H 2 2π π ( ) dω = c cosθ sinθdθdφ π 2 0 = 2cπ cosθ sinθdθ c = 1 π p( θ,φ) = 1 cosθ sinθ π

44 Cosine Weighted Hemisphere Could use marginal and conditional densities, but use Malley s method instead: uniformly generate points on the unit disk Generate directions by projecting the points on the disk up to the hemisphere above it dw da/cosq rejected samples dw cosq q da

45 Cosine Weighted Hemisphere Why does this work? Unit disk: p(r, f) = r/p Map to hemisphere: r = sin q Jacobian of this mapping (r, f) -> (sin q, f) Hence: J T # ( x) = cosθ 0 & % ( = cosθ $ 0 1' p θ,φ ( ) = J T p r,φ ( ) = cosθ sinθ π

46 Performance Measure Key issue of graphics algorithm time-accuracy tradeoff! Efficiency measure of Monte-Carlo: ε F ( ) = V: variance T: rendering time Better algorithm if Better variance in same time or Faster for same variance Variance reduction techniques wanted! 1 V F [ ]T F [ ]

47 Russian Roulette Don t evaluate integral if the value is small (doesn t add much!) Example - lighting integral L o p,ω o S 2 ( ) = f r p,ω o,ω i ( )L ( i p,ω i )cosθ i dω i Using N sample directions and distribution of p(w i ) 1 N N i=1 f r ( p,ω o,ω i )L ( i p,ω i )cosθ i p( ω ) i Avoid evaluations where f r is small or q close to 90 degrees

48 Russian Roulette cannot just leave these samples out With some probability q we will replace with a constant c With some probability (1-q) we actually do the normal evaluation, but weigh the result accordingly % ' F qc ξ > q F " = & 1 q (' c otherwise The expected value works out fine E [ F "] = ( 1 q) E [ F ] qc $ & % 1 q ' ) + qc = E F ( [ ]

49 Russian Roulette Increases variance Improves speed dramatically Don t pick q to be high though!!

50 Stratified Sampling - Revisited domain L consists of a bunch of strata L i Take n i samples in each strata and a uniform distribution p i = 1/v i General MC estimator for strata i: Overall integral: F 1 F 2 F 3 F 4 F = N i=1 v i F i F i = 1 n i n i j=1 f ( X ) i, j v i - volume of strata i n i - number of samples in strata i F i - integral of strata i P i - distribution in strata i

51 Stratified Sampling - Revisited General MC estimator: j=1 Expected value and variance (assuming v i is the volume of one strata): [ ] = 1 v i f x µ i = E f ( X ) i, j F i = 1 n i Λ i n i ( )dx f ( X ) i, j σ i 2 = 1 v i Λ i ( f ( x) µ ) 2 i dx Variance for one strata with n i samples: 2 σ i n i

52 Stratified Sampling - Revisited Overall estimator / variance: [ ] = V v i F i V F [ ] = V v i F i [ ] [ ] = v 2 i V F i = v i 2 σ i 2 n i Assuming number of samples proportional to volume of strata - n i =v i N: V [ F] = 1 N v i σ i 2

53 Stratified Sampling - Revisited Variance of stratified sampling: V [ F] = 1 2 v i σ i N Compared to no-strata (Q is the mean of f over the whole domain L): Pick strata i first, then distribute sample X within strata [ ] = E X V i [ F] V F V [ F] = 1 N [ ] [ ] +V [ X E i F ] ( v i σ 2 i + v i ( µ i Q) 2 )

54 Stratified Sampling - Revisited V [ F] = 1 2 v i σ i V [ F] = 1 v N i σ 2 i + v i ( µ i Q) 2 N Stratified sampling never increases variance Right hand side minimized, when strata are close to the mean of the whole function I.e. pick strata so they reflect local behaviour, not global (I.e. compact) Which is better? ( )

55 Stratified Sampling - Revisited Improved glossy highlights Random sampling stratified sampling

56 Stratified Sampling - Problems Curse of dimensionality Alternative - Latin Hypercubes Better variance than uniform random Worse variance than stratified Sample clustering

57 Quasi Monte Carlo Doesn t use real random numbers Replaced by low-discrepancy sequences Works well for many techniques including importance sampling Doesn t work as well for Russian Roulette and rejection sampling Better convergence rate than regular MC

58 Bias β = E F [ ] F If b is zero - unbiased, otherwise biased Example - pixel filtering ( ) = f x s,y t I x, y ( )L( s,t)dsdt Unbiased MC estimator, with distribution p I( x, y) 1 Np N i=1 Biased (regular) filtering: I( x, y) i i f ( x s i, y t i )L( s i,t i ) f ( x s i,y t i )L s i,t i ( ) ( ) f x s i,y t i

59 Bias typically Np i f ( x s i, y t i ) I.e. the biased estimator is preferred Essentially trading bias for variance

60 Importance Sampling MC Can improve our chances by sampling areas, that we expect have a great influence called importance sampling find a (known) function p, that comes close to the function we want to compute the integral of, 1 f then evaluate: ( ) ( x I = p x ) p( x) dx 0

61 Importance Sampling MC Crude MC: F = n i=1 λ i f ( x ) i For importance sampling, actually probe a new function f/p. I.e. we compute our new estimates to be: F = 1 N N i=1 f ( x ) i p( x ) i

62 Importance Sampling MC For which p does this make any sense? Well p should be close to f. If p = f, then we would get F = 1 N f ( x ) i =1 N p( x ) i Hence, if we choose p = f/µ, (I.e. p is the normalized distribution function of f) then we d get: F = 1 N f ( x ) i = µ = 1 f ( x)dx N f ( x ) i µ 0 i=1 i=1

63 Optimal Probability Density Variance V[f(x)/p(x)] should be small Optimal: f(x)/p(x) is constant, variance is 0 p(x) µ f (x) and ò p(x) dx = 1 p(x) = f (x) / ò f (x) dx Optimal selection is impossible since it needs the integral Practice: where f is large p is large

64 Are These Optimal? π θ = ω π = ω ) cos( ) pr( 2 1 ) pr( ) R(p)L(p, (p) L (p) L e r ω + = ) ( )cos, ( ) ( ) ( ) ( ω θ ω π pr p L p R p L p L e r + =

65 Importance Sampling MC Since we are finding random samples distributed by a probability given by p and we are actually evaluating in our experiments f/p, we find the variance of this experiment to be: σ 2 imp = = ' % & f p ( x) ( x) ( x) ( x) f p 2 $ " # 2 p dx ( x) I 2 dx improves error behaviour (just plug in p = f/i) I 2

66 Multiple Importance Sampling Importance strategy for f and g, but how to sample f*g?, e.g. L o ( p,w o ) = W Should we sample according to f or according to L i? Either one isn t good f ( p,w i,w o )L i ( p,w i )cosq i dw i Use Multiple Importance Sampling (MIS)

67 Multiple Importance Sampling Importance sampling f Importance sampling L Multiple Importance sampling

68 Multiple Importance Sampling In order to evaluate f (x)g(x)dx Pick n f samples according to p f and n g samples according to p g Use new MC estimator: n f 1 f (X i )g(x i )w f (X i ) + 1 f (Y j )g(y j )w g (Y j ) n f p f (X i ) n g p g (Y j ) i=1 Balance heuristic vs. power heuristic: w s (x) = i n sp s (x) n i p i (x) w s (x) = n g j=1 ( n s p s (x)) β i ( n i p i (x)) β

69 MC for global illumination We know the basics of MC How to apply MC in global illumination? How to apply to BxDF s How to apply to light source

70 MC for GI - general case General problem - evaluate: L o ( p,ω o ) = Ω f ( p,ω i,ω o )L i ( p,ω i )cosθ i dω i We don t know much about f and L, hence use cos-weighted sampling of hemisphere in order to find a w i Use Malley s method Make sure that w o and w i lie in same hemisphere

71 MC for GI - microfacet BRDFs Typically based on microfacet distribution (Fresnel and Geometry terms not statistical measures) Example - Blinn: D( ω ) h = ( n + 2) ( ω h N) n We know how to sample a spherical / power distribution: cosθ h = n +1 ξ 1 φ = 2πξ 2 This sampling is over w h, we need a distribution over w i

72 MC for GI - microfacet BRDFs This sampling is over w h, we need a distribution over w i : dω i = sinθ i dθ i dφ i dω h = sinθ h dθ h dφ h Which yields (using that q i =2q h and f i =f h ): dω h dω i = sinθ hdθ h dφ h sinθ i dθ i dφ i = sinθ hdθ h dφ h sin2θ h 2dθ h dφ h = = 1 4cosθ h sinθ h 4cosθ h sinθ h

73 MC for GI - microfacet BRDFs Isotropic microfacet model: p θ ( ) = p ( h θ) 4( ω o ω ) h

74 MC for GI - microfacet BRDFs Anisotropic model (after Ashikhmin and Shirley) for a quarter disk: % φ = arctan ' & e x +1 e y +1 tan % πξ (( 1 ' * & 2 )* ) ( e cosθ h = ξ x cos 2 φ +e y sin 2 φ +1) 1 2 If e x = e y, then we get Blinn s model

75 MC for GI - Specular Delta-function - special treatment 1 N N i=1 f ( p,ω i,ω o )L i ( p,ω i )cosθ i p( ω i ) Since p is also a delta function this simplifies to = 1 N N i=1 p( ω i ) = δ( ω ω i ) ρ hd (ω o )L i ( p,ω) ( ) ρ hd (ω o ) δ ω ω i L i (p,ω i )cosθ i cosθ i p( ω i )

76 MC for GI - Multiple BxDF s Sum up distribution densities p ω ( ) = 1 N N i=1 p i ω ( ) Have three unified samples - the first one determines according to which BxDF to distribute the spherical direction; the other two specify the actual direction

77 Light Sources We need to evaluate Sp: Cone of directions from point p to light (for evaluating the rendering equation for direct illuminations), I.e. w i L o ( p,ω o ) = Ω f ( p,ω,ω )L ( p,ω )cosθ dω i o i i i i Sr: Generate random rays from the light source (Bi-directional Path Tracing or Photon Mapping)

78 Point Lights Source is a point uniform power in all directions hard shadows Sp: Delta-light source Treat similar to specular BxDF Sr: sampling of a uniform sphere

79 Spot Lights Like point light, but only emits light in a cone-like direction Sp: like point light, i.e. delta function Sr: sampling of a cone p θ,φ ( ) = p θ ( ) p φ ( ) p φ ( ) =1 2π p θ ( ) = c θ max 1 = c sinθdθ = c 1 cosθ max 0 p( θ) =1 ( 1 cosθ ) max ( )

80 Projection Lights Like spot light, but with a texture in front of it Sp: like spot light, i.e. delta function Sr: like spot light, i.e. sampling of a cone p θ,φ ( ) = p θ ( ) p φ ( ) p φ ( ) =1 2π p θ ( ) = c θ max 1 = c sinθdθ = c 1 cosθ max 0 p( θ) =1 ( 1 cosθ ) max ( )

81 Goniophotometric Lights Like point light (hard shadows) Non-uniform power in all directions - given by distribution map Sp: like point-light Delta-light source Treat similar to specular BxDF Sr: like point light, i.e. sampling of a uniform sphere

82 Directional Lights Infinite light source, i.e. only one distinct light direction hard shadows Sp: like point-light Sr: Delta function create virtual disk of the size of the scene sample disk uniformly (e.g. Shirley)

83 Area Lights Defined by shape Soft shadows Sp: distribution over solid angle q o is the angle between w i and (light) shape normal dω i = cosθ o da A is the area of the shape r 2 Sr: Sampling over area of the shape Sampling distribution depends on the area of the shape p( x) = 1 A

84 Area Lights If v(p,p ) determines visibility: L o ( p,ω o ) = dω i = cosθ o r 2 Hence: Ω da f ( p,ω i,ω o )L i ( p,ω i )cosθ i dω i = v p, p & A L o ( p,ω o ) 1 p x ( ) f (p,ω i,ω o )L i p( x) = 1 A (p,ω i )cosθ i cosθ o r 2 ( ) v ( p, p $ cosθ ) f ( p,ω i,ω o )L i ( p,ω i )cosθ o i r 2 A r v p, p $ 2 ( ) f (p,ω i,ω o )L i (p,ω i )cosθ i cosθ o da

85 Spherical Lights Special area shape Not all of the sphere is visible from outside of the sphere Only sample the area, that is visible from p Sp: distribution over solid angle Use cone sampling sinθ max = r p c Sr: Simply sample a uniform sphere r c p

86 Infinite Area Lights Typically environment light (spherical) Encloses the whole scene Sp: Normal given - cos-weighted sampling Otherwise - uniform spherical distribution Sr: uniformly sample sphere at two points p 1 and p 2 The direction p 1 -p 2 is the uniformaly distributed ray

87 Infinite Area Lights Area light + directional light Morning skylight Midday skylight Sunset environment map