softened probabilistic

Transcription

1 Justin Solomon MIT

2 Understand geometry from a softened probabilistic standpoint.

3 Somewhere over here.

4 Exactly here.

5 One of these two places.

6 Query 1 2 Which is closer, 1 or 2?

7 Query 1 2 Which is closer, 1 or 2?

8 Query 1 2 Which is closer, 1 or 2?

9 L p norm KL divergence

10 Query 1 2 Which is closer, 1 or 2?

11 Query 1 2 Neither!

12 Measured overlap, not displacement.

13 Permuting histogram bins has no effect on these distances.

14 Image courtesy M. Cuturi Geometric theory of probability

15 Compare in this direction Not in this direction

16 y Cost to move mass m from x to y: x m d(x, y) Match mass from the distributions

17 Supply distribution p 0 Demand distribution p 1

18 m d(x, y) Starts at p Ends at q Positive mass

19 EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas; IJCV 40.2 (2000): Revised in: Ground Metric Learning Cuturi and Avis; JMLR 15 (2014)

20 Underlying map!

21 Useful conclusions: 1. Practical Can do better than generic solvers. 2. Theoretical T 0, 1 n n usually contains O(n) nonzeros. Min-cost flow

22

23 TRICKY NOTATION Monge-Kantorovich Problem

24 Function from sets to probability

25 x y

26 Shortest path distance Expectation Geodesic distance d(x,y) Continuous analog of EMD

27 Image courtesy M. Cuturi Not always well-posed!

28 PDF [CDF] CDF -1

29 Explains shortest path. Image from Optimal Transport with Proximal Splitting (Papadakis, Peyré, and Oudet) Mass moves along shortest paths

30 Slide courtesy M. Cuturi

31 Slide courtesy M. Cuturi

32 \begin{discussion}

33 Min-cost flow

34 Step 1: Compute D ij Step 2: Solve linear program Simplex Interior point Hungarian algorithm

35 Demand Supply -1 1

36

37

38

39 Orient edges arbitrarily

40 In computer science: Network flow problem

41 We used the structure of D.

42 Probabilities advect along the surface Eulerian Solomon, Rustamov, Guibas, and Butscher. Earth Mover s Distances on Discrete Surfaces. SIGGRAPH 2014 Think of probabilities like a fluid

43 Beckmann problem Total work Advects from ρ 0 to ρ 1 Scales linearly

44 Curl free

45 Curl-free Div-free

46 1. Sparse SPD linear solve for f 2. Unconstrained and convex optimization for g

47 x y

48 0 eigenfunctions 100 eigenfunctions Proposition: Satisfies triangle inequality.

49 McCann. A Convexity Principle for Interacting Gases. Advances in Mathematics 128 (1997). No displacement interpolation

50 W 1 ineffective for averaging tasks

51 Explains shortest path. Image from Optimal Transport with Proximal Splitting (Papadakis, Peyré, and Oudet) Mass moves along shortest paths

52

53 Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport (NIPS 2013)

54 Prove on the board:

55 Sinkhorn & Knopp. "Concerning nonnegative matrices and doubly stochastic matrices". Pacific J. Math. 21, (1967). Alternating projection

56 1. Supply vector p 2. Demand vector q 3. Multiplication by K

57 Fish image from borisfx.com Gaussian convolution

58 No need to store K

59 No need to store K

60 Geodesics in heat Crane, Weischedel, and Wardetzky; TOG 2013

61 Solomon et al. "Convolutional Wasserstein Distances: Efficient Optimal Transportation on Geometric Domains." SIGGRAPH Replace K with heat kernel

62 Similar problems, different algorithms

63 Benamou & Brenier A computational fluid mechanics solution of the Monge-Kantorovich mass transfer problem Numer. Math. 84 (2000), pp

64 Tangent space/inner product at μ

65 Consider set of distributions as a manifold Tangent spaces from advection Geodesics from displacement interpolation

66 Topics in Optimal Transportation Villani, 2003 Giant field in modern math

67 Lévy. A numerical algorithm for L2 semi-discrete optimal transport in 3D. (2014) Example: Semi-discrete transport

68 Example: Monge formulation

69 v V 0 v V 0 Wasserstein Propagation for Semi-Supervised Learning (Solomon et al.) Fast Computation of Wasserstein Barycenters (Cuturi and Doucet) Learning

70 Displacement Interpolation Using Lagrangian Mass Transport (Bonneel et al.) An Optimal Transport Approach to Robust Reconstruction and Simplification of 2D Shapes (de Goes et al.) Morphing and registration

71 Earth Mover s Distances on Discrete Surfaces (Solomon et al.) Blue Noise Through Optimal Transport (de Goes et al.) Graphics

72 Geodesic Shape Retrieval via Optimal Mass Transport (Rabin, Peyré, and Cohen) Adaptive Color Transfer with Relaxed Optimal Transport (Rabin, Ferradans, and Papadakis) Vision and image processing

73 Justin Solomon MIT