Scattered Interpolation Survey

Transcription

1 Scattered Interpolation Survey j.p.lewis u. southern california Scattered Interpolation Survey p.1/53

2 Scattered vs. Regular domain Scattered Interpolation Survey p.2/53

3 Motivation modeling animated character deformation texture synthesis stock market prediction neural networks machine learning... Scattered Interpolation Survey p.3/53

4 Machine Learning Score credit card applicants Each person has N attributes: income, age, gender, credit rating, zip code,... i.e. each person is a point in an N-dimensional space training data: some individuals have a score 1 = grant card, others -1 = deny card Scattered Interpolation Survey p.4/53

5 Machine Learning From training data, learn a function R N 1, 1... by interpolating the training data Scattered Interpolation Survey p.5/53

6 Texture Synthesis (blackboard drawing) Scattered Interpolation Survey p.6/53

7 Stock Market Prediction (blackboard drawing) Scattered Interpolation Survey p.7/53

8 Modeling Deforming a face mesh Images: Jun-Yong Noh and Ulrich Neumann, CGIT lab Scattered Interpolation Survey p.8/53

9 Shepard Interpolation ˆd(x) = wk (x)d k wk (x) weights set to an inverse power of the distance: w k (x) = x x k p. Note: singular at the data points x k. Scattered Interpolation Survey p.9/53

10 Shepard Interpolation improved higher order versions in Lancaster Curve and Surface Fitting book Scattered Interpolation Survey p.10/53

11 Natural Neighbor Interpolation Image: N. Sukmar, Natural Neighbor Interpolation and the Natural Element Method (NEM) Scattered Interpolation Survey p.11/53

12 Wiener interpolation linear estimator ˆx t = w k x t+k orthogonality E[(x t ˆx t )x m ] = 0 E[x t x m ] = E[ w k x t+k x m ] autocovariance E[x t x m ] = R(t m) linear system R(t m) = w k R(t + k m) Note no requirement on the actual spacing of the data. Related to the Kriging method in geology. Scattered Interpolation Survey p.12/53

13 Applications: Wiener interpolation Lewis, Generalized Stochastic Subdivision, ACM TOG July 1987 Scattered Interpolation Survey p.13/53

14 Laplace/Poisson Interpolation Objective: Minimize a roughness measure, the integrated derivative (or gradient) squared: df 2 dx dx f 2 ds Scattered Interpolation Survey p.14/53

15 Laplace/Poisson interpolation minimize R (f (x)) 2 dx F(y, y, x) = y 2 δf = F dy dy dɛ δɛ = F dy q δɛ de R F dɛ = R F dy q dx dy q dx = F dy q R F dy q b = 0 a de dɛ = R d dx = 2 d dx d dx F dy qdx = 0 F dy qdx df dx = 2 d2 f dx 2 = 2 2 f = 0 should come out like d2 f dx 2 = 2 = 0 F dy = 2y = 2 df dx now change q to q integration by parts because q is zero at both ends variation of functional is zero at minimum Scattered Interpolation Survey p.15/53

16 Laplace/Poisson: Discrete Local viewpoint: roughness R = u 2 du (u k+1 u k ) 2 for a particular k: dr = d [(u k u k 1 ) 2 + (u k+1 u k ) 2 ] du k du k = 2(u k u k 1 ) 2(u k+1 u k ) = 0 u k+1 2u k + u k 1 = 0 2 u = 0 Notice: D T D =...1, 2, 1 Scattered Interpolation Survey p.16/53

17 Laplace/Poisson Interpolation Discrete/matrix viewpoint: Encode derivative operator in a matrix D D = min f f T D T Df Scattered Interpolation Survey p.17/53

18 Laplace/Poisson Interpolation min f f T D T Df 2D T Df = 0 i.e. d 2 f dx = 0 or 2 2 = 0 f = 0 is a solution; last eigenvalue is zero, corresponds to a constant solution. Scattered Interpolation Survey p.18/53

19 Laplace/Poisson: solution approaches direct matrix inverse (better: Choleski) Jacobi (because matrix is quite sparse) Jacobi variants (SOR) Multigrid Scattered Interpolation Survey p.19/53

20 Jacobi iteration matrix viewpoint Ax = b (D + E)x = b split into diagonal D, non-diagonal E Dx = Ex + b x = D 1 Ex + D 1 b call B = D 1 E,z = D 1 b x Bx + z D 1 is easy hope that largest eigenvalue of B is less than 1 Scattered Interpolation Survey p.20/53

21 Jacobi iteration Local viewpoint Jacobi iteration sets each f k to the solution of its row of the matrix equation, independent of all other rows: Arc f c = b r A rk f k = b k j k A rj f j f k b k A kk j k A kj /A kk f j Scattered Interpolation Survey p.21/53

22 Jacobi iteration apply to Laplace eqn Jacobi iteration sets each f k to the solution of its row of the matrix equation, independent of all other rows: In 2D,...f t 1 2f t + f t+1 = 0 2f t = f t 1 + f t+1 f k 0.5 (f[k 1] + f[k + 1]) f[y][x] = 0.25 * ( f[y+1][x] + f[y-1][x] + f[y][x-1] + f[y][x+1] ) Scattered Interpolation Survey p.22/53

23 But now let s interpolate 1D case, say f 3 is known. Three eqns involve f 3. Subtract (a multiple of) f 3 from both sides of these equations: f 1 2f 2 + f 3 = 0 f 1 2f = f 3 f 2 2f 3 + f 4 = 0 f f 4 = 2f 3 f 3 2f 4 + f 5 = 0 0 2f 4 + f 5 = f 3 L = one column is zeroed 7 5 Scattered Interpolation Survey p.23/53

24 Multigrid r is known, e is not Ax = b x = x + e r = A x b r = Ax + Ae b r = Ae For Laplace/Poisson, r is smooth. So decimate, solve for e, interpolate. And recurse... Scattered Interpolation Survey p.24/53

25 Exciting demo Scattered Interpolation Survey p.25/53

26 Recovered fur Scattered Interpolation Survey p.26/53

27 Recovered fur: detail Scattered Interpolation Survey p.27/53

28 Poor interpolation Scattered Interpolation Survey p.28/53

29 Membrane vs. Thin Plate Left - membrane interpolation, right - thin plate. Scattered Interpolation Survey p.29/53

30 Thin plate spline Minimize the integrated second derivative squared (approximate curvature) min f ( d 2 f dx 2 ) 2 dx Scattered Interpolation Survey p.30/53

31 Radial Basis Functions ˆd(x) = N k w k φ( x x k ) Scattered Interpolation Survey p.31/53

32 Radial Basis Functions (RBFs) any function other than constant can be used! common choices: Gaussian φ(r) = exp( r 2 /σ 2 ) Thin plate spline φ(r) = r 2 log r Hardy multiquadratic φ(r) = (r 2 + c 2 ),c > 0 Notice: the last two increase as a function of radius Scattered Interpolation Survey p.32/53

33 RBF versus Shepard s Scattered Interpolation Survey p.33/53

34 Solving Thin plate interpolation if few known points: use RBF if many points use multigrid instead but Carr/Beatson et. al. (SIGGRAPH 01) use Greengart FMM for RBF with large numbers of points Scattered Interpolation Survey p.34/53

35 Radial Basis Functions NX ˆd(x) = w k φ( x x k ) e = X (d(x j ) ˆd(x j )) 2 j k = X NX (d(x j ) w k φ( x j x k )) 2 j k NX = (d(x 1 ) w k φ( x 1 x k )) 2 + (d(x 2 ) k NX w k φ( x 2 x k )) 2 + k Scattered Interpolation Survey p.35/53

36 Radial Basis Functions define R jk = φ( x j x k ) = (d(x 1 ) (w 1 R 11 + w 2 R 12 + w 3 R 13 + )) 2 +(d(x 2 ) (w 1 R 21 + w 2 R 22 + w 3 R 23 + )) 2 + +(d(x m ) (w 1 R m1 + w 2 R m2 + w 3 R m3 + )) 2 + d dw m = 2(d(x 1 ) (w 1 R 11 + w 2 R 12 + w 3 R 13 + ))R 1m +2(d(x 2 ) (w 1 R 21 + w 2 R 22 + w 3 R 23 + ))R 2m + +2(d(x m ) (w 1 R m1 + w 2 R m2 + w 3 R m3 + )) + = 0 put R k1,r k2,r k3, in row m of matrix. Scattered Interpolation Survey p.36/53

37 Radial Basis Functions ˆd(x) = N k e = (d Φw) 2 w k φ( x x k ) e = (d Φw) T (d Φw) de dw = 0 = ΦT (d Φw) Φ T d = Φ T Φw w = (Φ T Φ) 1 Φ T d Scattered Interpolation Survey p.37/53

38 Where does TPS kernel come from Fit an unknown function f to the data y k, regularized by minimizing a smoothness term. E[f] = (f k y k ) 2 + λ Pf 2 ( ) d e.g. Pf f = dx dx 2 Variational derivative of E wrt f leads to a differential equation P Pf(x) = 1 (f(x) yk )δ(x x k ) λ Scattered Interpolation Survey p.38/53

39 Where does TPS kernel come from Solve linear differential equation by finding Green s function of the differential operator, convolving it with the RHS (works only for a linear operator). Schematically, Lf = rhs L is the operator P P, rhs is the data fidelity f = g rhs f obtained by convolving g rhs Lg = δ choosing rhs = δ gives this eqn g is the convolutional inverse of L. Scattered Interpolation Survey p.39/53

40 Where does TPS kernel come from In summary, the kernel g is the inverse Fourier transform of the reciprocal of the Fourier transform of the adjoint-squared smoothing operator P. Scattered Interpolation Survey p.40/53

41 Where does TPS kernel come from Fit an unknown function f to the data y k, regularized by minimizing a smoothness term. E[f] = (f k y k ) 2 + λ Pf 2 e.g. Pf 2 = A similar discrete version. ( d 2 f dx 2 ) 2 dx E[f] = (f y) S S(f y) + λf P Pf Scattered Interpolation Survey p.41/53

42 Where does TPS kernel come from (continued) A similar discrete version. E[f] = (f y) S S(f y) + λf P Pf To simplify things, here the data points to interpolate are required to be at discrete sample locations in the vector y, so the length of this vector defines a sample rate (reasonable). S is a selection matrix with 1s and 0s on the diagonal (zeros elsewhere). It has 1s corresponding to the locations of data in y. y can be zero (or any other value) where there is no data. P is a diagonal-constant matrix that encodes the discrete form of the regularization operator. E.g. to minimize the integrated curvature, rows of P will contain: 2 3 2,1,0,0, , 2,1,0, ,1, 2,1,... Scattered Interpolation Survey p.42/53

43 Where does TPS kernel come from Take the derivative of E with respect to the vector f, 2S(f y) + λ2p Pf = 0 P Pf = 1 S(f y) λ Multiply by G, being the inverse of P P: f = GP Pf = 1 GS(f y) λ So the RBF kernel comes from G = (P P) 1. Scattered Interpolation Survey p.43/53

44 Where does TPS kernel come from: D (Discrete version) RBF kernel is G = (P P) 1. Take SVD P = UDV P P = V D 2 V The inverse of V D 2 V is V D 2 V. eigenvectors of a circulant matrix are sinusoids, and P is diagonal-constant (toeplitz?), or nearly circulant. So V D 2 V is approximately the same as taking the Fourier transform and then the reciprocal (remembering that D are the singular values of P not P P ) Scattered Interpolation Survey p.44/53

45 Matrix regularization Find w to minimize (Rw b) T (Rw b). If the training points are very close together, the corresponding columns of R are nearly parallel. Difficult to control if points are chosen by a user. Add a term to keep the weights small: w T w. minimize (Rw b) T (Rw b) + λw T w R T (Rw b) + 2λw = 0 R T Rw + 2λw = R T b (R T R + 2λI)w = R T b w = (R T R + 2λI) 1 R T b Scattered Interpolation Survey p.45/53

46 Applications: Pose Space Deformation Lewis/Cordner/Fong, SIGGRAPH 2000 incorporated in Softimage Scattered Interpolation Survey p.46/53

47 Applications: Pose Space Deformation Scattered Interpolation Survey p.47/53

48 Pose Space Deformation Scattered Interpolation Survey p.48/53

49 Applications: Matrix virtual city Scattered Interpolation Survey p.49/53

50 Smart Point Placement for Thin Plate Scattered Interpolation Survey p.50/53

51 Smart Point Placement for Thin Plate Scattered Interpolation Survey p.51/53

52 Smart Point Placement for Thin Plate Scattered Interpolation Survey p.52/53

53 Smart Point Placement for Thin Plate Scattered Interpolation Survey p.53/53