Numerical Minimization of Potential Energies on Specific Manifolds

Transcription

1 Numerical Minimization of Potential Energies on Specific Manifolds SIAM Conference on Applied Linear Algebra June 2012, Valencia Manuel Gra f 1 1 Chemnitz University of Technology, Germany, supported by DFG

2 Outline 1 The General Problem 2 Optimization on Riemannian Manifolds 3 Efficient Evaluation Approaches 4 Discrepancies and Potential Energies 5 Numerical Examples

3 The General Problem General Task For a given compact set X R n, and functions K : X X R, h : X R minimize the potential energy E(P, w) := M M w i w j K(p i, p j ) 2 w i h(p i ) + C i,j=1 for points P := (p i ) M i=1 X M and weights w := (w i ) M i=1 Ω RM Applications and Relations optimal quadrature points in reproducing kernel Hilbert spaces (spherical t-designs, low-discrepancy points, Gauß-quadrature) halftoning/dithering of images i=1 optimal point configurations (Thomson problem on S 2 )...

4 Optimization on Riemannian Manifolds minimization of E : X M Ω R for manifolds X R n, Ω R M by a nonlinear CG method on Riemannian manifolds Advantages no boundary issues for compact manifolds the method is invariant under change of coordinate system no need of local coordinates for specific manifolds and optimization problems group symmetries may be easily incorporated superlinear convergence rate Disadvantages calculations of geodesics may be too expansive or impractically

5 Optimization on Riemannian Manifolds Tangent Spaces and Geodesic Curves let M R n be a compact d-dimensional manifold M is characterized by smooth curves γ : R M tangent vector v R n at point x M v := γ(0) := d dt γ(t) t=0, γ(0) = x tangent space of M at x M defined by T x M := {v = γ(0) : γ(0) = x} R n is a d-dimensional subspace of R n geodesic curves defined by the relation γ(t) γ(t) = 0, t R for any point x M and vector v T x M there exists a unique geodesic γ x,v : R M with γ x,v (0) = x, γ x,v (0) = v

6 Optimization on Riemannian Manifolds Example of a geodesic on M = S 2

7 Optimization on Riemannian Manifolds Example of a geodesic on M = S 2

8 Optimization on Riemannian Manifolds Example of a geodesic on M = S 2

9 Optimization on Riemannian Manifolds Gradient and Hessian smooth function f : M R gradient of f at x is the unique tangent vector M f (x) T x M satisfying M f (x) v = d dt f γ x,v(t), t=0 v T x M Hessian of f at x is the unique symmetric bilinear form H M f (x) : T x M T x M R satisfying H M f (x)(v, v) = d2 dt 2 f γ x,v(t), t=0 v T x M

10 Optimization on Riemannian Manifolds Representations of the Gradient and Hessian smooth function f := f M induced by function f : R n R gradient and Hessian of f at x := (x 1,..., x n ) R n ( ) n ( f f (x) := (x) R n 2 f, H f (x) := (x) x i x i x j i=1 gradient of f at x M M f (x) = P TxM f (x) T x M ) n i,j=1 where P TxM : R n T x M is the ortho. projection onto T x M matrix representation of the Hessian of f at x M H M f (x) = H f (x) + N f (x) R n n where N f (x) R n n is a symmetric matrix satisfying R n n v N f (x)v = f (x) γ x,v (0), v T x M

11 Optimization on Riemannian Manifolds Remarks optimization methods on Riemannian manifolds M rely on closed form expressions of geodesics γ x,v, x M, v T x M for M = S d := {x R d+1 : x 2 = 1} the geodesics are v γ x,v (t) := cos(t v 2 )x+sin(t v 2 ), x M, v T x M, v 2 closed form expressions are known for specific manifolds, e.g., matrix Lie groups M R n n = R n2 have the geodesics γ X,V (t) := X exp X 1 Vt, X M, V T X M, where the matrix exponential is defined by exp A := k=0 A k k!, A Rn n

12 Optimization on Riemannian Manifolds nonlinear CG method 1 : local minimization of f : M R n R Input: initial guess x 0 M, accuracy ε > 0 1: g 0 := M f (x 0 ) T x0 M, d 0 = g 0, k = 0 2: while ε < g k 2 do 3: determine α k > 0 by a line search in direction d k : M f (γ xk,d k (α k )) γ xk,d k (α k ) = 0 4: x k+1 := γ xk,d k (α k ) M 5: dk := γ xk,d k (α k ) T xk+1 M 6: g k+1 := M f (x k+1 ) T xk+1 M H M f (x k+1 )(g k+1, d k ) 7: β k :=, k + 1 mod d 0 H M f (x k+1 )( d k, d k ) 0 k + 1 mod d 0 8: d k+1 := g k+1 + β k dk 9: k := k : end while Output: approximate stationary point x k 1 S. T. Smith (1994)

13 Optimization on Riemannian Manifolds Example of a CG-iteration on M = S 2

14 Optimization on Riemannian Manifolds Example of a CG-iteration on M = S 2

15 Optimization on Riemannian Manifolds Example of a CG-iteration on M = S 2

16 Optimization on Riemannian Manifolds Example of a CG-iteration on M = S 2

17 Efficient Evaluation Approaches Minimization of the energy E(P, w), P := (p 1,..., p M ) X M, w := (w 1,..., w M ) Ω needs optimization on the product manifold M := X M Ω R M(n+1) x := (x 1,..., x M, w) M, x j X, j = 1,..., M v := (v 1,..., v M, v 0 ) T x M, v j T xj X, v 0 T w Ω geodesics are given by γ x,v (t) = (γ x1,v 1 (t),..., γ xm,v M (t), γ w,v0 (t)) M, t R gradient M E(P, w) and Hessian H M E(P, w) can be computed by the geodesic γ x,v

18 Efficient Evaluation Approaches Naive Evaluation fixed compact manifold X R n energy E(P, w) needs O(M 2 ) function evaluations of K : X X R, h : X R, cf. E(P, w) = w Kw 2h w + C, K(p 1, p 1 )... K(p 1, p M ) h(p K := )..., h :=... K(p M, p 1 )... K(p M, p M ) h(p M ) the gradient M E(P, w) and the Hessian H M E(P, w) can be computed using O(M 2 ) function evaluation of the derivatives of K and h

19 Efficient Evaluation Approaches Nonequispaced Fast Fourier Trasnsforms 1 X = S 2, SO(3), or T d, d N polynomial functions of the form K(x, y) = λ l ψ l (x)ψ l (y), h(y) = ˆν l ψ X,l (y) x, y X, N d l=0 for certain polynomial bases ψ l L 2 (X ), l N, on X evaluation of the energy E(P, w), its gradient M E(P, w), and the matrix-vector multiplication H M E(P, w)v, v T x M, need 2 S 2 : O(M log 2 M), (M N 2 ) N d l=0 SO(3) : O(M log 2 M), (M N 3 ) T d : O(M log M), (M N d ) arithmetic operations 1 NFFT Library (J. Keiner, S. Kunis, D. Potts): http// potts/nfft 2 MG, D. Potts (2011); MG, D. Potts, G. Steidl (2011)

20 Efficient Evaluation Approaches Local Kernels fixed compact d-dimensional manifold X R n kernel satisfies for R > 0 K R (x, y) = 0, x y 2 R, x, y X for point set P := {p 1,..., p M } X M define N R (x) := {p P : p x 2 R} points P X M are well distributed if, for fixed c > 0, M R d N R (p i ) < c, i = 1,..., M for well distributed points, evaluation of the energy E(P, w), M E(P, w), and H M E(P, w)v, v T x M : the determination of the sets N R (p i ), i = 1,..., M, in O(M log M) operations O(M) function evaluations of K, h, and their derivatives

21 Discrepancies and Potential Energies compact set X R n with standard Borel σ-algebra σ(x ) Borel measure ν : σ(x ) R with ν(x ) = 1 normalized point measure at P X M { ν P := 1 M 1, x Ω, δ pi, δ x (Ω) := Ω X, x X M 0, else, i=1 discrepancy system 1 D σ(x ) of measurable test sets L 2 -discrepancy between ν P and ν (formally) ( ( ) 1 2dµD 2 D D (P, ν) := ν P (B) ν(b)) (B) D for certain choices of K : X X R, h : X R, C one calculates E(P) := E (P, 1M ) e = DD(P, 2 ν), P X M 1 M. Drmota, R. F. Tichy Sequences, Discrepancies and Applications

22 Discrepancies and Potential Energies L 2 -Discrepancies over Halfspaces discrepancy system D consists of halfspaces h + (n, d) := {x R n : x n d}, (n, d) D := S n 1 R such that the correpsonding halfplanes have non-empty intersection with X, i.e., D := H X := {h + (n, d) X : (n, d) D X }, D X := {(n, d) D : h + (n, d) h ( n, d) X } canonical measure on D induced by µ S n 1 µ R the L 2 -discrepancy over halfspaces reads as 1 D 2 H X (P, ν) = 1 M 2 M K H (p i, p j ) 2 M i,j=1 with the Euclidean distance kernel 1 R. Alexander (1975) M i=1 K H (x, y) := C X c n x y 2, X K H (p i, x)d ν (x)+c K,ν x, y X

23 Discrepancies and Potential Energies L 2 -Discrepancies over Euclidean Balls for fixed radius R > 0, the discrepancy system D consists of Euclidean balls B n (c, r) := {x R n : c x 2 r}, c D := R n, r > 0, which have non-empty intersection with X, i.e., D := B R := {B n (c, R) X : c D} canonical measure on D induced by µ R n the L 2 -discrepancy over Euclidean balls reads as DB 2 R (P, ν) = 1 M M 2 K BR (p i, p j ) 2 M K BR (p M i, x)d ν (x)+c K,ν with kernel i,j=1 i=1 K BR (x, y) := µ R n( Bn (x, R) B n (y, R) ), x, y X X

24 Discrepancies and Potential Energies Remarks the kernels K BR, R > 0, are local kernels with K BR (x, y) = 0, x y 2 2R, x, y X, positive definite, and radial, i.e, K BR (x, y) = k n R ( x y 2), x, y X R n, where kr n : [0, ) R is n 2-times differentiable for m n the kernels defined by K R (x, y) := k m R ( x y 2), x, y X R n, are positive definite and correspond to some L 2 -discrepancy the kernels K R, R > 0, are used for computing low-discrepancy points of D H (P, ν)

25 Numerical Examples Low-Discrepancy Points on the Sphere S d X = S d R d+1 with measure ν := µ S d ω d, ω d := µ S d (S d ) E(P) = DH 2 (P, ν) for S d K(x, y) := ω d 1 d x y 2, h(x) := C := 2ω 2dω 2 d 1 dω 2d 1 ω d optimal points P M X M of the L 2 -Discrepancy over halfspaces satisfy 1 c d M d+1 2d D HS d (P M, ν) C dm d+1 2d, M N, cd, C d > 0 low-discrepancy points P M X M have optimal order M d+1 2d random points with respect to ν have expected order M W. M. Schmidt (1969); J. Beck (1984)

26 Numerical Examples Computation of Low-Discrepancy Points Fourier approximation for X = S 2 : for given M N minimize the energy E(P) with h C = 0 for polynomial kernels K N (x, y) := N n n=1 k= n 16π 2 Y k n (x)y k n (y) (2n 1)(2n + 1)(2n + 3), where N := cm 1 2 for some fixed constant c 1 Heuristic with local kernels for X = S d, d N: for given M N minimize the energy E(P) with h C = 0 for local kernels K R (x, y) := k m R ( x y 2), m := max{d + 1, 4}, where R := cm 1 d for some fixed constant c ( ) d ω 1 d d ω d 1

27 Numerical Examples Low-Discrepancy Points by Polynomial Kernels K N on S 2 L 2 -discrepancy DH S (P, ν) Number of points M Average order: 4 3 π M 1 2 Random points Optimal order: c M 3 4, c := Local min.: N = 2M J. S. Brauchart, D. P. Hardin, E. B. Saff (2012)

28 Numerical Examples Low-Discrepancy Points by Local Kernels K R on S d, d = 2, 3, 4, (P, ν) L 2 -discrepancy DH S d Number of points M S 5 : 1.78 M 3 5 Local min.: R = 1.6/M 1 5 S 4 : 1.79 M 5 8 Local min.: R = 1.7/M 1 4 S 3 : 1.74 M 2 3 Local min.: R = 1.8/M 1 3 S 2 : 1.58 M 3 4 Local min.: R = 2.4/M 1 2

29 Numerical Examples Different Geometric Characteristics of Local Minimizers on S 2 polynomial kernels K N for M = 400 points P N = 2M 1 2 N = 3M 1 2 D (P, ν) = D HS 2 H S 2 (P, ν) =

30 Numerical Examples Different Geometric Characteristics of Local Minimizers on S 2 local kernels K R for M = 400 points P R = 2.9/M 1 2 R = 2.4/M 1 2 D (P, ν) = D HS 2 H S 2 (P, ν) =

31 Conclusion general framework for optimizing potential energies on Riemannian manifolds efficient optimization by the nonlinear CG method efficient evaluation of the energy, its gradient, and Hessian times vector multiplication: for polynomial functions on S 2, SO(3), T d, d N for local kernels on S d, d N computation of low-discrepancy points on the sphere S d, d = 2, 3, 4, 5 publications: grman/

32 Conclusion general framework for optimizing potential energies on Riemannian manifolds efficient optimization by the nonlinear CG method efficient evaluation of the energy, its gradient, and Hessian times vector multiplication: for polynomial functions on S 2, SO(3), T d, d N for local kernels on S d, d N computation of low-discrepancy points on the sphere S d, d = 2, 3, 4, 5 publications: grman/ Thank you for your attention!