FFTs in Graphics and Vision. Spherical Convolution and Axial Symmetry Detection

Transcription

1 FFTs in Graphics and Vision Spherica Convoution and Axia Symmetry Detection

2 Outine Math Review Symmetry Genera Convoution Spherica Convoution Axia Symmetry Detection

3 Math Review Symmetry: Given a unitary representation of a group G on a vector space V, we say that a vector v V is invariant under the action of G if for a g G: ρ g v = v The set of G-invariant vector V G is a vector space.

4 Math Review Symmetry: The inear map π G is a projection onto V G, if: π G v V G for a v V π G v = v for a v V G v, w π G w = 0 for a v V G, w V. The map π G is the map sending a vector v to the cosest G-invariant vector.

5 Math Review Symmetry: The measure of symmetry of a vector v with respect to the group G is the size of its projection onto the space of G-invariant vectors: Sym 2 v, G = π G v 2

6 Math Review Convoution: Given two functions f(p) and g(p), we define the convoution of the two functions to be: f g q = f p g q p dp

7 Math Review Convoution: If we hod the function g fixed we can define a map from the space of functions back into itsef: C g f = f g Caim: The map C g is a inear operator.

8 Math Review Convoution: If we hod the function g fixed we can define a map from the space of functions back into itsef: C g f = f g Caim: Given functions f and h and scaars α and β: C g αf + βh q = α f p + β h p g q p dp = α f p g q p dp + β h p g q p dp = α C g f + β C g h

9 Math Review Convoution: Assume that the function g is rea-vaued and radia, i.e. the vaue of g at a point p is competey determined by the distance of p from the origin: g p = g( p ) Exampe: The function g is a Gaussian f g f g =

10 Math Review Convoution: Assume that the function g is rea-vaued and radia, i.e. the vaue of g at a point p is competey determined by the distance of p from the origin: g p = g( p ) Caim: In this case, C g is sef-adjoint (i.e. symmetric).

11 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Expanding the eft side, we get: C g f, h = C g f p h(p) dp

12 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Writing out the operator C g, we get: C g f, h = C g f p h(p) dp C g f, h = f g p h p dp

13 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Expressing the convoution as an integra gives: C g f, h = f g p h p dp C g f, h = f q g p q dq h p dp

14 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Changing the order of integration, we get: C g f, h = f q g p q dq h p dp C g f, h = f q g p q h p dp dq

15 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Using the fact that g is rea-vaued and radia: C g f, h = f q g p q h p dp dq C g f, h = f q h p g(q p) dp dq

16 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Using the equation for convoution, we get: C g f, h = f q h p g(q p) dp dq C g f, h = f q h g q dq

17 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h Using the equation for C g, we get: C g f, h = f q h g q dq C g f, h = f q C g h q dq

18 Math Review Convoution: Proof: We need to show that for any functions f and h: C g f, h = f, C g h And finay, using the equation for the dot-product: C g f, h = f q C g h q dq C g f, h = f, C g h

19 Outine Math Review Spherica Convoution Axia Symmetry Detection

20 Spherica Convoution/Correation In the case of the circe we used convoution / correation for two different tasks:

21 Spherica Convoution/Correation In the case of the circe we used convoution / correation for two different tasks: 1. We used convoution for operations ike smoothing circuar functions

22 Spherica Convoution/Correation In the case of the circe we used convoution / correation for two different tasks: 1. We used convoution for operations ike smoothing circuar functions 2. We used correation for operations ike aignment and symmetry detection k-fod Symmetry

23 Spherica Convoution/Correation Up to now, we thought of these two operations as essentiay the same. The situation changes as we move to functions on a sphere.

24 Spherica Convoution/Correation When we perform an operation ike smoothing, the input is: A function on the circe defining the signa, and A function on the circe defining the smoothing fiter The output of the operation is: A function on the circe = Signa Fiter Smoothed Signa

25 Spherica Convoution/Correation When we perform an operation ike aignment, the input is: Two functions on a circe The output is: A function on the space of 2D rotations

26 Spherica Convoution/Correation In the case of a circe, the situation is simper because the space of rotations is itsef a circe: There is a one-to-one mapping from points on a circe to rotations, with a point on a circe with ange θ corresponding to a rotation by an ange of θ. In the case of the sphere, the situation becomes more compicated: The sphere is a 2D space whie the rotations are a 3D space, so there can t be a one-to-one mapping.

27 Spherica Convoution In the case of a circe, we compute the vaue of the smoothed function at p by rotating the fiter so that (1,0) maps to p and then we compute the inner product of the signa with the rotated fiter. = *

28 Spherica Convoution In the case of a circe, we compute the vaue of the smoothed function at p by rotating the fiter so that (1,0) maps to p and then we compute the inner product of the signa with the rotated fiter. p =,

29 Spherica Convoution We can try an appy the same type of approach to the case of spherica functions. (0,1,0) Signa Fiter

30 Spherica Convoution We woud ike to define a new function on the sphere whose vaue at the point p is obtained by: Finding a rotation R that maps the North poe to p and then compute the inner product of the signa with the rotated fiter. p, R(0,1,0) Signa Rotated Fiter

31 Spherica Convoution The probem is that there are many different rotations that send the North poe to the point p, so this does not ead to a we-defined notion of smoothing. p, S(0,1,0) Signa Rotated Fiter

32 Spherica Convoution Reca: If we have two rotations R and S mapping the North poe to the point p, the rotations must differ by an initia rotation about the y-axis: S = R R y ψ (0,1,0) S R 1 p

33 Spherica Convoution Reca: Thus, we can make the notion of smoothing wedefined by ensuring that the initia rotation about the y-axis does not change the fiter.

34 Spherica Convoution Reca: This means that we can extend the circuar notion of smoothing to the sphere if we ensure that the fiter is symmetric about the y-axis: (0,1,0) Signa Fiter

35 Spherica Convoution Reca: This means that we can extend the circuar notion of smoothing to the sphere if we ensure that the fiter is symmetric about the y-axis: (0,1,0) If R and S are rotations mapping the North poe to p, then the rotation of the fiter by either R or S wi give Signa the same spherica Fiter function!

36 Spherica Convoution Convoution: Using the Euer ange representation, we know that the rotation taking the North poe to the point p = Φ(θ, φ) is the rotation: R θ, φ = R y θ R z φ y z x p θ φ

37 Spherica Convoution Convoution: Thus, given A spherica function f θ, φ A spherica fiter g(θ, φ) that is rotationaysymmetric about the y-axis The convoution of f with g at p = Φ θ, φ can be expressed by rotating g so the North poe gets mapped to p and computing the inner product: f g θ, φ = f, ρ R θ,φ (g)

38 Spherica Convoution Convoution: Expressing the spherica functions f and g in terms of the spherica harmonic basis, we get: f θ, φ = f, m Y m (θ, φ) g θ, φ = m= m= g, m Y m (θ, φ)

39 Spherica Convoution Convoution: Reca that the spherica harmonics can be expressed as a compex exponentia in θ times a poynomia in cos φ: Y m θ, φ = P m cos φ e imθ So a rotation by α degrees about the y-axis acts on the (, m)-th spherica harmonics by: ρ Ry α Y m = e imα Y m

40 Spherica Convoution Convoution: Thus, if the fiter g is rotationay symmetric about the y-axis, any rotation about the y-axis must not change g. That is, for a α we must have: ρ Ry α g = g Or in terms of the spherica harmonics: m= g, m Y m (θ, φ) = m= g, m = g, m e imα g, m e imα Y m θ, φ

41 Spherica Convoution Convoution: g, m = g, m e imα Thus, either: e imα = 1 for a α m = 0, or g, m = 0 Thus, in terms of the spherica harmonics, we get: g θ, φ = g, 0 Y 0 θ, φ

42 Spherica Convoution Y 0 0 θ, φ Im Y 1 1 θ, φ Y 1 0 θ, φ Re Y 1 1 θ, φ Im Y 2 2 θ, φ Im Y 2 1 θ, φ Y 2 0 θ, φ Re Y 2 1 θ, φ Re Y 2 2 θ, φ Im Y 3 3 θ, φ Im Y 3 2 θ, φ Im Y 3 1 θ, φ Y 3 0 θ, φ Re Y 2 1 θ, φ Re Y 3 2 θ, φ Re Y 3 3 θ, φ

43 Spherica Convoution g θ, φ = g, 0 Y 0 (θ, φ) Y 0 0 θ, φ Im Y 1 1 θ, φ Y 1 0 θ, φ Re Y 1 1 θ, φ Im Y 2 2 θ, φ Im Y 2 1 θ, φ Y 2 0 θ, φ Re Y 2 1 θ, φ Re Y 2 2 θ, φ Im Y 3 3 θ, φ Im Y 3 2 θ, φ Im Y 3 1 θ, φ Y 3 0 θ, φ Re Y 2 1 θ, φ Re Y 3 2 θ, φ Re Y 3 3 θ, φ

44 Spherica Convoution Convoution: Thus, the expression for the functions in terms of their spherica harmonic decomposition becomes: f θ, φ = f, m Y m (θ, φ) m= g θ, φ = g, 0 Y 0 (θ, φ) and we get an expression for the convoution: f g θ, φ = f, m Y m 0, ρ R θ,φ g, 0 Y m=

45 Spherica Convoution Convoution: By everaging the conjugate-inearity of the inner product and using the fact that the transformation ρ R is inear, we get: f g θ, φ = f g θ, φ = m=, m= f, m Y m, ρ R θ,φ g, 0 Y 0 f, m g(, 0) Y m, ρ R θ,φ Y 0

46 Spherica Convoution Convoution: Additionay, we know that: A rotation of an -th frequency function wi sti be an -th frequency function The space of -th frequency functions is orthogona to the space of -th frequency functions (if ) Thus, for a, we have: Y m, ρ R Y m = 0

47 Spherica Convoution Convoution: This ets us simpify the expression for the convoution: f g θ, φ = f g θ, φ =, m= m= f, m g(, 0) Y m, ρ R θ,φ Y 0 f, m g(, 0) Y m, ρ R θ,φ Y 0

48 Spherica Convoution Convoution: f g θ, φ = m= f, m g(, 0) Y m, ρ R θ,φ Y 0 To compute the convoution, we need to be abe to evauate the inner product: Y m, ρ R θ,φ Y 0

49 Spherica Convoution Convoution: What is the meaning of the function: Y m, ρ R θ,φ Y 0 This is a function on the sphere whose vaue at the point p = Φ(θ, φ) is the inner product of Y m with the rotation of Y 0, where the rotation takes the North poe to p.

50 Spherica Convoution Convoution: We woud ike to show that this function acts very simpy on the spherica harmonics: Y m, ρ R θ,φ Y 0 = λ Y m θ, φ Im Y 2 2 θ, φ Im Y 2 1 θ, φ Y 2 0 θ, φ Re Y 2 1 θ, φ Re Y 2 2 θ, φ

51 Spherica Convoution Convoution: Let s consider the operator C that maps spherica functions to spherica functions, defined by: C f θ, φ = f, ρ R θ,φ Y 0 As before, it turns out this map is a symmetric inear operator on the space of functions. Thus, there exists an orthonorma basis with respect to which C is diagona.

52 Spherica Convoution Convoution: Let s consider the operator C that maps spherica functions to spherica functions, defined by: C f θ, φ = f, ρ R θ,φ Y 0 This operator aso has the property that it commutes with rotations: Rotating a spherica function and then convoving with Y 0 is the same as first convoving with Y 0 and then rotating.

53 Spherica Convoution Convoution: So, as with the Lapacian, we have a case in which we are given a symmetric operator which commutes with rotations.

54 Spherica Convoution L: a symmetric operator R SO(3): a rotation V λ : the space of e. functions of L with e.vaue λ R L f = L R f λ R f = L R f f V λ R f V λ

55 Spherica Convoution Convoution: Thus, the subspace of -th frequency functions is a space of functions that are eigenvectors of C, a with the same eigenvaue: C Y m = λ, Y m Thus we have: C Y m = Y m 0 if λ otherwise

56 Spherica Convoution Convoution: Putting this a together, we get: Y m, ρ R θ,φ Y 0 = λ Y m (θ, φ) Thus, the equation for the convoution becomes: f g θ, φ = f g θ, φ = m= m= f, m g, 0 Y m, ρ R θ,φ Y 0 f, m g, 0 λ Y m (θ, φ)

57 Spherica Convoution Convoution: f g θ, φ = m= f, m g, 0 λ Y m (θ, φ) Thus, the convoution of f with g can be obtained by mutipying the (, m)-th spherica harmonic coefficients of f by λ g(, 0). As in the case of functions on a circe, this means that convoution in the spatia domain amounts to mutipication in the frequency domain.

58 Spherica Convoution Convoution: In order to be abe to use the convoution theorem for spherica functions, we need to know what the eigenvaues λ are.

59 Spherica Convoution Convoution: In order to be abe to use the convoution theorem for spherica functions, we need to know what the eigenvaues λ are. It turns out that these are: λ = 4π 2 + 1

60 Spherica Convoution Convoution: Which gives us the equation: f g, m = 4π f, m g(, 0) =

61 Outine Math Review Spherica Convoution Axia Symmetry Detection

62 Axia Symmetry Detection Given a spherica function f, we woud ike to compute the measure of the axia symmetry of f with respect to every axis through the origin. f(θ, φ) AxiaSym f (θ, φ)

63 Axia Symmetry Detection What is the measure of the axia symmetry of f about the y-axis? f(θ, φ)

64 Axia Symmetry Detection What is the measure of the axia symmetry of f about the y-axis? We know that f is axiay symmetric about the yaxis if it can be expressed as the sum of the Y 0 : Y 0 0 θ, φ Im Y 1 1 θ, φ Y 1 0 θ, φ Re Y 1 1 θ, φ Im Y 2 2 θ, φ Im Y 2 1 θ, φ Y 2 0 θ, φ Re Y 2 1 θ, φ Re Y 2 2 θ, φ

65 Axia Symmetry Detection What is the measure of the axia symmetry of f about the y-axis? We know that f is axiay symmetric about the yaxis if it can be expressed as the sum of the Y 0. We aso know that for m 0: Y 0, Y m = 0 So the projection onto the space of functions that are axiay symmetric about the y-axis is: π y k m= f, m Y m = f, 0 Y 0

66 Axia Symmetry Detection What is the measure of the axia symmetry of f about the y-axis? Thus, the measure of the axia symmetry of f about the y-axis is defined as: YAxiaSym 2 f = = f, 0 Y 0 f, 0 2 2

67 Axia Symmetry Detection More generay, we woud ike to be abe to compute the measure of the axia symmetry of f with respect to any axis. To compute the symmetry measure about the ine through p = Φ(θ, φ) we: Rotate so that p goes to the North poe, and Compute the symmetry measure about the y-axis. R(p) R p p

68 Axia Symmetry Detection More generay, we woud ike to be abe to compute the measure of the axia symmetry of f with respect Since to the any rotation axis. R(θ, φ) maps the To compute North the poe symmetry to p, the measure rotation we about are the ine through interested p = Φ(θ, in φ) is we: the inverse, R 1 (θ, φ). Rotate so that p goes to the North poe, and Compute the symmetry measure about the y-axis. R(p) R p p

69 Axia Symmetry Detection Using the fact that the spherica harmonics form an orthonorma basis, we know that the (, m)-th harmonic coefficient of f is defined by: f, m = f, Y m Thus, to compute the measure of axia symmetry about the axis through p we need to compute: AxiaSym f 2 θ, φ = ρ R 1 θ,φ f, Y 0 2

70 Axia Symmetry Detection Using the fact that ρ is a unitary representation we can re-write this equation as: AxiaSym f 2 θ, φ = AxiaSym f 2 θ, φ = ρ R 1 θ,φ f, Y 0 2 f, ρ R θ,φ Y 0 2

71 Axia Symmetry Detection Expressing f in terms of its spherica harmonic decomposition, we get: AxiaSym f 2 θ, φ = AxiaSym f 2 θ, φ = m= f, ρ R θ,φ Y 0 2 f(, m) Y m, ρ R θ,φ Y 0 2

72 Axia Symmetry Detection Appying the identity: Y m, ρ R θ,φ Y 0 = 4π Y m θ, φ we get an expression for the symmetry measure: AxiaSym f 2 θ, φ = AxiaSym f 2 θ, φ = 4π m= f(, m) Y m, ρ R θ,φ Y 0 m= f, m Y m θ, φ 2 2

73 Axia Symmetry Detection AxiaSym f 2 θ, φ = 4π m= f, m Y m θ, φ 2 Thus, the measure of axia symmetry can be computed by taking the weighted sum of the squares of the frequency components of f.

74 Axia Symmetry Detection AxiaSym f 2 θ, φ = 4π m= f, m Y m θ, φ 2 Initia Function Frequency Decomposition = = Axia Symmetry Descriptor Weighted Square Norms