A complex onential Fourier transform approach to gradient density estimation Karthik S. Gurumoorthy a,, Anand Rangarajan a, Arunava Banerjee a a Department of Computer and Information Science and Engineering, University of Florida, Gainesville, Florida, USA Abstract We prove a novel result wherein the density function of the gradients of a thrice differentiable function S obtained via a random variable transformation of a uniformly distributed random variable defined on a closed, bounded interval Ω R is accurately approximated by the normalized power spectrum of ϕ = is as the free parameter 0. The result is shown using the well known stationary phase approximation and standard integration techniques and requires proper ordering of limits. Experimental results provide anecdotal visual evidence corroborating the result.. Introduction The literature is replete with techniques which attempt to estimate a non-observable probability density function using observed data believed to be sampled from an underlying distribution. Density estimation techniques have a long history and run the gamut of histogramming, Parzen windows [see Parzen 962], vector quantization, wavelets etc. In the present work, we prove a novel mathematical result relating the density function of the gradients of a function S [viewed as a random variable transformation Y = S X where X is uniformly distributed] with the normalized power spectrum of is as the quantity 0. Corresponding Author Address: E30, CSE Building, University of Florida, P.O. Box 620, Gainesville, FL 326-620, USA. Ph: 00-352-392-200, Fax: 00-352-392-220 Email addresses: ksg@cise.ufl.edu Karthik S. Gurumoorthy, anand@cise.ufl.edu Anand Rangarajan, arunava@cise.ufl.edu Arunava Banerjee Preprint submitted to Statistics and Probability Letters March, 20
We provide a simple Fourier transform-based technique to determine the density of Y. Our method computes the gradient density directly from the function S, circumventing the need to compute its derivative S. The approach is based on ressing the function S as the phase of a wave function ϕ, specifically ϕx = for isx small values of and then considering the normalized power spectrum magnitude squared of the Fourier transform of ϕ [see Bracewell 999]. Using the stationary phase approximation a well known technique in asymptotic analysis [please see Olver 974a] we show that in the limiting case as 0, the power spectrum of ϕ converges to the density of Y and hence can serve as its density estimator at small, non-zero values of. Our new mathematical relationship is motivated by the classical-quantum relation, wherein classical physics is ressed as a limiting case of quantum mechanics [please refer to Griffiths 2004; Feynman and Hibbs 965]. When S is treated as the Hamilton-Jacobi scalar field, the gradient of S corresponds to the classical momentum of a particle [see Goldstein et al. 200]. In the parlance of quantum mechanics, the magnitude square of the wave function ressed either in its position or momentum basis corresponds to its position or momentum density respectively. Since these representations either in the position or momentum basis are simply the suitably scaled Fourier transforms of each other, the magnitude square of the Fourier transform of the wave function ressed in its position basis, is its quantum momentum density [see Griffiths 2004]. The principal theorem proved in the article [Theorem 2.5] states that the classical momentum density denoted by P can be ressed as a limiting case as 0 of its corresponding quantum momentum density denoted by P, in complete agreement with the correspondence principle [see Griffiths 2004]. 2. Equivalence of the gradient density and the power spectrum As stated above, the density function for the gradients of S denoted by Y can be obtained via a random variable transformation of a uniformly distributed random variable X using the derivative S as the transformation function, namely, Y = S X. We assume that S is thrice differentiable on a closed, bounded interval Ω = [b, b 2 ] with length L = b 2 b and has a non-vanishing second derivative almost everywhere 2
on Ω, i.e. µ {x : S x = 0} = 0, 2. where µ denotes the Lebesgue measure. The assumption in 2. is made in order to ensure that the density function of Y exists almost everywhere. This is further clarified in Lemma 2.2 below. Define the following sets: B {x : S x = 0}, C {S x : x B} {S b, S b 2 }, and A u {x : S x = u}. 2.2 Here, S b = lim x b + S x and S b 2 = lim x b S x. The higher derivatives 2 of S at the end points b, b 2 are also defined along similar lines using one-sided limits. The main purpose of defining these one-sided limits is to exactly determine the set C where the density of Y is not defined. Since µb = 0, we also have µc = 0. Lemma 2.. [Finiteness Lemma] A u is finite for every u / C. Lemma 2.2. [Density Lemma] The probability density of Y on R C exists and is given by P = L Nu 0 k= S x k, 2.3 where the summation is over A u0 which is the finite set of locations x k Ω where S x k = as per Lemma 2., with A u0 = N. Since the density is based on the transformation Y = S X, the probability density function in 2.3 assumes the existence of the inverse of the transformation function S [see Billingsley 995]. This is made licit in the following lemma which is required by the main theorem. Lemma 2.3. [Interval Lemma] For every u / C, η > 0 and a closed interval J η = [u η, u + η] such that J η C is empty. The proofs of the lemmas are available in AppendixA. We now prove the main result which relates the normalized power spectrum of in the limit as 0 with the probability density of the random variable isx Y = S X denoted by P. 3
Define a function F : R R + C as ˆ b2 isx F u, 2πL For a fixed value of, define a function F : R C as b iux dx. 2.4 F u F u,. 2.5 Observe that F is closely related to the Fourier transform of isx. The scale factor 2πL is the normalizing term such that the L 2 norm of F is one as seen in the following lemma whose proof is straightforward and omitted for lack of space. Lemma 2.4. With F defined as above, F L 2 R and F =. Define a function P : R C R + as By definition, P P u F u 2 = F uf u. 2.6 0. Since µc = 0, from Lemma 2.4, P udu =. Hence, treating P u as a density function, we have the following theorem statement. Theorem 2.5. If P and P are defined as above, then the ˆ u0 +α lim α 0 α lim P udu = P, / C. 2.7 0 Before embarking on the proof, we would like to emphasize that the ordering of the limits and the integral as given in the theorem statement is crucial and cannot be arbitrarily interchanged. To press this point home, we show below that after solving for P, the lim 0 P does not exist. Hence, the order of the integral followed by the limit 0 cannot be interchanged. Furthermore, when we swap the limits between α and, we get lim lim 0 α 0 α ˆ u0 +α P udu = lim 0 P 2.8 which does not exist. Hence, the theorem statement can be valid only for the specified sequence of limits and the integral. 4
2.. Brief osition of the result To understand the result in simpler terms, let us reconsider the definition of the scaled Fourier transform given in 2.4. The first onential is is a varying complex sinusoid, whereas the second onential iux is a fixed complex sinusoid at frequency u. When we multiply these two complex onentials, at low values of, the two sinusoids are usually not in sync and tend to cancel each other out. However, around the locations where S x = u, the two sinusoids are in perfect sync as the combined onent is stationary with the approximate duration of this resonance depending on S x. The value of the integral in 2.4 can be approximated via the stationary phase approximation [please see Olver 974a] as F u ±i π L 4 Nu k= { } i Sx k ux k S x k 2.9 where Nu = A u. The approximation is increasingly tight as 0. The squared Fourier transform P gives us the required result L Nu k= S x k except for the cross phase factors Sx k Sx l ux k x l obtained as a byproduct of two or more remote locations x k and x l indexing into the same frequency bin u, i.e, x k x l, but S x k = S x l = u. Integrating the squared Fourier transform over a small frequency range [u, u + α] removes these cross phase factors and we obtain the desired result. 2.2. Formal proof We shall now provide the proof by considering different cases. case i: Let us consider the case in which no stationary points exist for the given, i.e, there is no x Ω for which S x =. Let tx = Sx x. Then, t x is of constant sign in [b, b 2 ] and hence tx is strictly monotonic. Defining v = tx, we have from 2.4, ˆ tb2 iv F = gvdv. 2.0 2πL tb 5
Here, gv = t x where x = t v. Integrating by parts, we get F 2πL = i i [ ˆ tb2 tb itb2 g tb 2 Then F 2πL S b 2 + S b + itb ] g tb iv g vdv. 2. ˆ tb2 tb g v dv. 2.2 Hence, F γ, where γ > 0 is a continuous function of. Then P γ 2. Since S x is continuous and / C, we can find a ρ > 0 such that for every u [ ρ, + ρ], no stationary points exist. The value ρ can also be chosen appropriately such that [ ρ, + ρ] C =. If α < ρ, then ˆ u0 +α lim P udu = 0. 2.3 0 Furthermore, from 2.3 we have P = 0. The result immediately follows. case ii: We now consider the case where stationary points exist. Since we are interested only in the situation as 0, the stationary phase method in Olver 974a,b can be used to obtain a good approximation for F defined in 2.4 and 2.5. The phase term in this function, Sx x, is stationary only when S x =. Consider the set A u0 defined in 2.2. Since it is finite by Lemma 2., let A u0 = {x, x 2,..., x Nu0} with x k < x k+, k. We break Ω into disjoint intervals such that each interval has utmost one stationary point. To this end, consider numbers {c, c 2,..., c Nu0 +} such that b < c < x, x k tx = Sx x. Then, < c k+ < x k+ and x Nu0 < c Nu0+ < b 2. Let F Nu 0 2πL = G + G 2 + K k + K k 2.4 k= 6
where G = G 2 = K k = K k = ˆ c b ˆ b2 itx dx, itx itx itx c Nu0 + ˆ xk c ˆ k ck+ x k dx, dx, and dx. 2.5 Note that the integrals G and G 2 do not have any stationary points. From case i above, we get G + G 2 = ϵ, = O 2.6 as 0. Furthermore, ϵ, γ 2, where γ 2 > 0 is a continuous function of. In order to evaluate K k and K k, observe that when we and the phase term up to second order, tx tx k Qx k x x k 2 as x x k, where Qx k = S x k 2. Furthermore, in the open intervals c k, x k and x k, c k+, t x = S x is continuous and is of constant sign. From Theorem 3. in Olver 974a, we get K k = K k = 2 ± iπ 4 Γ 2 itxk 2 S x k + ϵ 2,. 2.7 From Lemma 2.3 in Olver 974a, it can be verified that ϵ 2, = o as 0 and can also be uniformly bounded by a function of independent of for small values of. In 2.7, Γ is the Gamma function and the sign in the phase term depends on whether S x k > 0 or S x k < 0. Plugging the values of these integrals in 2.4 and noting that Γ 2 = π, we get F 2πL = N k= i [Sx 2π k x k ] S x k ± iπ 4 +ϵ, + ϵ 2,. 2.8 Hence, F = N L k= i [Sx k x k ] ± iπ S x k 4 2.9 +ϵ 3,, 2.20 7
where ϵ 3, = ϵ,+ϵ 2, 2πL = o as 0. From the definition of P u in 2.6, we have P = L + L N k= Nu 0 k= S x k Nu 0 l=;l k cos [Sx k Sx l x k x l ] + θx k, x l S x k 2 S x l 2 +ϵ 4,, 2.2 where ϵ 4, includes both the magnitude square of ϵ 3, and the cross terms involving the main first term in 2.9 and ϵ 3,. Notice that the main term in 2.9 can be bounded by a function of independent of as i [Sx k x k ] =, 2.22 and S x k 0, k. Since ϵ 3, = o, we get ϵ 4, = o as 0. Additionally, θx k, x l = 0, π 2 or π 2 and θx l, x k = θx k, x l. The first term in 2.2 exactly matches the ression for P as seen from Lemma 2.2. But, since lim 0 cos is not defined, lim 0 P does not exist and hence the cross cosine terms do not vanish when we take the limit. We now show that integrating P u over a small non-zero interval [, + α] and then taking the limit with respect to followed by the limit with respect to α does yield the density of Y. From Lemmas 2. and 2.3, we see that for a given a A u0, when u is varied over the interval J η = [ η, + η], the inverse function S u is well defined with S u N a, where N a is some small neighborhood around a. For each a A u0, define the inverse function S a u : J η N a as S a u = x iff u = S x and x N a. 2.23 Unfortunately, when we move from a fixed value to a variable u defined in the interval J η, the locations x k and x l which previously were fixed in 2.2 now also vary over the interval J η. This makes the notation somewhat cumbersome and unwieldy. Using the inverse functions defined in 2.23 and defining a k x k [and consequently 8
a l = x l ], we define x k u S a k u [and therefore xl u = S a l u] for u J η. Finally, define the functions p u and q u over J η as Observe that p u S x k u S x l u u x k u x l u, and 2.24 q u. S x k u 2 S x l u 2 2.25 p u = S x k ux ku S x l ux lu x k u x l u u x ku x lu = x l u x k u 2.26 as u = S x k u = S x l u. In particular, if x l > x k, then x l u > x k u and vice versa. Hence, p u never vanishes and is also of constant sign over J η. Then, it follows that p u is strictly monotonic and specifically bijective on J η. We will use this result in the subsequent steps. Now, let α < η. Since the additional error term ϵ 4, in 2.2 converges to zero as 0 and can also be uniformly bounded by a function of for small values of, we have Then, we get where I L I 2 L Here, I 3 k, l is given by I 3 k, l ˆ u0+α ˆ u0+α lim 0 ϵ 4, du = 0. 2.27 ˆ u0 +α lim 0 P udu = I + I 2 2.28 N k= N k= ˆ u0 +α S du, and 2.29 x k u N l=;l k lim I 3k, l. 2.30 0 [ ] p u q u cos + θ x k u, x l u du. 2.3 When α < η, the sign of S x k u around x k and the sign of S x l u around x l do not change over the interval [, + α]. Since θx k u, x l u depends on the sign of S, θ is constant on [, +α] and equals θ = θx k, x l. 9
Now, anding the cosine term in 2.3, we get ˆ u0+α p u I 3 k, l = cosθ q u cos ˆ u0 +α p u sinθ q u sin Since p u is bijective, we get via a standard change of variables: du du. 2.32 I 3 k, l = cosθ I 4 k, l sinθ I 5 k, l 2.33 where I 4 k, l = I 5 k, l = ˆ β2 β ˆ β2 β v cos g vdv, and 2.34 v sin g vdv. 2.35 Here, β = p, β 2 Integrating I 4 k, l by parts, we get β 2 I 4 k, l = sin g Then, I 4 k, l g = p + α and g v = q p v p p v. ˆ β2 β β 2 β 2 sin β g β v sin g vdv. 2.36 + g β ˆ β2 + g v dv. 2.37 β It is worth mentioning that q and hence g are differentiable over their respective intervals as the sign of S x k us x l u does not change over the interval [, + α]. We then have I 4 k, l M where M is some constant independent of. Hence, lim 0 I 4 k, l = 0, k, l. By a similar argument, lim 0 I 5 k, l = 0, k, l. From 2.30 and 2.33, we get I 2 = 0. Since lim α 0 α I = Nu 0 L S x k = P, 2.38 k= the main result ressed in Theorem 2.5 follows. 0
Below, we show comparisons between our Fourier transform approach and the standard histogramming technique on some trigonometric and onential functions sampled on a regular grid between [ 0.25, 0.25] at a grid spacing of 2 5. For the sake of convenience, we normalized the functions such that its maximum gradient value is. Using the sampled values Ŝ, we computed the Fast Fourier transform of iŝ at = 0.0000, took its magnitude squared and then normalized it to compute the gradient density. We also computed the discrete derivative of S at the grid locations and then determined its gradient density by histogramming using around 220 bins. The plots shown in Figure provide anecdotal empirical evidence supporting the mathematical result stated in Theorem 2.5. Notice the near-perfect match between the gradient densities computed via standard histogramming and the gradient densities determined using our Fourier transform method. Figure : Comparison results. i Left: Gradient densities obtained from histogramming, ii Right : Gradient densities obtained from squared Fourier transform of the wave function 3. Discussion Observe that the integrals I = ˆ u0+α ˆ u0+α P udu, I = P udu 3.
give the interval measures of the density functions P and P respectively. Theorem 2.5 states that at small values of, both the interval measures are approximately equal, with the difference between them being o. Recall that by definition, P ϕx = isx is the normalized power spectrum [see Bracewell 999] of the wave function. Hence, we conclude that the power spectrum of ϕx can potentially serve as a density estimator for the gradients of S at small values of. The erimental results shown above serve as a demonstration, anecdotally attesting to the verity of the result. Our result is directly inspired by the three-way relationships between the classical momentum S, the quantum momentum operator i x and its spatial frequency spectrum. Since these relationships hold in higher dimensions as well, we are likewise interested in extending our density estimation result to higher dimensions. This is a fruitful avenue for future work. References Billingsley, P., 995. Probability and measure, 3rd Edition. Wiley-Interscience. Bracewell, R., 999. The Fourier transform and its applications, 3rd Edition. McGraw- Hill Science and Engineering. Feynman, R., Hibbs, A., 965. Quantum mechanics and path integrals. McGraw-Hill. Goldstein, H., Poole, C., Safko, J., 200. Classical mechanics, 3rd Edition. Addison- Wesley. Griffiths, D., 2004. Introduction to quantum mechanics, 2nd Edition. Benjamin- Cummings. Olver, F., 974a. Asymptotics and special functions. Academic Press, New York and London. Olver, F., 974b. Error bounds for stationary phase approximations. SIAM Journal of Mathematical Analysis 5, 9 29. 2
Parzen, E., 962. On the estimation of a probability density function and the mode. Annals of Mathematical Statistics 33, 065 076. Rudin, W., 976. Principles of mathematical analysis, 3rd Edition. McGraw-Hill. AppendixA. Proof of Lemmas. Proof of Finiteness Lemma We prove the result by contradiction. Observe that A u is a subset of the compact set Ω. If A u is not finite, then by Theorem 2.37 in Rudin 976, A u has a limit point x 0 Ω. Consider a sequence {x n } n=, with each x n A u, converging to x 0. Since S x n = u, n, from the continuity of S we get S x 0 = u and hence x 0 A u. Then S x 0 S x n lim = 0 = S x 0. A. n x 0 x n Based on the definitions given in 2.2, we have x 0 B and hence u C resulting in a contradiction. 2. Proof of Interval Lemma Observe that B is closed because if x 0 is a limit point of B, from the continuity of S we have S x 0 = 0 and hence x 0 B. B is also compact as it is a closed subset of Ω. Since S is continuous, C = S B {S b, S b 2 } is also compact and hence R C is open. Then for u / C, there exists an open neighborhood N r u for some r > 0 around u such that N r u C =. By defining η = r 2, the proof is complete. 3. Proof of Density Lemma Since the random variable X is assumed to have a uniform distribution on Ω, its density is given by f X x = L for every x Ω. Recall that the random variable Y is obtained via a random variable transformation from X, using the function S. Hence, its density function exists on R C where we have banished the image under S of the measure zero set of points where S vanishes and is given by 2.3. The reader may refer to Billingsley 995 for a detailed lanation. 3