The Gaussian transform. ALECU, Teodor, VOLOSHYNOVSKYY, Svyatoslav, PUN, Thierry. Abstract

Transcription

1 Proceedings Chapter The aussian transform ALECU, Teodor, VOLOSHYNOVSKYY, Svyatoslav, PUN, Thierry Abstract This paper introduces the general purpose aussian Transform, which aims at representing a generic symmetric distribution as an infinite mixture of aussian distributions We start by the mathematical formulation of the problem and continue with the investigation of the conditions of existence of such a transform Our analysis leads to the derivation of analytical and numerical tools for the computation of the aussian Transform, mainly based on the Laplace and Fourier transforms, as well as of the afferent properties set (eg the transform of sums of independent variables Finally, the aussian Transform is exemplified in analytical form for typical distributions (eg aussian, Laplacian, and in numerical form for the eneralized aussian and eneralized Cauchy distributions families Reference ALECU, Teodor, VOLOSHYNOVSKYY, Svyatoslav, PUN, Thierry The aussian transform In: Proceedings of the 3th European Signal Processing Conference, EUSIPCO 5 5 p -4 Available at: Disclaimer: layout of this document may differ from the published version

2 THE AUSSIAN TRANSFORM Teodor Iulian Alecu, Sviatoslav Voloshynovskiy and Thierry Pun Computer Vision and Multimedia Laboratory, University of eneva, 4 Rue énéral-dufour, 4 eneva, Switzerland phone: + ( , fax: + ( , TeodorAlecu@cuiunigech ABSTRACT This paper introduces the general purpose aussian Transform, which aims at representing a generic symmetric distribution as an infinite mixture of aussian distributions We start by the mathematical formulation of the problem and continue with the investigation of the conditions of existence of such a transform Our analysis leads to the derivation of analytical and numerical tools for the computation of the aussian Transform, mainly based on the Laplace and Fourier transforms, as well as of the afferent properties set (eg the transform of sums of independent variables Finally, the aussian Transform is exemplified in analytical form for typical distributions (eg aussian, Laplacian, and in numerical form for the eneralized aussian and eneralized Cauchy distributions families INTRODUCTION aussian distributions are extensively used in the (broad sense signal processing community, mainly for computational benefits For instance, in an estimation problem aussian priors yield quadratic functionals and linear solutions In rate-distortion and coding theories, closed form results are mostly available for aussian source and channel descriptions Real data, however, is generally not aussian distributed The goal of the work presented in this paper is to describe non-aussian distributions through an infinite mixture of aussian distributions In a related work [4], it was proven that any distribution can be approximated through a mixture of aussian up to an arbitrary level of precision However, no hint was given by the author on how to obtain the desired mixture in the general case In [5], an analytical formula is given for an infinite mixture of aussians equivalent to the Laplacian distribution, and used in a source coding application Unfortunately, no generalization was attempted by the authors The work presented here has the roots in their proof and extends the concept to a wide range of symmetric distributions through the introduced aussian Transform The aussian Transform concept and the results presented in this paper can be extensively used in various applications of signal and image processing and communications including estimation, detection, source and channel coding etc The rest of the paper is divided in two main blocks In Section we define the aussian Transform, analyze its existence, investigate its properties and derive the mathematical tools for analytical and/or numerical computation, whereas in Section 3 we exemplify both the transform for some typical distributions such as eneralized aussian and eneralized Cauchy, and some of the properties deduced in Section AUSSIAN TRANSFORM Definition and existence We consider a generic symmetric distribution p(x As we are aiming at representing it through an infinite mixture of aussians, we can safely disregard the mean, and assume for simplicity reasons that p(x is zero-mean We are looking for an integral representation in the form: ( N ( x d p( x, ( where N ( x is the zero-mean aussian distribution: x N ( x e, π and ( is the mixing function that should reproduce the original p(x We can now introduce the aussian Transform Definition (aussian Transform The direct aussian Transform is defined as the operator which transforms p( x into, and the Inverse aussian Transform - is defined as the operator which maps ( to p( x : : p( x ( ; : ( p( x Obviously, - is simply given by (: ( ( N x d ( We look now for the direct aussian Transform We need to prove that for a given p x a mixing distribution exists such as to comply with ( This can be summarized in three conditions Condition For a given p(x, a function ( defined according to ( exists Condition This function is non-negative Condition 3 Its integral ( d is equal to The last condition is a consequence of Condition Indeed, if exists, then, integrating both sides of ( with respect to x and inverting the integration order of the left side:

3 ( N ( x d dx p( x dx, thus ( N ( x dxd Finally, since ( x ( N is a distribution: d In order to investigate Condition, the existence of (, perform the following variable substitutions: s x and Since p(x is symmetric, it can be rewritten as: p( x p( x p( s The left hand side of ( transforms to: ( ( st N x d e dt tt π t According to the definition of the Laplace Transform L [], equation ( finally takes the form: L p( s (3 tt π t Thus, ( is linked to the original probability distribution p(x through the Laplace Transform and can be computed using the Inverse Laplace Transform L - The direct aussian Transform is therefore given by: π ( p( x ( p( s ( t L (4 t Consequently, the existence of the aussian Transform is conditioned by the existence of the Inverse Laplace Transform of p( s From the general properties of the Laplace Transform, it is sufficient [] to prove that the limit at infinity of s p( s is bounded, or equivalently : lim x x p x < (5 The above condition is satisfied by all the distributions from the exponential family, as well as by all the distributions with finite variance (4 allows for straightforward identification of aussian Transforms for distributions whose Laplace Transforms are known, by simply using handbook tables Unfortunately, it does not guarantee compliance with the Condition : non-negativity As it is rather difficult to verify a priori this constraint, the test should be performed a posteriori, either analytically or numerically Properties of the aussian Transform We derive the first property of the aussian Transform using the initial value theorem for the Laplace Transform [], the direct formula (4 and the existence condition (5 Final Value Property The aussian Transform tends asymptotically to when tends to infinity: ( lim (6 We also expect the transform to preserve the data variance Mean Value Property The mean value of the aussian Transform is equal to the variance of the original distribution: x p( x dx ( d (7 x p( x dx x ( N ( x d dx Proof: N x x dx d d qed Another crucial property of the aussian Transform applies to the transform of the sum of independent variables Convolution Property If and are independent random variables with ( p ( x ( and p x, then the aussian Transform of their sum is the convolution of their respective aussian Transforms (the result can be generalized for the sum of multiple independent variables: ( p + x + x (8 Proof: consider the random variable + Since exists, is a random aussian variable with variance distributed according to the distribution probability Similarly, is a random aussian variable with variance distributed according to Then is also a random aussian variable with variance + But and are independent variables drawn from and It follows that is a random variable described by the probability distribution, qed Corollary If and are independent random variables and is aussian distributed p ( x N x, then the aussian transform of their sum is a shifted version of :, < + (9 (, 3 Numerical computation The computation of the aussian Transform for distributions not available in handbooks is still possible through the complex inversion method for Laplace Transforms known as the Bromwich integral [3]: ( ε + i ut L f s f ( u e du π i, ( ε i where ε is a real positive constant satisfying ε > sup Re poles f and u is an auxiliary variable ( ( ( can be rearranged as a Fourier Transform, allowing the use of the numerous numerical and/or symbolical packages available (with ω the variable in the Fourier space:

4 εt ( f ( s e ( f ( ε + iω L F ( Very often p(x has no poles, being a continuous and bounded function, and in this case it might be very practical to evaluate ( at the limit ε Using (4 and (: π ( ( F ( p( iω ( When the original distribution is only numerically known, approximation of ( is still possible either through analytical approximations of p(x, followed by (4 (or (, or through solving the inverse problem yielded by ( 3 EAMPLES OF AUSSIAN TRANSFORMS L λ λx L λ λ s p x λ e ; p s λ e Laplacian: λ L λ ( p ( s λ e As mentioned, the result (4 was already proven in [5] C b C b p ( x b ; p ( s b π b + x π b + s Cauchy: b C ( p ( x b e π The results (3, (4 and (5 are plotted in Fig (4 (5 3 Analytic aussian transforms The most obvious and natural example is the aussian Transform of a aussian distribution One would expect to have (with δ the Dirac function: ( N ( x δ ( (3 Indeed, from (4: s t ( N ( x t L e Then, using the Laplace Transform tables: ( N ( x δ t Since the aussian Transform is a function of []: δ ( ( δ, and ( ( x ( ( δ δ N Thus the aussian Transform of a aussian distribution N x is a Dirac function centered in ( The convolution property (8 can now be used to prove a well known result in statistics: the sum of two independent aussian variables, with respective probability laws x N x, is another aussian variable N ( and ( with probability distribution ( x + N (the extension to non-zero mean distributions is trivial Proof: ( p δ ( ; p δ thus ( p * ( + p p δ + Then, inverting the aussian Transform: p + δ + N x + ( ( Similarly to (3, it is possible to compute the aussian transforms of other usual symmetric distributions using the Laplace transform tables [] We exemplify with the Laplacian and Cauchy distributions Figure aussian Transforms To exemplify the Convolution Property, consider now Cauchy data contaminated with independent additive white aussian noise Then the aussian Transform of the measured data (Fig 3 is a shifted version of the original data transform (9 Figure 3 Shift of the aussian Transform The previous analytical results will be generalized in subsection B through numerical computations 3 Numeric aussian transforms This part illustrates the numerical computation of the aussian Transform through ( for the eneralized aussian and eneralized Cauchy distributions families eneralized aussian Distribution (D The D family is described by an exponential probability density function with parameters and : η ( x p ( x, e η, Γ ( where η( 3 ( ( Γ Γ For the D particularizes to the Laplacian distribution, while for one obtains the aussian distribution The aussian Transform of the eneralized aussian distribution can not be obtained in

5 analytical form using (4 However, it does exist (5 and can be calculated numerically through (: ( ( π ( F ( ω, p i, η ( η ( p ( iω e ( iω, Γ ( The aussian Transforms for ranging from 5 to with fixed are plotted in Fig 6 The transforms evolve from a Dirac-like distribution centered on for small to exponential for, then Rayleigh-like for, bellshaped for 5 and again Dirac-like centered on for As expected, the aussian Transform of the Laplacian distribution ( is exponential (4 Corresponding plots are given in Fig7 Figure 7 aussian Transforms of CD As a remark, the variance of the Cauchy distribution being infinite, its aussian Transform has infinite mean value (7 The CD family possesses finite variance only for ν > 4 CONCLUSION We introduced in this paper the aussian Transform, which allows for the representation of symmetric distributions as an infinite aussian mixture The scope of applicability of the aussian Transform is potentially very broad, from denoising and regularization to filtering, coding,compression, watermarking etc However, an extension of the concept to nonsymmetric distribution would be required for some specific applications Further investigation of the existence conditions, especially non-negativity, is also necessary Finally, one would need adapted numerical packages (most likely based on existing Laplace and Fourier transform computational packages for the computation of aussian Transforms of both analytically and numerically defined distributions ACKNOWLEDMENT This work is supported in part by the Swiss NCCR (National Centre of Research IM (Interactive Multimodal Information Management, and the EU Network of Excellence Similar ( The authors thank Dr Oleksiy Koval for careful reading and fruitful discussions Figure 6 aussian Transforms of D Unfortunately, the real part of the complex probability function diverges for periodical values of, which impedes on the computation of the transform through this method for > However, real data is mostly confined to << [6] eneralized Cauchy Distribution (CD The CD probability density function is given by Γ ( ν + ν C b px x ν, b, ν + 5 πγ ( ν ( b + x and it particularizes to the Cauchy distribution for ν 5 Its aussian Transform can be computed through: ν b Γ ( ν + 5 ν 5 ν, b( ( F (( b + iω Γ ν REFERENCES A D Poularikas, S Seely, Laplace Transforms The Transforms and Applications Handbook: Second Edition Ed Alexander D Poularikas Boca Raton: CRC Press LLC, Arfken, Mathematical Methods for Physicists, 3rd ed Orlando, FL: Academic Press, pp , Arfken, "Inverse Laplace Transformation" 5 in Mathematical Methods for Physicists, 3rd ed Orlando, FL: Academic Press, pp , R Wilson, MMM: Multiresolution aussian Mixture Models for Computer Vision, in IEEE International Conference on Pattern Recognition, Barcelona, Spain, September 5 A Hjørungnes, J M Lervik, and T A Ramstad, Entropy Coding of Composite Sources Modeled by Infinite Mixture aussian Distributions, in Proc 996 IEEE Digital Signal Processing Workshop, Loen, Norway, September S Mallat, A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, in IEEE Transactions on Pattern Analysis and Machine Intelligence Volume, Issue 7 (July 989 p