SPECKLE: MODELING AND FILTERING. Torbjørn Eltoft. Department of Physics, University of Tromsø, 9037 Tromsø, Norway

Size: px
Start display at page:

Download "SPECKLE: MODELING AND FILTERING. Torbjørn Eltoft. Department of Physics, University of Tromsø, 9037 Tromsø, Norway"

Transcription

1 F % F C SPECKLE: MODELNG AND FLTERNG Torbjørn Eltoft Department of Physics, University of Tromsø, 93 Tromsø, Norway ASTRACT A new statistical distribution for modeling non-rayleigh amplitude statistics, which is formed as a compound distribution of a the Rician distribution and the nverse Gaussian (G), is introduced. This new model has been called the Rician nverse Gaussian (RiG) distribution. The theoretical basis of the model is briefly presented, and we give an algorithm for estimating its parameters from data. The modeling capabilities of the RiG distribution is tested by fitting the model to s generated from segments of a medical ultrasound image and a single-look SAR image. Finally, a maximum a posteriori (MAP) speckle filter has been constructed using the RiG model. The filtering performance of this filter is shown to be excellent. 1. NTRODUCTON Modeling non-rayleigh amplitude statistics in coherent imaging systems has been a research area for almost three decades, and many interesting distributions have been proposed. Some of these are the Rice-distribution, the K-distribution and the homodined K-distribution. Accurate statistical models for the amplitude statistics are important both with respect to characterization/classification of image regions [1], and with respect to speckle filtering [2, 3]. n this paper we introduce a new model for modeling the amplitude statistics of coherent imagery, which made up as a compound distribution of a the Rician distribution and the nverse Gaussian (G) [4]. We have for this reason denoted this new distribution the Rician nverse Gaussian (RiG) distribution. We give some details of the theoretical basis of this new model, describe how its parameters can be estimated, and test its ability to fit to s of segments of ultrasound images. We include comparisons with the well-known K-distribution. We also develop a maximum aposteriori speckle filter based on the RiG model, and show its filtering performance. 2. REVEW OF SPECKLE MODELS f the scattering medium is modeled as a collection of discrete scatterers, we may write the detected signal as is the number of scatterers, (1) where is the amplitude and is the phase of the! th scatterer. i: Now, if the scatterers phases are random and independent of the amplitudes, and if the number is very large, the probability density function (pdf) of, the amplitude of, will be a Rayleigh distribution "$#&% ( )+ -,/.1 32 (2) where ) is the the standard deviation of the Gaussian distributed 4 - and 5 -components of. This distribution represents the fully-developed speckle case, or the diffuse scattering signal. ii: f a specular component 6 is added to (1), the pdf of the amplitude gets Rician, i.e. "8#&% ( )+ -,. :9-;-< =2 4> % 6 )+ (3) where 4 > is the modified essel function of first kind and zero order. iii: The well-known -distribution [5, 6] results, if we allow the variance of the Rayleigh-distribution to itself being a random variable, distributed according to a A -distribution. n this case, the amplitude of is a compound variable of a Rayleigh variable and a A -distributed variable. The pdf is explicitly given as where %HG and order D, C " # % ( -C A %ED C 1F, % ( (4) is the modified essel function of second kind J D$KML N is a scale parameter. D is a shape parameter. iv: The homodyned K-distribution results when a coherent

2 D L G T > % % component, 6, is added to the diffuse scattering signal in (1), and we allow the number of terms in the summation,, to itself being random following a negative binomial distribution. The resulting pdf, which can not be obtained in closed form, is given by the integral "$# % ( > % > % 6 > % )+ K ( F The models reviewed above have some shortcomings: the Rayleigh model certainly is a proper model for fully developed speckle, but this case does not often occur in highresolution imagery. Also, the K-model has its limitations, because, often the signal has a specular component, resulting in bad fits to the data. The homodyned model has good modeling capabilities, but its use is restricted by the lack of a tractable pdf. 3. NEW MODEL We will in the following introduce a unifying model for non-rayleigh amplitude data, which is able to overcome the problems mentioned above. n addition, its parameters can easily be estimated from data. When the amplitude of a detected signal, modeled as in (1), is non-rayleigh, this implies that the 4 - and 5 - components of the complex signal deviate from Gaussian statistics. We propose to use the Normal nverse Gaussian (NG) distribution to model these components. A NGdistribution is defined as a variance-mean mixture of a normal distribution with the inverse Gaussian (4 ) as a mixing distribution []. The probability density function is explicitly given as where " % and %HG "% $(% (! #" % " % $(% 1 %&$(% 1 ) + % ) % ) + (5) (6) is the modified essel function of the second kind and order. As noted, the distribution is characterized by the four parameters %,, which are bounded such that -/.1 2.,, and )3 L4 L 3. controls the steepness of the distribution; a small means a steep distribution, a larger means a less steep distribution. determines the skewness; L - implies skewed to the left, N - implies skewed to the right, and - implies a symmetric distribution. is a location parameter, and is a dispersion parameter similar to the variance in the Gaussian distribution. Let be defined by the relation () where :9<; % - =. Hence, the conditional distribution 6 is normal ; % >J,. The marginal distribution of 6 is nverse Gaussian (G) with parameters and A). This probability density is given by " %, =C"%,EDF = #" % ) (8) The G-distribution was first introduced as the first passage time distribution of rownian motion with drift. t has recently attained increasing interest in signal processing and statistics (financial) due to its ability to model highly skewed densities, and its many analogies with the Gaussian distribution. ts mean and variance are GH 6KJ ML N and OPH 6KJ N&Q, respectively. The distribution is well documented in the literature [4]. Consider the case when the - and -components of in (1) are uncorrelated and 2-dimensional NG-distributed, i.e: 9 4R % - ES, and 9 4R % -, ET. Note that we assume that both S and UT are zero. Accordingly, and are both compound random variables of the form: > S 6 6 S and > T 6 6 T, where V % S T 9W; %XY, with Y [Z 4 ( Z -dimensional identity matrix.), and % ]\ ) % S ^ 4 is the, with. t is now easy to show that the distribution of the envelope, conditioned on 6, i.e. 6, is Rician distributed, with pdf given by " #`_ a % where \ S ^! #" % ) ^ 4> % 1 ( (9) T. From (9) we observe that the skewness parameters of the NG-distributions will have the effect of a coherent component on the amplitude statistics. Now, the amplitude will accordingly be a compound variable consisting of the Rice-distribution in (9), and the G( ) distribution, i.e. " # % ( " #`_ a % " a % (1) The integral in (1) can be evaluated to give the following closed form pdf for : "8# % ( /b D,F c " % % D,Fd D,F % 4> % ( (11) We call this distribution the Rician nverse Gaussian (RiG) distribution. As can be seen, the pdf has three parameter.

3 L 9 1 ) [1,.98, 3 ] [1,.8, 5 ] [1,.6, 2 ] [1,.8, 1 ] Fig. 1. Examples of pdfs residing to the RiG model. The numbers in the brackets correspond to,,, respectively Fig.1 displays some realizations of the pdf for some selected values of the parameters. The numbers in brackets in the upper right corner of Fig.1 give the values of,, and for each curve. We observe that by varying the parameters,, and, we may model a wide range of distributions, from a Rayleigh-shaped pdf to a Gaussian-shaped pdf. 4. PARAMETER ESTMATON The parameters of the RiG distribution can be estimated using an EM-type algorithm. n the previous section we assumed that the speckle-free image, represented by the vari- distribution with parameters able, follows an G % and. ts th-order moments are given by GH 6 J %,, N L From (12) we easily find that GH 6 J and G G 6 GH 6KJ, a are: (12) (13) (14) t is further noted that the posterior distribution 6 is given as: " a_ # % ( _ ; ;, Q &% Q! L )( " $# + Q! L ", = #"- ) EL " #.# (15) This is the pdf of a generalized inverse Gaussian (GG) []. variable specified by the parameters % ) D The moments of a GG % +/ distribution are given by [] GH 6 J - % " % (16) Let 1. We thus have that G8H 6 KJ (1) $2 G 6 1 % &% & 1 (18) Now, given N independent observations! - GG G+. From some initial values for,, and, we may estimate G8H 6KJ as and G6 6 a as N L 6 N % / 53 &% & 1 Using (13) and (14), our estimators for and are : 89 ; L 6 N < a>= L56 N (19) (2) (21) (22) respectively. The parameter is estimated using the maximum likelihood method. Using previous estimates of,, and, define A % A % as DCFEHG "8#&% is defined in (11). Our estimate for is ac- where " # % cordingly given as arg max JKA % + (23) This maximization must be performed numerically. 5. MODELNG DATA (24) Fig.2 shows a medical ultrasound image of the human heart. Two test regions are seen as marked rectangles. The s of the amplitudes of the test regions have been displayed in Fig.3, along with the best fitted RiG- and K- models. The parameters of the K-models were estimated

4 ) Fig. 2. Ultrasound image (log-compressed). The marked areas define the image regions which the RiG-model was tested on Fig. 4. Single-look SAR image (displayed log-compressed). The marked areas define the image regions which were tested against the K and the RiG models..4 RiG model K model.12.1 RiG model K model.2.18 RiG K.3 RiG K Fig. 3. Fitted RiG-model (solid), (thin solid) and fitted K-model (dotted-dashed) corresponding to test regions Fig. 5. Fitted RiG-model (solid), (thin solid) and fitted K-model (dotted-dashed) corresponding to test regions using a moment based estimator, or by matching the higherorder moments of the data. We note that the RiG-model fits the s of the data much better than the K-model. Fig.3 also demonstrates the high flexibility of the the RiG model. Fig.4 shows a single-look SAR image of a mountainous area in southern Norway. The same conclusion drawn from the analysis of the the ultrasound data, is valid for the SAR data. The RiG model also has a very good fit to the s of the SAR data, as can be noted from fig MAP-FLTERNG The problem in ayesian de-speckling is the following: We want to estimate the amplitude, where is the spatial location, which is most likely to occur in a specklefree image, given an observed amplitude value. This is known as MAP-filtering, and neglecting the spatial coor- dinates the estimation is stated in mathematical terms as " a_ # % arg max " a_ # % (25) is generally not known, and one has to restate the expression in terms of known pdfs using ayes rule " a_ # % " #`_ a % G1"Ea % "$#&% ( (26) f we assume that is RiG distributed, (15) explicitly gives us " a_ # %, and the de-speckled is simply obtained by maximizing( the pdf of (15) with respect to. This gives R % (2) is explicitly given in terms of the parameters and Hence, of the RiG model. When MAP filtering is performed, the RiG-parameters are estimated from the observed data in a local window around each pixel. The estimated

5 speckle-free amplitude is obtained from (2) with and replaced by their local estimates, and equal to the observed amplitude. Fig. (6) shows the original ultrasound image, which shows the interior of the human heart. This image has been filtered using the proposed model in an MAP filtering approach. Note that no thresholding is applied, as is usual in the traditional A -MAP filter. All pixel-values in the output image has been estimated using (2). The filtered image is shown in Fig. () Figs. (8) and (9) show the original and the filtered SAR images, respectively. [6] C. J. Oliver, A model for non-rayleigh scattering statistics, Optica acta., vol. 31, no. 6, pp. 1 22, [] O. E. arndorff-nielsen, Normal nverse Gaussian Distributions and Stochastic Volatility Modelling, Scand. J. Statist., vol. 24, pp. 1 13, CONCLUSON We have presented a new statistical distribution for modeling amplitude data, which do not follow a Rayleigh distribution. We call this distribution the Rician nverse Gaussian distribution. The model has three parameters, which gives the distribution rich flexibility. These parameters may be estimated from real data using an EM-type algorithm. The experimental analysis shows that the new pdf is well suited for modeling medical ultrasound and SAR data. We have also developed a MAP filter based on the new distribution. This is easy to implement, due to the fact that the a posterior distribution is given in closed form. The filter output is hence given explicitly. Our initial analysis of the performance of this speckle filter seems very promising. 8. REFERENCES [1] R. M. Cramblitt and M. R. ell, Surface characterization using frequency diverse scattering measurements and regularity models, EEE Trans. Signal Processing, vol. 44, no. 3, pp , March [2] D. T. Kuan, A. A. Sawchuck, T. C. Strand, and P. Chavel, Adaptive noise smoothing filter for images with signal dependent noise, EEE Trans. Pattern Anal. and Machine ntell., vol., pp , [3] S. Foucher, G.. énié, and J.-M. oucher, Multiscale MAP Filtering of SAR mages, EEE Trans. mage Pros., vol. 1, no. 1, pp. 49 6, Jan. 21. [4] M. T. Wasan, First passage time distribution of brownian motion with positive drift (inverse gaussian distribution), Queen s papers in pure and applied matematics -no.19, Queen s University, [5] E. Jakeman and P. N. Pusey, Significance of K distributions in scattering experiments, Phys. Rev. Lett., vol. 4, pp , 198.

6 Fig. 6. Original ultrasound image (log-compressed). Fig. 8. Original single-look SAR image Fig.. MAP-filtered version of the image in Fig.(6). Window size (, ) Fig. 9. MAP-filtered version of the image in Fig.(8). Window size (,.

SIMPLIFIED MAP DESPECKLING BASED ON LAPLACIAN-GAUSSIAN MODELING OF UNDECIMATED WAVELET COEFFICIENTS

SIMPLIFIED MAP DESPECKLING BASED ON LAPLACIAN-GAUSSIAN MODELING OF UNDECIMATED WAVELET COEFFICIENTS 9th European Signal Processing Conference (EUSIPCO 0) Barcelona, Spain, August 9 - September, 0 SIMPLIFIED MAP DESPECKLING BASED ON LAPLACIAN-GAUSSIAN MODELING OF UNDECIMATED WAVELET COEFFICIENTS Fabrizio

More information

Bayesian-based Wavelet Shrinkage for SAR Image Despeckling Using Cycle Spinning *

Bayesian-based Wavelet Shrinkage for SAR Image Despeckling Using Cycle Spinning * Jun. 006 Journal of Electronic Science and Technology of China Vol.4 No. ayesian-based Wavelet Shrinkage for SAR Image Despeckling Using Cycle Spinning * ZHANG De-xiang,, GAO Qing-wei,, CHEN Jun-ning.

More information

Linear Dynamical Systems

Linear Dynamical Systems Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations

More information

Sequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes

Sequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes Sequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes Ellida M. Khazen * 13395 Coppermine Rd. Apartment 410 Herndon VA 20171 USA Abstract

More information

SEC: Stochastic ensemble consensus approach to unsupervised SAR sea-ice segmentation

SEC: Stochastic ensemble consensus approach to unsupervised SAR sea-ice segmentation 2009 Canadian Conference on Computer and Robot Vision SEC: Stochastic ensemble consensus approach to unsupervised SAR sea-ice segmentation Alexander Wong, David A. Clausi, and Paul Fieguth Vision and Image

More information

IMPROVED TECHNIQUES FOR STATISTICAL ANALYSIS OF THE ENVELOPE OF BACKSCATTERED ULTRASOUND USING THE HOMODYNED K DISTRIBUTION DAVID P.

IMPROVED TECHNIQUES FOR STATISTICAL ANALYSIS OF THE ENVELOPE OF BACKSCATTERED ULTRASOUND USING THE HOMODYNED K DISTRIBUTION DAVID P. IMPROVED TECHNIQUES FOR STATISTICAL ANALYSIS OF THE ENVELOPE OF BACKSCATTERED ULTRASOUND USING THE HOMODYNED K DISTRIBUTION BY DAVID P. HRUSKA B.S., University of Illinois at Urbana-Champaign, 7 THESIS

More information

PHYS 6710: Nuclear and Particle Physics II

PHYS 6710: Nuclear and Particle Physics II Data Analysis Content (~7 Classes) Uncertainties and errors Random variables, expectation value, (co)variance Distributions and samples Binomial, Poisson, and Normal Distribution Student's t-distribution

More information

Mixture Models and EM

Mixture Models and EM Mixture Models and EM Goal: Introduction to probabilistic mixture models and the expectationmaximization (EM) algorithm. Motivation: simultaneous fitting of multiple model instances unsupervised clustering

More information

Envelope PDF in Multipath Fading Channels with Random Number of Paths and Nonuniform Phase Distributions

Envelope PDF in Multipath Fading Channels with Random Number of Paths and Nonuniform Phase Distributions Envelope PDF in Multipath Fading Channels with andom umber of Paths and onuniform Phase Distributions ALI ABDI AD MOSTAFA KAVEH DEPT. OF ELEC. AD COMP. EG., UIVESITY OF MIESOTA 4-74 EE/CSCI BLDG., UIO

More information

Reliability Theory of Dynamic Loaded Structures (cont.) Calculation of Out-Crossing Frequencies Approximations to the Failure Probability.

Reliability Theory of Dynamic Loaded Structures (cont.) Calculation of Out-Crossing Frequencies Approximations to the Failure Probability. Outline of Reliability Theory of Dynamic Loaded Structures (cont.) Calculation of Out-Crossing Frequencies Approximations to the Failure Probability. Poisson Approximation. Upper Bound Solution. Approximation

More information

Impact of channel-state information on coded transmission over fading channels with diversity reception

Impact of channel-state information on coded transmission over fading channels with diversity reception Impact of channel-state information on coded transmission over fading channels with diversity reception Giorgio Taricco Ezio Biglieri Giuseppe Caire September 4, 1998 Abstract We study the synergy between

More information

Using the MBES for classification of riverbed sediments

Using the MBES for classification of riverbed sediments Acoustics 8 Paris Using the MBES for classification of riverbed sediments A. Amiri-Simkooei a, M. Snellen a and D. G Simons b a Acoustic Remote Sensing Group, Delft Institute of Earth Observation and Space

More information

Modeling conditional distributions with mixture models: Applications in finance and financial decision-making

Modeling conditional distributions with mixture models: Applications in finance and financial decision-making Modeling conditional distributions with mixture models: Applications in finance and financial decision-making John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università

More information

Optimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator

Optimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator 1 Optimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator Israel Cohen Lamar Signal Processing Ltd. P.O.Box 573, Yokneam Ilit 20692, Israel E-mail: icohen@lamar.co.il

More information

Lecture 3: Pattern Classification

Lecture 3: Pattern Classification EE E6820: Speech & Audio Processing & Recognition Lecture 3: Pattern Classification 1 2 3 4 5 The problem of classification Linear and nonlinear classifiers Probabilistic classification Gaussians, mixtures

More information

Lecture 4: Probabilistic Learning

Lecture 4: Probabilistic Learning DD2431 Autumn, 2015 1 Maximum Likelihood Methods Maximum A Posteriori Methods Bayesian methods 2 Classification vs Clustering Heuristic Example: K-means Expectation Maximization 3 Maximum Likelihood Methods

More information

Signal Detection Basics - CFAR

Signal Detection Basics - CFAR Signal Detection Basics - CFAR Types of noise clutter and signals targets Signal separation by comparison threshold detection Signal Statistics - Parameter estimation Threshold determination based on the

More information

LIKELIHOOD RECEIVER FOR FH-MFSK MOBILE RADIO*

LIKELIHOOD RECEIVER FOR FH-MFSK MOBILE RADIO* LIKELIHOOD RECEIVER FOR FH-MFSK MOBILE RADIO* Item Type text; Proceedings Authors Viswanathan, R.; S.C. Gupta Publisher International Foundation for Telemetering Journal International Telemetering Conference

More information

On the estimation of the K parameter for the Rice fading distribution

On the estimation of the K parameter for the Rice fading distribution On the estimation of the K parameter for the Rice fading distribution Ali Abdi, Student Member, IEEE, Cihan Tepedelenlioglu, Student Member, IEEE, Mostafa Kaveh, Fellow, IEEE, and Georgios Giannakis, Fellow,

More information

Machine Learning 4771

Machine Learning 4771 Machine Learning 4771 Instructor: Tony Jebara Topic 11 Maximum Likelihood as Bayesian Inference Maximum A Posteriori Bayesian Gaussian Estimation Why Maximum Likelihood? So far, assumed max (log) likelihood

More information

Towards inference for skewed alpha stable Levy processes

Towards inference for skewed alpha stable Levy processes Towards inference for skewed alpha stable Levy processes Simon Godsill and Tatjana Lemke Signal Processing and Communications Lab. University of Cambridge www-sigproc.eng.cam.ac.uk/~sjg Overview Motivation

More information

The Expectation-Maximization Algorithm

The Expectation-Maximization Algorithm 1/29 EM & Latent Variable Models Gaussian Mixture Models EM Theory The Expectation-Maximization Algorithm Mihaela van der Schaar Department of Engineering Science University of Oxford MLE for Latent Variable

More information

Computer Vision & Digital Image Processing

Computer Vision & Digital Image Processing Computer Vision & Digital Image Processing Image Restoration and Reconstruction I Dr. D. J. Jackson Lecture 11-1 Image restoration Restoration is an objective process that attempts to recover an image

More information

Automated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling

Automated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling Automated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling Abstract An automated unsupervised technique, based upon a Bayesian framework, for the segmentation of low light level

More information

The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space

The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space Robert Jenssen, Deniz Erdogmus 2, Jose Principe 2, Torbjørn Eltoft Department of Physics, University of Tromsø, Norway

More information

Bayesian Paradigm. Maximum A Posteriori Estimation

Bayesian Paradigm. Maximum A Posteriori Estimation Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)

More information

2D Image Processing. Bayes filter implementation: Kalman filter

2D Image Processing. Bayes filter implementation: Kalman filter 2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

Parameter Estimation

Parameter Estimation 1 / 44 Parameter Estimation Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay October 25, 2012 Motivation System Model used to Derive

More information

Lecture 4: Importance of Noise and Fluctuations

Lecture 4: Importance of Noise and Fluctuations Lecture 4: Importance of Noise and Fluctuations Jordi Soriano Fradera Dept. Física de la Matèria Condensada, Universitat de Barcelona UB Institute of Complex Systems September 2016 1. Noise in biological

More information

Bayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Bayesian statistics. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics

More information

2 Statistical Estimation: Basic Concepts

2 Statistical Estimation: Basic Concepts Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:

More information

THE problem of phase noise and its influence on oscillators

THE problem of phase noise and its influence on oscillators IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 54, NO. 5, MAY 2007 435 Phase Diffusion Coefficient for Oscillators Perturbed by Colored Noise Fergal O Doherty and James P. Gleeson Abstract

More information

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory Statistical Inference Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory IP, José Bioucas Dias, IST, 2007

More information

Spectral Clustering of Polarimetric SAR Data With Wishart-Derived Distance Measures

Spectral Clustering of Polarimetric SAR Data With Wishart-Derived Distance Measures Spectral Clustering of Polarimetric SAR Data With Wishart-Derived Distance Measures STIAN NORMANN ANFINSEN ROBERT JENSSEN TORBJØRN ELTOFT COMPUTATIONAL EARTH OBSERVATION AND MACHINE LEARNING LABORATORY

More information

K distribution: an appropriate substitute for Rayleigh-lognormal. distribution in fading-shadowing wireless channels

K distribution: an appropriate substitute for Rayleigh-lognormal. distribution in fading-shadowing wireless channels K distribution: an appropriate substitute for Rayleigh-lognormal distribution in fading-shadowing wireless channels A. Abdi and M. Kaveh Indexing terms: Rayleigh-lognormal distribution, K distribution,

More information

Density Estimation: ML, MAP, Bayesian estimation

Density Estimation: ML, MAP, Bayesian estimation Density Estimation: ML, MAP, Bayesian estimation CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Maximum-Likelihood Estimation Maximum

More information

Lecture 7: Wireless Channels and Diversity Advanced Digital Communications (EQ2410) 1

Lecture 7: Wireless Channels and Diversity Advanced Digital Communications (EQ2410) 1 Wireless : Wireless Advanced Digital Communications (EQ2410) 1 Thursday, Feb. 11, 2016 10:00-12:00, B24 1 Textbook: U. Madhow, Fundamentals of Digital Communications, 2008 1 / 15 Wireless Lecture 1-6 Equalization

More information

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature

More information

Expectation propagation for signal detection in flat-fading channels

Expectation propagation for signal detection in flat-fading channels Expectation propagation for signal detection in flat-fading channels Yuan Qi MIT Media Lab Cambridge, MA, 02139 USA yuanqi@media.mit.edu Thomas Minka CMU Statistics Department Pittsburgh, PA 15213 USA

More information

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian

More information

Contrastive Divergence

Contrastive Divergence Contrastive Divergence Training Products of Experts by Minimizing CD Hinton, 2002 Helmut Puhr Institute for Theoretical Computer Science TU Graz June 9, 2010 Contents 1 Theory 2 Argument 3 Contrastive

More information

Introduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf

Introduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf 1 Introduction to Machine Learning Maximum Likelihood and Bayesian Inference Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf 2013-14 We know that X ~ B(n,p), but we do not know p. We get a random sample

More information

MODULE -4 BAYEIAN LEARNING

MODULE -4 BAYEIAN LEARNING MODULE -4 BAYEIAN LEARNING CONTENT Introduction Bayes theorem Bayes theorem and concept learning Maximum likelihood and Least Squared Error Hypothesis Maximum likelihood Hypotheses for predicting probabilities

More information

Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart

Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart 1 Motivation and Problem In Lecture 1 we briefly saw how histograms

More information

Introduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Lior Wolf

Introduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Lior Wolf 1 Introduction to Machine Learning Maximum Likelihood and Bayesian Inference Lecturers: Eran Halperin, Lior Wolf 2014-15 We know that X ~ B(n,p), but we do not know p. We get a random sample from X, a

More information

Optimisation using measured Green s function for improving spatial coherence in acoustic measurements

Optimisation using measured Green s function for improving spatial coherence in acoustic measurements Ultrasonics 42 (2004) 205 212 www.elsevier.com/locate/ultras Optimisation using measured Green s function for improving spatial coherence in acoustic measurements Matthew Clark *, Steve D. Sharples, Mike

More information

Course content (will be adapted to the background knowledge of the class):

Course content (will be adapted to the background knowledge of the class): Biomedical Signal Processing and Signal Modeling Lucas C Parra, parra@ccny.cuny.edu Departamento the Fisica, UBA Synopsis This course introduces two fundamental concepts of signal processing: linear systems

More information

ICML Scalable Bayesian Inference on Point processes. with Gaussian Processes. Yves-Laurent Kom Samo & Stephen Roberts

ICML Scalable Bayesian Inference on Point processes. with Gaussian Processes. Yves-Laurent Kom Samo & Stephen Roberts ICML 2015 Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes Machine Learning Research Group and Oxford-Man Institute University of Oxford July 8, 2015 Point Processes

More information

2. (a) What is gaussian random variable? Develop an equation for guassian distribution

2. (a) What is gaussian random variable? Develop an equation for guassian distribution Code No: R059210401 Set No. 1 II B.Tech I Semester Supplementary Examinations, February 2007 PROBABILITY THEORY AND STOCHASTIC PROCESS ( Common to Electronics & Communication Engineering, Electronics &

More information

I HH. I data, and the PWF are compared with the

I HH. I data, and the PWF are compared with the Optimal Speckle Reduction in Polarimetric SAR Imagery* Leslie M. Novak Michael C. Burl ABSTRACT Speckle is a major cause of degradation in synthetic aperture radar (SAR) imagery. With the availability

More information

Statistics and Data Analysis

Statistics and Data Analysis Statistics and Data Analysis The Crash Course Physics 226, Fall 2013 "There are three kinds of lies: lies, damned lies, and statistics. Mark Twain, allegedly after Benjamin Disraeli Statistics and Data

More information

Intensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER? Intensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER?

Intensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER? Intensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER? : WHICH ONE LOOKS BETTER? 3.1 : WHICH ONE LOOKS BETTER? 3.2 1 Goal: Image enhancement seeks to improve the visual appearance of an image, or convert it to a form suited for analysis by a human or a machine.

More information

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem

More information

Statistícal Methods for Spatial Data Analysis

Statistícal Methods for Spatial Data Analysis Texts in Statistícal Science Statistícal Methods for Spatial Data Analysis V- Oliver Schabenberger Carol A. Gotway PCT CHAPMAN & K Contents Preface xv 1 Introduction 1 1.1 The Need for Spatial Analysis

More information

Lecture 6: Gaussian Mixture Models (GMM)

Lecture 6: Gaussian Mixture Models (GMM) Helsinki Institute for Information Technology Lecture 6: Gaussian Mixture Models (GMM) Pedram Daee 3.11.2015 Outline Gaussian Mixture Models (GMM) Models Model families and parameters Parameter learning

More information

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions DD2431 Autumn, 2014 1 2 3 Classification with Probability Distributions Estimation Theory Classification in the last lecture we assumed we new: P(y) Prior P(x y) Lielihood x2 x features y {ω 1,..., ω K

More information

Machine Learning for Signal Processing Bayes Classification and Regression

Machine Learning for Signal Processing Bayes Classification and Regression Machine Learning for Signal Processing Bayes Classification and Regression Instructor: Bhiksha Raj 11755/18797 1 Recap: KNN A very effective and simple way of performing classification Simple model: For

More information

GAUSSIAN PROCESS REGRESSION

GAUSSIAN PROCESS REGRESSION GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The

More information

Multivariate Asset Return Prediction with Mixture Models

Multivariate Asset Return Prediction with Mixture Models Multivariate Asset Return Prediction with Mixture Models Swiss Banking Institute, University of Zürich Introduction The leptokurtic nature of asset returns has spawned an enormous amount of research into

More information

Question Paper Code : AEC11T03

Question Paper Code : AEC11T03 Hall Ticket No Question Paper Code : AEC11T03 VARDHAMAN COLLEGE OF ENGINEERING (AUTONOMOUS) Affiliated to JNTUH, Hyderabad Four Year B Tech III Semester Tutorial Question Bank 2013-14 (Regulations: VCE-R11)

More information

ABSTRACT INTRODUCTION

ABSTRACT INTRODUCTION ABSTRACT Presented in this paper is an approach to fault diagnosis based on a unifying review of linear Gaussian models. The unifying review draws together different algorithms such as PCA, factor analysis,

More information

Written by David Niedzwiecki ( )

Written by David Niedzwiecki ( ) Written by David Niedzwiecki (0-0-08) Protocol: Noise Analysis The use of noise analysis is typically needed in electrophysiology experiments when translocation or blockage times are smaller than the sampling

More information

2D Image Processing. Bayes filter implementation: Kalman filter

2D Image Processing. Bayes filter implementation: Kalman filter 2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

SYNTHETIC aperture radar (SAR) images provide useful

SYNTHETIC aperture radar (SAR) images provide useful 1642 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 42, NO. 8, AUGUST 2004 Bayesian Wavelet Shrinkage With Edge Detection for SAR Image Despeckling Min Dai, Student Member, IEEE, Cheng Peng,

More information

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1 EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle

More information

CONDITIONAL MORPHOLOGY FOR IMAGE RESTORATION

CONDITIONAL MORPHOLOGY FOR IMAGE RESTORATION CONDTONAL MORPHOLOGY FOR MAGE RESTORATON C.S. Regauoni, E. Stringa, C. Valpreda Department of Biophysical and Electronic Engineering University of Genoa - Via all opera Pia 11 A, Genova taly 16 145 ABSTRACT

More information

CFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE

CFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE CFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE Anshul Sharma and Randolph L. Moses Department of Electrical Engineering, The Ohio State University, Columbus, OH 43210 ABSTRACT We have developed

More information

Transmission matrices of random media: Means for spectral polarimetric measurements

Transmission matrices of random media: Means for spectral polarimetric measurements Reprint (R40) Transmission matrices of random media: Means for spectral polarimetric measurements Reprinted with permission by Thomas W. Kohlgraf-Owens, Aristide Dogariu CREOL, the College of Optics and

More information

ADVERTIMENT ADVERTENCIA WARNING

ADVERTIMENT ADVERTENCIA WARNING ADVERTIMENT. La consulta d aquesta tesi queda condicionada a l acceptació de les següents condicions d'ús: La difusió d aquesta tesi per mitjà del servei TDX (www.tesisenxarxa.net) ha estat autoritzada

More information

Introduction. Chapter 1

Introduction. Chapter 1 Chapter 1 Introduction In this book we will be concerned with supervised learning, which is the problem of learning input-output mappings from empirical data (the training dataset). Depending on the characteristics

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project

More information

Gaussian Processes for Audio Feature Extraction

Gaussian Processes for Audio Feature Extraction Gaussian Processes for Audio Feature Extraction Dr. Richard E. Turner (ret26@cam.ac.uk) Computational and Biological Learning Lab Department of Engineering University of Cambridge Machine hearing pipeline

More information

Contents Preface iii 1 Origins and Manifestations of Speckle 2 Random Phasor Sums 3 First-Order Statistical Properties

Contents Preface iii 1 Origins and Manifestations of Speckle 2 Random Phasor Sums 3 First-Order Statistical Properties Contents Preface iii 1 Origins and Manifestations of Speckle 1 1.1 General Background............................. 1 1.2 Intuitive Explanation of the Cause of Speckle................ 2 1.3 Some Mathematical

More information

New Statistical Model for the Enhancement of Noisy Speech

New Statistical Model for the Enhancement of Noisy Speech New Statistical Model for the Enhancement of Noisy Speech Electrical Engineering Department Technion - Israel Institute of Technology February 22, 27 Outline Problem Formulation and Motivation 1 Problem

More information

Problem 1 (20) Log-normal. f(x) Cauchy

Problem 1 (20) Log-normal. f(x) Cauchy ORF 245. Rigollet Date: 11/21/2008 Problem 1 (20) f(x) f(x) 0.0 0.1 0.2 0.3 0.4 0.0 0.2 0.4 0.6 0.8 4 2 0 2 4 Normal (with mean -1) 4 2 0 2 4 Negative-exponential x x f(x) f(x) 0.0 0.1 0.2 0.3 0.4 0.5

More information

Mixture Models and Expectation-Maximization

Mixture Models and Expectation-Maximization Mixture Models and Expectation-Maximiation David M. Blei March 9, 2012 EM for mixtures of multinomials The graphical model for a mixture of multinomials π d x dn N D θ k K How should we fit the parameters?

More information

CSC411 Fall 2018 Homework 5

CSC411 Fall 2018 Homework 5 Homework 5 Deadline: Wednesday, Nov. 4, at :59pm. Submission: You need to submit two files:. Your solutions to Questions and 2 as a PDF file, hw5_writeup.pdf, through MarkUs. (If you submit answers to

More information

Probability and statistics; Rehearsal for pattern recognition

Probability and statistics; Rehearsal for pattern recognition Probability and statistics; Rehearsal for pattern recognition Václav Hlaváč Czech Technical University in Prague Czech Institute of Informatics, Robotics and Cybernetics 166 36 Prague 6, Jugoslávských

More information

Dimension Reduction. David M. Blei. April 23, 2012

Dimension Reduction. David M. Blei. April 23, 2012 Dimension Reduction David M. Blei April 23, 2012 1 Basic idea Goal: Compute a reduced representation of data from p -dimensional to q-dimensional, where q < p. x 1,...,x p z 1,...,z q (1) We want to do

More information

arxiv: v1 [cs.cv] 10 Feb 2016

arxiv: v1 [cs.cv] 10 Feb 2016 GABOR WAVELETS IN IMAGE PROCESSING David Bařina Doctoral Degree Programme (2), FIT BUT E-mail: xbarin2@stud.fit.vutbr.cz Supervised by: Pavel Zemčík E-mail: zemcik@fit.vutbr.cz arxiv:162.338v1 [cs.cv]

More information

Using Entropy and 2-D Correlation Coefficient as Measuring Indices for Impulsive Noise Reduction Techniques

Using Entropy and 2-D Correlation Coefficient as Measuring Indices for Impulsive Noise Reduction Techniques Using Entropy and 2-D Correlation Coefficient as Measuring Indices for Impulsive Noise Reduction Techniques Zayed M. Ramadan Department of Electronics and Communications Engineering, Faculty of Engineering,

More information

GARCH Models Estimation and Inference

GARCH Models Estimation and Inference GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in

More information

Some Model Selection Problems. in Image Analysis. Mário A. T. Figueiredo. Instituto Superior Técnico PORTUGAL. Lisboa

Some Model Selection Problems. in Image Analysis. Mário A. T. Figueiredo. Instituto Superior Técnico PORTUGAL. Lisboa Some Model Selection Problems in Image Analysis Mário A. T. Figueiredo Instituto Superior Técnico Lisboa PORTUGAL People involved in work here presented: José M. N. Leitão, Instituto Superior Técnico,

More information

Fundamentals of Statistical Signal Processing Volume II Detection Theory

Fundamentals of Statistical Signal Processing Volume II Detection Theory Fundamentals of Statistical Signal Processing Volume II Detection Theory Steven M. Kay University of Rhode Island PH PTR Prentice Hall PTR Upper Saddle River, New Jersey 07458 http://www.phptr.com Contents

More information

Applied Probability and Stochastic Processes

Applied Probability and Stochastic Processes Applied Probability and Stochastic Processes In Engineering and Physical Sciences MICHEL K. OCHI University of Florida A Wiley-Interscience Publication JOHN WILEY & SONS New York - Chichester Brisbane

More information

Financial Econometrics and Quantitative Risk Managenent Return Properties

Financial Econometrics and Quantitative Risk Managenent Return Properties Financial Econometrics and Quantitative Risk Managenent Return Properties Eric Zivot Updated: April 1, 2013 Lecture Outline Course introduction Return definitions Empirical properties of returns Reading

More information

Sparse Linear Models (10/7/13)

Sparse Linear Models (10/7/13) STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine

More information

A Bayesian method for the analysis of deterministic and stochastic time series

A Bayesian method for the analysis of deterministic and stochastic time series A Bayesian method for the analysis of deterministic and stochastic time series Coryn Bailer-Jones Max Planck Institute for Astronomy, Heidelberg DPG, Berlin, March 2015 Time series modelling heteroscedastic,

More information

Chapter 4. Continuous Random Variables 4.1 PDF

Chapter 4. Continuous Random Variables 4.1 PDF Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will

More information

The Kalman Filter. Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience. Sarah Dance

The Kalman Filter. Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience. Sarah Dance The Kalman Filter Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience Sarah Dance School of Mathematical and Physical Sciences, University of Reading s.l.dance@reading.ac.uk July

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 20: Expectation Maximization Algorithm EM for Mixture Models Many figures courtesy Kevin Murphy s

More information

Introduction...2 Chapter Review on probability and random variables Random experiment, sample space and events

Introduction...2 Chapter Review on probability and random variables Random experiment, sample space and events Introduction... Chapter...3 Review on probability and random variables...3. Random eperiment, sample space and events...3. Probability definition...7.3 Conditional Probability and Independence...7.4 heorem

More information

Random Fatigue. ! Module 3. ! Lecture 23 :Random Vibrations & Failure Analysis

Random Fatigue. ! Module 3. ! Lecture 23 :Random Vibrations & Failure Analysis !! Module 3! Lecture 23 :Random Vibrations & Failure Analysis Random Fatigue!! Sayan Gupta Department of Applied Mechanics Indian Institute of Technology Madras Random fatigue As has been mentioned earlier,

More information

SPECKLE is a phenomenon inherent in coherent imaging

SPECKLE is a phenomenon inherent in coherent imaging IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 4, APRIL 008 119 A Stationary Wavelet-Domain Wiener Filter for Correlated Speckle Stian Solbø and Torbjørn Eltoft, Member, IEEE Abstract

More information

ECE521 week 3: 23/26 January 2017

ECE521 week 3: 23/26 January 2017 ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear

More information

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2015 http://www.astro.cornell.edu/~cordes/a6523 Lecture 4 See web page later tomorrow Searching for Monochromatic Signals

More information

The Gaussian distribution

The Gaussian distribution The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a

More information

Stat 5102 Final Exam May 14, 2015

Stat 5102 Final Exam May 14, 2015 Stat 5102 Final Exam May 14, 2015 Name Student ID The exam is closed book and closed notes. You may use three 8 1 11 2 sheets of paper with formulas, etc. You may also use the handouts on brand name distributions

More information