SPECKLE: MODELING AND FILTERING. Torbjørn Eltoft. Department of Physics, University of Tromsø, 9037 Tromsø, Norway
|
|
- Augustus Grant
- 5 years ago
- Views:
Transcription
1 F % F C SPECKLE: MODELNG AND FLTERNG Torbjørn Eltoft Department of Physics, University of Tromsø, 93 Tromsø, Norway ASTRACT A new statistical distribution for modeling non-rayleigh amplitude statistics, which is formed as a compound distribution of a the Rician distribution and the nverse Gaussian (G), is introduced. This new model has been called the Rician nverse Gaussian (RiG) distribution. The theoretical basis of the model is briefly presented, and we give an algorithm for estimating its parameters from data. The modeling capabilities of the RiG distribution is tested by fitting the model to s generated from segments of a medical ultrasound image and a single-look SAR image. Finally, a maximum a posteriori (MAP) speckle filter has been constructed using the RiG model. The filtering performance of this filter is shown to be excellent. 1. NTRODUCTON Modeling non-rayleigh amplitude statistics in coherent imaging systems has been a research area for almost three decades, and many interesting distributions have been proposed. Some of these are the Rice-distribution, the K-distribution and the homodined K-distribution. Accurate statistical models for the amplitude statistics are important both with respect to characterization/classification of image regions [1], and with respect to speckle filtering [2, 3]. n this paper we introduce a new model for modeling the amplitude statistics of coherent imagery, which made up as a compound distribution of a the Rician distribution and the nverse Gaussian (G) [4]. We have for this reason denoted this new distribution the Rician nverse Gaussian (RiG) distribution. We give some details of the theoretical basis of this new model, describe how its parameters can be estimated, and test its ability to fit to s of segments of ultrasound images. We include comparisons with the well-known K-distribution. We also develop a maximum aposteriori speckle filter based on the RiG model, and show its filtering performance. 2. REVEW OF SPECKLE MODELS f the scattering medium is modeled as a collection of discrete scatterers, we may write the detected signal as is the number of scatterers, (1) where is the amplitude and is the phase of the! th scatterer. i: Now, if the scatterers phases are random and independent of the amplitudes, and if the number is very large, the probability density function (pdf) of, the amplitude of, will be a Rayleigh distribution "$#&% ( )+ -,/.1 32 (2) where ) is the the standard deviation of the Gaussian distributed 4 - and 5 -components of. This distribution represents the fully-developed speckle case, or the diffuse scattering signal. ii: f a specular component 6 is added to (1), the pdf of the amplitude gets Rician, i.e. "8#&% ( )+ -,. :9-;-< =2 4> % 6 )+ (3) where 4 > is the modified essel function of first kind and zero order. iii: The well-known -distribution [5, 6] results, if we allow the variance of the Rayleigh-distribution to itself being a random variable, distributed according to a A -distribution. n this case, the amplitude of is a compound variable of a Rayleigh variable and a A -distributed variable. The pdf is explicitly given as where %HG and order D, C " # % ( -C A %ED C 1F, % ( (4) is the modified essel function of second kind J D$KML N is a scale parameter. D is a shape parameter. iv: The homodyned K-distribution results when a coherent
2 D L G T > % % component, 6, is added to the diffuse scattering signal in (1), and we allow the number of terms in the summation,, to itself being random following a negative binomial distribution. The resulting pdf, which can not be obtained in closed form, is given by the integral "$# % ( > % > % 6 > % )+ K ( F The models reviewed above have some shortcomings: the Rayleigh model certainly is a proper model for fully developed speckle, but this case does not often occur in highresolution imagery. Also, the K-model has its limitations, because, often the signal has a specular component, resulting in bad fits to the data. The homodyned model has good modeling capabilities, but its use is restricted by the lack of a tractable pdf. 3. NEW MODEL We will in the following introduce a unifying model for non-rayleigh amplitude data, which is able to overcome the problems mentioned above. n addition, its parameters can easily be estimated from data. When the amplitude of a detected signal, modeled as in (1), is non-rayleigh, this implies that the 4 - and 5 - components of the complex signal deviate from Gaussian statistics. We propose to use the Normal nverse Gaussian (NG) distribution to model these components. A NGdistribution is defined as a variance-mean mixture of a normal distribution with the inverse Gaussian (4 ) as a mixing distribution []. The probability density function is explicitly given as where " % and %HG "% $(% (! #" % " % $(% 1 %&$(% 1 ) + % ) % ) + (5) (6) is the modified essel function of the second kind and order. As noted, the distribution is characterized by the four parameters %,, which are bounded such that -/.1 2.,, and )3 L4 L 3. controls the steepness of the distribution; a small means a steep distribution, a larger means a less steep distribution. determines the skewness; L - implies skewed to the left, N - implies skewed to the right, and - implies a symmetric distribution. is a location parameter, and is a dispersion parameter similar to the variance in the Gaussian distribution. Let be defined by the relation () where :9<; % - =. Hence, the conditional distribution 6 is normal ; % >J,. The marginal distribution of 6 is nverse Gaussian (G) with parameters and A). This probability density is given by " %, =C"%,EDF = #" % ) (8) The G-distribution was first introduced as the first passage time distribution of rownian motion with drift. t has recently attained increasing interest in signal processing and statistics (financial) due to its ability to model highly skewed densities, and its many analogies with the Gaussian distribution. ts mean and variance are GH 6KJ ML N and OPH 6KJ N&Q, respectively. The distribution is well documented in the literature [4]. Consider the case when the - and -components of in (1) are uncorrelated and 2-dimensional NG-distributed, i.e: 9 4R % - ES, and 9 4R % -, ET. Note that we assume that both S and UT are zero. Accordingly, and are both compound random variables of the form: > S 6 6 S and > T 6 6 T, where V % S T 9W; %XY, with Y [Z 4 ( Z -dimensional identity matrix.), and % ]\ ) % S ^ 4 is the, with. t is now easy to show that the distribution of the envelope, conditioned on 6, i.e. 6, is Rician distributed, with pdf given by " #`_ a % where \ S ^! #" % ) ^ 4> % 1 ( (9) T. From (9) we observe that the skewness parameters of the NG-distributions will have the effect of a coherent component on the amplitude statistics. Now, the amplitude will accordingly be a compound variable consisting of the Rice-distribution in (9), and the G( ) distribution, i.e. " # % ( " #`_ a % " a % (1) The integral in (1) can be evaluated to give the following closed form pdf for : "8# % ( /b D,F c " % % D,Fd D,F % 4> % ( (11) We call this distribution the Rician nverse Gaussian (RiG) distribution. As can be seen, the pdf has three parameter.
3 L 9 1 ) [1,.98, 3 ] [1,.8, 5 ] [1,.6, 2 ] [1,.8, 1 ] Fig. 1. Examples of pdfs residing to the RiG model. The numbers in the brackets correspond to,,, respectively Fig.1 displays some realizations of the pdf for some selected values of the parameters. The numbers in brackets in the upper right corner of Fig.1 give the values of,, and for each curve. We observe that by varying the parameters,, and, we may model a wide range of distributions, from a Rayleigh-shaped pdf to a Gaussian-shaped pdf. 4. PARAMETER ESTMATON The parameters of the RiG distribution can be estimated using an EM-type algorithm. n the previous section we assumed that the speckle-free image, represented by the vari- distribution with parameters able, follows an G % and. ts th-order moments are given by GH 6 J %,, N L From (12) we easily find that GH 6 J and G G 6 GH 6KJ, a are: (12) (13) (14) t is further noted that the posterior distribution 6 is given as: " a_ # % ( _ ; ;, Q &% Q! L )( " $# + Q! L ", = #"- ) EL " #.# (15) This is the pdf of a generalized inverse Gaussian (GG) []. variable specified by the parameters % ) D The moments of a GG % +/ distribution are given by [] GH 6 J - % " % (16) Let 1. We thus have that G8H 6 KJ (1) $2 G 6 1 % &% & 1 (18) Now, given N independent observations! - GG G+. From some initial values for,, and, we may estimate G8H 6KJ as and G6 6 a as N L 6 N % / 53 &% & 1 Using (13) and (14), our estimators for and are : 89 ; L 6 N < a>= L56 N (19) (2) (21) (22) respectively. The parameter is estimated using the maximum likelihood method. Using previous estimates of,, and, define A % A % as DCFEHG "8#&% is defined in (11). Our estimate for is ac- where " # % cordingly given as arg max JKA % + (23) This maximization must be performed numerically. 5. MODELNG DATA (24) Fig.2 shows a medical ultrasound image of the human heart. Two test regions are seen as marked rectangles. The s of the amplitudes of the test regions have been displayed in Fig.3, along with the best fitted RiG- and K- models. The parameters of the K-models were estimated
4 ) Fig. 2. Ultrasound image (log-compressed). The marked areas define the image regions which the RiG-model was tested on Fig. 4. Single-look SAR image (displayed log-compressed). The marked areas define the image regions which were tested against the K and the RiG models..4 RiG model K model.12.1 RiG model K model.2.18 RiG K.3 RiG K Fig. 3. Fitted RiG-model (solid), (thin solid) and fitted K-model (dotted-dashed) corresponding to test regions Fig. 5. Fitted RiG-model (solid), (thin solid) and fitted K-model (dotted-dashed) corresponding to test regions using a moment based estimator, or by matching the higherorder moments of the data. We note that the RiG-model fits the s of the data much better than the K-model. Fig.3 also demonstrates the high flexibility of the the RiG model. Fig.4 shows a single-look SAR image of a mountainous area in southern Norway. The same conclusion drawn from the analysis of the the ultrasound data, is valid for the SAR data. The RiG model also has a very good fit to the s of the SAR data, as can be noted from fig MAP-FLTERNG The problem in ayesian de-speckling is the following: We want to estimate the amplitude, where is the spatial location, which is most likely to occur in a specklefree image, given an observed amplitude value. This is known as MAP-filtering, and neglecting the spatial coor- dinates the estimation is stated in mathematical terms as " a_ # % arg max " a_ # % (25) is generally not known, and one has to restate the expression in terms of known pdfs using ayes rule " a_ # % " #`_ a % G1"Ea % "$#&% ( (26) f we assume that is RiG distributed, (15) explicitly gives us " a_ # %, and the de-speckled is simply obtained by maximizing( the pdf of (15) with respect to. This gives R % (2) is explicitly given in terms of the parameters and Hence, of the RiG model. When MAP filtering is performed, the RiG-parameters are estimated from the observed data in a local window around each pixel. The estimated
5 speckle-free amplitude is obtained from (2) with and replaced by their local estimates, and equal to the observed amplitude. Fig. (6) shows the original ultrasound image, which shows the interior of the human heart. This image has been filtered using the proposed model in an MAP filtering approach. Note that no thresholding is applied, as is usual in the traditional A -MAP filter. All pixel-values in the output image has been estimated using (2). The filtered image is shown in Fig. () Figs. (8) and (9) show the original and the filtered SAR images, respectively. [6] C. J. Oliver, A model for non-rayleigh scattering statistics, Optica acta., vol. 31, no. 6, pp. 1 22, [] O. E. arndorff-nielsen, Normal nverse Gaussian Distributions and Stochastic Volatility Modelling, Scand. J. Statist., vol. 24, pp. 1 13, CONCLUSON We have presented a new statistical distribution for modeling amplitude data, which do not follow a Rayleigh distribution. We call this distribution the Rician nverse Gaussian distribution. The model has three parameters, which gives the distribution rich flexibility. These parameters may be estimated from real data using an EM-type algorithm. The experimental analysis shows that the new pdf is well suited for modeling medical ultrasound and SAR data. We have also developed a MAP filter based on the new distribution. This is easy to implement, due to the fact that the a posterior distribution is given in closed form. The filter output is hence given explicitly. Our initial analysis of the performance of this speckle filter seems very promising. 8. REFERENCES [1] R. M. Cramblitt and M. R. ell, Surface characterization using frequency diverse scattering measurements and regularity models, EEE Trans. Signal Processing, vol. 44, no. 3, pp , March [2] D. T. Kuan, A. A. Sawchuck, T. C. Strand, and P. Chavel, Adaptive noise smoothing filter for images with signal dependent noise, EEE Trans. Pattern Anal. and Machine ntell., vol., pp , [3] S. Foucher, G.. énié, and J.-M. oucher, Multiscale MAP Filtering of SAR mages, EEE Trans. mage Pros., vol. 1, no. 1, pp. 49 6, Jan. 21. [4] M. T. Wasan, First passage time distribution of brownian motion with positive drift (inverse gaussian distribution), Queen s papers in pure and applied matematics -no.19, Queen s University, [5] E. Jakeman and P. N. Pusey, Significance of K distributions in scattering experiments, Phys. Rev. Lett., vol. 4, pp , 198.
6 Fig. 6. Original ultrasound image (log-compressed). Fig. 8. Original single-look SAR image Fig.. MAP-filtered version of the image in Fig.(6). Window size (, ) Fig. 9. MAP-filtered version of the image in Fig.(8). Window size (,.
SIMPLIFIED MAP DESPECKLING BASED ON LAPLACIAN-GAUSSIAN MODELING OF UNDECIMATED WAVELET COEFFICIENTS
9th European Signal Processing Conference (EUSIPCO 0) Barcelona, Spain, August 9 - September, 0 SIMPLIFIED MAP DESPECKLING BASED ON LAPLACIAN-GAUSSIAN MODELING OF UNDECIMATED WAVELET COEFFICIENTS Fabrizio
More informationBayesian-based Wavelet Shrinkage for SAR Image Despeckling Using Cycle Spinning *
Jun. 006 Journal of Electronic Science and Technology of China Vol.4 No. ayesian-based Wavelet Shrinkage for SAR Image Despeckling Using Cycle Spinning * ZHANG De-xiang,, GAO Qing-wei,, CHEN Jun-ning.
More informationLinear Dynamical Systems
Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations
More informationSequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes
Sequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes Ellida M. Khazen * 13395 Coppermine Rd. Apartment 410 Herndon VA 20171 USA Abstract
More informationSEC: Stochastic ensemble consensus approach to unsupervised SAR sea-ice segmentation
2009 Canadian Conference on Computer and Robot Vision SEC: Stochastic ensemble consensus approach to unsupervised SAR sea-ice segmentation Alexander Wong, David A. Clausi, and Paul Fieguth Vision and Image
More informationIMPROVED TECHNIQUES FOR STATISTICAL ANALYSIS OF THE ENVELOPE OF BACKSCATTERED ULTRASOUND USING THE HOMODYNED K DISTRIBUTION DAVID P.
IMPROVED TECHNIQUES FOR STATISTICAL ANALYSIS OF THE ENVELOPE OF BACKSCATTERED ULTRASOUND USING THE HOMODYNED K DISTRIBUTION BY DAVID P. HRUSKA B.S., University of Illinois at Urbana-Champaign, 7 THESIS
More informationPHYS 6710: Nuclear and Particle Physics II
Data Analysis Content (~7 Classes) Uncertainties and errors Random variables, expectation value, (co)variance Distributions and samples Binomial, Poisson, and Normal Distribution Student's t-distribution
More informationMixture Models and EM
Mixture Models and EM Goal: Introduction to probabilistic mixture models and the expectationmaximization (EM) algorithm. Motivation: simultaneous fitting of multiple model instances unsupervised clustering
More informationEnvelope PDF in Multipath Fading Channels with Random Number of Paths and Nonuniform Phase Distributions
Envelope PDF in Multipath Fading Channels with andom umber of Paths and onuniform Phase Distributions ALI ABDI AD MOSTAFA KAVEH DEPT. OF ELEC. AD COMP. EG., UIVESITY OF MIESOTA 4-74 EE/CSCI BLDG., UIO
More informationReliability Theory of Dynamic Loaded Structures (cont.) Calculation of Out-Crossing Frequencies Approximations to the Failure Probability.
Outline of Reliability Theory of Dynamic Loaded Structures (cont.) Calculation of Out-Crossing Frequencies Approximations to the Failure Probability. Poisson Approximation. Upper Bound Solution. Approximation
More informationImpact of channel-state information on coded transmission over fading channels with diversity reception
Impact of channel-state information on coded transmission over fading channels with diversity reception Giorgio Taricco Ezio Biglieri Giuseppe Caire September 4, 1998 Abstract We study the synergy between
More informationUsing the MBES for classification of riverbed sediments
Acoustics 8 Paris Using the MBES for classification of riverbed sediments A. Amiri-Simkooei a, M. Snellen a and D. G Simons b a Acoustic Remote Sensing Group, Delft Institute of Earth Observation and Space
More informationModeling conditional distributions with mixture models: Applications in finance and financial decision-making
Modeling conditional distributions with mixture models: Applications in finance and financial decision-making John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università
More informationOptimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator
1 Optimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator Israel Cohen Lamar Signal Processing Ltd. P.O.Box 573, Yokneam Ilit 20692, Israel E-mail: icohen@lamar.co.il
More informationLecture 3: Pattern Classification
EE E6820: Speech & Audio Processing & Recognition Lecture 3: Pattern Classification 1 2 3 4 5 The problem of classification Linear and nonlinear classifiers Probabilistic classification Gaussians, mixtures
More informationLecture 4: Probabilistic Learning
DD2431 Autumn, 2015 1 Maximum Likelihood Methods Maximum A Posteriori Methods Bayesian methods 2 Classification vs Clustering Heuristic Example: K-means Expectation Maximization 3 Maximum Likelihood Methods
More informationSignal Detection Basics - CFAR
Signal Detection Basics - CFAR Types of noise clutter and signals targets Signal separation by comparison threshold detection Signal Statistics - Parameter estimation Threshold determination based on the
More informationLIKELIHOOD RECEIVER FOR FH-MFSK MOBILE RADIO*
LIKELIHOOD RECEIVER FOR FH-MFSK MOBILE RADIO* Item Type text; Proceedings Authors Viswanathan, R.; S.C. Gupta Publisher International Foundation for Telemetering Journal International Telemetering Conference
More informationOn the estimation of the K parameter for the Rice fading distribution
On the estimation of the K parameter for the Rice fading distribution Ali Abdi, Student Member, IEEE, Cihan Tepedelenlioglu, Student Member, IEEE, Mostafa Kaveh, Fellow, IEEE, and Georgios Giannakis, Fellow,
More informationMachine Learning 4771
Machine Learning 4771 Instructor: Tony Jebara Topic 11 Maximum Likelihood as Bayesian Inference Maximum A Posteriori Bayesian Gaussian Estimation Why Maximum Likelihood? So far, assumed max (log) likelihood
More informationTowards inference for skewed alpha stable Levy processes
Towards inference for skewed alpha stable Levy processes Simon Godsill and Tatjana Lemke Signal Processing and Communications Lab. University of Cambridge www-sigproc.eng.cam.ac.uk/~sjg Overview Motivation
More informationThe Expectation-Maximization Algorithm
1/29 EM & Latent Variable Models Gaussian Mixture Models EM Theory The Expectation-Maximization Algorithm Mihaela van der Schaar Department of Engineering Science University of Oxford MLE for Latent Variable
More informationComputer Vision & Digital Image Processing
Computer Vision & Digital Image Processing Image Restoration and Reconstruction I Dr. D. J. Jackson Lecture 11-1 Image restoration Restoration is an objective process that attempts to recover an image
More informationAutomated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling
Automated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling Abstract An automated unsupervised technique, based upon a Bayesian framework, for the segmentation of low light level
More informationThe Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space
The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space Robert Jenssen, Deniz Erdogmus 2, Jose Principe 2, Torbjørn Eltoft Department of Physics, University of Tromsø, Norway
More informationBayesian Paradigm. Maximum A Posteriori Estimation
Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)
More information2D Image Processing. Bayes filter implementation: Kalman filter
2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationParameter Estimation
1 / 44 Parameter Estimation Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay October 25, 2012 Motivation System Model used to Derive
More informationLecture 4: Importance of Noise and Fluctuations
Lecture 4: Importance of Noise and Fluctuations Jordi Soriano Fradera Dept. Física de la Matèria Condensada, Universitat de Barcelona UB Institute of Complex Systems September 2016 1. Noise in biological
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationTHE problem of phase noise and its influence on oscillators
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 54, NO. 5, MAY 2007 435 Phase Diffusion Coefficient for Oscillators Perturbed by Colored Noise Fergal O Doherty and James P. Gleeson Abstract
More informationParametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory
Statistical Inference Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory IP, José Bioucas Dias, IST, 2007
More informationSpectral Clustering of Polarimetric SAR Data With Wishart-Derived Distance Measures
Spectral Clustering of Polarimetric SAR Data With Wishart-Derived Distance Measures STIAN NORMANN ANFINSEN ROBERT JENSSEN TORBJØRN ELTOFT COMPUTATIONAL EARTH OBSERVATION AND MACHINE LEARNING LABORATORY
More informationK distribution: an appropriate substitute for Rayleigh-lognormal. distribution in fading-shadowing wireless channels
K distribution: an appropriate substitute for Rayleigh-lognormal distribution in fading-shadowing wireless channels A. Abdi and M. Kaveh Indexing terms: Rayleigh-lognormal distribution, K distribution,
More informationDensity Estimation: ML, MAP, Bayesian estimation
Density Estimation: ML, MAP, Bayesian estimation CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Maximum-Likelihood Estimation Maximum
More informationLecture 7: Wireless Channels and Diversity Advanced Digital Communications (EQ2410) 1
Wireless : Wireless Advanced Digital Communications (EQ2410) 1 Thursday, Feb. 11, 2016 10:00-12:00, B24 1 Textbook: U. Madhow, Fundamentals of Digital Communications, 2008 1 / 15 Wireless Lecture 1-6 Equalization
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationExpectation propagation for signal detection in flat-fading channels
Expectation propagation for signal detection in flat-fading channels Yuan Qi MIT Media Lab Cambridge, MA, 02139 USA yuanqi@media.mit.edu Thomas Minka CMU Statistics Department Pittsburgh, PA 15213 USA
More informationDS-GA 1002 Lecture notes 11 Fall Bayesian statistics
DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian
More informationContrastive Divergence
Contrastive Divergence Training Products of Experts by Minimizing CD Hinton, 2002 Helmut Puhr Institute for Theoretical Computer Science TU Graz June 9, 2010 Contents 1 Theory 2 Argument 3 Contrastive
More informationIntroduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf
1 Introduction to Machine Learning Maximum Likelihood and Bayesian Inference Lecturers: Eran Halperin, Yishay Mansour, Lior Wolf 2013-14 We know that X ~ B(n,p), but we do not know p. We get a random sample
More informationMODULE -4 BAYEIAN LEARNING
MODULE -4 BAYEIAN LEARNING CONTENT Introduction Bayes theorem Bayes theorem and concept learning Maximum likelihood and Least Squared Error Hypothesis Maximum likelihood Hypotheses for predicting probabilities
More informationStatistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart
Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart 1 Motivation and Problem In Lecture 1 we briefly saw how histograms
More informationIntroduction to Machine Learning. Maximum Likelihood and Bayesian Inference. Lecturers: Eran Halperin, Lior Wolf
1 Introduction to Machine Learning Maximum Likelihood and Bayesian Inference Lecturers: Eran Halperin, Lior Wolf 2014-15 We know that X ~ B(n,p), but we do not know p. We get a random sample from X, a
More informationOptimisation using measured Green s function for improving spatial coherence in acoustic measurements
Ultrasonics 42 (2004) 205 212 www.elsevier.com/locate/ultras Optimisation using measured Green s function for improving spatial coherence in acoustic measurements Matthew Clark *, Steve D. Sharples, Mike
More informationCourse content (will be adapted to the background knowledge of the class):
Biomedical Signal Processing and Signal Modeling Lucas C Parra, parra@ccny.cuny.edu Departamento the Fisica, UBA Synopsis This course introduces two fundamental concepts of signal processing: linear systems
More informationICML Scalable Bayesian Inference on Point processes. with Gaussian Processes. Yves-Laurent Kom Samo & Stephen Roberts
ICML 2015 Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes Machine Learning Research Group and Oxford-Man Institute University of Oxford July 8, 2015 Point Processes
More information2. (a) What is gaussian random variable? Develop an equation for guassian distribution
Code No: R059210401 Set No. 1 II B.Tech I Semester Supplementary Examinations, February 2007 PROBABILITY THEORY AND STOCHASTIC PROCESS ( Common to Electronics & Communication Engineering, Electronics &
More informationI HH. I data, and the PWF are compared with the
Optimal Speckle Reduction in Polarimetric SAR Imagery* Leslie M. Novak Michael C. Burl ABSTRACT Speckle is a major cause of degradation in synthetic aperture radar (SAR) imagery. With the availability
More informationStatistics and Data Analysis
Statistics and Data Analysis The Crash Course Physics 226, Fall 2013 "There are three kinds of lies: lies, damned lies, and statistics. Mark Twain, allegedly after Benjamin Disraeli Statistics and Data
More informationIntensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER? Intensity Transformations and Spatial Filtering: WHICH ONE LOOKS BETTER?
: WHICH ONE LOOKS BETTER? 3.1 : WHICH ONE LOOKS BETTER? 3.2 1 Goal: Image enhancement seeks to improve the visual appearance of an image, or convert it to a form suited for analysis by a human or a machine.
More informationMachine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io
Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem
More informationStatistícal Methods for Spatial Data Analysis
Texts in Statistícal Science Statistícal Methods for Spatial Data Analysis V- Oliver Schabenberger Carol A. Gotway PCT CHAPMAN & K Contents Preface xv 1 Introduction 1 1.1 The Need for Spatial Analysis
More informationLecture 6: Gaussian Mixture Models (GMM)
Helsinki Institute for Information Technology Lecture 6: Gaussian Mixture Models (GMM) Pedram Daee 3.11.2015 Outline Gaussian Mixture Models (GMM) Models Model families and parameters Parameter learning
More informationLecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions
DD2431 Autumn, 2014 1 2 3 Classification with Probability Distributions Estimation Theory Classification in the last lecture we assumed we new: P(y) Prior P(x y) Lielihood x2 x features y {ω 1,..., ω K
More informationMachine Learning for Signal Processing Bayes Classification and Regression
Machine Learning for Signal Processing Bayes Classification and Regression Instructor: Bhiksha Raj 11755/18797 1 Recap: KNN A very effective and simple way of performing classification Simple model: For
More informationGAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
More informationMultivariate Asset Return Prediction with Mixture Models
Multivariate Asset Return Prediction with Mixture Models Swiss Banking Institute, University of Zürich Introduction The leptokurtic nature of asset returns has spawned an enormous amount of research into
More informationQuestion Paper Code : AEC11T03
Hall Ticket No Question Paper Code : AEC11T03 VARDHAMAN COLLEGE OF ENGINEERING (AUTONOMOUS) Affiliated to JNTUH, Hyderabad Four Year B Tech III Semester Tutorial Question Bank 2013-14 (Regulations: VCE-R11)
More informationABSTRACT INTRODUCTION
ABSTRACT Presented in this paper is an approach to fault diagnosis based on a unifying review of linear Gaussian models. The unifying review draws together different algorithms such as PCA, factor analysis,
More informationWritten by David Niedzwiecki ( )
Written by David Niedzwiecki (0-0-08) Protocol: Noise Analysis The use of noise analysis is typically needed in electrophysiology experiments when translocation or blockage times are smaller than the sampling
More information2D Image Processing. Bayes filter implementation: Kalman filter
2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationSYNTHETIC aperture radar (SAR) images provide useful
1642 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 42, NO. 8, AUGUST 2004 Bayesian Wavelet Shrinkage With Edge Detection for SAR Image Despeckling Min Dai, Student Member, IEEE, Cheng Peng,
More informationEEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1
EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle
More informationCONDITIONAL MORPHOLOGY FOR IMAGE RESTORATION
CONDTONAL MORPHOLOGY FOR MAGE RESTORATON C.S. Regauoni, E. Stringa, C. Valpreda Department of Biophysical and Electronic Engineering University of Genoa - Via all opera Pia 11 A, Genova taly 16 145 ABSTRACT
More informationCFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE
CFAR TARGET DETECTION IN TREE SCATTERING INTERFERENCE Anshul Sharma and Randolph L. Moses Department of Electrical Engineering, The Ohio State University, Columbus, OH 43210 ABSTRACT We have developed
More informationTransmission matrices of random media: Means for spectral polarimetric measurements
Reprint (R40) Transmission matrices of random media: Means for spectral polarimetric measurements Reprinted with permission by Thomas W. Kohlgraf-Owens, Aristide Dogariu CREOL, the College of Optics and
More informationADVERTIMENT ADVERTENCIA WARNING
ADVERTIMENT. La consulta d aquesta tesi queda condicionada a l acceptació de les següents condicions d'ús: La difusió d aquesta tesi per mitjà del servei TDX (www.tesisenxarxa.net) ha estat autoritzada
More informationIntroduction. Chapter 1
Chapter 1 Introduction In this book we will be concerned with supervised learning, which is the problem of learning input-output mappings from empirical data (the training dataset). Depending on the characteristics
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
More informationGaussian Processes for Audio Feature Extraction
Gaussian Processes for Audio Feature Extraction Dr. Richard E. Turner (ret26@cam.ac.uk) Computational and Biological Learning Lab Department of Engineering University of Cambridge Machine hearing pipeline
More informationContents Preface iii 1 Origins and Manifestations of Speckle 2 Random Phasor Sums 3 First-Order Statistical Properties
Contents Preface iii 1 Origins and Manifestations of Speckle 1 1.1 General Background............................. 1 1.2 Intuitive Explanation of the Cause of Speckle................ 2 1.3 Some Mathematical
More informationNew Statistical Model for the Enhancement of Noisy Speech
New Statistical Model for the Enhancement of Noisy Speech Electrical Engineering Department Technion - Israel Institute of Technology February 22, 27 Outline Problem Formulation and Motivation 1 Problem
More informationProblem 1 (20) Log-normal. f(x) Cauchy
ORF 245. Rigollet Date: 11/21/2008 Problem 1 (20) f(x) f(x) 0.0 0.1 0.2 0.3 0.4 0.0 0.2 0.4 0.6 0.8 4 2 0 2 4 Normal (with mean -1) 4 2 0 2 4 Negative-exponential x x f(x) f(x) 0.0 0.1 0.2 0.3 0.4 0.5
More informationMixture Models and Expectation-Maximization
Mixture Models and Expectation-Maximiation David M. Blei March 9, 2012 EM for mixtures of multinomials The graphical model for a mixture of multinomials π d x dn N D θ k K How should we fit the parameters?
More informationCSC411 Fall 2018 Homework 5
Homework 5 Deadline: Wednesday, Nov. 4, at :59pm. Submission: You need to submit two files:. Your solutions to Questions and 2 as a PDF file, hw5_writeup.pdf, through MarkUs. (If you submit answers to
More informationProbability and statistics; Rehearsal for pattern recognition
Probability and statistics; Rehearsal for pattern recognition Václav Hlaváč Czech Technical University in Prague Czech Institute of Informatics, Robotics and Cybernetics 166 36 Prague 6, Jugoslávských
More informationDimension Reduction. David M. Blei. April 23, 2012
Dimension Reduction David M. Blei April 23, 2012 1 Basic idea Goal: Compute a reduced representation of data from p -dimensional to q-dimensional, where q < p. x 1,...,x p z 1,...,z q (1) We want to do
More informationarxiv: v1 [cs.cv] 10 Feb 2016
GABOR WAVELETS IN IMAGE PROCESSING David Bařina Doctoral Degree Programme (2), FIT BUT E-mail: xbarin2@stud.fit.vutbr.cz Supervised by: Pavel Zemčík E-mail: zemcik@fit.vutbr.cz arxiv:162.338v1 [cs.cv]
More informationUsing Entropy and 2-D Correlation Coefficient as Measuring Indices for Impulsive Noise Reduction Techniques
Using Entropy and 2-D Correlation Coefficient as Measuring Indices for Impulsive Noise Reduction Techniques Zayed M. Ramadan Department of Electronics and Communications Engineering, Faculty of Engineering,
More informationGARCH Models Estimation and Inference
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in
More informationSome Model Selection Problems. in Image Analysis. Mário A. T. Figueiredo. Instituto Superior Técnico PORTUGAL. Lisboa
Some Model Selection Problems in Image Analysis Mário A. T. Figueiredo Instituto Superior Técnico Lisboa PORTUGAL People involved in work here presented: José M. N. Leitão, Instituto Superior Técnico,
More informationFundamentals of Statistical Signal Processing Volume II Detection Theory
Fundamentals of Statistical Signal Processing Volume II Detection Theory Steven M. Kay University of Rhode Island PH PTR Prentice Hall PTR Upper Saddle River, New Jersey 07458 http://www.phptr.com Contents
More informationApplied Probability and Stochastic Processes
Applied Probability and Stochastic Processes In Engineering and Physical Sciences MICHEL K. OCHI University of Florida A Wiley-Interscience Publication JOHN WILEY & SONS New York - Chichester Brisbane
More informationFinancial Econometrics and Quantitative Risk Managenent Return Properties
Financial Econometrics and Quantitative Risk Managenent Return Properties Eric Zivot Updated: April 1, 2013 Lecture Outline Course introduction Return definitions Empirical properties of returns Reading
More informationSparse Linear Models (10/7/13)
STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine
More informationA Bayesian method for the analysis of deterministic and stochastic time series
A Bayesian method for the analysis of deterministic and stochastic time series Coryn Bailer-Jones Max Planck Institute for Astronomy, Heidelberg DPG, Berlin, March 2015 Time series modelling heteroscedastic,
More informationChapter 4. Continuous Random Variables 4.1 PDF
Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will
More informationThe Kalman Filter. Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience. Sarah Dance
The Kalman Filter Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience Sarah Dance School of Mathematical and Physical Sciences, University of Reading s.l.dance@reading.ac.uk July
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 20: Expectation Maximization Algorithm EM for Mixture Models Many figures courtesy Kevin Murphy s
More informationIntroduction...2 Chapter Review on probability and random variables Random experiment, sample space and events
Introduction... Chapter...3 Review on probability and random variables...3. Random eperiment, sample space and events...3. Probability definition...7.3 Conditional Probability and Independence...7.4 heorem
More informationRandom Fatigue. ! Module 3. ! Lecture 23 :Random Vibrations & Failure Analysis
!! Module 3! Lecture 23 :Random Vibrations & Failure Analysis Random Fatigue!! Sayan Gupta Department of Applied Mechanics Indian Institute of Technology Madras Random fatigue As has been mentioned earlier,
More informationSPECKLE is a phenomenon inherent in coherent imaging
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 4, APRIL 008 119 A Stationary Wavelet-Domain Wiener Filter for Correlated Speckle Stian Solbø and Torbjørn Eltoft, Member, IEEE Abstract
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More informationA6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring
A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2015 http://www.astro.cornell.edu/~cordes/a6523 Lecture 4 See web page later tomorrow Searching for Monochromatic Signals
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationStat 5102 Final Exam May 14, 2015
Stat 5102 Final Exam May 14, 2015 Name Student ID The exam is closed book and closed notes. You may use three 8 1 11 2 sheets of paper with formulas, etc. You may also use the handouts on brand name distributions
More information