Bayesian inference & Markov chain Monte Carlo. Note 1: Many slides for this lecture were kindly provided by Paul Lewis and Mark Holder
|
|
- Polly Hart
- 5 years ago
- Views:
Transcription
1 Bayesian inference & Markov chain Monte Carlo Note 1: Many slides for this lecture were kindly rovided by Paul Lewis and Mark Holder Note 2: Paul Lewis has written nice software for demonstrating Markov chain Monte Carlo idea. Software is called MCRobot and is freely available at: htt://hydrodictyon.eeb.uconn.edu/eole/lewis/software.h (unfortunately software only works for windows oerating system, but see the imcmc rogram by John Huelsenbeck at: htt://cteg.berkeley.edu/software.html... also try the MCMC robot iad a ) Assume we want to estimate a arameter θ with data X. Maximum likelihood aroach to estimating θ finds value of θ that maximizes Pr (X θ). Before observing data, we may have some idea of how lausible are values of θ. This idea is called our rior distribution of θ and we ll denote it Pr (θ). Bayesians base estimate of θ on the osterior distribution Pr (θ X).
2 Pr (θ X) = Pr (θ, X) Pr (X) = Pr (X θ)pr (θ) θ Pr (X, θ)dθ = Pr (X θ)pr (θ) θ Pr (X θ)pr (θ)dθ = likelihood rior difficult quantity to calculate Often, determining the exact value of the above integral is difficult. Problems with Bayesian aroachs in general: 1. Disagreements about hilosohy of inference & Disagreements over riors 2. Heavy Comutational Requirements (roblem 2 is raidly becoming less noteworthy)
3 Potential advantages of Bayesian hylogeny inference Interretation of osterior robabilities of toologies is more straightforward than interretation of bootstra suort. If rior distributions for arameters are far from diffuse, very comlicated and realistic models can be used and the roblem of overarameterization can be simultaneously avoided. MrBayes software for hylogeny inference is at: htt://mrbayes.csit.fsu.edu/ Let be the robability of heads. Then 1- is the robability of tails Imagine a data set X with these results from fliing a coin Toss Result H T H T T T Probability P(X ) = (1-) 2 4 almost binomial distribution form
4 Likelihood with 2 heads and 4 tails Likelihood P(X ) Log-Likelihood with 2 heads and 4 tails Log-Likelihood log(p(x ))
5 For integers a and b, Beta density B(a,b) is a-1 b-1 P()= (a+b-1)!/((a-1)!(b-1)!) (1-) where is between 0 and 1. Exected value of is a/(a+b) 2 Variance of is ab/((a+b+1)(a+b) ) Beta distribution is conjugate rior for data from binomial distribution Uniform Prior Distribution (i.e., Beta(1,1) distribution) Prior Density P()
6 Beta(3,5) osterior from Uniform rior + data (2 heads and 4 tails) Posterior P( X) Posterior Mean = 3/(3+5) Beta(20,20) rior distribution Prior Mean = 0.5 rior density P()
7 Beta(22,24) osterior from Beta(20,20) rior + data (2 heads and 4 tails) Posterior P( X) Beta(30,10) rior distribution Prior Mean = 0.75 rior density P()
8 Beta(32,14) osterior from Beta(30,10) rior + data (2 heads and 4 tails) Posterior Density P( X) Posterior Mean = 32/(32+14) Likelihood with 20 Heads and 40 Tails Likelihood P(X ) 0.0e e e e e e - 17
9 Log-Likelihood with 20 heads and 40 tails Log-Likelihood log(p(x )) Uniform Prior Distribution (i.e., Beta(1,1) distribution) Prior Density P()
10 Beta(21,41) osterior from Uniform rior + data (20 heads and 40 tails) Posterior P( X) Beta(20,20) rior distribution Prior Mean = 0.5 rior density P()
11 Beta(40,60) osterior from Beta(20,20) rior + data (20 heads and 40 tails) Posterior P( X) Beta(30,10) rior distribution Prior Mean = 0.75 rior density P()
12 Beta(50,50) osterior from Beta(30,10) rior + data (20 heads and 40 tails) Posterior P( X) Markov chain Monte Carlo (MCMC) idea aroximates Pr (θ X) by samling a large number of θ values from Pr (θ X). So, θ values with higher osterior robability are more likely to be samled than θ values with low osterior robability.
13 Question: How is this samling achieved? Answer: A Markov chain is constructed and simulated. The states of this chain reresent values of θ. The stationary distribution of this chain is Pr (θ X). In other words, we start chain at some initial value of θ. After running chain for a long enough time, the robability of the chain being at some articular state will be aroximately equal to the osterior robability of the state. Let θ (t) be the value of θ after t stesofthemarkovchainwhere θ (0) is the initial value. Each ste of the Markov chain involves randomly roosing a new value of θ based on the current value of θ. Call the roosed value θ. We decide with some robability to either accet θ as our new stateortorejecttheroosedθ and remain at our current state. The Hastings (Hastings 1970) algorithm is a way to make this decision and force the stationary distribution of the chain to be Pr (θ X). According to the Hastings algorithm, what state should we adot at ste t +1 if θ (t) is the current state and θ is the roosed state?
14 Let J(θ θ (t) ) be the juming distribution, i.e. the robability of roosing θ given that the current state is θ (t). Define r as r = Pr (X θ )Pr (θ )J(θ (t) θ ) Pr ( X θ (t)) Pr ( θ (t)) J(θ θ (t) ) With robability equal to the minimum of r and 1, weset Otherwise, we set θ (t+1) = θ. θ (t+1) = θ (t). For the Hastings algorithm to yield the stationary distribution Pr (θ X), there are a few required conditions. The most imortant condition is that it must be ossible to reach each state from any other in a finite number of stes. Also, the Markov chain can t be eriodic. MCMC imlementation details: The Markov chain should be run as long as ossible. We may have T total samles after running our Markov chain. They would be θ (1), θ (2),..., θ (T ). The first B (1 B < T) of these samles are often discarded (i.e. not used to aroximate the osterior). The eriod before the chain has gotten these B samles that will be discarded is referred to as the burn in eriod. The reason for discarding these samles is that the early samles tyically are largely deendent on the initial state of the Markov chain and often the initial state of the chain is (either intentionally or unintentionally) atyical with resect to the osterior distribution. The remaining samles θ (B+1), θ (B+2),..., θ (T ) are used to aroximate the osterior distribution. For examle, the average among the samled values for a arameter might be a good estimate of its osterior mean.
15 Markov Chain Monte Carlo and Relatives (some imortant aers) CARLIN, B.P., and T.A. LOUIS Bayes and Emirical Bayes Methods for Data Analysis. Chaman and Hall, London. GELMAN, A., J.B. CARLIN, H.S. STERN, and D.B. RUBIN Bayesian Data Analysis. Chaman and Hall, London. GEYER, C Markov chain Monte Carlo maximum likelihood. Pages in Comuting Science and Statistics: Proceedings of the 23rd Symosium on the Interface. Keramidas, ed. Fairfax Station: Interface Foundation HASTINGS, W.K. (1970) Monte Carlo samling methods using Markov chains and their alications. Biometrika 57: METROPOLIS, N., A.W. ROSENBLUTH, M.N. ROSENBLUTH, A.H. TELLER, and E. TELLER Equations of state calculations by fast comuting machines. J. Chem. Phys. 21: Posterior redictive inference (notice resemblance to arametric bootstra) 1. Via MCMC or some other technique, get N samled arameter sets θ (1),...,θ (N) from osterior distribution (θ X) 2. For each samled arameter set θ (k), simulate a new data set X (k) from (X θ (k) ) 3. Calculate a test statistic value T (X (k) ) from each simulated data set and see where test statistic value for actual data T (X) is relative to simulated distribution of test statistic values.
16 From Huelsenbeck et al Syst Biol 52(2): Notation for following ages: X data M i,m j :Modelsiand j θ i,θ j : arameters for models i and j (X θ i,m i ),(X θ j,m j ): likelihoods
17 Bayes factor (M i X) (M j X) = (M i)(x M i )/(X) (M j )(X M j )/(X) = (M i) (M j ) (X M i) (X M j ) Left factor is called rior odds and right factor is called Bayes factor. Bayes factor is ratio of marginal likelihoods of the two models. BF ij = (X M i) (X M j ) According to wikiedia, Jeffreys (1961) interretation of BF 12 (1 reresenting one model and 2 being the other): BF 12 Interretation < 1 : 1 Negative (suorts M 2 ) 1 : 1 to 3 : 1 Barely worth mentioning 3 : 1 to 10 : 1 Substantial 10:1to30:1 Strong 30 : 1 to 100 : 1 Very Strong > 100 : 1 Decisive
18 BF ij = (X M i) (X M j ) Bayes factors hard to comute because marginal likelihoods hard to comute: (X M i )= θ i (X M i,θ i )(θ i M i )dθ i Imortant oint to note from above: Bayes factors deend on riors (θ i M i ) because marginal likelihoods deend on riors! How to aroximate/comute marginal likelihood? (X M i )= θ i (X M i,θ i )(θ i M i )dθ i Harmonic mean estimator of marginal likelihood (widely used but likely to be terrible and should be avoided): 1 (X M i ). = 1 N N k=1 1 (X θ (k) i,m i ) where θ (k) i are samled from osterior (θ i X, M i ).
19 Imortant aers regarding Bayesian Model Comarison... Posterior Predictive Inference in Phylogenetics: J.P. Bollback Molecular Biology and Evolution. 19: Harmonic Mean and other techniques for estimating Bayes factors: Newton and Raftery Journal of the Royal Statistical Society. Series B. 56(1):3-48. more reliable ways to aroximate marginal likelihood Thermodynamic Integration to Aroximate Bayes Factors (adated to molecular evolution data): Lartillot and Philie Syst. Biol. 55: Imroving marginal likelihood estimation for Bayesian hylogenetic model selection. W. Xie, P.O. Lewis, Y. Fan, L. Kao, M-H Chen Syst Biol. 60(2): Choosing among artition models in Bayesian hylogenetics. Y. Fan, R. Wu, M-H Chen, L Kuo, P.O. Lewis Mol. Biol. Evol. 28(1): Markov chain Monte Carlo without likelihoods. P. Marjoram, J. Molitor, V. Plagnol, and S. Tavare PNAS USA. 100(26): H. Jeffreys. The Theory of Probability (3e). Oxford (1961);. 432 M.A. Beaumont, W. Zhang, D.J. Balding. Aroximate Bayesian Comutation in Poulation Genetics Genetics 162: Paul Lewis MCMC Robot Demo Target distribution: Mixture of bivariate normal hills inner contours: 50% of the robability outer contours: 95% Proosal scheme: random direction gamma-distributed ste length (mean 45 ixels, s.d. 40 ixels) reflection at edges
20 MCMC robot rules Slightly downhill stes are usually acceted Drastic off the cliff downhill stes are almost never acceted Uhill stes are always acceted With these rules, it is easy to see that the robot tends to stay near the tos of hills Burn-in First 100 stes Note that first few stes are not at all reresentative of the distribution. Starting oint
21 Problems with MCMC aroaches: 1. They are difficult to imlement. Imlementation may need to be clever to be comutationally tractable and rogramming bugs are a serious ossibility. 2. For the kinds of comlicated situations that biologists face, it may be very difficult to know how fast the Markov chain converges to the desired osterior distribution. There are diagnostics for evaluating whether a chain has converged to the osterior distribution but the diagnostics do not rovide a guarantee of convergence. A GOOD DIAGNOSTIC : MULTIPLE RUNS!! Just how long is a long run? What would you conclude about the target distribution had you stoed the robot at this oint? One way to detect this mistake is to erform several indeendent runs. Results different among runs? Probably none of them were run long enough!
22 History lots Ln Likelihood White noise aearance is a good sign Burn in is over right about here Imortant! This is a lot of first 1000 stes, and there is no indication that anything is wrong (but we know for a fact that we didn t let this one run long enough) Slow mixing Chain is sending long eriods of time stuck in one lace Indicates ste size is too large, and most roosed stes would take the robot off the cliff
23 The roblem of co-linearity Parameter Joint osterior density for a model having two highly correlated arameters is a narrow ridge If we have searate roosals for and, even small stes may be too large! Parameter Some material on Bayesian model comarison and hyothesis testing 1. Some Bayesians dislike much hyothesis testing because null hyotheses often are known a riori to be false and value deends both on how wrong null is and on amount of data. 2. Posterior redictive inference for assessing fit of models (see next ages) 3. Bayes factors for comaring models (see next ages)
24 The Tradeoff Pro: Proosing big stes hels in juming from one island in the osterior density to another Con: Proosing big stes often results in oor mixing Solution: Better roosals - MCMCMC Huelsenbeck has found that a technique called Metroolis-Couled Markov chain Monte Carlo (i.e., MCMCMC!! or MC 3) suggested by C.J. Geyer is useful for getting convergence with hylogeny reconstruction. The idea of MCMCMC is to run multile Markov chains in arallel. One chain will have stationary distribution that is the osterior of interest. The other chains will aroximate osterior distributions that are various degrees more smooth that than the osterior distribution of interest. Each chain is run searately, excet that occasionally 2 chains are randomly icked and a roosal to switch the states of these two chains is made. This roosal is randomly acceted or reject with the aroriate robability
25 Metroolis-couled Markov chain Monte Carlo (MCMCMC, or MC 3 ) MC 3 involves running several chains simultaneously The cold chain is the one that counts, the rest are heated chains. What is a heated chain? Instead of using R, to make accetance or rejection decisions, heated chains use: In MrBayes: H = Temerature*(Chain's index) The cold chain has index 0 Heated chains exlore the surface more freely Occasionally, you roose to switch the ositions of 2 of the chains
26 Heated chains act as scouts Phylogeny Priors: For hylogeny inference, arameters might reresent toology, branch lengths, base frequencies, transition-transversion ratio, etc. Each arameter needs secified rior distribution. For examle All unrooted toologies can be considered equally robable a riori. Given toology, all branch lengths between 0 and some big number could be considered equally likely a riori 2. All combinations of base frequencies could be considered equally likely a riori 3. The transition-transversion ratio could have a rior distribution that is uniform between 0 & some big number.... and so on.
27 Moving through Tree Sace Larget Simon Local Move Moving through Tree Sace Larget Simon Local Move
28 Moving through Tree Sace Larget Simon Local Move Moving through Tree Sace Larget Simon Local Move
29 Moving through arameter sace Using (ratio of the transition rate to the transversion rate) as an examle of a model arameter. Proosal distribution is the uniform distribution on the interval ( -, + ) A larger means the samler will attemt to make larger jums on average. Putting it all together Start with an initial tree and model arameters (often chosen randomly). Proose a new, randomly-selected move. Accet or reject the move (Walking). Every k generations, save tree, branch lengths and all model arameters (Thinning). After n generations, summarize the samle using histograms, means, credibility intervals, etc. (Summarizing).
30 Samling the chain tells us: Which tree has the highest osterior robability? What is the robability that tree X is the true tree? What values of the arameters are most robable? What if we are only interested in one grouing? Which of the trees in the MCMC run contained the clade (e.g. A + C)? The roortion of trees with A and C together in our samle aroximates the osterior robability that A and C are sister to each other.
31 Slit (a.k.a. clade) robabilities A slit is a artitioning of taxa that corresonds with a articular branch. Slits are usually reresented by strings: asterisks (*) show which taxa are on one side of the branch, and the hyhens ( ) show the taxa on the other side. Posteriors of model arameters lower = uer = % credible interval Histogram created from a samle of 1000 kaa values From: Lewis, L., & Flechtner, V. (2002. Taxon 51: ) mean = 3.234
Bayesian inference & Markov chain Monte Carlo. Note 1: Many slides for this lecture were kindly provided by Paul Lewis and Mark Holder
Bayesian inference & Markov chain Monte Carlo Note 1: Many slides for this lecture were kindly provided by Paul Lewis and Mark Holder Note 2: Paul Lewis has written nice software for demonstrating Markov
More informationBayesian Phylogenetics
Bayesian Phylogenetics Paul O. Lewis Department of Ecology & Evolutionary Biology University of Connecticut Woods Hole Molecular Evolution Workshop, July 27, 2006 2006 Paul O. Lewis Bayesian Phylogenetics
More informationA Bayesian Approach to Phylogenetics
A Bayesian Approach to Phylogenetics Niklas Wahlberg Based largely on slides by Paul Lewis (www.eeb.uconn.edu) An Introduction to Bayesian Phylogenetics Bayesian inference in general Markov chain Monte
More informationSome of these slides have been borrowed from Dr. Paul Lewis, Dr. Joe Felsenstein. Thanks!
Some of these slides have been borrowed from Dr. Paul Lewis, Dr. Joe Felsenstein. Thanks! Paul has many great tools for teaching phylogenetics at his web site: http://hydrodictyon.eeb.uconn.edu/people/plewis
More informationLearning Sequence Motif Models Using Gibbs Sampling
Learning Sequence Motif Models Using Gibbs Samling BMI/CS 776 www.biostat.wisc.edu/bmi776/ Sring 2018 Anthony Gitter gitter@biostat.wisc.edu These slides excluding third-arty material are licensed under
More informationBayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies
Bayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies 1 What is phylogeny? Essay written for the course in Markov Chains 2004 Torbjörn Karfunkel Phylogeny is the evolutionary development
More informationOn split sample and randomized confidence intervals for binomial proportions
On slit samle and randomized confidence intervals for binomial roortions Måns Thulin Deartment of Mathematics, Usala University arxiv:1402.6536v1 [stat.me] 26 Feb 2014 Abstract Slit samle methods have
More informationIntroduction to Probability and Statistics
Introduction to Probability and Statistics Chater 8 Ammar M. Sarhan, asarhan@mathstat.dal.ca Deartment of Mathematics and Statistics, Dalhousie University Fall Semester 28 Chater 8 Tests of Hyotheses Based
More informationSTA 250: Statistics. Notes 7. Bayesian Approach to Statistics. Book chapters: 7.2
STA 25: Statistics Notes 7. Bayesian Aroach to Statistics Book chaters: 7.2 1 From calibrating a rocedure to quantifying uncertainty We saw that the central idea of classical testing is to rovide a rigorous
More informationCombining Logistic Regression with Kriging for Mapping the Risk of Occurrence of Unexploded Ordnance (UXO)
Combining Logistic Regression with Kriging for Maing the Risk of Occurrence of Unexloded Ordnance (UXO) H. Saito (), P. Goovaerts (), S. A. McKenna (2) Environmental and Water Resources Engineering, Deartment
More information4. Score normalization technical details We now discuss the technical details of the score normalization method.
SMT SCORING SYSTEM This document describes the scoring system for the Stanford Math Tournament We begin by giving an overview of the changes to scoring and a non-technical descrition of the scoring rules
More informationSAS for Bayesian Mediation Analysis
Paer 1569-2014 SAS for Bayesian Mediation Analysis Miočević Milica, Arizona State University; David P. MacKinnon, Arizona State University ABSTRACT Recent statistical mediation analysis research focuses
More informationBayesian Model Averaging Kriging Jize Zhang and Alexandros Taflanidis
HIPAD LAB: HIGH PERFORMANCE SYSTEMS LABORATORY DEPARTMENT OF CIVIL AND ENVIRONMENTAL ENGINEERING AND EARTH SCIENCES Bayesian Model Averaging Kriging Jize Zhang and Alexandros Taflanidis Why use metamodeling
More informationInfer relationships among three species: Outgroup:
Infer relationships among three species: Outgroup: Three possible trees (topologies): A C B A B C Model probability 1.0 Prior distribution Data (observations) probability 1.0 Posterior distribution Bayes
More informationCHAPTER-II Control Charts for Fraction Nonconforming using m-of-m Runs Rules
CHAPTER-II Control Charts for Fraction Nonconforming using m-of-m Runs Rules. Introduction: The is widely used in industry to monitor the number of fraction nonconforming units. A nonconforming unit is
More informationTests for Two Proportions in a Stratified Design (Cochran/Mantel-Haenszel Test)
Chater 225 Tests for Two Proortions in a Stratified Design (Cochran/Mantel-Haenszel Test) Introduction In a stratified design, the subects are selected from two or more strata which are formed from imortant
More informationSystem Reliability Estimation and Confidence Regions from Subsystem and Full System Tests
009 American Control Conference Hyatt Regency Riverfront, St. Louis, MO, USA June 0-, 009 FrB4. System Reliability Estimation and Confidence Regions from Subsystem and Full System Tests James C. Sall Abstract
More information8 STOCHASTIC PROCESSES
8 STOCHASTIC PROCESSES The word stochastic is derived from the Greek στoχαστικoς, meaning to aim at a target. Stochastic rocesses involve state which changes in a random way. A Markov rocess is a articular
More informationBayesian Spatially Varying Coefficient Models in the Presence of Collinearity
Bayesian Satially Varying Coefficient Models in the Presence of Collinearity David C. Wheeler 1, Catherine A. Calder 1 he Ohio State University 1 Abstract he belief that relationshis between exlanatory
More informationAI*IA 2003 Fusion of Multiple Pattern Classifiers PART III
AI*IA 23 Fusion of Multile Pattern Classifiers PART III AI*IA 23 Tutorial on Fusion of Multile Pattern Classifiers by F. Roli 49 Methods for fusing multile classifiers Methods for fusing multile classifiers
More informationMonte Carlo Studies. Monte Carlo Studies. Sampling Distribution
Monte Carlo Studies Do not let yourself be intimidated by the material in this lecture This lecture involves more theory but is meant to imrove your understanding of: Samling distributions and tests of
More informationCHAPTER 5 STATISTICAL INFERENCE. 1.0 Hypothesis Testing. 2.0 Decision Errors. 3.0 How a Hypothesis is Tested. 4.0 Test for Goodness of Fit
Chater 5 Statistical Inference 69 CHAPTER 5 STATISTICAL INFERENCE.0 Hyothesis Testing.0 Decision Errors 3.0 How a Hyothesis is Tested 4.0 Test for Goodness of Fit 5.0 Inferences about Two Means It ain't
More informationResearch Note REGRESSION ANALYSIS IN MARKOV CHAIN * A. Y. ALAMUTI AND M. R. MESHKANI **
Iranian Journal of Science & Technology, Transaction A, Vol 3, No A3 Printed in The Islamic Reublic of Iran, 26 Shiraz University Research Note REGRESSION ANALYSIS IN MARKOV HAIN * A Y ALAMUTI AND M R
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationDETC2003/DAC AN EFFICIENT ALGORITHM FOR CONSTRUCTING OPTIMAL DESIGN OF COMPUTER EXPERIMENTS
Proceedings of DETC 03 ASME 003 Design Engineering Technical Conferences and Comuters and Information in Engineering Conference Chicago, Illinois USA, Setember -6, 003 DETC003/DAC-48760 AN EFFICIENT ALGORITHM
More informationLecture 23 Maximum Likelihood Estimation and Bayesian Inference
Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Thais Paiva STA 111 - Summer 2013 Term II August 7, 2013 1 / 31 Thais Paiva STA 111 - Summer 2013 Term II Lecture 23, 08/07/2013 Lecture
More informationA MIXED CONTROL CHART ADAPTED TO THE TRUNCATED LIFE TEST BASED ON THE WEIBULL DISTRIBUTION
O P E R A T I O N S R E S E A R C H A N D D E C I S I O N S No. 27 DOI:.5277/ord73 Nasrullah KHAN Muhammad ASLAM 2 Kyung-Jun KIM 3 Chi-Hyuck JUN 4 A MIXED CONTROL CHART ADAPTED TO THE TRUNCATED LIFE TEST
More informationApproximate Bayesian Computation: a simulation based approach to inference
Approximate Bayesian Computation: a simulation based approach to inference Richard Wilkinson Simon Tavaré 2 Department of Probability and Statistics University of Sheffield 2 Department of Applied Mathematics
More informationUsing the Divergence Information Criterion for the Determination of the Order of an Autoregressive Process
Using the Divergence Information Criterion for the Determination of the Order of an Autoregressive Process P. Mantalos a1, K. Mattheou b, A. Karagrigoriou b a.deartment of Statistics University of Lund
More informationarxiv: v3 [physics.data-an] 23 May 2011
Date: October, 8 arxiv:.7v [hysics.data-an] May -values for Model Evaluation F. Beaujean, A. Caldwell, D. Kollár, K. Kröninger Max-Planck-Institut für Physik, München, Germany CERN, Geneva, Switzerland
More informationPublished: 14 October 2013
Electronic Journal of Alied Statistical Analysis EJASA, Electron. J. A. Stat. Anal. htt://siba-ese.unisalento.it/index.h/ejasa/index e-issn: 27-5948 DOI: 1.1285/i275948v6n213 Estimation of Parameters of
More informationIntroductory Bayesian Analysis
Introductor Baesian Analsis Jaa M. Sataoan Memorial Sloan-Ketterin Cancer Center Aril, 008 Baesian Inference Fit robabilit model to observed data Unknown arameters Summarize usin robabilit distribution
More informationEcon 3790: Business and Economics Statistics. Instructor: Yogesh Uppal
Econ 379: Business and Economics Statistics Instructor: Yogesh Ual Email: yual@ysu.edu Chater 9, Part A: Hyothesis Tests Develoing Null and Alternative Hyotheses Tye I and Tye II Errors Poulation Mean:
More informationEcon 3790: Business and Economics Statistics. Instructor: Yogesh Uppal
Econ 379: Business and Economics Statistics Instructor: Yogesh Ual Email: yual@ysu.edu Chater 9, Part A: Hyothesis Tests Develoing Null and Alternative Hyotheses Tye I and Tye II Errors Poulation Mean:
More informationOne-way ANOVA Inference for one-way ANOVA
One-way ANOVA Inference for one-way ANOVA IPS Chater 12.1 2009 W.H. Freeman and Comany Objectives (IPS Chater 12.1) Inference for one-way ANOVA Comaring means The two-samle t statistic An overview of ANOVA
More informationIntroduction to Probability for Graphical Models
Introduction to Probability for Grahical Models CSC 4 Kaustav Kundu Thursday January 4, 06 *Most slides based on Kevin Swersky s slides, Inmar Givoni s slides, Danny Tarlow s slides, Jaser Snoek s slides,
More informationHotelling s Two- Sample T 2
Chater 600 Hotelling s Two- Samle T Introduction This module calculates ower for the Hotelling s two-grou, T-squared (T) test statistic. Hotelling s T is an extension of the univariate two-samle t-test
More informationarxiv: v1 [physics.data-an] 26 Oct 2012
Constraints on Yield Parameters in Extended Maximum Likelihood Fits Till Moritz Karbach a, Maximilian Schlu b a TU Dortmund, Germany, moritz.karbach@cern.ch b TU Dortmund, Germany, maximilian.schlu@cern.ch
More informationScaling Multiple Point Statistics for Non-Stationary Geostatistical Modeling
Scaling Multile Point Statistics or Non-Stationary Geostatistical Modeling Julián M. Ortiz, Steven Lyster and Clayton V. Deutsch Centre or Comutational Geostatistics Deartment o Civil & Environmental Engineering
More informationSlides Prepared by JOHN S. LOUCKS St. Edward s s University Thomson/South-Western. Slide
s Preared by JOHN S. LOUCKS St. Edward s s University 1 Chater 11 Comarisons Involving Proortions and a Test of Indeendence Inferences About the Difference Between Two Poulation Proortions Hyothesis Test
More informationBayesian Networks Practice
Bayesian Networks Practice Part 2 2016-03-17 Byoung-Hee Kim, Seong-Ho Son Biointelligence Lab, CSE, Seoul National University Agenda Probabilistic Inference in Bayesian networks Probability basics D-searation
More informationPerformance of lag length selection criteria in three different situations
MPRA Munich Personal RePEc Archive Performance of lag length selection criteria in three different situations Zahid Asghar and Irum Abid Quaid-i-Azam University, Islamabad Aril 2007 Online at htts://mra.ub.uni-muenchen.de/40042/
More informationPlotting the Wilson distribution
, Survey of English Usage, University College London Setember 018 1 1. Introduction We have discussed the Wilson score interval at length elsewhere (Wallis 013a, b). Given an observed Binomial roortion
More informationA Bound on the Error of Cross Validation Using the Approximation and Estimation Rates, with Consequences for the Training-Test Split
A Bound on the Error of Cross Validation Using the Aroximation and Estimation Rates, with Consequences for the Training-Test Slit Michael Kearns AT&T Bell Laboratories Murray Hill, NJ 7974 mkearns@research.att.com
More informationNamed Entity Recognition using Maximum Entropy Model SEEM5680
Named Entity Recognition using Maximum Entroy Model SEEM5680 Named Entity Recognition System Named Entity Recognition (NER): Identifying certain hrases/word sequences in a free text. Generally it involves
More informationJohn Weatherwax. Analysis of Parallel Depth First Search Algorithms
Sulementary Discussions and Solutions to Selected Problems in: Introduction to Parallel Comuting by Viin Kumar, Ananth Grama, Anshul Guta, & George Karyis John Weatherwax Chater 8 Analysis of Parallel
More informationYixi Shi. Jose Blanchet. IEOR Department Columbia University New York, NY 10027, USA. IEOR Department Columbia University New York, NY 10027, USA
Proceedings of the 2011 Winter Simulation Conference S. Jain, R. R. Creasey, J. Himmelsach, K. P. White, and M. Fu, eds. EFFICIENT RARE EVENT SIMULATION FOR HEAVY-TAILED SYSTEMS VIA CROSS ENTROPY Jose
More informationHypothesis Test-Confidence Interval connection
Hyothesis Test-Confidence Interval connection Hyothesis tests for mean Tell whether observed data are consistent with μ = μ. More secifically An hyothesis test with significance level α will reject the
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationSPARSE SIGNAL RECOVERY USING A BERNOULLI GENERALIZED GAUSSIAN PRIOR
3rd Euroean Signal Processing Conference EUSIPCO SPARSE SIGNAL RECOVERY USING A BERNOULLI GENERALIZED GAUSSIAN PRIOR Lotfi Chaari 1, Jean-Yves Tourneret 1 and Caroline Chaux 1 University of Toulouse, IRIT
More informationBiitlli Biointelligence Laboratory Lb School of Computer Science and Engineering. Seoul National University
Monte Carlo Samling Chater 4 2009 Course on Probabilistic Grahical Models Artificial Neural Networs, Studies in Artificial i lintelligence and Cognitive i Process Biitlli Biointelligence Laboratory Lb
More informationMATHEMATICAL MODELLING OF THE WIRELESS COMMUNICATION NETWORK
Comuter Modelling and ew Technologies, 5, Vol.9, o., 3-39 Transort and Telecommunication Institute, Lomonosov, LV-9, Riga, Latvia MATHEMATICAL MODELLIG OF THE WIRELESS COMMUICATIO ETWORK M. KOPEETSK Deartment
More informationUnderstanding and Using Availability
Understanding and Using Availability Jorge Luis Romeu, Ph.D. ASQ CQE/CRE, & Senior Member C. Stat Fellow, Royal Statistical Society Past Director, Region II (NY & PA) Director: Juarez Lincoln Marti Int
More informationHENSEL S LEMMA KEITH CONRAD
HENSEL S LEMMA KEITH CONRAD 1. Introduction In the -adic integers, congruences are aroximations: for a and b in Z, a b mod n is the same as a b 1/ n. Turning information modulo one ower of into similar
More informationLower Confidence Bound for Process-Yield Index S pk with Autocorrelated Process Data
Quality Technology & Quantitative Management Vol. 1, No.,. 51-65, 15 QTQM IAQM 15 Lower onfidence Bound for Process-Yield Index with Autocorrelated Process Data Fu-Kwun Wang * and Yeneneh Tamirat Deartment
More informationBayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007
Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.
More informationThe Longest Run of Heads
The Longest Run of Heads Review by Amarioarei Alexandru This aer is a review of older and recent results concerning the distribution of the longest head run in a coin tossing sequence, roblem that arise
More informationModeling and Estimation of Full-Chip Leakage Current Considering Within-Die Correlation
6.3 Modeling and Estimation of Full-Chi Leaage Current Considering Within-Die Correlation Khaled R. eloue, Navid Azizi, Farid N. Najm Deartment of ECE, University of Toronto,Toronto, Ontario, Canada {haled,nazizi,najm}@eecg.utoronto.ca
More informationUse of Transformations and the Repeated Statement in PROC GLM in SAS Ed Stanek
Use of Transformations and the Reeated Statement in PROC GLM in SAS Ed Stanek Introduction We describe how the Reeated Statement in PROC GLM in SAS transforms the data to rovide tests of hyotheses of interest.
More information7.2 Inference for comparing means of two populations where the samples are independent
Objectives 7.2 Inference for comaring means of two oulations where the samles are indeendent Two-samle t significance test (we give three examles) Two-samle t confidence interval htt://onlinestatbook.com/2/tests_of_means/difference_means.ht
More informationCollaborative Place Models Supplement 1
Collaborative Place Models Sulement Ber Kaicioglu Foursquare Labs ber.aicioglu@gmail.com Robert E. Schaire Princeton University schaire@cs.rinceton.edu David S. Rosenberg P Mobile Labs david.davidr@gmail.com
More informationBayesian classification CISC 5800 Professor Daniel Leeds
Bayesian classification CISC 5800 Professor Daniel Leeds Classifying with robabilities Examle goal: Determine is it cloudy out Available data: Light detector: x 0,25 Potential class (atmosheric states):
More informationProbability Estimates for Multi-class Classification by Pairwise Coupling
Probability Estimates for Multi-class Classification by Pairwise Couling Ting-Fan Wu Chih-Jen Lin Deartment of Comuter Science National Taiwan University Taiei 06, Taiwan Ruby C. Weng Deartment of Statistics
More informationEvaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models
Evaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models Ketan N. Patel, Igor L. Markov and John P. Hayes University of Michigan, Ann Arbor 48109-2122 {knatel,imarkov,jhayes}@eecs.umich.edu
More informationUnderstanding and Using Availability
Understanding and Using Availability Jorge Luis Romeu, Ph.D. ASQ CQE/CRE, & Senior Member Email: romeu@cortland.edu htt://myrofile.cos.com/romeu ASQ/RD Webinar Series Noviembre 5, J. L. Romeu - Consultant
More informationAn Analysis of Reliable Classifiers through ROC Isometrics
An Analysis of Reliable Classifiers through ROC Isometrics Stijn Vanderlooy s.vanderlooy@cs.unimaas.nl Ida G. Srinkhuizen-Kuyer kuyer@cs.unimaas.nl Evgueni N. Smirnov smirnov@cs.unimaas.nl MICC-IKAT, Universiteit
More informationarxiv:cond-mat/ v2 25 Sep 2002
Energy fluctuations at the multicritical oint in two-dimensional sin glasses arxiv:cond-mat/0207694 v2 25 Se 2002 1. Introduction Hidetoshi Nishimori, Cyril Falvo and Yukiyasu Ozeki Deartment of Physics,
More informationA Time-Varying Threshold STAR Model of Unemployment
A Time-Varying Threshold STAR Model of Unemloyment michael dueker a michael owyang b martin sola c,d a Russell Investments b Federal Reserve Bank of St. Louis c Deartamento de Economia, Universidad Torcuato
More informationRound-off Errors and Computer Arithmetic - (1.2)
Round-off Errors and Comuter Arithmetic - (.). Round-off Errors: Round-off errors is roduced when a calculator or comuter is used to erform real number calculations. That is because the arithmetic erformed
More informationA Qualitative Event-based Approach to Multiple Fault Diagnosis in Continuous Systems using Structural Model Decomposition
A Qualitative Event-based Aroach to Multile Fault Diagnosis in Continuous Systems using Structural Model Decomosition Matthew J. Daigle a,,, Anibal Bregon b,, Xenofon Koutsoukos c, Gautam Biswas c, Belarmino
More informationBayesian Phylogenetics
Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 Bayesian Phylogenetics 1 / 27 Who was Bayes? The Reverand Thomas Bayes was born
More informationUnsupervised Hyperspectral Image Analysis Using Independent Component Analysis (ICA)
Unsuervised Hyersectral Image Analysis Using Indeendent Comonent Analysis (ICA) Shao-Shan Chiang Chein-I Chang Irving W. Ginsberg Remote Sensing Signal and Image Processing Laboratory Deartment of Comuter
More informationShort course A vademecum of statistical pattern recognition techniques with applications to image and video analysis. Agenda
Short course A vademecum of statistical attern recognition techniques with alications to image and video analysis Lecture 6 The Kalman filter. Particle filters Massimo Piccardi University of Technology,
More informationMODELING THE RELIABILITY OF C4ISR SYSTEMS HARDWARE/SOFTWARE COMPONENTS USING AN IMPROVED MARKOV MODEL
Technical Sciences and Alied Mathematics MODELING THE RELIABILITY OF CISR SYSTEMS HARDWARE/SOFTWARE COMPONENTS USING AN IMPROVED MARKOV MODEL Cezar VASILESCU Regional Deartment of Defense Resources Management
More informationWho was Bayes? Bayesian Phylogenetics. What is Bayes Theorem?
Who was Bayes? Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 The Reverand Thomas Bayes was born in London in 1702. He was the
More informationAn Improved Calibration Method for a Chopped Pyrgeometer
96 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 17 An Imroved Calibration Method for a Choed Pyrgeometer FRIEDRICH FERGG OtoLab, Ingenieurbüro, Munich, Germany PETER WENDLING Deutsches Forschungszentrum
More informationMODEL-BASED MULTIPLE FAULT DETECTION AND ISOLATION FOR NONLINEAR SYSTEMS
MODEL-BASED MULIPLE FAUL DEECION AND ISOLAION FOR NONLINEAR SYSEMS Ivan Castillo, and homas F. Edgar he University of exas at Austin Austin, X 78712 David Hill Chemstations Houston, X 77009 Abstract A
More informationA Comparison between Biased and Unbiased Estimators in Ordinary Least Squares Regression
Journal of Modern Alied Statistical Methods Volume Issue Article 7 --03 A Comarison between Biased and Unbiased Estimators in Ordinary Least Squares Regression Ghadban Khalaf King Khalid University, Saudi
More informationEmpirical Bayesian EM-based Motion Segmentation
Emirical Bayesian EM-based Motion Segmentation Nuno Vasconcelos Andrew Liman MIT Media Laboratory 0 Ames St, E5-0M, Cambridge, MA 09 fnuno,lig@media.mit.edu Abstract A recent trend in motion-based segmentation
More informationSupplementary Materials for Robust Estimation of the False Discovery Rate
Sulementary Materials for Robust Estimation of the False Discovery Rate Stan Pounds and Cheng Cheng This sulemental contains roofs regarding theoretical roerties of the roosed method (Section S1), rovides
More information¼ ¼ 6:0. sum of all sample means in ð8þ 25
1. Samling Distribution of means. A oulation consists of the five numbers 2, 3, 6, 8, and 11. Consider all ossible samles of size 2 that can be drawn with relacement from this oulation. Find the mean of
More informationMorten Frydenberg Section for Biostatistics Version :Friday, 05 September 2014
Morten Frydenberg Section for Biostatistics Version :Friday, 05 Setember 204 All models are aroximations! The best model does not exist! Comlicated models needs a lot of data. lower your ambitions or get
More informationPhylogenetics: Bayesian Phylogenetic Analysis. COMP Spring 2015 Luay Nakhleh, Rice University
Phylogenetics: Bayesian Phylogenetic Analysis COMP 571 - Spring 2015 Luay Nakhleh, Rice University Bayes Rule P(X = x Y = y) = P(X = x, Y = y) P(Y = y) = P(X = x)p(y = y X = x) P x P(X = x 0 )P(Y = y X
More informationMaximum Entropy and the Stress Distribution in Soft Disk Packings Above Jamming
Maximum Entroy and the Stress Distribution in Soft Disk Packings Above Jamming Yegang Wu and S. Teitel Deartment of Physics and Astronomy, University of ochester, ochester, New York 467, USA (Dated: August
More informationApproximating min-max k-clustering
Aroximating min-max k-clustering Asaf Levin July 24, 2007 Abstract We consider the roblems of set artitioning into k clusters with minimum total cost and minimum of the maximum cost of a cluster. The cost
More informationCharacteristics of Beam-Based Flexure Modules
Shorya Awtar e-mail: shorya@mit.edu Alexander H. Slocum e-mail: slocum@mit.edu Precision Engineering Research Grou, Massachusetts Institute of Technology, Cambridge, MA 039 Edi Sevincer Omega Advanced
More informationInformation collection on a graph
Information collection on a grah Ilya O. Ryzhov Warren Powell February 10, 2010 Abstract We derive a knowledge gradient olicy for an otimal learning roblem on a grah, in which we use sequential measurements
More informationSTK4900/ Lecture 7. Program
STK4900/9900 - Lecture 7 Program 1. Logistic regression with one redictor 2. Maximum likelihood estimation 3. Logistic regression with several redictors 4. Deviance and likelihood ratio tests 5. A comment
More informationBayesian Inference. Chapter 1. Introduction and basic concepts
Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master
More informationEstimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
More informationThe Poisson Regression Model
The Poisson Regression Model The Poisson regression model aims at modeling a counting variable Y, counting the number of times that a certain event occurs during a given time eriod. We observe a samle
More informationOn parameter estimation in deformable models
Downloaded from orbitdtudk on: Dec 7, 07 On arameter estimation in deformable models Fisker, Rune; Carstensen, Jens Michael Published in: Proceedings of the 4th International Conference on Pattern Recognition
More informationAnalysis of execution time for parallel algorithm to dertmine if it is worth the effort to code and debug in parallel
Performance Analysis Introduction Analysis of execution time for arallel algorithm to dertmine if it is worth the effort to code and debug in arallel Understanding barriers to high erformance and redict
More information6 Stationary Distributions
6 Stationary Distributions 6. Definition and Examles Definition 6.. Let {X n } be a Markov chain on S with transition robability matrix P. A distribution π on S is called stationary (or invariant) if π
More informationEstimation of Separable Representations in Psychophysical Experiments
Estimation of Searable Reresentations in Psychohysical Exeriments Michele Bernasconi (mbernasconi@eco.uninsubria.it) Christine Choirat (cchoirat@eco.uninsubria.it) Raffaello Seri (rseri@eco.uninsubria.it)
More informationOutline. Markov Chains and Markov Models. Outline. Markov Chains. Markov Chains Definitions Huizhen Yu
and Markov Models Huizhen Yu janey.yu@cs.helsinki.fi Det. Comuter Science, Univ. of Helsinki Some Proerties of Probabilistic Models, Sring, 200 Huizhen Yu (U.H.) and Markov Models Jan. 2 / 32 Huizhen Yu
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More informationRobust Bayesian analysis of loss reserves data using the generalized-t distribution
QUANTITATIVE FINANCE RESEARCH CENTRE QUANTITATIVE FINANCE RESEARCH CENTRE Research Paer 96 May 007 Robust Bayesian analysis of loss reserves data using the generalized-t distribution Jennifer Chan, Boris
More informationPER-PATCH METRIC LEARNING FOR ROBUST IMAGE MATCHING. Sezer Karaoglu, Ivo Everts, Jan C. van Gemert, and Theo Gevers
PER-PATCH METRIC LEARNING FOR ROBUST IMAGE MATCHING Sezer Karaoglu, Ivo Everts, Jan C. van Gemert, and Theo Gevers Intelligent Systems Lab, Amsterdam, University of Amsterdam, 1098 XH Amsterdam, The Netherlands
More informationThe Binomial Approach for Probability of Detection
Vol. No. (Mar 5) - The e-journal of Nondestructive Testing - ISSN 45-494 www.ndt.net/?id=7498 The Binomial Aroach for of Detection Carlos Correia Gruo Endalloy C.A. - Caracas - Venezuela www.endalloy.net
More information