Statistics for Particle Physics. Kyle Cranmer. New York University. Kyle Cranmer (NYU) CERN Academic Training, Feb 2-5, 2009
|
|
- Edwina Carroll
- 5 years ago
- Views:
Transcription
1 Statistics for Particle Physics Kyle Cranmer New York University 1
2 Hypothesis Testing 55
3 Hypothesis testing One of the most common uses of statistics in particle physics is Hypothesis Testing! assume one has pdf for data under two hypotheses: " Null-Hypothesis, H 0: eg. background-only " Alternate-Hypothesis H 1: eg. signal-plus-background! one makes a measurement and then needs to decide whether to reject or accept H0 Probability ±10 50 Events Events Observed 56
4 Hypothesis testing One of the most common uses of statistics in particle physics is Hypothesis Testing! assume one has pdf for data under two hypotheses: " Null-Hypothesis, H 0: eg. background-only " Alternate-Hypothesis H 1: eg. signal-plus-background! one makes a measurement and then needs to decide whether to reject or accept H0 Probability ±10 50 Events Events Observed 56
5 Hypothesis testing Before we can make much progress with statistics, we need to decide what it is that we want to do.! first let us define a few terms: " Rate of Type I error α " Rate of Type II " Power = 1 β β 57
6 Hypothesis testing Before we can make much progress with statistics, we need to decide what it is that we want to do.! first let us define a few terms: " Rate of Type I error α " Rate of Type II " Power = 1 β β Treat the two hypotheses asymmetrically! the Null is special. " Fix rate of Type I error, call it the size of the test 57
7 Hypothesis testing Before we can make much progress with statistics, we need to decide what it is that we want to do.! first let us define a few terms: " Rate of Type I error α " Rate of Type II " Power = 1 β β Treat the two hypotheses asymmetrically! the Null is special. " Fix rate of Type I error, call it the size of the test Now one can state a well-defined goal! Maximize power for a fixed rate of Type I error 57
8 Hypothesis testing The idea of a 5σ discovery criteria for particle physics is really a conventional way to specify the size of the test! usually 5σ corresponds to α = " eg. a very small chance we reject the standard model In the simple case of number counting it is obvious what region is sensitive to the presence of a new signal! but in higher dimensions it is not so easy Probability Events Events Observed ±10 58
9 Hypothesis testing The idea of a 5σ discovery criteria for particle physics is really a conventional way to specify the size of the test! usually 5σ corresponds to α = " eg. a very small chance we reject the standard model In the simple case of number counting it is obvious what region is sensitive to the presence of a new signal! but in higher dimensions it is not so easy x 1 x 2 x x x x 1 58
10 Hypothesis testing The idea of a 5σ discovery criteria for particle physics is really a conventional way to specify the size of the test! usually 5σ corresponds to α = " eg. a very small chance we reject the standard model In the simple case of number counting it is obvious what region is sensitive to the presence of a new signal! but in higher dimensions it is not so easy Probability Events Events Observed ±10 59
11 Hypothesis testing The idea of a 5σ discovery criteria for particle physics is really a conventional way to specify the size of the test! usually 5σ corresponds to α = " eg. a very small chance we reject the standard model In the simple case of number counting it is obvious what region is sensitive to the presence of a new signal! but in higher dimensions it is not so easy ecision bounda H 1 [G. Cowan] ion boundary: H Probability 50 Events Events Observed ±10 59
12 Hypothesis testing The idea of a 5σ discovery criteria for particle physics is really a conventional way to specify the size of the test! usually 5σ corresponds to α = " eg. a very small chance we reject the standard model In the simple case of number counting it is obvious what region is sensitive to the presence of a new signal! but in higher dimensions it is not so easy ecision bounda H 1 [G. Cowan] ion boundary: H H Probability 50 Events Events Observed ±10 H 0 H 1 59
13 The Neyman-Pearson Lemma In Neyman & Pearson developed a theory in which one must consider competing Hypotheses: - the Null Hypothesis H 0 (background only) - the Alternate Hypothesis H 1 (signal-plus-background) Given some probability that we wrongly reject the Null Hypothesis α = P (x / W H 0 ) (Convention: if data falls in W then we accept H0) Find the region W such that we minimize the probability of wrongly accepting the H 0 (when H 1 is true) β = P (x W H 1 ) 60
14 The Neyman-Pearson Lemma The region accepting H 0 W that minimizes the probability of wrongly is just a contour of the Likelihood Ratio P (x H 1 ) P (x H 0 ) >k α Any other region of the same size will have less power The likelihood ratio is an example of a Test Statistic, eg. a real-valued function that summarizes the data in a way relevant to the hypotheses that are being tested 61
15 A short proof of Neyman-Pearson W W C P (x H 1 ) P (x H 0 ) >k α Consider the contour of the likelihood ratio that has size a given size (eg. probability under H0 is 1- ) α 62
16 A short proof of Neyman-Pearson Now consider a variation on the contour that has the same size 63
17 A short proof of Neyman-Pearson P ( H 0 )=P ( H 0 ) Now consider a variation on the contour that has the same size (eg. same probability under H0) 64
18 A short proof of Neyman-Pearson P (x H 1 ) P (x H 0 ) <k α P ( H 0 )=P ( H 0 ) P ( H 1 ) <P( H 0 )k α Because the new area is outside the contour of the likelihood ratio, we have an inequality 65
19 A short proof of Neyman-Pearson P (x H 1 ) P (x H 0 ) <k α P ( H 0 )=P ( H 0 ) P (x H 1 ) P (x H 0 ) >k α P ( H 1 ) <P( H 0 ) k α P ( H 1 ) >P( H 0 ) k α And for the region we lost, we also have an inequality Together they give... 66
20 A short proof of Neyman-Pearson P (x H 1 ) P (x H 0 ) <k α P ( H 0 )=P ( H 0 ) P (x H 1 ) P (x H 0 ) >k α P ( H 1 ) <P( H 0 ) k α P ( H 1 ) >P( H 0 ) k α P ( H 1 ) <P( H 1 ) The new region region has less power. 67
21 Decision Theory One of the deficiencies of the Neyman-Pearson approach is that one must specify the size of the test α! But where does come from? " is it purely conventional or is there a reason? α A great deal of literature related to statistics (and economics, etc.) is devoted to making decisions.! need to consider Utility or Risk of di#erent outcomes In the context of decision and utility theory there can be a justification, but this is rarely done in particle physics 68
22 Probability density An explicit likelihood ratio In that case: (a) Q = L(x H 1) L(x H 0 ) = Nchan LEP Observed m H = 115 GeV/c 2 Expected for background Expected for signal plus background i P ois(n i s i + b i ) n i Nchan N chan q = ln Q = s tot -2 ln(q) -2 ln(q) i i j s i f s (x ij )+b i f b (x ij ) s i +b i P ois(n i b i ) n i j f b (x ij ) ( ln 1+ s ) if s (x ij ) b i f b (x ij ) n i j Observed Expected for background Expected for signal plus background LEP m (GeV/c 2 ) 69
23 Kernel Estimation Kernel estimation is the generalization of Average Shifted Histograms ˆf 1 (x) = h(x i )= n i ( nh(x i ) K ( ) x xi h(x i ) ) 1/5 σ ˆf 0 (x i ) n 1/5 Probability Density K.Cranmer, Comput.Phys.Commun. 136 (2001). [hep-ex/ ] the data is the model Neural Network Output Adaptive Kernel estimation puts wider kernels in regions of low probability Used at LEP for describing pdfs from Monte Carlo (KEYS) 33
24 2 discriminating variables Often one uses the output of a neural network or multivariate algorithm in place of a true likelihood ratio.! That s fine, but what do you do with it?! If you have a fixed cut for all events, this is what you are doing: y 1 y 2 x 1 x 2 ( q = ln Q = s + ln 1+ sf ) s(x, y) bf b (x, y) q 70
25 Experiments vs. Events Ideally, you want to cut on the likelihood ratio for your experiment! equivalent to a sum of log likelihood ratios Easy to see that includes experiments where one event had a high LR and the other one was relatively small q 2 q 12 = q 1 + q 2 q 1 q 1 q 2 q 12 y 1 y 2 x 1 x 2 71
26 Decision Theory α α From Fred James lectures 72
27 Decisions: Bayesian & Frequentist Structure of P(x H0) & P(x H1) puts limits on allowable ranges of alpha, beta! Bayesians want to minimize expected risk based on priors and risk/utility of outcomes Frequentists don t have priors to work with, so they only have risk/ utility in two situations! minimax approach aims to minimize maximum risk " most conservative F. James, Ch. 6 " paranoid for games against nature Frequentist choice of alpha interpreted in Bayesian framework implies this ratio: OD OA = l 0 µ l 1 (1 µ) l 1 (1 µ)p ( X H 1 ) < l 0 µp ( X H 0 ) P (X H 1 ) P (X H 0 ) < l 0 µ l 1 (1 µ) 73
28 A Few Slides on Multivariate Algorithms 74
29 Use of Multivariate Methods Multivariate methods are now ubiquitous in high-energy physics, the nagging problem is that:! most multivariate techniques are borrowed from other fields, and they optimize some heuristic that physicists aren t interested in (like a score, or ad hoc training error)! the di#erence can be quite large when systematic uncertainties are taken into account A few recent developments! Evolutionary techniques! Matrix Element techniques ] 2 Statistical Uncertainty [GeV/c Whiteson & Whiteson, hep-ex/ ± ± ± 0.5 Heuristic Binary-C Multi-class 9.1 ± 0.4 Binary-M 7.3 ± ± 0.2 NEAT classes NEAT features 75
30 The Neyman-Pearson Lemma The region W that minimizes the probability of wrongly accepting the H 0 is just a contour of the Likelihood Ratio: L(x H 0 ) L(x H 1 ) > k α This is the goal! The problem is we don t have access to L(x H 0 ) & L(x H 1 ) 76
31 The Neyman-Pearson Lemma The region W that minimizes the probability of wrongly accepting the H 0 is just a contour of the Likelihood Ratio: L(x H 0 ) L(x H 1 ) > k α This is the goal! W L(x H 0 ) = H µ+ The problem is we don t have access to L(x H 0 ) & L(x H 1 ) W µ 76
32 Matrix Element Techniques Instead of using generic machine learning algorithms, some members of the Tevatron experiments are starting to attack this convolution numerically L(x H 0 ) = W Phase-space Integral W H µ+ Matrixµ Element Transfer Functions 77
33 Matrix Element Techniques Instead of using generic machine learning algorithms, some members of the Tevatron experiments are starting to attack this convolution numerically Phase-space Integral Matrix Element Transfer Functions 77
34 Matrix Element Techniques for Theorists A few years ago, I realized that phenomenologists doing sensitivity studies can use the Neyman-Pearson lemma directly! directly integrate likelihood ratio! model detector e#ects with transfer functions " numerically much easier than experimental situation because one generates hypothetical data! just as one computes a cross-section for a new signal, one can compute a maximum significance (at leading order) Experimental: x ~ observable Q(x) = L(x H n 1) Pois(n s + b) j = f s+b(x j ) L(x H 0 ) Pois(n b) n j f b(x j ) n ( q(x) ln Q(x) = s + ln 1 + sf ) s(x j ) bf b (x j ) j=1 Theoretical: r ~ phase space q( r) = σ tot,s L + ln ( 1 + dσ ) s( r) dσ b ( r) Cranmer, Plehn. EPJ & hep-ph/
35 79
36 Learning Machines 80
37 Examples of Learning Machines y 4 Cuts can be viewed as learning machines x f = 1 <x< 1 2 and <y< else 1 2 Neural Nets can be viewed as learning machines weights & biases make up the parameters Input Units Output Unit Hidden Layers: Processing Units 81
38 Statistical Learning Theory & Hypothesis Testing exactly 82
39 Limits on Risk h(log(2l/h) + 1) log(η/4) l VC Confidence For Sample Size of 10,000 95% Confidence Level h / l = VC Dimension / Sample Size 83
40 Limits on Risk h(log(2l/h) + 1) log(η/4) l VC Confidence For Sample Size of 10,000 95% Confidence Level h / l = VC Dimension / Sample Size Support Vector Machines aim to minimize the limit on Risk by balancing Remp and complexity of learning machine characterized by h 83
41 Some personal history Archbishop of Canterbury Thomas Cranmer (born: 1489, executed: 1556) author of the Book of Common Prayer Kyle Cranmer (NYU) Two centuries later (when this Book had become an official prayer book of the Church of England) Thomas Bayes was a non-conformist minister (Presbyterian) who refused to use Cranmer!s book CERN Academic Training, Feb 2-5,
42 VC Dimension 85
43 Importance of VC Dimension 86
44 Importance of VC Dimension Because we usually have an independent testing set, the limit on true Risk is often not very useful in practice 86
45 Genetic Programming R.S. Bowman and I brought a technique called Genetic Programming to HEP. It s a program that actually writes programs to search for the Higgs! Comput. Phys. Commun [physics/ ] XOR AND <=> min > Iso1 POT XOR σm The FOCUS collaboration has recently used Genetic Programming to study doubly Cabibbo suppressed decay of D + K + π + π relative to Cabbibo favored D + K π + π + Iso2 p OoT π e2 NOT POT σm hep-ex/ a) Selected CF b) Selected DCS 2 Events/5 MeV/c Yield = ± Yield = 466 ± GeV/c GeV/c 87
46 Remaining Lectures Lecture 3:! The Neyman-Construction (illustrated)! Inverted hypothesis tests: A dictionary for limits (intervals)! Coverage as a calibration for our statistical device! Compound hypotheses, nuisance parameters, & similar tests! Systematics, Systematics, Systematics Lecture 4:! Generalizing our procedures to include systematics! Eliminating nuisance parameters: profiling and marginalization! Introduction to ancillary statistics & conditioning! High dimensional models, Markov Chain Monte Carlo, and Hierarchical Bayes! The look elsewhere e#ect and false discovery rate 88
Statistics for Particle Physics. Kyle Cranmer. New York University. Kyle Cranmer (NYU) CERN Academic Training, Feb 2-5, 2009
Statistics for Particle Physics Kyle Cranmer New York University 91 Remaining Lectures Lecture 3:! Compound hypotheses, nuisance parameters, & similar tests! The Neyman-Construction (illustrated)! Inverted
More informationStatistics for the LHC Lecture 1: Introduction
Statistics for the LHC Lecture 1: Introduction Academic Training Lectures CERN, 14 17 June, 2010 indico.cern.ch/conferencedisplay.py?confid=77830 Glen Cowan Physics Department Royal Holloway, University
More informationStatistics for the LHC Lecture 2: Discovery
Statistics for the LHC Lecture 2: Discovery Academic Training Lectures CERN, 14 17 June, 2010 indico.cern.ch/conferencedisplay.py?confid=77830 Glen Cowan Physics Department Royal Holloway, University of
More informationStatistical Methods for Particle Physics (I)
Statistical Methods for Particle Physics (I) https://agenda.infn.it/conferencedisplay.py?confid=14407 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationHypothesis testing (cont d)
Hypothesis testing (cont d) Ulrich Heintz Brown University 4/12/2016 Ulrich Heintz - PHYS 1560 Lecture 11 1 Hypothesis testing Is our hypothesis about the fundamental physics correct? We will not be able
More informationStatistical Methods in Particle Physics Lecture 1: Bayesian methods
Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationLecture 5. G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1
Lecture 5 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationStatistical Methods for Particle Physics Lecture 1: parameter estimation, statistical tests
Statistical Methods for Particle Physics Lecture 1: parameter estimation, statistical tests http://benasque.org/2018tae/cgi-bin/talks/allprint.pl TAE 2018 Benasque, Spain 3-15 Sept 2018 Glen Cowan Physics
More informationLecture 2. G. Cowan Lectures on Statistical Data Analysis Lecture 2 page 1
Lecture 2 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationHypothesis Testing - Frequentist
Frequentist Hypothesis Testing - Frequentist Compare two hypotheses to see which one better explains the data. Or, alternatively, what is the best way to separate events into two classes, those originating
More informationStatistical Methods in Particle Physics Lecture 2: Limits and Discovery
Statistical Methods in Particle Physics Lecture 2: Limits and Discovery SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationMultivariate statistical methods and data mining in particle physics
Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general
More informationStatistical Challenges of the LHC. Kyle Cranmer
Statistical Challenges of the LHC Outline: - The state of the art of frequentist hypothesis testing with no systematics - Including systematics: translating confidence intervals to hypothesis testing -
More informationUse of the likelihood principle in physics. Statistics II
Use of the likelihood principle in physics Statistics II 1 2 3 + Bayesians vs Frequentists 4 Why ML does work? hypothesis observation 5 6 7 8 9 10 11 ) 12 13 14 15 16 Fit of Histograms corresponds This
More informationSystematic uncertainties in statistical data analysis for particle physics. DESY Seminar Hamburg, 31 March, 2009
Systematic uncertainties in statistical data analysis for particle physics DESY Seminar Hamburg, 31 March, 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationPrimer on statistics:
Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood
More informationStatistical Methods for Particle Physics Lecture 2: statistical tests, multivariate methods
Statistical Methods for Particle Physics Lecture 2: statistical tests, multivariate methods www.pp.rhul.ac.uk/~cowan/stat_aachen.html Graduierten-Kolleg RWTH Aachen 10-14 February 2014 Glen Cowan Physics
More informationStatistical Data Analysis Stat 3: p-values, parameter estimation
Statistical Data Analysis Stat 3: p-values, parameter estimation London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway,
More informationAdvanced statistical methods for data analysis Lecture 1
Advanced statistical methods for data analysis Lecture 1 RHUL Physics www.pp.rhul.ac.uk/~cowan Universität Mainz Klausurtagung des GK Eichtheorien exp. Tests... Bullay/Mosel 15 17 September, 2008 1 Outline
More informationStatistical Methods for Particle Physics Lecture 4: discovery, exclusion limits
Statistical Methods for Particle Physics Lecture 4: discovery, exclusion limits www.pp.rhul.ac.uk/~cowan/stat_aachen.html Graduierten-Kolleg RWTH Aachen 10-14 February 2014 Glen Cowan Physics Department
More informationJourneys of an Accidental Statistician
Journeys of an Accidental Statistician A partially anecdotal account of A Unified Approach to the Classical Statistical Analysis of Small Signals, GJF and Robert D. Cousins, Phys. Rev. D 57, 3873 (1998)
More informationPhysics 403. Segev BenZvi. Classical Hypothesis Testing: The Likelihood Ratio Test. Department of Physics and Astronomy University of Rochester
Physics 403 Classical Hypothesis Testing: The Likelihood Ratio Test Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Bayesian Hypothesis Testing Posterior Odds
More informationSome Statistical Tools for Particle Physics
Some Statistical Tools for Particle Physics Particle Physics Colloquium MPI für Physik u. Astrophysik Munich, 10 May, 2016 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More informationIntroduction to Statistical Methods for High Energy Physics
Introduction to Statistical Methods for High Energy Physics 2011 CERN Summer Student Lectures Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationStatistical Methods for Particle Physics Lecture 3: Systematics, nuisance parameters
Statistical Methods for Particle Physics Lecture 3: Systematics, nuisance parameters http://benasque.org/2018tae/cgi-bin/talks/allprint.pl TAE 2018 Centro de ciencias Pedro Pascual Benasque, Spain 3-15
More informationPhysics 403. Segev BenZvi. Credible Intervals, Confidence Intervals, and Limits. Department of Physics and Astronomy University of Rochester
Physics 403 Credible Intervals, Confidence Intervals, and Limits Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Summarizing Parameters with a Range Bayesian
More informationHypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006
Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312 This draft, April 20, 2006 1 1 A Brief Review of Hypothesis Testing and Its Uses values and pure significance tests (R.A. Fisher)
More informationParameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
More informationSome Topics in Statistical Data Analysis
Some Topics in Statistical Data Analysis Invisibles School IPPP Durham July 15, 2013 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan G. Cowan
More informationTopics in Statistical Data Analysis for HEP Lecture 1: Bayesian Methods CERN-JINR European School of High Energy Physics Bautzen, June 2009
Topics in Statistical Data Analysis for HEP Lecture 1: Bayesian Methods CERN-JINR European School of High Energy Physics Bautzen, 14 27 June 2009 Glen Cowan Physics Department Royal Holloway, University
More informationMODIFIED FREQUENTIST ANALYSIS OF SEARCH RESULTS (THE CL s METHOD)
MODIFIED FREQUENTIST ANALYSIS OF SEARCH RESULTS (THE CL s METHOD) A. L. Read University of Oslo, Department of Physics, P.O. Box 148, Blindern, 316 Oslo 3, Norway Abstract The statistical analysis of direct
More informationEstimating the accuracy of a hypothesis Setting. Assume a binary classification setting
Estimating the accuracy of a hypothesis Setting Assume a binary classification setting Assume input/output pairs (x, y) are sampled from an unknown probability distribution D = p(x, y) Train a binary classifier
More informationRecent developments in statistical methods for particle physics
Recent developments in statistical methods for particle physics Particle Physics Seminar Warwick, 17 February 2011 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More informationStatistical Methods for Particle Physics Lecture 2: multivariate methods
Statistical Methods for Particle Physics Lecture 2: multivariate methods http://indico.ihep.ac.cn/event/4902/ istep 2015 Shandong University, Jinan August 11-19, 2015 Glen Cowan ( 谷林 科恩 ) Physics Department
More informationFundamental Probability and Statistics
Fundamental Probability and Statistics "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are
More informationStatistical Methods for Particle Physics Lecture 3: systematic uncertainties / further topics
Statistical Methods for Particle Physics Lecture 3: systematic uncertainties / further topics istep 2014 IHEP, Beijing August 20-29, 2014 Glen Cowan ( Physics Department Royal Holloway, University of London
More informationETH Zurich HS Mauro Donegà: Higgs physics meeting name date 1
Higgs physics - lecture 4 ETH Zurich HS 2015 Mauro Donegà Mauro Donegà: Higgs physics meeting name date 1 Outline 1 2 3 4 5 6 Introduction Accelerators Detectors EW constraints Search at LEP1 / LEP 2 Statistics:
More informationFYST17 Lecture 8 Statistics and hypothesis testing. Thanks to T. Petersen, S. Maschiocci, G. Cowan, L. Lyons
FYST17 Lecture 8 Statistics and hypothesis testing Thanks to T. Petersen, S. Maschiocci, G. Cowan, L. Lyons 1 Plan for today: Introduction to concepts The Gaussian distribution Likelihood functions Hypothesis
More informationOVERVIEW OF PRINCIPLES OF STATISTICS
OVERVIEW OF PRINCIPLES OF STATISTICS F. James CERN, CH-1211 eneva 23, Switzerland Abstract A summary of the basic principles of statistics. Both the Bayesian and Frequentist points of view are exposed.
More informationIntroductory Statistics Course Part II
Introductory Statistics Course Part II https://indico.cern.ch/event/735431/ PHYSTAT ν CERN 22-25 January 2019 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationConfidence Limits and Intervals 3: Various other topics. Roger Barlow SLUO Lectures on Statistics August 2006
Confidence Limits and Intervals 3: Various other topics Roger Barlow SLUO Lectures on Statistics August 2006 Contents 1.Likelihood and lnl 2.Multidimensional confidence regions 3.Systematic errors: various
More informationE. Santovetti lesson 4 Maximum likelihood Interval estimation
E. Santovetti lesson 4 Maximum likelihood Interval estimation 1 Extended Maximum Likelihood Sometimes the number of total events measurements of the experiment n is not fixed, but, for example, is a Poisson
More informationDetection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010
Detection Theory Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Outline Neyman-Pearson Theorem Detector Performance Irrelevant Data Minimum Probability of Error Bayes Risk Multiple
More informationIrr. Statistical Methods in Experimental Physics. 2nd Edition. Frederick James. World Scientific. CERN, Switzerland
Frederick James CERN, Switzerland Statistical Methods in Experimental Physics 2nd Edition r i Irr 1- r ri Ibn World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI CONTENTS
More informationStatistical Methods for Particle Physics
Statistical Methods for Particle Physics Invisibles School 8-13 July 2014 Château de Button Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationIntroduction to Likelihoods
Introduction to Likelihoods http://indico.cern.ch/conferencedisplay.py?confid=218693 Likelihood Workshop CERN, 21-23, 2013 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More informationSTAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01
STAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01 Nasser Sadeghkhani a.sadeghkhani@queensu.ca There are two main schools to statistical inference: 1-frequentist
More informationEvaluation. Andrea Passerini Machine Learning. Evaluation
Andrea Passerini passerini@disi.unitn.it Machine Learning Basic concepts requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain
More informationDetection theory 101 ELEC-E5410 Signal Processing for Communications
Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off
More informationIntroduction to Statistical Inference
Structural Health Monitoring Using Statistical Pattern Recognition Introduction to Statistical Inference Presented by Charles R. Farrar, Ph.D., P.E. Outline Introduce statistical decision making for Structural
More informationEvaluation requires to define performance measures to be optimized
Evaluation Basic concepts Evaluation requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain (generalization error) approximation
More informationWhy Try Bayesian Methods? (Lecture 5)
Why Try Bayesian Methods? (Lecture 5) Tom Loredo Dept. of Astronomy, Cornell University http://www.astro.cornell.edu/staff/loredo/bayes/ p.1/28 Today s Lecture Problems you avoid Ambiguity in what is random
More informationarxiv: v1 [hep-ex] 9 Jul 2013
Statistics for Searches at the LHC Glen Cowan Physics Department, Royal Holloway, University of London, Egham, Surrey, TW2 EX, UK arxiv:137.2487v1 [hep-ex] 9 Jul 213 Abstract These lectures 1 describe
More informationStatistical Tools in Collider Experiments. Multivariate analysis in high energy physics
Statistical Tools in Collider Experiments Multivariate analysis in high energy physics Pauli Lectures - 06/0/01 Nicolas Chanon - ETH Zürich 1 Main goals of these lessons - Have an understanding of what
More informationLectures on Statistical Data Analysis
Lectures on Statistical Data Analysis London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More informationStatistical Methods in Particle Physics
Statistical Methods in Particle Physics Lecture 11 January 7, 2013 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2012 / 13 Outline How to communicate the statistical uncertainty
More informationLecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis
Lecture 3 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationIntelligent Systems Statistical Machine Learning
Intelligent Systems Statistical Machine Learning Carsten Rother, Dmitrij Schlesinger WS2014/2015, Our tasks (recap) The model: two variables are usually present: - the first one is typically discrete k
More informationMachine Learning Lecture 2
Machine Perceptual Learning and Sensory Summer Augmented 15 Computing Many slides adapted from B. Schiele Machine Learning Lecture 2 Probability Density Estimation 16.04.2015 Bastian Leibe RWTH Aachen
More informationLecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary
ECE 830 Spring 207 Instructor: R. Willett Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we saw that the likelihood
More informationLecture Testing Hypotheses: The Neyman-Pearson Paradigm
Math 408 - Mathematical Statistics Lecture 29-30. Testing Hypotheses: The Neyman-Pearson Paradigm April 12-15, 2013 Konstantin Zuev (USC) Math 408, Lecture 29-30 April 12-15, 2013 1 / 12 Agenda Example:
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationDiscovery and Significance. M. Witherell 5/10/12
Discovery and Significance M. Witherell 5/10/12 Discovering particles Much of what we know about physics at the most fundamental scale comes from discovering particles. We discovered these particles by
More informationGenerative Techniques: Bayes Rule and the Axioms of Probability
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2016/2017 Lesson 8 3 March 2017 Generative Techniques: Bayes Rule and the Axioms of Probability Generative
More informationCSE 546 Final Exam, Autumn 2013
CSE 546 Final Exam, Autumn 0. Personal info: Name: Student ID: E-mail address:. There should be 5 numbered pages in this exam (including this cover sheet).. You can use any material you brought: any book,
More informationStatistical Methods for Discovery and Limits in HEP Experiments Day 3: Exclusion Limits
Statistical Methods for Discovery and Limits in HEP Experiments Day 3: Exclusion Limits www.pp.rhul.ac.uk/~cowan/stat_freiburg.html Vorlesungen des GK Physik an Hadron-Beschleunigern, Freiburg, 27-29 June,
More informationIntroduction: MLE, MAP, Bayesian reasoning (28/8/13)
STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this
More informationHypothesis testing:power, test statistic CMS:
Hypothesis testing:power, test statistic The more sensitive the test, the better it can discriminate between the null and the alternative hypothesis, quantitatively, maximal power In order to achieve this
More informationMidterm Review CS 6375: Machine Learning. Vibhav Gogate The University of Texas at Dallas
Midterm Review CS 6375: Machine Learning Vibhav Gogate The University of Texas at Dallas Machine Learning Supervised Learning Unsupervised Learning Reinforcement Learning Parametric Y Continuous Non-parametric
More informationAlgorithmisches Lernen/Machine Learning
Algorithmisches Lernen/Machine Learning Part 1: Stefan Wermter Introduction Connectionist Learning (e.g. Neural Networks) Decision-Trees, Genetic Algorithms Part 2: Norman Hendrich Support-Vector Machines
More informationConstructing Ensembles of Pseudo-Experiments
Constructing Ensembles of Pseudo-Experiments Luc Demortier The Rockefeller University, New York, NY 10021, USA The frequentist interpretation of measurement results requires the specification of an ensemble
More informationThe Bayes classifier
The Bayes classifier Consider where is a random vector in is a random variable (depending on ) Let be a classifier with probability of error/risk given by The Bayes classifier (denoted ) is the optimal
More informationPART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
More informationMidterm Review CS 7301: Advanced Machine Learning. Vibhav Gogate The University of Texas at Dallas
Midterm Review CS 7301: Advanced Machine Learning Vibhav Gogate The University of Texas at Dallas Supervised Learning Issues in supervised learning What makes learning hard Point Estimation: MLE vs Bayesian
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationIntelligent Systems Discriminative Learning, Neural Networks
Intelligent Systems Discriminative Learning, Neural Networks Carsten Rother, Dmitrij Schlesinger WS2014/2015, Outline 1. Discriminative learning 2. Neurons and linear classifiers: 1) Perceptron-Algorithm
More informationChapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)
Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis
More informationApplied Statistics. Multivariate Analysis - part II. Troels C. Petersen (NBI) Statistics is merely a quantization of common sense 1
Applied Statistics Multivariate Analysis - part II Troels C. Petersen (NBI) Statistics is merely a quantization of common sense 1 Fisher Discriminant You want to separate two types/classes (A and B) of
More informationStatistical Data Analysis Stat 2: Monte Carlo Method, Statistical Tests
Statistical Data Analysis Stat 2: Monte Carlo Method, Statistical Tests London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway,
More informationECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters
ECE531 Lecture 6: Detection of Discrete-Time Signals with Random Parameters D. Richard Brown III Worcester Polytechnic Institute 26-February-2009 Worcester Polytechnic Institute D. Richard Brown III 26-February-2009
More informationCSE 473: Artificial Intelligence Autumn Topics
CSE 473: Artificial Intelligence Autumn 2014 Bayesian Networks Learning II Dan Weld Slides adapted from Jack Breese, Dan Klein, Daphne Koller, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 473 Topics
More informationStatistical Methods in Particle Physics Day 4: Discovery and limits
Statistical Methods in Particle Physics Day 4: Discovery and limits 清华大学高能物理研究中心 2010 年 4 月 12 16 日 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationStatistical Methods for LHC Physics
RHUL Physics www.pp.rhul.ac.uk/~cowan Particle Physics Seminar University of Glasgow 26 February, 2009 1 Outline Quick overview of physics at the Large Hadron Collider (LHC) New multivariate methods for
More informationFINAL: CS 6375 (Machine Learning) Fall 2014
FINAL: CS 6375 (Machine Learning) Fall 2014 The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run out of room for
More informationPart III. A Decision-Theoretic Approach and Bayesian testing
Part III A Decision-Theoretic Approach and Bayesian testing 1 Chapter 10 Bayesian Inference as a Decision Problem The decision-theoretic framework starts with the following situation. We would like to
More informationLecturer: Dr. Adote Anum, Dept. of Psychology Contact Information:
Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information: aanum@ug.edu.gh College of Education School of Continuing and Distance Education 2014/2015 2016/2017 Session Overview In this Session
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More informationDetection theory. H 0 : x[n] = w[n]
Detection Theory Detection theory A the last topic of the course, we will briefly consider detection theory. The methods are based on estimation theory and attempt to answer questions such as Is a signal
More informationStatistical Inference
Statistical Inference Robert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USA Spring, 2006 1. DeGroot 1973 In (DeGroot 1973), Morrie DeGroot considers testing the
More informationtheta a framework for template-based modeling and inference
theta a framework for template-based modeling and inference Thomas Müller Jochen Ott Jeannine Wagner-Kuhr Institut für Experimentelle Kernphysik, Karlsruhe Institute of Technology (KIT), Germany June 17,
More informationDetection and Estimation Chapter 1. Hypothesis Testing
Detection and Estimation Chapter 1. Hypothesis Testing Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, 2015 1/20 Syllabus Homework:
More informationStatistical Methods in Particle Physics. Lecture 2
Statistical Methods in Particle Physics Lecture 2 October 17, 2011 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2011 / 12 Outline Probability Definition and interpretation Kolmogorov's
More informationHYPOTHESIS TESTING: FREQUENTIST APPROACH.
HYPOTHESIS TESTING: FREQUENTIST APPROACH. These notes summarize the lectures on (the frequentist approach to) hypothesis testing. You should be familiar with the standard hypothesis testing from previous
More informationPhysics 403. Segev BenZvi. Choosing Priors and the Principle of Maximum Entropy. Department of Physics and Astronomy University of Rochester
Physics 403 Choosing Priors and the Principle of Maximum Entropy Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Odds Ratio Occam Factors
More information2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?
ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we
More informationModel Averaging (Bayesian Learning)
Model Averaging (Bayesian Learning) We want to predict the output Y of a new case that has input X = x given the training examples e: p(y x e) = m M P(Y m x e) = m M P(Y m x e)p(m x e) = m M P(Y m x)p(m
More informationFirst two sided limit on BR(B s μ + μ - ) Matthew Herndon, University of Wisconsin Madison SUSY M. Herndon, SUSY
First two sided limit on BR(B s μ + μ - ) Matthew Herndon, University of Wisconsin Madison SUSY 2011 M. Herndon, SUSY 2011 1 B s(d) μ + μ - Beyond the SM Indirect searches for new physics Look at processes
More informationMachine Learning Lecture 3
Announcements Machine Learning Lecture 3 Eam dates We re in the process of fiing the first eam date Probability Density Estimation II 9.0.207 Eercises The first eercise sheet is available on L2P now First
More informationParameter Estimation, Sampling Distributions & Hypothesis Testing
Parameter Estimation, Sampling Distributions & Hypothesis Testing Parameter Estimation & Hypothesis Testing In doing research, we are usually interested in some feature of a population distribution (which
More informationStat 135 Fall 2013 FINAL EXAM December 18, 2013
Stat 135 Fall 2013 FINAL EXAM December 18, 2013 Name: Person on right SID: Person on left There will be one, double sided, handwritten, 8.5in x 11in page of notes allowed during the exam. The exam is closed
More information