Gaussians. Andrew W. Moore Professor School of Computer Science Carnegie Mellon University.
|
|
|
- Osborne Miles
- 5 years ago
- Views:
Transcription
1 Note to other teachers and users of these slides. Andrew would be delighted if you found this source aterial useful in giing your own lectures. Feel free to use these slides erbati, or to odify the to fit your own needs. PowerPoint originals are aailable. If you ake use of a significant portion of these slides in your own lecture, please include this essage, or the following link to the source repository of Andrew s tutorials: Coents and corrections gratefully receied. Gaussians Andrew W. oore Professor School of Coputer Science Carnegie ellon niersity aw@cs.cu.edu Copyright Andrew W. oore Slide Gaussians in Data ining Why we should care he entropy of a PDF niariate Gaussians ultiariate Gaussians Bayes Rule and Gaussians aiu ikelihood and AP using Gaussians Copyright Andrew W. oore Slide
2 Why we should care Gaussians are as natural as Orange Juice and Sunshine We need the to understand Bayes Optial Classifiers We need the to understand regression We need the to understand neural nets We need the to understand iture odels (You get the idea Copyright Andrew W. oore Slide 3 he bo distribution w if if w w > /w -w/ w/ Copyright Andrew W. oore Slide 4
3 he bo distribution w if if w w > /w -w/ w/ E[ ] Var[ ] w Copyright Andrew W. oore Slide 5 Entropy of Entropy of a PDF H[ ] log d Natural log (ln or log e he larger the entropy of a distribution the harder it is to predict the harder it is to copress it the less spiky the distribution Copyright Andrew W. oore Slide 6 3
4 he bo distribution w if if w w > /w -w/ w/ w/ H[ ] log d log d log d log w w w w w w/ w/ w/ Copyright Andrew W. oore Slide 7 nit ariance bo distribution w if if w w > 3 E[ ] w Var[ ] 3 3 if w 3 then Var[ ] and H[ ].4 Copyright Andrew W. oore Slide 8 4
5 he Hat distribution w w if if w > w w E[ ] w Var[ ] 6 w w Copyright Andrew W. oore Slide 9 nit ariance hat distribution w w if if w > w 6 E[ ] w Var[ ] if w 6 then Var[ ] and H[ ].396 Copyright Andrew W. oore Slide 5
6 he spikes distribution Dirac Delta δ ( + δ ( δ ( δ ( E[ ] Var[ ] - H [ ] log d Copyright Andrew W. oore Slide Entropies of unit-ariance distributions Distribution Entropy Bo Hat spikes??? infinity.489 argest possible entropy of any unitariance distribution Copyright Andrew W. oore Slide 6
7 nit ariance Gaussian ep π E[ ] Var[ ] H[ ] log d.489 Copyright Andrew W. oore Slide 3 General Gaussian ( ep π 5 E [ ] Var[ ] Copyright Andrew W. oore Slide 4 7
8 General Gaussian Also known as the noral distribution or Bellshaped cure ( ep π 5 E [ ] Var[ ] Shorthand: We say ~ N(, to ean is distributed as a Gaussian with paraeters and. In the aboe figure, ~ N(,5 Copyright Andrew W. oore Slide 5 he Error Function Assue ~ N(, Define ERF( P(< Cuulatie Distribution of ERF ( z dz π z z z ep dz Copyright Andrew W. oore Slide 6 8
9 sing he Error Function Assue ~ N(, ERF ( P(<, Copyright Andrew W. oore Slide 7 he Central iit heore If(,, n are i.i.d. continuous rando ariables n hen define z f (,,... n i n i As n-->infinity, z--->gaussian with ean E[ i ] and ariance Var[ i ] Soewhat of a justification for assuing Gaussian noise is coon Copyright Andrew W. oore Slide 8 9
10 Other aazing facts about Gaussians Wouldn t you like to know? We will not eaine the until we need to. Copyright Andrew W. oore Slide 9 Write r.. Biariate Gaussians Y hen define ~ N (, ( ( ( ep π Where the Gaussian s paraeters are to ean y y y y Where we insist that is syetric non-negatie definite Copyright Andrew W. oore Slide
11 Write r.. Biariate Gaussians Y hen define ~ N (, to ean ( ( ( ep π Where the Gaussian s paraeters are y y y y Where we insist that is syetric non-negatie definite It turns out that E[] and Co[]. (Note that this is a resulting property of Gaussians, not a definition* *his note rates 7.4 on the pedanticness scale Copyright Andrew W. oore Slide Ealuating : Step ( ( ( ep π. Begin with ector Copyright Andrew W. oore Slide
12 Ealuating : Step. Begin with ector. Define δ - ( ( ( ep π δ Copyright Andrew W. oore Slide 3 Ealuating : Step 3. Begin with ector. Define δ - 3. Count the nuber of contours crossed of the ellipsoids fored - D this count sqrt(δ - δ ahalonobis Distance between and δ ( ( ( ep π Contours defined by sqrt(δ - δ constant Copyright Andrew W. oore Slide 4
13 Ealuating : Step 4 ( ( ( ep π. Begin with ector. Define δ - 3. Count the nuber of contours crossed of the ellipsoids fored - D this count sqrt(δ - δ ahalonobis Distance between and 4. Define w e-d / e-d / D close to in squared ahalonobis space gets a large weight. Far away gets a tiny weight Copyright Andrew W. oore Slide 5 Ealuating : Step 5 ( ( ( ep π. Begin with ector. Define δ - 3. Count the nuber of contours crossed of the ellipsoids fored - D this count sqrt(δ - δ ahalonobis Distance between and 4. Define w e-d / e-d / 5. D ultiply w by to ensure d π Copyright Andrew W. oore Slide 6 3
14 Eaple Obsere: ean, Principal aes, iplication of off-diagonal coariance ter, a gradient zone of Coon conention: show contour corresponding to standard deiations fro ean Copyright Andrew W. oore Slide 7 Eaple Copyright Andrew W. oore Slide 8 4
15 Eaple In this eaple, and y are alost independent Copyright Andrew W. oore Slide 9 Eaple In this eaple, and +y are clearly not independent Copyright Andrew W. oore Slide 3 5
16 6 Copyright Andrew W. oore Slide 3 Eaple In this eaple, and +y are clearly not independent Copyright Andrew W. oore Slide 3 ultiariate Gaussians ( ( ( ep ( ( p π r.. Write O hen define, ( ~ N to ean Where the Gaussian s paraeters hae Where we insist that is syetric non-negatie definite Again, E[] and Co[]. (Note that this is a resulting property of Gaussians, not a definition
17 7 Copyright Andrew W. oore Slide 33 General Gaussians O Copyright Andrew W. oore Slide 34 Ais-Aligned Gaussians 3 O j i i i for
18 8 Copyright Andrew W. oore Slide 35 Spherical Gaussians O j i i i for Copyright Andrew W. oore Slide 36 Degenerate Gaussians What s so wrong with clipping one s toenails in public?
19 9 Copyright Andrew W. oore Slide 37 Where are we now? We e seen the forulae for Gaussians We hae an intuition of how they behae We hae soe eperience of reading a Gaussian s coariance atri Coing net: Soe useful tricks with Gaussians Copyright Andrew W. oore Slide 38 Subsets of ariables where as Write ( ( + u u V V his will be our standard notation for breaking an - diensional distribution into subsets of ariables
20 Copyright Andrew W. oore Slide 39 Gaussian arginals are Gaussian + u u ( (, where as Write V V HEN is also distributed as a Gaussian u u uu u V, N ~ IF ( u uu, N ~ V arginalize Copyright Andrew W. oore Slide 4 Gaussian arginals are Gaussian + u u ( (, where as Write V V HEN is also distributed as a Gaussian u u uu u V, N ~ IF ( u uu, N ~ V arginalize his fact is not iediately obious Obious, once we know it s a Gaussian (why?
21 Gaussian arginals are Gaussian Write IF ~ N V as u, uu u where V HEN is also distributed as a Gaussian ~ N, ( u uu u V arginalize ( u (, V How would you proe this? u u, d (snore... u + Copyright Andrew W. oore Slide 4 inear ransfors reain Gaussian atri A ultiply A Assue is an -diensional Gaussian r.. ( ~ N, Define Y to be a p-diensional r.. thusly (note : Y A p where A is a p atri. hen Y ~ N, ( A A A Note: the subset result is a special case of this result Copyright Andrew W. oore Slide 4
22 Adding saples of independent Gaussians is Gaussian if ~ N (, and Y ~ N(, and Y Y y ( + then + Y ~ N, + y y + + Y y Why doesn t this hold if and Y are dependent? Which of the below stateents is true? If and Y are dependent, then +Y is Gaussian but possibly with soe other coariance If and Y are dependent, then +Y ight be non-gaussian Copyright Andrew W. oore Slide 43 Conditional of Gaussian is Gaussian V Conditionalize V IF ~ V N u, uu u u ( u, where HEN V ~ N u + ( V u u u u uu u u Copyright Andrew W. oore Slide 44
23 IF ~ N V u, uu u u ( u, where HEN V ~ N u + ( V u u u u uu u u IF w ~ N, y (, where HEN w y ~ N w y w y w y w y 976( y Copyright Andrew W. oore Slide 45 IF ~ N V u, uu u u ( u, where HEN V ~ N u + ( V u u u u uu u u IF w ~ N, y (, where HEN w y ~ N w y w w y w y y 976( y P(w 8 P(w 76 P(w Copyright Andrew W. oore Slide 46 3
24 IF ~ N V u, uu u u ( u, where HEN V ~ N u + ( V u u u u uu u u w IF ~ N, y Note: when 76 gien 967 alue 3.68 of HEN w is y ~, N the ( conditional w y, w y where ean of u is u 976( y 76 w y w y Note: arginal 3.68 ean is a linear function of Note: conditional ariance is independent of the gien alue of P(w 8 Note: conditional ariance can only be equal to or saller than P(w 76 arginal ariance P(w Copyright Andrew W. oore Slide 47 Gaussians and the chain rule et A be a constant atri ( AV, and V ~ N( IF V ~ N u, HEN A ~ N V (,, with A A + ( A u V V A Chain Rule V Copyright Andrew W. oore Slide 48 4
25 V Y V V V Aailable Gaussian tools Conditionalize arginalize atri A ultiply + + Y Chain Rule A V V u ~ N, V uu u IF u HEN ~ N (, u uu ( IF ~ N, IF AND Y A HEN Y ~ N( A, A A (, and Y ~ N( y, y and Y Y ~ N( + if ~ N then +, + where ~ N V u, + uu u y u y ( V u u u HEN u ( AV, and V ~ N( IF V ~ N u, HEN ~ N V (,, with uu A A + ( A ( V ~ N u u u, u A u Copyright Andrew W. oore Slide 49 Assue You are an intellectual snob You hae a child Copyright Andrew W. oore Slide 5 5
26 Intellectual snobs with children are obsessed with IQ In the world as a whole, IQs are drawn fro a Gaussian N(,5 Copyright Andrew W. oore Slide 5 IQ tests If you take an IQ test you ll get a score that, on aerage (oer any tests will be your IQ But because of noise on any one test the score will often be a few points lower or higher than your true IQ. SCORE IQ ~ N(IQ, Copyright Andrew W. oore Slide 5 6
27 Assue You drag your kid off to get tested She gets a score of 3 Yippee you screech and start deciding how to casually refer to her ebership of the top % of IQs in your Christas newsletter. P(<3, 5 P(<, erf(.977 Copyright Andrew W. oore Slide 53 Assue You drag your kid off to get tested She gets a score of 3 You are thinking: Well sure the test isn t accurate, so Yippee you screech she ight and hae an start IQ of deciding or she how ight hae an Q of 4, but the to casually refer ost to her likely IQ ebership gien the eidence of the top % of IQs in score3 your Christas is, of course, newsletter. 3. P(<3, 5 P(<, erf(.977 Can we trust this reasoning? Copyright Andrew W. oore Slide 54 7
28 aiu ikelihood IQ IQ~N(,5 S IQ ~ N(IQ, S3 IQ le arg a s 3 iq iq he E is the alue of the hidden paraeter that akes the obsered data ost likely In this case le IQ 3 Copyright Andrew W. oore Slide 55 IQ~N(,5 S IQ ~ N(IQ, S3 B. IQ le he E is the alue of the hidden paraeter that akes the obsered data ost likely In this case arg a s 3 iq iq le IQ 3 his is not the sae as he ost likely alue of the paraeter gien the obsered data Copyright Andrew W. oore Slide 56 8
29 What we really want: IQ~N(,5 S IQ ~ N(IQ, S3 Question: What is IQ (S3? Called the Posterior Distribution of IQ Copyright Andrew W. oore Slide 57 IQ~N(,5 S IQ ~ N(IQ, S3 Question: What is IQ (S3? Which tool or tools? V Y V Conditionalize arginalize atri A ultiply + + Y A V V V Chain Rule V Copyright Andrew W. oore Slide 58 9
30 IQ~N(,5 S IQ ~ N(IQ, S3 Plan Question: What is IQ (S3? S IQ IQ Chain Rule S IQ Swap IQ S Conditionalize I Q S Copyright Andrew W. oore Slide 59 Working IQ~N(,5 S IQ ~ N(IQ, S3 Question: What is IQ (S3? IF ~ N V u, + uu u u HEN ( V u u u ( AV, and V ~ N( IF V ~ N u, HEN ~ N V (,, with A A + ( A u A Copyright Andrew W. oore Slide 6 3
31 Your pride and joy s posterior IQ If you did the working, you now hae IQ S3 If you hae to gie the ost likely IQ gien the score you should gie IQ ap arg a iq s 3 iq where AP eans aiu A-posteriori Copyright Andrew W. oore Slide 6 What you should know he Gaussian PDF forula off by heart nderstand the workings of the forula for a Gaussian Be able to understand the Gaussian tools described so far Hae a rough idea of how you could proe the Be happy with how you could use the Copyright Andrew W. oore Slide 6 3
Feature Extraction Techniques
Feature Extraction Techniques Unsupervised Learning II Feature Extraction Unsupervised ethods can also be used to find features which can be useful for categorization. There are unsupervised ethods that
CS Lecture 13. More Maximum Likelihood
CS 6347 Lecture 13 More Maxiu Likelihood Recap Last tie: Introduction to axiu likelihood estiation MLE for Bayesian networks Optial CPTs correspond to epirical counts Today: MLE for CRFs 2 Maxiu Likelihood
Estimating Parameters for a Gaussian pdf
Pattern Recognition and achine Learning Jaes L. Crowley ENSIAG 3 IS First Seester 00/0 Lesson 5 7 Noveber 00 Contents Estiating Paraeters for a Gaussian pdf Notation... The Pattern Recognition Proble...3
Support Vector Machines. Maximizing the Margin
Support Vector Machines Support vector achines (SVMs) learn a hypothesis: h(x) = b + Σ i= y i α i k(x, x i ) (x, y ),..., (x, y ) are the training exs., y i {, } b is the bias weight. α,..., α are the
1 Generalization bounds based on Rademacher complexity
COS 5: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #0 Scribe: Suqi Liu March 07, 08 Last tie we started proving this very general result about how quickly the epirical average converges
Understanding Machine Learning Solution Manual
Understanding Machine Learning Solution Manual Written by Alon Gonen Edited by Dana Rubinstein Noveber 17, 2014 2 Gentle Start 1. Given S = ((x i, y i )), define the ultivariate polynoial p S (x) = i []:y
1 Bounding the Margin
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #12 Scribe: Jian Min Si March 14, 2013 1 Bounding the Margin We are continuing the proof of a bound on the generalization error of AdaBoost
Kernel Methods and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley ENSIAG 2 / osig 1 Second Seester 2012/2013 Lesson 20 2 ay 2013 Kernel ethods and Support Vector achines Contents Kernel Functions...2 Quadratic
Probabilistic Machine Learning
Probabilistic Machine Learning by Prof. Seungchul Lee isystes Design Lab http://isystes.unist.ac.kr/ UNIST Table of Contents I.. Probabilistic Linear Regression I... Maxiu Likelihood Solution II... Maxiu-a-Posteriori
Probabilistic and Bayesian Analytics
This ersion of the Powerpoint presentation has been labeled to let you know which bits will by coered in the ain class (Tue/Thu orning lectures and which parts will be coered in the reiew sessions. Probabilistic
Support Vector Machines. Goals for the lecture
Support Vector Machines Mark Craven and David Page Coputer Sciences 760 Spring 2018 www.biostat.wisc.edu/~craven/cs760/ Soe of the slides in these lectures have been adapted/borrowed fro aterials developed
Improved Hidden Clique Detection by Optimal Linear Fusion of Multiple Adjacency Matrices
Iproed Hidden Clique Detection by Optial Linear Fusion of Multiple Adjacency Matrices Hianshu Nayar, Benjain A. Miller, Kelly Geyer, Rajonda S. Caceres, Steen T. Sith, and Raj Rao Nadakuditi Departent
Pattern Recognition and Machine Learning. Learning and Evaluation for Pattern Recognition
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lesson 1 4 October 2017 Outline Learning and Evaluation for Pattern Recognition Notation...2 1. The Pattern Recognition
E0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
Finite fields. and we ve used it in various examples and homework problems. In these notes I will introduce more finite fields
Finite fields I talked in class about the field with two eleents F 2 = {, } and we ve used it in various eaples and hoework probles. In these notes I will introduce ore finite fields F p = {,,...,p } for
A Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
1 Proof of learning bounds
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #4 Scribe: Akshay Mittal February 13, 2013 1 Proof of learning bounds For intuition of the following theore, suppose there exists a
Support Vector Machines MIT Course Notes Cynthia Rudin
Support Vector Machines MIT 5.097 Course Notes Cynthia Rudin Credit: Ng, Hastie, Tibshirani, Friedan Thanks: Şeyda Ertekin Let s start with soe intuition about argins. The argin of an exaple x i = distance
Problem Set 2 Due Sept, 21
EE6: Rando Processes in Sstes Lecturer: Jean C. Walrand Proble Set Due Sept, Fall 6 GSI: Assane Guee This proble set essentiall reviews notions of conditional epectation, conditional distribution, and
Principal Components Analysis
Principal Coponents Analysis Cheng Li, Bingyu Wang Noveber 3, 204 What s PCA Principal coponent analysis (PCA) is a statistical procedure that uses an orthogonal transforation to convert a set of observations
Lecture 12: Ensemble Methods. Introduction. Weighted Majority. Mixture of Experts/Committee. Σ k α k =1. Isabelle Guyon
Lecture 2: Enseble Methods Isabelle Guyon guyoni@inf.ethz.ch Introduction Book Chapter 7 Weighted Majority Mixture of Experts/Coittee Assue K experts f, f 2, f K (base learners) x f (x) Each expert akes
8.1 Force Laws Hooke s Law
8.1 Force Laws There are forces that don't change appreciably fro one instant to another, which we refer to as constant in tie, and forces that don't change appreciably fro one point to another, which
Probability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
Non-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
Bayesian Learning. Chapter 6: Bayesian Learning. Bayes Theorem. Roles for Bayesian Methods. CS 536: Machine Learning Littman (Wu, TA)
Bayesian Learning Chapter 6: Bayesian Learning CS 536: Machine Learning Littan (Wu, TA) [Read Ch. 6, except 6.3] [Suggested exercises: 6.1, 6.2, 6.6] Bayes Theore MAP, ML hypotheses MAP learners Miniu
CSE555: Introduction to Pattern Recognition Midterm Exam Solution (100 points, Closed book/notes)
CSE555: Introduction to Pattern Recognition Midterm Exam Solution (00 points, Closed book/notes) There are 5 questions in this exam. The last page is the Appendix that contains some useful formulas.. (5pts)
Detection and Estimation Theory
ESE 54 Detection and Estiation Theory Joseph A. O Sullivan Sauel C. Sachs Professor Electronic Systes and Signals Research Laboratory Electrical and Systes Engineering Washington University 11 Urbauer
Block designs and statistics
Bloc designs and statistics Notes for Math 447 May 3, 2011 The ain paraeters of a bloc design are nuber of varieties v, bloc size, nuber of blocs b. A design is built on a set of v eleents. Each eleent
VC-dimension for characterizing classifiers
VC-dimension for characterizing classifiers Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to
Cross-validation for detecting and preventing overfitting
Cross-validation for detecting and preventing overfitting A Regression Problem = f() + noise Can we learn f from this data? Note to other teachers and users of these slides. Andrew would be delighted if
Using EM To Estimate A Probablity Density With A Mixture Of Gaussians
Using EM To Estiate A Probablity Density With A Mixture Of Gaussians Aaron A. D Souza adsouza@usc.edu Introduction The proble we are trying to address in this note is siple. Given a set of data points
Machine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
Econometrics II - EXAM Outline Solutions All questions have 25pts Answer each question in separate sheets
Econometrics II - EXAM Outline Solutions All questions hae 5pts Answer each question in separate sheets. Consider the two linear simultaneous equations G with two exogeneous ariables K, y γ + y γ + x δ
Astrometric Errors Correlated Strongly Across Multiple SIRTF Images
Astrometric Errors Correlated Strongly Across Multiple SIRTF Images John Fowler 28 March 23 The possibility exists that after pointing transfer has been performed for each BCD (i.e. a calibrated image
Intelligent Systems: Reasoning and Recognition. Perceptrons and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley osig 1 Winter Seester 2018 Lesson 6 27 February 2018 Outline Perceptrons and Support Vector achines Notation...2 Linear odels...3 Lines, Planes
Estimation of the Mean of the Exponential Distribution Using Maximum Ranked Set Sampling with Unequal Samples
Open Journal of Statistics, 4, 4, 64-649 Published Online Septeber 4 in SciRes http//wwwscirporg/ournal/os http//ddoiorg/436/os4486 Estiation of the Mean of the Eponential Distribution Using Maiu Ranked
ON REGULARITY, TRANSITIVITY, AND ERGODIC PRINCIPLE FOR QUADRATIC STOCHASTIC VOLTERRA OPERATORS MANSOOR SABUROV
ON REGULARITY TRANSITIVITY AND ERGODIC PRINCIPLE FOR QUADRATIC STOCHASTIC VOLTERRA OPERATORS MANSOOR SABUROV Departent of Coputational & Theoretical Sciences Faculty of Science International Islaic University
CHAPTER 8 CONSTRAINED OPTIMIZATION 2: SEQUENTIAL QUADRATIC PROGRAMMING, INTERIOR POINT AND GENERALIZED REDUCED GRADIENT METHODS
CHAPER 8 CONSRAINED OPIMIZAION : SEQUENIAL QUADRAIC PROGRAMMING, INERIOR POIN AND GENERALIZED REDUCED GRADIEN MEHODS 8. Introduction In the previous chapter we eained the necessary and sufficient conditions
Bayesian Networks: Independencies and Inference
Bayesian Networks: Independencies and Inference Scott Davies and Andrew Moore Note to other teachers and users of these slides. Andrew and Scott would be delighted if you found this source material useful
Computational and Statistical Learning Theory
Coputational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 2: PAC Learning and VC Theory I Fro Adversarial Online to Statistical Three reasons to ove fro worst-case deterinistic
16 Independence Definitions Potential Pitfall Alternative Formulation. mcs-ftl 2010/9/8 0:40 page 431 #437
cs-ftl 010/9/8 0:40 page 431 #437 16 Independence 16.1 efinitions Suppose that we flip two fair coins siultaneously on opposite sides of a roo. Intuitively, the way one coin lands does not affect the way
Assignment 4 (Solutions) NPTEL MOOC (Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications)
Assignment 4 Solutions NPTEL MOOC Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications The system model can be written as, y hx + The MSE of the MMSE estimate ĥ of the aboe mentioned system
A Note on the Applied Use of MDL Approximations
A Note on the Applied Use of MDL Approxiations Daniel J. Navarro Departent of Psychology Ohio State University Abstract An applied proble is discussed in which two nested psychological odels of retention
Lean Walsh Transform
Lean Walsh Transfor Edo Liberty 5th March 007 inforal intro We show an orthogonal atrix A of size d log 4 3 d (α = log 4 3) which is applicable in tie O(d). By applying a rando sign change atrix S to the
Computational and Statistical Learning Theory
Coputational and Statistical Learning Theory Proble sets 5 and 6 Due: Noveber th Please send your solutions to learning-subissions@ttic.edu Notations/Definitions Recall the definition of saple based Radeacher
Model Fitting. CURM Background Material, Fall 2014 Dr. Doreen De Leon
Model Fitting CURM Background Material, Fall 014 Dr. Doreen De Leon 1 Introduction Given a set of data points, we often want to fit a selected odel or type to the data (e.g., we suspect an exponential
Support Vector Machine Classification of Uncertain and Imbalanced data using Robust Optimization
Recent Researches in Coputer Science Support Vector Machine Classification of Uncertain and Ibalanced data using Robust Optiization RAGHAV PAT, THEODORE B. TRAFALIS, KASH BARKER School of Industrial Engineering
VC-dimension for characterizing classifiers
VC-dimension for characterizing classifiers Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to
Kinetic Theory of Gases: Elementary Ideas
Kinetic Theory of Gases: Eleentary Ideas 17th February 2010 1 Kinetic Theory: A Discussion Based on a Siplified iew of the Motion of Gases 1.1 Pressure: Consul Engel and Reid Ch. 33.1) for a discussion
Handout 7. and Pr [M(x) = χ L (x) M(x) =? ] = 1.
![Handout 7. and Pr [M(x) = χ L (x) M(x) =? ] = 1.](/thumbs/77/76481813.jpg "Handout 7. and Pr [M(x) = χ L (x) M(x) =? ] = 1.")
Notes on Coplexity Theory Last updated: October, 2005 Jonathan Katz Handout 7 1 More on Randoized Coplexity Classes Reinder: so far we have seen RP,coRP, and BPP. We introduce two ore tie-bounded randoized
Probabilistic and Bayesian Analytics
Probabilistic and Bayesian Analytics Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to use these
CS 331: Artificial Intelligence Naïve Bayes. Naïve Bayes
CS 33: Artificial Intelligence Naïe Bayes Thanks to Andrew Moore for soe corse aterial Naïe Bayes A special type of Bayesian network Makes a conditional independence assption Typically sed for classification
NB1140: Physics 1A - Classical mechanics and Thermodynamics Problem set 2 - Forces and energy Week 2: November 2016
NB1140: Physics 1A - Classical echanics and Therodynaics Proble set 2 - Forces and energy Week 2: 21-25 Noveber 2016 Proble 1. Why force is transitted uniforly through a assless string, a assless spring,
This model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.
![This model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.](/thumbs/73/69472191.jpg "This model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.")
CS 493: Algoriths for Massive Data Sets Feb 2, 2002 Local Models, Bloo Filter Scribe: Qin Lv Local Models In global odels, every inverted file entry is copressed with the sae odel. This work wells when
Problem Set 2. Chapter 1 Numerical:
Chapter 1 Nuerical: roble Set 16. The atoic radius of xenon is 18 p. Is that consistent with its b paraeter of 5.15 1 - L/ol? Hint: what is the volue of a ole of xenon atos and how does that copare to
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 11 10/15/2008 ABSTRACT INTEGRATION I
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 11 10/15/2008 ABSTRACT INTEGRATION I Contents 1. Preliinaries 2. The ain result 3. The Rieann integral 4. The integral of a nonnegative
Kinetic Theory of Gases: Elementary Ideas
Kinetic Theory of Gases: Eleentary Ideas 9th February 011 1 Kinetic Theory: A Discussion Based on a Siplified iew of the Motion of Gases 1.1 Pressure: Consul Engel and Reid Ch. 33.1) for a discussion of
What is Probability? (again)
INRODUCTION TO ROBBILITY Basic Concepts and Definitions n experient is any process that generates well-defined outcoes. Experient: Record an age Experient: Toss a die Experient: Record an opinion yes,
The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Parameters
journal of ultivariate analysis 58, 96106 (1996) article no. 0041 The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Paraeters H. S. Steyn
Pattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016/2017 Lessons 9 11 Jan 2017 Outline Artificial Neural networks Notation...2 Convolutional Neural Networks...3
CSE525: Randomized Algorithms and Probabilistic Analysis May 16, Lecture 13
CSE55: Randoied Algoriths and obabilistic Analysis May 6, Lecture Lecturer: Anna Karlin Scribe: Noah Siegel, Jonathan Shi Rando walks and Markov chains This lecture discusses Markov chains, which capture
Will Monroe August 9, with materials by Mehran Sahami and Chris Piech. image: Arito. Parameter learning
Will Monroe August 9, 07 with aterials by Mehran Sahai and Chris Piech iage: Arito Paraeter learning Announceent: Proble Set #6 Goes out tonight. Due the last day of class, Wednesday, August 6 (before
OBJECTIVES INTRODUCTION
M7 Chapter 3 Section 1 OBJECTIVES Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance, and
Physics 221A: HW3 solutions
Physics 22A: HW3 solutions October 22, 202. a) It will help to start things off by doing soe gaussian integrals. Let x be a real vector of length, and let s copute dxe 2 xt Ax, where A is soe real atrix.
One Dimensional Collisions
One Diensional Collisions These notes will discuss a few different cases of collisions in one diension, arying the relatie ass of the objects and considering particular cases of who s oing. Along the way,
Page 1 Lab 1 Elementary Matrix and Linear Algebra Spring 2011
Page Lab Eleentary Matri and Linear Algebra Spring 0 Nae Due /03/0 Score /5 Probles through 4 are each worth 4 points.. Go to the Linear Algebra oolkit site ransforing a atri to reduced row echelon for
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
Introduction to Optimization Techniques. Nonlinear Programming
Introduction to Optiization echniques Nonlinear Prograing Optial Solutions Consider the optiization proble in f ( x) where F R n xf Definition : x F is optial (global iniu) for this proble, if f( x ) f(
Physically Based Modeling CS Notes Spring 1997 Particle Collision and Contact
Physically Based Modeling CS 15-863 Notes Spring 1997 Particle Collision and Contact 1 Collisions with Springs Suppose we wanted to ipleent a particle siulator with a floor : a solid horizontal plane which
Pattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016 Lessons 7 14 Dec 2016 Outline Artificial Neural networks Notation...2 1. Introduction...3... 3 The Artificial
Pseudo-marginal Metropolis-Hastings: a simple explanation and (partial) review of theory
Pseudo-arginal Metropolis-Hastings: a siple explanation and (partial) review of theory Chris Sherlock Motivation Iagine a stochastic process V which arises fro soe distribution with density p(v θ ). Iagine
Reading from Young & Freedman: For this topic, read the introduction to chapter 25 and sections 25.1 to 25.3 & 25.6.
PHY10 Electricity Topic 6 (Lectures 9 & 10) Electric Current and Resistance n this topic, we will cover: 1) Current in a conductor ) Resistivity 3) Resistance 4) Oh s Law 5) The Drude Model of conduction
Kinetic Molecular Theory of Ideal Gases
Lecture -3. Kinetic Molecular Theory of Ideal Gases Last Lecture. IGL is a purely epirical law - solely the consequence of experiental obserations Explains the behaior of gases oer a liited range of conditions.
3.3 Variational Characterization of Singular Values
3.3. Variational Characterization of Singular Values 61 3.3 Variational Characterization of Singular Values Since the singular values are square roots of the eigenvalues of the Heritian atrices A A and
N-Point. DFTs of Two Length-N Real Sequences
Coputation of the DFT of In ost practical applications, sequences of interest are real In such cases, the syetry properties of the DFT given in Table 5. can be exploited to ake the DFT coputations ore
Physics 18 Spring 2011 Homework 3 - Solutions Wednesday February 2, 2011
Phsics 18 Spring 2011 Hoework 3 - s Wednesda Februar 2, 2011 Make sure our nae is on our hoework, and please bo our final answer. Because we will be giving partial credit, be sure to attept all the probles,
Chapter 2 General Properties of Radiation Detectors
Med Phys 4RA3, 4RB3/6R3 Radioisotopes and Radiation Methodology -1 Chapter General Properties of Radiation Detectors Ionizing radiation is ost coonly detected by the charge created when radiation interacts
Support Vector Machines. Machine Learning Series Jerry Jeychandra Blohm Lab
Support Vector Machines Machine Learning Series Jerry Jeychandra Bloh Lab Outline Main goal: To understand how support vector achines (SVMs) perfor optial classification for labelled data sets, also a
Page 1. t F t m v. N s kg s. J F t SPH4U. From Newton Two New Concepts Impulse & Momentum. Agenda
SPH4U Agenda Fro Newton Two New Concepts Ipulse & oentu Ipulse Collisions: you gotta consere oentu! elastic or inelastic (energy consering or not) Inelastic collisions in one diension and in two diensions
Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers Roy D. Yates and David J.
Probability and Stochastic Processes: A Friendly Introduction for Electrical and oputer Engineers Roy D. Yates and David J. Goodan Proble Solutions : Yates and Goodan,1..3 1.3.1 1.4.6 1.4.7 1.4.8 1..6
PHY 171. Lecture 14. (February 16, 2012)
PHY 171 Lecture 14 (February 16, 212) In the last lecture, we looked at a quantitative connection between acroscopic and icroscopic quantities by deriving an expression for pressure based on the assuptions
A nonstandard cubic equation
MATH-Jan-05-0 A nonstandard cubic euation J S Markoitch PO Box West Brattleboro, VT 050 Dated: January, 05 A nonstandard cubic euation is shown to hae an unusually econoical solution, this solution incorporates
PROBABILITY OF RUIN FOR A DEPENDENT, TWO-DIMENSIONAL POISSON PROCESS. 1. Introduction
B A D A N A O P E R A C J N E D E C Z J E Nr 9 Stanisław HELPERN* PROBABLT O RUN OR A DEPENDENT, TWO-DMENSONAL POSSON PROCESS A two-diensional, dependent Poisson risk process is inestigated in the paper.
NON-PARAMETRIC METHODS IN ANALYSIS OF EXPERIMENTAL DATA
NON-PARAMETRIC METHODS IN ANALYSIS OF EXPERIMENTAL DATA Rajender Parsad I.A.S.R.I., Library Aenue, New Delhi 0 0. Introduction In conentional setup the analysis of experimental data is based on the assumptions
(Received 2006 May 29; revised 2007 September 26; accepted 2007 September 26)
Estiation of the iniu integration tie for eterining the equialent continuous soun leel with a gien leel of uncertainty consiering soe statistical hypotheses for roa traffic S. R. e Donato a) (Receie 006
e = n 1 ( ) 3 [ m 3] = n [ m 3] n
![e = n 1 ( ) 3 [ m 3] = n [ m 3] n](/thumbs/91/107171701.jpg "e = n 1 ( ) 3 [ m 3] = n [ m 3] n")
Magnetospheric Physics - Hoework Solutions, /7/4 7. Plasa definition Can a plasa be aintained at teperatures of T e K Hint: Calculate the density liit using the plasa paraeter and explain your result).
ESE 523 Information Theory
ESE 53 Inforation Theory Joseph A. O Sullivan Sauel C. Sachs Professor Electrical and Systes Engineering Washington University 11 Urbauer Hall 10E Green Hall 314-935-4173 (Lynda Marha Answers) jao@wustl.edu
Information Gain. Andrew W. Moore Professor School of Computer Science Carnegie Mellon University.
te to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them
Midterm, Fall 2003
5-78 Midterm, Fall 2003 YOUR ANDREW USERID IN CAPITAL LETTERS: YOUR NAME: There are 9 questions. The ninth may be more time-consuming and is worth only three points, so do not attempt 9 unless you are
SF Chemical Kinetics.
SF Cheical Kinetics. Lecture 5. Microscopic theory of cheical reaction inetics. Microscopic theories of cheical reaction inetics. basic ai is to calculate the rate constant for a cheical reaction fro first
In this chapter, we consider several graph-theoretic and probabilistic models
THREE ONE GRAPH-THEORETIC AND STATISTICAL MODELS 3.1 INTRODUCTION In this chapter, we consider several graph-theoretic and probabilistic odels for a social network, which we do under different assuptions
Polygonal Designs: Existence and Construction
Polygonal Designs: Existence and Construction John Hegean Departent of Matheatics, Stanford University, Stanford, CA 9405 Jeff Langford Departent of Matheatics, Drake University, Des Moines, IA 5011 G
Quadratic forms and a some matrix computations
Linear Algebra or Wireless Conications Lectre: 8 Qadratic ors and a soe atri coptations Ove Edors Departent o Electrical and Inoration echnology Lnd University it Stationary points One diension ( d d =
Biostatistics Department Technical Report
Biostatistics Departent Technical Report BST006-00 Estiation of Prevalence by Pool Screening With Equal Sized Pools and a egative Binoial Sapling Model Charles R. Katholi, Ph.D. Eeritus Professor Departent
Physics 11 HW #7 Solutions
hysics HW #7 Solutions Chapter 7: Focus On Concepts: 2, 6, 0, 3 robles: 8, 7, 2, 22, 32, 53, 56, 57 Focus On Concepts 7-2 (d) Moentu is a ector quantity that has a agnitude and a direction. The agnitudes
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
Due to Fabrication Errors in AWGs
An Estiation of Perforance Degradation Due to Fabrication Errors in AWGs Thoas Kaalais, Thoas Sphicopoulos and Diitris Syridis The authors are with the departent of Inforatics and Telecounications of the
Asymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias
Asymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias Zhiyi Zhang Department of Mathematics and Statistics Uniersity of North Carolina at Charlotte Charlotte, NC 28223 Abstract
THE FIFTH DIMENSION EQUATIONS
JP Journal of Mathematical Sciences Volume 7 Issues 1 & 013 Pages 41-46 013 Ishaan Publishing House This paper is aailable online at http://www.iphsci.com THE FIFTH DIMENSION EQUATIONS Niittytie 1B16 03100
Lesson 3: Free fall, Vectors, Motion in a plane (sections )
Lesson 3: Free fall, Vectors, Motion in a plane (sections.6-3.5) Last time we looked at position s. time and acceleration s. time graphs. Since the instantaneous elocit is lim t 0 t the (instantaneous)