Review (Probability & Linear Algebra)
|
|
- Preston Walton
- 6 years ago
- Views:
Transcription
1 Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani
2 Outline Axioms of probability theory Conditional probability, Joint probability, Bayes rule Discrete and continuous random variables Probability mass and density functions Expected value, variance, standard deviation Expectation for two variables covariance, correlation Some probability distributions Gaussian distribution Linear Algebra 2
3 Basic Probability Elements Sample space ( ): set of all possible outcomes (or worlds) Outcomes are assumed to be mutually exclusive. An event is a certain set of outcomes (i.e., subset of ). A random variable is a function defined over the sample space Gender: {, } Height: R 3
4 Probability Space A probability space is defined as a triple : A sample space Ω that contains the set of all possible outcomes (outcomes also called states of nature) Aset whose elements are called events. The events are subsets of Ω. should be a Borel Field. represents the probability measure that assigns probabilities to events. 4
5 Probability and Logic Probability theory can be viewed as a generalization of propositional logic Probabilistic logic : is a propositional logic sentence Belief of agent in is no longer restricted to true, false, unknown () can range from 0 to 1 5
6 Probability Axioms (Kolomogrov) Axioms define a reasonable theory of uncertainty Kolomogrov s probability axioms (propositional notation) () 0 ( ) If is a tautology = 1 = + ( ) (, ) True a b ^ 6
7 Random Variables Random variables: Variables in probability theory Domain of random variables: Boolean, discrete or continuous Probability distribution: the function describing probabilities of possible values of a random variable = 1 =,(=2)=, 7
8 Random Variables Random variable is a function that maps every outcome in to a real (complex) number. To define probabilities easily as functions defined on (real) numbers. Expectation, variance, Head Tail 0 1 Real line 8
9 Probabilistic Inference Joint probability distribution Can specify probability of every atomic event We can find every probabilities from it (by summing over atomic events). Prior and posterior probabilities belief in the absence or presence of evidences Bayes rule used when it is difficult to compute ( ) but we have information about ( ) Independence new evidence may be irrelevant 9
10 Joint Probability Distribution Probability of all combinations of the values for a set of random variables. If two or more random variables are considered together, they can be described in terms of their joint probability Example: Joint probability of features (,,, ) 10
11 Prior and Posterior Probabilities Prior or unconditional probabilities of propositions: belief in the absence of any other evidence e.g., = = 0.14 (h = ) = 0.6 Posterior or conditional probabilities: belief in the presence of evidences ( ) is the probability of given that is true e.g., h = = ) 11
12 Conditional Probability if Product rule (an alternative formulation): ( ) = ( ) () = ( ) () ( ) obeys the same rules as probabilities E.g., ( ) + (~ ) = 1 12
13 Conditional Probability For statistically dependent variables, knowing the value of one variable may allow us to better estimate the other. All probabilities in effect are conditional probabilities E.g., () = ( ) (. ) Ω Type equation here. Ω Renormalize the probability of events jointly occur with b 13
14 Conditional Probability: Example Rolling a fair dice : the outcome is an even number : the outcome is a prime number P a b Pab Pb
15 Chain rule Chain rule is derived by successive application of product rule: ( 1,, ) = ( 1,, ) ( 1,, ) = ( 1,, ) ( 1,, ) ( 1,, ) = =( ) ( 1,, ) 15
16 Law of Total Probability mutually exclusive and is an event (subset of ) = ( )+ ( )+ + ( ) 16
17 Independence Propositions and are independent iff ( ) = () = (, ) = () () Knowing tells us nothing about (and vice versa) 17
18 Bayes Rule Bayes rule: Obtained form product rule: = = In some problems, it may be difficult to compute ( ) directly, yet we might have information about ( ). ( ) = ( ) () / () 18
19 Bayes Rule Often it would be useful to derive the rule a bit further: 19
20 Bayes Rule: Example Meningitis( )&Stiffneck( ) = = 0.01 =0.7 ( ) = () /() = /0.01 =
21 Bayes Rule: General Form 21
22 Calculating Probabilities from Joint Distribution Useful techniques Marginalization = (, ) Conditioning = () Normalization X = X, 22
23 Probability Mass Function (PMF) Probability Mass Function (PMF) shows the probability for each value of a discrete random variable Each impulse magnitude is equal to the probability of the corresponding outcome Example: PMF of a fair dice () ( = ) 0 (=) =1 23
24 Probability Density Function (PDF) Probability Density Function (PDF) is defined for continuous random variables 2 2 The probability of x x, x is ( ) px ( x 0) lim P( x 0 X x 0 ) () () 0 = 1 ( ) 24
25 Cumulative Distribution Function (CDF) Cumulative Distribution Function (CDF) Defined as the integration of PDF Similarly defined on discrete variables (summation instead of integration) () CDF x F x P X x x X. FX x px d Non-decreasing Right Continuous ( ) = 0 ( ) = 1 () = () P( u x v) F ( v) F ( u) X X 25
26 Distribution Statistics Basic descriptors of spatial distributions: Mean Value Variance & Standard Deviation Moments Covariance & Correlation 26
27 Expected Value Expected (or mean) value: weighted average of all possible values of the random variable 27 Expectation of a discrete random variable : xpx E X Expectation of a function of a discrete random variable : Expected value of a continuous random variable : Expectation of a function of a continuous random variable : x ( ) g( x) Px E g X E X xpx x dx x E g X g x px x dx
28 Variance Variance: a measure of how far values of a random variable are spread out around its expected value 2 Var X E X E X X 2 2 X X E X Standard deviation: square root of variance X var( X ) 28
29 Moments Moments n th order moment of a random variable : M E X n n Normalized n th order moment: E X X n The first order moment is the mean value. The second order moment is the variance added by the square of the mean. 29
30 Correlation & Covariance Correlation Crr ( X, Y ) E XY Discrete RVs x xyp ( x, y ) y Covariance is the correlation of mean removed variables: Cov ( X, Y ) E X X Y Y E XY X Y 30
31 Covariance: Example, = 0, =
32 Covariance Properties The covariance value shows the tendency of the pair of RVs to increase together >0 and tend to increase together <0 : tends to decrease when increases =0 : no linear correlation between and 32
33 Orthogonal, Uncorrelated & Independent RVs Orthogonal random variables ( ), = 0 Uncorrelated random variables ( ), = 0 Independent random variables, = 0 Independent random variables 33
34 Pearson s Product Moment Correlation Defined only if both and are finite and nonzero. shows the degree of linear dependence between and. 1 1 according to Cauchy- Schwarz inequality ( ) =1shows a perfect positive linear relationship and = 1 shows a perfect negative linear relationship 34
35 Pearson s Correlation: Examples = (, ) [Wikipedia] 35
36 Covariance Matrix If is a vector of random variables ( -dim random vector): Covariance matrix indicates the tendency of each pair of RVs to vary together =[ ] = = ( ) ( ) = (( )( )) (( )( )) (( )( )) (( )( )) 36
37 Covariance Matrix: Two Variables Cov ( X, Y ) = =
38 Covariance Matrix shows the covariance of and : ij Cov ( X i, X j ) E X i i. X j j = 38
39 Sums of Random Variables Mean: = + =+ Variance: () = () + () + 2(, ) If, independent: () = () + () Distribution: p ( z) p ( x, z x) dx Z independence X, Y p ( x) p ( y ) p ( x) p ( z x) dx X Y X Y 39
40 Some Famous Probability Mass and Density Functions Uniform () ~(, ) 1 p x U a U b b a 1 Gaussian (Normal) p x e N. 2 x 2, () 1 2 ~(, ) 40
41 Some Famous Probability Mass and Density Functions Binomial n P X k 1p p k nk k ( = ) ~(, ) Exponential () x x e x 0 px e U x 0 x
42 Gaussian (Normal) Distribution It is widely used to model the distribution of continuous variables 68% within,+ 95% within 2,+2 Standard Normal distribution: 42
43 Multivariate Gaussian Distribution is a vector of Gaussian variables 1 ( ) 1 1 xμ T ( xμ) p() x N (, μ ) e 2 (2 ) d /2 1/2 μe[] x E( x ) E( x ) 1 d T E[( xμ)( xμ) ] Mahalanobis Distance: 43 r 2 T x μ x μ T 1 ( ) ( ) Probability Density x x
44 Multivariate Gaussian Distribution The covariance matrix is always symmetric and positive semidefinite Multivariate Gaussian is completely specified by +( + 1)/2 parameters Special cases of : = : Independent random variables with the same variance (circularly symmetric Gaussian) Digonal matrix = different variances : Independent random variables with 44
45 Multivariate Gaussian Distribution Level Surfaces The Gaussian distribution will be constant on surfaces in -space for which: = Hyper-ellipsoid Principal axes of the hyper-ellipsoids are the eigenvectors of. Bivariate Gaussian: Curves of constant density are ellipses. 45
46 Bivariate Gaussian distribution and are the eigenvalues of ( )and and are the corresponding eigenvectors = 46
47 Linear Transformations on Multivariate Gaussian : Whitening transform = 47
48 Gaussian Distribution Properties Some attracting properties of the Gaussian distribution: Marginal and conditional distributions of a Gaussian are also Gaussian After a linear transformation, a Gaussian distribution is again Gaussian There exists a linear transformation that diagonalizes the covariance matrix (whitening transform). It converts the multivariate normal distribution into a spherical one. Gaussian is a distribution that maximizes the entropy Gaussian is stable and infinitely divisible Central Limit Theorem Some distributions can be estimated by Gaussian distribution when their parameter value is sufficiently large (e.g., Binomial) 48
49 Central Limit Theorem (under mild conditions) Suppose i.i.d. (Independent Identically Distributed) RVs ( =1,,) with finite variances Let S N be the sum of these RVs Distribution of converges to a normal distribution as increases, regardless to the distribution of the RVs. Example: N X i 1 i ~ uniform, =1,, = 1 49
50 Linear Algebra: Basic Definitions Matrix : Matrix Transpose A a a a a a a a a... a n n [ aij ] mn m1 m2 mn T BA b a 1 i n,1 j m Symetric matrix = ij ji Vector 50 a a1 T... a [ a1,..., an ] a n
51 Linear Mapping Linear function ( + ) = () + () () = (),, A linear function: In general, a matrix = can be used to denote a map : R R where = + + = 51
52 Linear Algebra: Basic Definitions Inner (dot) product ab, Matrix multiplication A T a b i 1 [ a ] B [ b ] ij mp ij pn n ab i i AB = C [ c ] c A, B T ij mn ij i.. j i-th row of A j-th column of B 52
53 Inner Product Inner (dot) product Length (Euclidean norm) of a vector a is normalized iff a = 1 Angle between vectors Orthogonal vectors and : n T ab i 1 ab n T a a i 1 Orthonormal set of vectors,,, :, = 1 = a cos T ab i T ab a b 0 i a 2 i
54 Linear Independence A set of vectors is linearly independent if no vector is a linear combination of other vectors = 0 = =...= = 0 54
55 Matrix Determinant and Trace Determinant () = () () n det( A) a A ; i 1,... n; A ij j 1 i j ( 1) det( M ) ij ij ij Trace tr[ A ] n a j 1 jj 55
56 Matrix Inversion Inverse of : AB BA I B A n 1 exists iff det () 0 (A is nonsingular) Singular: det () = 0 ill-conditioned:a is nonsingular but close to being singular Pseudo-inverse for a non square matrix # = is not singular # = 56
57 Matrix Rank : maximum number of linearly independent columns or rows of A. : Full rank : iff is nonsingular ( ) 57
58 Eigenvectors and Eigenvalues Characteristic equation: n-th order polynomial, with n roots det( AI ) 0 n tr( A) n j j 1 det( A) A n j 1 j 58
59 Eigenvectors and Eigenvalues: Symmetric Matrix For a symmetric matrix, the eigenvectors corresponding to distinct eigenvalues are orthogonal These eigenvectors can be used to form an orthonormal set ( and ) 59
60 Eigen Decomposition: Symmetric Matrix = = = Eigen decomposition of a symmetric matrix: 60
61 Positive definite matrix Symmetric is positive definite: Eigen values of a positive define matrix are positive:, >0 61
62 Simple Vector Derivatives 62
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationStatistical Pattern Recognition
Statistical Pattern Recognition A Brief Mathematical Review Hamid R. Rabiee Jafar Muhammadi, Ali Jalali, Alireza Ghasemi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Probability theory
More informationL2: Review of probability and statistics
Probability L2: Review of probability and statistics Definition of probability Axioms and properties Conditional probability Bayes theorem Random variables Definition of a random variable Cumulative distribution
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationProbability Theory Review Reading Assignments
Probability Theory Review Reading Assignments R. Duda, P. Hart, and D. Stork, Pattern Classification, John-Wiley, 2nd edition, 2001 (appendix A.4, hard-copy). "Everything I need to know about Probability"
More informationLinear Algebra in Computer Vision. Lecture2: Basic Linear Algebra & Probability. Vector. Vector Operations
Linear Algebra in Computer Vision CSED441:Introduction to Computer Vision (2017F Lecture2: Basic Linear Algebra & Probability Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Mathematics in vector space Linear
More informationRecitation. Shuang Li, Amir Afsharinejad, Kaushik Patnaik. Machine Learning CS 7641,CSE/ISYE 6740, Fall 2014
Recitation Shuang Li, Amir Afsharinejad, Kaushik Patnaik Machine Learning CS 7641,CSE/ISYE 6740, Fall 2014 Probability and Statistics 2 Basic Probability Concepts A sample space S is the set of all possible
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2
MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationFundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner
Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization
More informationProbability Theory for Machine Learning. Chris Cremer September 2015
Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares
More informationA Tutorial on Data Reduction. Principal Component Analysis Theoretical Discussion. By Shireen Elhabian and Aly Farag
A Tutorial on Data Reduction Principal Component Analysis Theoretical Discussion By Shireen Elhabian and Aly Farag University of Louisville, CVIP Lab November 2008 PCA PCA is A backbone of modern data
More information1 Proof techniques. CS 224W Linear Algebra, Probability, and Proof Techniques
1 Proof techniques Here we will learn to prove universal mathematical statements, like the square of any odd number is odd. It s easy enough to show that this is true in specific cases for example, 3 2
More informationCommunication Theory II
Communication Theory II Lecture 5: Review on Probability Theory Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt Febraury 22 th, 2015 1 Lecture Outlines o Review on probability theory
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More informationDependence. MFM Practitioner Module: Risk & Asset Allocation. John Dodson. September 11, Dependence. John Dodson. Outline.
MFM Practitioner Module: Risk & Asset Allocation September 11, 2013 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationLecture 11. Probability Theory: an Overveiw
Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationBasic Concepts in Matrix Algebra
Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More informationReview: mostly probability and some statistics
Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Uncertainty & Probabilities & Bandits Daniel Hennes 16.11.2017 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Uncertainty Probability
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More information3. Review of Probability and Statistics
3. Review of Probability and Statistics ECE 830, Spring 2014 Probabilistic models will be used throughout the course to represent noise, errors, and uncertainty in signal processing problems. This lecture
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationBasics on Probability. Jingrui He 09/11/2007
Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability
More informationStatistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationGaussian random variables inr n
Gaussian vectors Lecture 5 Gaussian random variables inr n One-dimensional case One-dimensional Gaussian density with mean and standard deviation (called N, ): fx x exp. Proposition If X N,, then ax b
More informationProbability theory. References:
Reasoning Under Uncertainty References: Probability theory Mathematical methods in artificial intelligence, Bender, Chapter 7. Expert systems: Principles and programming, g, Giarratano and Riley, pag.
More informationAppendix A : Introduction to Probability and stochastic processes
A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationMultivariate Distributions (Hogg Chapter Two)
Multivariate Distributions (Hogg Chapter Two) STAT 45-1: Mathematical Statistics I Fall Semester 15 Contents 1 Multivariate Distributions 1 11 Random Vectors 111 Two Discrete Random Variables 11 Two Continuous
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 PROBABILITY. Prof. Steven Waslander
ME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 Prof. Steven Waslander p(a): Probability that A is true 0 pa ( ) 1 p( True) 1, p( False) 0 p( A B) p( A) p( B) p( A B) A A B B 2 Discrete Random Variable X
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationIntroduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
More informationHuman-Oriented Robotics. Probability Refresher. Kai Arras Social Robotics Lab, University of Freiburg Winter term 2014/2015
Probability Refresher Kai Arras, University of Freiburg Winter term 2014/2015 Probability Refresher Introduction to Probability Random variables Joint distribution Marginalization Conditional probability
More informationMath 302 Outcome Statements Winter 2013
Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the Three-Dimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationCS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro
CS37300 Class Notes Jennifer Neville, Sebastian Moreno, Bruno Ribeiro 2 Background on Probability and Statistics These are basic definitions, concepts, and equations that should have been covered in your
More informationDependence. Practitioner Course: Portfolio Optimization. John Dodson. September 10, Dependence. John Dodson. Outline.
Practitioner Course: Portfolio Optimization September 10, 2008 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y ) (x,
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More informationBasic Probability. Robert Platt Northeastern University. Some images and slides are used from: 1. AIMA 2. Chris Amato
Basic Probability Robert Platt Northeastern University Some images and slides are used from: 1. AIMA 2. Chris Amato (Discrete) Random variables What is a random variable? Suppose that the variable a denotes
More informationMultivariate Statistics
Multivariate Statistics Chapter 2: Multivariate distributions and inference Pedro Galeano Departamento de Estadística Universidad Carlos III de Madrid pedro.galeano@uc3m.es Course 2016/2017 Master in Mathematical
More informationcomponent risk analysis
273: Urban Systems Modeling Lec. 3 component risk analysis instructor: Matteo Pozzi 273: Urban Systems Modeling Lec. 3 component reliability outline risk analysis for components uncertain demand and uncertain
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationMachine Learning for Large-Scale Data Analysis and Decision Making A. Week #1
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Week #1 Today Introduction to machine learning The course (syllabus) Math review (probability + linear algebra) The future
More informationSingle Maths B: Introduction to Probability
Single Maths B: Introduction to Probability Overview Lecturer Email Office Homework Webpage Dr Jonathan Cumming j.a.cumming@durham.ac.uk CM233 None! http://maths.dur.ac.uk/stats/people/jac/singleb/ 1 Introduction
More informationSummary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016
8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying
More informationStochastic Processes. Review of Elementary Probability Lecture I. Hamid R. Rabiee Ali Jalali
Stochastic Processes Review o Elementary Probability bili Lecture I Hamid R. Rabiee Ali Jalali Outline History/Philosophy Random Variables Density/Distribution Functions Joint/Conditional Distributions
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationModèles stochastiques II
Modèles stochastiques II INFO 154 Gianluca Bontempi Département d Informatique Boulevard de Triomphe - CP 1 http://ulbacbe/di Modéles stochastiques II p1/50 The basics of statistics Statistics starts ith
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationIntroduction to Machine Learning
Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationLinear Algebra V = T = ( 4 3 ).
Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationProbability theory basics
Probability theory basics Michael Franke Basics of probability theory: axiomatic definition, interpretation, joint distributions, marginalization, conditional probability & Bayes rule. Random variables:
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationChap 3. Linear Algebra
Chap 3. Linear Algebra Outlines 1. Introduction 2. Basis, Representation, and Orthonormalization 3. Linear Algebraic Equations 4. Similarity Transformation 5. Diagonal Form and Jordan Form 6. Functions
More information02 Background Minimum background on probability. Random process
0 Background 0.03 Minimum background on probability Random processes Probability Conditional probability Bayes theorem Random variables Sampling and estimation Variance, covariance and correlation Probability
More informationOutline. Random Variables. Examples. Random Variable
Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More information01 Probability Theory and Statistics Review
NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement
More information1 Linear Regression and Correlation
Math 10B with Professor Stankova Worksheet, Discussion #27; Tuesday, 5/1/2018 GSI name: Roy Zhao 1 Linear Regression and Correlation 1.1 Concepts 1. Often when given data points, we want to find the line
More informationProbability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3
Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationJoint Distribution of Two or More Random Variables
Joint Distribution of Two or More Random Variables Sometimes more than one measurement in the form of random variable is taken on each member of the sample space. In cases like this there will be a few
More informationQuantifying uncertainty & Bayesian networks
Quantifying uncertainty & Bayesian networks CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2016 Soleymani Artificial Intelligence: A Modern Approach, 3 rd Edition,
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationa b = a T b = a i b i (1) i=1 (Geometric definition) The dot product of two Euclidean vectors a and b is defined by a b = a b cos(θ a,b ) (2)
This is my preperation notes for teaching in sections during the winter 2018 quarter for course CSE 446. Useful for myself to review the concepts as well. More Linear Algebra Definition 1.1 (Dot Product).
More informationJoint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete
More informationChapter 5. The multivariate normal distribution. Probability Theory. Linear transformations. The mean vector and the covariance matrix
Probability Theory Linear transformations A transformation is said to be linear if every single function in the transformation is a linear combination. Chapter 5 The multivariate normal distribution When
More informationMULTIVARIATE DISTRIBUTIONS
Chapter 9 MULTIVARIATE DISTRIBUTIONS John Wishart (1898-1956) British statistician. Wishart was an assistant to Pearson at University College and to Fisher at Rothamsted. In 1928 he derived the distribution
More informationLinear Algebra for Machine Learning. Sargur N. Srihari
Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationVAR Model. (k-variate) VAR(p) model (in the Reduced Form): Y t-2. Y t-1 = A + B 1. Y t + B 2. Y t-p. + ε t. + + B p. where:
VAR Model (k-variate VAR(p model (in the Reduced Form: where: Y t = A + B 1 Y t-1 + B 2 Y t-2 + + B p Y t-p + ε t Y t = (y 1t, y 2t,, y kt : a (k x 1 vector of time series variables A: a (k x 1 vector
More informationPROBABILITY AND INFORMATION THEORY. Dr. Gjergji Kasneci Introduction to Information Retrieval WS
PROBABILITY AND INFORMATION THEORY Dr. Gjergji Kasneci Introduction to Information Retrieval WS 2012-13 1 Outline Intro Basics of probability and information theory Probability space Rules of probability
More informationAarti Singh. Lecture 2, January 13, Reading: Bishop: Chap 1,2. Slides courtesy: Eric Xing, Andrew Moore, Tom Mitchell
Machine Learning 0-70/5 70/5-78, 78, Spring 00 Probability 0 Aarti Singh Lecture, January 3, 00 f(x) µ x Reading: Bishop: Chap, Slides courtesy: Eric Xing, Andrew Moore, Tom Mitchell Announcements Homework
More informationProbability, Random Processes and Inference
INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx
More informationBayes Decision Theory
Bayes Decision Theory Minimum-Error-Rate Classification Classifiers, Discriminant Functions and Decision Surfaces The Normal Density 0 Minimum-Error-Rate Classification Actions are decisions on classes
More information