ECON 7335 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO LECTURE 1: BASICS. 1. Bayes Rule. p(b j A)p(A) p(b)
|
|
- Ashley Miles
- 5 years ago
- Views:
Transcription
1 ECON 7335 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO LECTURE : BASICS KRISTOFFER P. NIMARK. Bayes Rule De nition. Bayes Rule. The probability of event A occurring conditional on the event B having occurred is given by p(a j B) = p(b j A)p(A) p(b) (.) as long as p(b) 6= 0. It can be derived from the de nition of conditional probability p(a j B) p (A \ B) p(b) (.) and that the probability of A and B occurring is the same as the probability of B and A occurring implying p(a j B)p(B) = p(b j A)p(A) (.3) which can be rearranged to yield Bayes Rule. Bayes Rule is general and applies whether A and B are discrete or continuous random variables. It can be used to update a prior belief p(x) about the latent variable x conditional on the signal y: Bayes Rule then gives the posterior belief p (x j y) = p(y j x)p(x) p(y) (.4) Date: January 7, 06.
2 KRISTOFFER P. NIMARK.. Example with binary variables and signals (adapted from O Hara 995). Consider a market maker that holds inventory of an asset that can have either a High or Low value. The market maker s prior belief that the value of the asset is high is denoted p(h) = : There are two types of traders, informed and uninformed and the market maker is equally likely to meet either type. An informed trader always buys when the value is high and always sells when the value is low. An uninformed trader is equally likely to buy or sell. What is the posterior probability that the value is high if the rst trade is a sale (S)? Bayes Rule states that p(h j S) = p(s j H)p(H) p(s) Start by nding the probability of a sale conditional on the value being high p(s j H) = 0 + = 4 (.5) and low p(s j L) = + = 3 4 We also need to nd the (unconditional) probability of sale (.6) p(s) = p(s j H)p(H) + p(s j L)p(L) (.7) Plug into Bayes Rule to get the posterior conditional on a sale p (H j S) = = p(s j H)p(H) p(s j H)p(H) + p(s j L)p(L) = 4 (.8) (.9)
3 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO 3 The same steps can be used to nd the posterior probability that the value is high if the rst transaction is a buy (B) p (H j B) = = p(b j H)p(H) p(b j H)p(H) + p(b j L)p(L) = 3 4 (.0) (.)... Updating beliefs when new information arrives. If the arrival of traders is independent over time, it is straighforward to update the posterior after a second transaction takes place by simply using the posterior after the rst observation as the prior in the update. If both the rst and second transaction are sales, the posterior probability is then given by p (H j S; S) = = 0 (.) However, if the rst transaction is a sale and the second a buy, we get p (H j S; B) = = (.3) i.e. the second signal cancels the rst and psoterior equals the original prior belief. Useful fact: Under quite general conditions, beliefs that are updated using Bayes Rule converge almost surely to the truth... Bayes Rule and jointly normally distributed variables. To illustrate the usefulness of Bayes Rule, we can apply it to a simple bivariate setting. Let the prior about the latent variable x be normally distributed x N(0; x) (.4) and the signal y be the sum of the true x plus a normally distributed noise term so that y = x + : N(0; ) (.5)
4 4 KRISTOFFER P. NIMARK We then have all the ingredients need to apply Bayes Rule to nd the posterior p(x j y): p(x) = p(y) = p e x x (.6) x p p x + e (x+) ( x + ) (.7) p(y j x) = p e (y x) (.8) Plugging these expressions into Bayes Rule gives p(x j y) = p e (y x) p p e x + p x e x x (x+) ( x + ) (.9) which can be simpli ed to p(x j y) = p x + e x! x ( x + ) y + x! (.0) The posterior distribution p(x j y) is thus normally distributed with mean x y and variance + : This illustrates two points: First, normally distributed priors combined ( x+ ) x with normally distributed noise in the signal result in normally distributed posteriors. This is an extremely useful property of normal distributions. Second, the posterior variance of x is lower than the prior variance, i.e. x + More information thus reduces uncertainty about x. < x (.)
5 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO 5. The Projection Theorem This section explains how orthogonal projections can be used to nd least squares predictions of random variables. We start by de ning some concepts needed for stating the projection theorem. For more details about the projection theorem, see for instance Chapter of Brockwell and Davis (006) or Chapter 3 in Luenberger (969). De nition. (Inner Product Space) A real vector space H is said to be an inner product space if for each pair of elements x and y in H there is a number hx; yi called the inner product of x and y such that hx; yi = hy; xi (.) hx + y; zi = hx; zi + hy; zi for all x; y; z H (.) hx; yi = hx; yi for all x; y H and R (.3) hx; xi 0 for all x H (.4) hx; xi = 0 if and only if x = 0 (.5) De nition 3. (Norm) The norm of an element x of an inner product space is de ned to be kxk = p hx; xi (.6) De nition 4. (Cauchy Sequence) A sequence fx n ; n = ; ; :::g of elements of an inner product space is said to be Cauchy sequence if kx n x m k! 0 as m; n! i.e. for every " > 0 there exists a positive integer N(") such that kx n x m k < " as m; n > N(")
6 6 KRISTOFFER P. NIMARK De nition 5. (Hilbert Space) A Hilbert space H is an inner product space which is complete, i.e. every Cauchy sequence fx n g converges in norm to some element x H: Theorem. (The Projection Theorem) If M is a closed subspace of the Hilbert Space H and x H; then (i) there is a unique element bx M such that kx bxk = inf ym kx yk and (ii) bx M and kx bxk = inf ym kx yk if and only if bx M and (x bx) M? where M? is the orthogonal complement to M in H: The element bx is called the orthogonal projection of x onto M: Proof. We rst show that if bx is a minimizing vector then x bx must be orthogonal to M: Suppose to the contrary that there is an element m M which is not orthogonal to the error x bx: Without loss of generality we may assume that kmk = and that hx bx; mi = 6= 0: De ne the vector m M m bx + m (.7) We then have that kx m k = kx bx mk (.8) = kx bxk hx bx; mi hm; x bxi + jj (.9) = kx bxk jj (.0) < kx bxk (.) where the second line follows from (.) and the de nition of the norm, the third line comes from the fact that hx bx; mi = hm; x bxi = jj : The inequality on the last line follows
7 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO 7 from the fact that jj > 0: We then have a contradiction: bx cannot be the element in M that minimizes the norm of the error if 6= 0 since kx m k then is smaller than kx bxk : We now show that if x bx is orthogonal to M then it is the unique minimizing vector. For any m M we have that kx mk = kx bx + bx mk (.) = kx bxk + kbx mk (.3) > kx bxk for bx 6= m (.4) Properties of Projection Mappings. Let H be a Hilbert space and and let P M be a projection mapping onto a closed subspace M: Then (i) each x H has a unique representation as a sum of an element in M and an element in M? ; i.e. x = P M x + (I P M )x (.5) (ii) x M if and only if P M x = x (iii) x M? if and only if P M x = 0 (iv) M M if and only if P M P M x = P M (v) kxk = kp M xk + k(i P M ) xk The de nitions and the proofs above refer to Hilbert spaces in general. We now de ne the space relevant for most of time series analysis. De nition 6. (The space L (; F; P ) ) We can de ne the space L (; F; P ) as the space consisting of all collections C of random variables X de ned on the probability space (; F; P ) satisfying the condition Z EX = X(!)P (d!) < (.6)
8 8 KRISTOFFER P. NIMARK and de ne the inner product of this space as hx; Y i = E (XY ) for any X; Y C (.7) Least squares estimation via the projection theorem. The inner product space L satis es all of the axioms above. Noting that the inner product de nition (.7) corresponds to a covariance means that we can use the projection theorem to nd the minimum variance estimate of a vector of random variables with nite variances as a function of some other random variables with nite variances. That is, both the information set and the variables we are trying to predict must be elements of the relevant space and since hx; Y i = E (XY ) implies that an estimate bx that minimizes the norm of the estimation error kx bxk also minimizes the variance since kx bxk = q E (x bx) (x bx) 0 (.8) To nd the estimate bx as a linear function of y simply use that hx y; yi = E [(x y) y 0 ] (.9) = 0 and solve for = E (xy 0 ) [E (yy 0 )] (.0) The advantage of this approach is that once you have made sure that the variables y and x are in a well de ned inner product space, there is no need to minimize the variance directly. The projection theorem ensures that an estimate with orthogonal errors is the (linear) minimum variance estimate. Two useful properties of linear projections.
9 INFORMATION, LEARNING AND EXPECTATIONS IN MACRO 9 If two random variables X and Y are Gaussian, then the projection of Y onto X coincides withe the conditional expectation E(Y j X): If X and Y are not Gaussian, the linear projection of Y onto X is the minimum variance linear prediction of Y given X. References [] Brockwell, P.J. and R.A. Davis, 006, Time Series: Theory and Methods, Springer-Verlag. [] Luenberger, D., (969), Optimization by Vector Space Methods, John Wiley and Sons, Inc., New York. [3] O Hara, M., 995. Market microstructure theory, Blackwell, Cambridge, MA.
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector
More informationINTRODUCTION TO INFORMATION THEORY
INTRODUCTION TO INFORMATION THEORY KRISTOFFER P. NIMARK These notes introduce the machinery of information theory which is a eld within applied mathematics. The material can be found in most textbooks
More informationProjection Theorem 1
Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality
More informationFunctional Analysis: Assignment Set # 3 Spring 2009 Professor: Fengbo Hang February 25, 2009
duardo Corona Functional Analysis: Assignment Set # 3 Spring 9 Professor: Fengbo Hang February 5, 9 C6. Show that a norm that satis es the parallelogram identity: comes from a scalar product. kx + yk +
More informationMath 702 Problem Set 10 Solutions
Math 70 Problem Set 0 Solutions Unless otherwise stated, X is a complex Hilbert space with the inner product h; i the induced norm k k. The null-space range of a linear map T W X! X are denoted by N.T
More informationConvergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit
Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Evan Kwiatkowski, Jan Mandel University of Colorado Denver December 11, 2014 OUTLINE 2 Data Assimilation Bayesian Estimation
More informationECONOMETRIC METHODS II: TIME SERIES LECTURE NOTES ON THE KALMAN FILTER. The Kalman Filter. We will be concerned with state space systems of the form
ECONOMETRIC METHODS II: TIME SERIES LECTURE NOTES ON THE KALMAN FILTER KRISTOFFER P. NIMARK The Kalman Filter We will be concerned with state space systems of the form X t = A t X t 1 + C t u t 0.1 Z t
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationSIGNALS, BELIEFS AND UNUSUAL EVENTS ERID LECTURE DUKE UNIVERSITY
SIGNALS, BELIEFS AND UNUSUAL EVENTS ERID LECTURE DUKE UNIVERSITY KRISTOFFER P. NIMARK These notes presents and discusses ideas related to the modeling framework in my paper Man-bites-dog Business Cycles
More informationMeasure-theoretic probability
Measure-theoretic probability Koltay L. VEGTMAM144B November 28, 2012 (VEGTMAM144B) Measure-theoretic probability November 28, 2012 1 / 27 The probability space De nition The (Ω, A, P) measure space is
More informationIntro to Probability. Andrei Barbu
Intro to Probability Andrei Barbu Some problems Some problems A means to capture uncertainty Some problems A means to capture uncertainty You have data from two sources, are they different? Some problems
More informationCS 630 Basic Probability and Information Theory. Tim Campbell
CS 630 Basic Probability and Information Theory Tim Campbell 21 January 2003 Probability Theory Probability Theory is the study of how best to predict outcomes of events. An experiment (or trial or event)
More informationStochastic Processes
Introduction and Techniques Lecture 4 in Financial Mathematics UiO-STK4510 Autumn 2015 Teacher: S. Ortiz-Latorre Stochastic Processes 1 Stochastic Processes De nition 1 Let (E; E) be a measurable space
More informationMetric Spaces. DEF. If (X; d) is a metric space and E is a nonempty subset, then (E; d) is also a metric space, called a subspace of X:
Metric Spaces DEF. A metric space X or (X; d) is a nonempty set X together with a function d : X X! [0; 1) such that for all x; y; and z in X : 1. d (x; y) 0 with equality i x = y 2. d (x; y) = d (y; x)
More informationBayesian Methods for DSGE models Lecture 1 Macro models as data generating processes
Bayesian Methods for DSGE models Lecture 1 Macro models as data generating processes Kristoffer Nimark CREI, Universitat Pompeu Fabra and Barcelona GSE June 30, 2014 Bayesian Methods for DSGE models Course
More informationLeast Squares Regression
CIS 50: Machine Learning Spring 08: Lecture 4 Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may not cover all the
More informationEcon 325: Introduction to Empirical Economics
Econ 325: Introduction to Empirical Economics Lecture 2 Probability Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 3-1 3.1 Definition Random Experiment a process leading to an uncertain
More informationReview (Probability & Linear Algebra)
Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationLeast Squares Regression
E0 70 Machine Learning Lecture 4 Jan 7, 03) Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are a brief summary of the topics covered in the lecture. They are not a substitute
More information2 Interval-valued Probability Measures
Interval-Valued Probability Measures K. David Jamison 1, Weldon A. Lodwick 2 1. Watson Wyatt & Company, 950 17th Street,Suite 1400, Denver, CO 80202, U.S.A 2. Department of Mathematics, Campus Box 170,
More informationStatistical Learning Theory
Statistical Learning Theory Part I : Mathematical Learning Theory (1-8) By Sumio Watanabe, Evaluation : Report Part II : Information Statistical Mechanics (9-15) By Yoshiyuki Kabashima, Evaluation : Report
More informationMean-Variance Utility
Mean-Variance Utility Yutaka Nakamura University of Tsukuba Graduate School of Systems and Information Engineering Division of Social Systems and Management -- Tennnoudai, Tsukuba, Ibaraki 305-8573, Japan
More information4 Derivations of the Discrete-Time Kalman Filter
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof N Shimkin 4 Derivations of the Discrete-Time
More informationThe Geometric Meaning of the Notion of Joint Unpredictability of a Bivariate VAR(1) Stochastic Process
Econometrics 2013, 3, 207-216; doi:10.3390/econometrics1030207 OPEN ACCESS econometrics ISSN 2225-1146 www.mdpi.com/journal/econometrics Article The Geometric Meaning of the Notion of Joint Unpredictability
More informationProbability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.
Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel
More informationECON 4117/5111 Mathematical Economics
Test 1 September 23, 2016 1. Suppose that p and q are logical statements. The exclusive or, denoted by p Y q, is true when only one of p and q is true. (a) Construct the truth table of p Y q. (b) Prove
More informationSingle Maths B: Introduction to Probability
Single Maths B: Introduction to Probability Overview Lecturer Email Office Homework Webpage Dr Jonathan Cumming j.a.cumming@durham.ac.uk CM233 None! http://maths.dur.ac.uk/stats/people/jac/singleb/ 1 Introduction
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationUNIT-2: MULTIPLE RANDOM VARIABLES & OPERATIONS
UNIT-2: MULTIPLE RANDOM VARIABLES & OPERATIONS In many practical situations, multiple random variables are required for analysis than a single random variable. The analysis of two random variables especially
More informationProbability theory. References:
Reasoning Under Uncertainty References: Probability theory Mathematical methods in artificial intelligence, Bender, Chapter 7. Expert systems: Principles and programming, g, Giarratano and Riley, pag.
More informationProbability Theory Review Reading Assignments
Probability Theory Review Reading Assignments R. Duda, P. Hart, and D. Stork, Pattern Classification, John-Wiley, 2nd edition, 2001 (appendix A.4, hard-copy). "Everything I need to know about Probability"
More informationIntroduction to Linear Algebra. Tyrone L. Vincent
Introduction to Linear Algebra Tyrone L. Vincent Engineering Division, Colorado School of Mines, Golden, CO E-mail address: tvincent@mines.edu URL: http://egweb.mines.edu/~tvincent Contents Chapter. Revew
More informationStochastic integral. Introduction. Ito integral. References. Appendices Stochastic Calculus I. Geneviève Gauthier.
Ito 8-646-8 Calculus I Geneviève Gauthier HEC Montréal Riemann Ito The Ito The theories of stochastic and stochastic di erential equations have initially been developed by Kiyosi Ito around 194 (one of
More informationLecture 22: Variance and Covariance
EE5110 : Probability Foundations for Electrical Engineers July-November 2015 Lecture 22: Variance and Covariance Lecturer: Dr. Krishna Jagannathan Scribes: R.Ravi Kiran In this lecture we will introduce
More informationNear convexity, metric convexity, and convexity
Near convexity, metric convexity, and convexity Fred Richman Florida Atlantic University Boca Raton, FL 33431 28 February 2005 Abstract It is shown that a subset of a uniformly convex normed space is nearly
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 1 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationFall TMA4145 Linear Methods. Exercise set 10
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 207 Exercise set 0 Please justify your answers! The most important part is how you arrive at
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationProbability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3
Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationIntroduction to Bayesian Learning. Machine Learning Fall 2018
Introduction to Bayesian Learning Machine Learning Fall 2018 1 What we have seen so far What does it mean to learn? Mistake-driven learning Learning by counting (and bounding) number of mistakes PAC learnability
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationFundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner
Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationRSMG Working Paper Series. TITLE: The value of information and the value of awareness. Author: John Quiggin. Working Paper: R13_2
2013 TITLE: The value of information and the value of awareness 2011 RSMG Working Paper Series Risk and Uncertainty Program Author: John Quiggin Working Paper: R13_2 Schools of Economics and Political
More informationTopics in Mathematical Economics. Atsushi Kajii Kyoto University
Topics in Mathematical Economics Atsushi Kajii Kyoto University 26 June 2018 2 Contents 1 Preliminary Mathematics 5 1.1 Topology.................................. 5 1.2 Linear Algebra..............................
More informationMath Real Analysis II
Math 4 - Real Analysis II Solutions to Homework due May Recall that a function f is called even if f( x) = f(x) and called odd if f( x) = f(x) for all x. We saw that these classes of functions had a particularly
More informationEconomics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
More information4.3 - Linear Combinations and Independence of Vectors
- Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be
More informationStable Process. 2. Multivariate Stable Distributions. July, 2006
Stable Process 2. Multivariate Stable Distributions July, 2006 1. Stable random vectors. 2. Characteristic functions. 3. Strictly stable and symmetric stable random vectors. 4. Sub-Gaussian random vectors.
More informationDEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER MACHINE LEARNING AND ADAPTIVE INTELLIGENCE
Data Provided: None DEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER 204 205 MACHINE LEARNING AND ADAPTIVE INTELLIGENCE hour Please note that the rubric of this paper is made different from many other papers.
More informationMachine learning - HT Maximum Likelihood
Machine learning - HT 2016 3. Maximum Likelihood Varun Kanade University of Oxford January 27, 2016 Outline Probabilistic Framework Formulate linear regression in the language of probability Introduce
More informationImplicit Function Theorem: One Equation
Natalia Lazzati Mathematics for Economics (Part I) Note 3: The Implicit Function Theorem Note 3 is based on postol (975, h 3), de la Fuente (2, h5) and Simon and Blume (994, h 5) This note discusses the
More informationECSE B Solutions to Assignment 8 Fall 2008
ECSE 34-35B Solutions to Assignment 8 Fall 28 Problem 8.1 A manufacturing system is governed by a Poisson counting process {N t ; t < } with rate parameter λ >. If a counting event occurs at an instant
More informationTopics in Mathematical Economics. Atsushi Kajii Kyoto University
Topics in Mathematical Economics Atsushi Kajii Kyoto University 25 November 2018 2 Contents 1 Preliminary Mathematics 5 1.1 Topology.................................. 5 1.2 Linear Algebra..............................
More informationLinear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space
Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................
More informationChris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
More informationRevisiting independence and stochastic dominance for compound lotteries
Revisiting independence and stochastic dominance for compound lotteries Alexander Zimper Working Paper Number 97 Department of Economics and Econometrics, University of Johannesburg Revisiting independence
More informationTheorem (4.11). Let M be a closed subspace of a Hilbert space H. For any x, y E, the parallelogram law applied to x/2 and y/2 gives.
Math 641 Lecture #19 4.11, 4.12, 4.13 Inner Products and Linear Functionals (Continued) Theorem (4.10). Every closed, convex set E in a Hilbert space H contains a unique element of smallest norm, i.e.,
More information1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,
Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional
More informationSTA 256: Statistics and Probability I
Al Nosedal. University of Toronto. Fall 2017 My momma always said: Life was like a box of chocolates. You never know what you re gonna get. Forrest Gump. There are situations where one might be interested
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationComputational Genomics
Computational Genomics http://www.cs.cmu.edu/~02710 Introduction to probability, statistics and algorithms (brief) intro to probability Basic notations Random variable - referring to an element / event
More informationBayesian Approaches Data Mining Selected Technique
Bayesian Approaches Data Mining Selected Technique Henry Xiao xiao@cs.queensu.ca School of Computing Queen s University Henry Xiao CISC 873 Data Mining p. 1/17 Probabilistic Bases Review the fundamentals
More informationComputational Biology Lecture #3: Probability and Statistics. Bud Mishra Professor of Computer Science, Mathematics, & Cell Biology Sept
Computational Biology Lecture #3: Probability and Statistics Bud Mishra Professor of Computer Science, Mathematics, & Cell Biology Sept 26 2005 L2-1 Basic Probabilities L2-2 1 Random Variables L2-3 Examples
More informationIf Y and Y 0 satisfy (1-2), then Y = Y 0 a.s.
20 6. CONDITIONAL EXPECTATION Having discussed at length the limit theory for sums of independent random variables we will now move on to deal with dependent random variables. An important tool in this
More informationEconomics 620, Lecture 8: Asymptotics I
Economics 620, Lecture 8: Asymptotics I Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 1 / 17 We are interested in the properties of estimators
More informationa b = a T b = a i b i (1) i=1 (Geometric definition) The dot product of two Euclidean vectors a and b is defined by a b = a b cos(θ a,b ) (2)
This is my preperation notes for teaching in sections during the winter 2018 quarter for course CSE 446. Useful for myself to review the concepts as well. More Linear Algebra Definition 1.1 (Dot Product).
More informationIntroduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationLecture 4 Bayes Theorem
1 / 24 Lecture 4 Bayes Theorem September 09, 2010 2 / 24 Lesson Plan 1. Bayes Theorem 2. Simpson s Paradox 3. More worked problems 3 / 24 Why Study Probability? A probability model describes the random
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 89 Part II
More informationLecture 4 Bayes Theorem
Lecture 4 Bayes Theorem Thais Paiva STA 111 - Summer 2013 Term II July 5, 2013 Lecture Plan 1 Probability Review 2 Bayes Theorem 3 More worked problems Why Study Probability? A probability model describes
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationLECTURE 12 UNIT ROOT, WEAK CONVERGENCE, FUNCTIONAL CLT
MARCH 29, 26 LECTURE 2 UNIT ROOT, WEAK CONVERGENCE, FUNCTIONAL CLT (Davidson (2), Chapter 4; Phillips Lectures on Unit Roots, Cointegration and Nonstationarity; White (999), Chapter 7) Unit root processes
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part X Factor analysis When we have data x (i) R n that comes from a mixture of several Gaussians, the EM algorithm can be applied to fit a mixture model. In this setting,
More informationData Mining Techniques
Data Mining Techniques CS 6220 - Section 2 - Spring 2017 Lecture 4 Jan-Willem van de Meent (credit: Yijun Zhao, Arthur Gretton Rasmussen & Williams, Percy Liang) Kernel Regression Basis function regression
More informationThe Normal Linear Regression Model with Natural Conjugate Prior. March 7, 2016
The Normal Linear Regression Model with Natural Conjugate Prior March 7, 2016 The Normal Linear Regression Model with Natural Conjugate Prior The plan Estimate simple regression model using Bayesian methods
More informationBayesian RL Seminar. Chris Mansley September 9, 2008
Bayesian RL Seminar Chris Mansley September 9, 2008 Bayes Basic Probability One of the basic principles of probability theory, the chain rule, will allow us to derive most of the background material in
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationNotes on Time Series Modeling
Notes on Time Series Modeling Garey Ramey University of California, San Diego January 17 1 Stationary processes De nition A stochastic process is any set of random variables y t indexed by t T : fy t g
More informationDiscrete Mathematics and Probability Theory Fall 2015 Lecture 21
CS 70 Discrete Mathematics and Probability Theory Fall 205 Lecture 2 Inference In this note we revisit the problem of inference: Given some data or observations from the world, what can we infer about
More informationThe main purpose of this chapter is to prove the rst and second fundamental theorem of asset pricing in a so called nite market model.
1 2. Option pricing in a nite market model (February 14, 2012) 1 Introduction The main purpose of this chapter is to prove the rst and second fundamental theorem of asset pricing in a so called nite market
More information5. Conditional Distributions
1 of 12 7/16/2009 5:36 AM Virtual Laboratories > 3. Distributions > 1 2 3 4 5 6 7 8 5. Conditional Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an
More informationHilbert Spaces: Infinite-Dimensional Vector Spaces
Hilbert Spaces: Infinite-Dimensional Vector Spaces PHYS 500 - Southern Illinois University October 27, 2016 PHYS 500 - Southern Illinois University Hilbert Spaces: Infinite-Dimensional Vector Spaces October
More informationLecture 11. Probability Theory: an Overveiw
Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the
More informationCorrections to Theory of Asset Pricing (2008), Pearson, Boston, MA
Theory of Asset Pricing George Pennacchi Corrections to Theory of Asset Pricing (8), Pearson, Boston, MA. Page 7. Revise the Independence Axiom to read: For any two lotteries P and P, P P if and only if
More informationGaussian Processes. 1. Basic Notions
Gaussian Processes 1. Basic Notions Let T be a set, and X : {X } T a stochastic process, defined on a suitable probability space (Ω P), that is indexed by T. Definition 1.1. We say that X is a Gaussian
More informationExpectation Propagation Algorithm
Expectation Propagation Algorithm 1 Shuang Wang School of Electrical and Computer Engineering University of Oklahoma, Tulsa, OK, 74135 Email: {shuangwang}@ou.edu This note contains three parts. First,
More informationTopics in Probability and Statistics
Topics in Probability and tatistics A Fundamental Construction uppose {, P } is a sample space (with probability P), and suppose X : R is a random variable. The distribution of X is the probability P X
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationDefinition and basic properties of heat kernels I, An introduction
Definition and basic properties of heat kernels I, An introduction Zhiqin Lu, Department of Mathematics, UC Irvine, Irvine CA 92697 April 23, 2010 In this lecture, we will answer the following questions:
More informationChapter 2. Probability
2-1 Chapter 2 Probability 2-2 Section 2.1: Basic Ideas Definition: An experiment is a process that results in an outcome that cannot be predicted in advance with certainty. Examples: rolling a die tossing
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 17: Continuous random variables: conditional PDF Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin
More informationBayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 2: Bayesian Basics https://people.orie.cornell.edu/andrew/orie6741 Cornell University August 25, 2016 1 / 17 Canonical Machine Learning
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2017 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More information