CS229 Lecture notes. Andrew Ng
|
|
- Valentine Phelps
- 5 years ago
- Views:
Transcription
1 CS229 Lecture notes Andrew Ng Part IX The EM agorithm In the previous set of notes, we taked about the EM agorithm as appied to fitting a mixture of Gaussians. In this set of notes, we give a broader view of the EM agorithm, and show how it can be appied to a arge famiy of estimation probems with atent variabes. We begin our discussion with a very usefu resut caed Jensen s inequaity 1 Jensen s inequaity Let f be a function whose domain is the set of rea numbers. Reca that f is a convex function if f (x) 0 (for a x R). In the case of f taking vector-vaued inputs, this is generaized to the condition that its hessian H is positive semi-definite (H 0). If f (x) > 0 for a x, then we say f is stricty convex (in the vector-vaued case, the corresponding statement is that H must be stricty positive semi-definite, written H > 0). Jensen s inequaity can then be stated as foows: Theorem. Let f be a convex function, and et X be a random variabe. Then: E[f(X)] f(ex). Moreover, if f is stricty convex, then E[f(X)] = f(ex) hods true if and ony if X = E[X] with probabiity 1 (i.e., if X is a constant). Reca our convention of occasionay dropping the parentheses when writing expectations, so in the theorem above, f(ex) = f(e[x]). For an interpretation of the theorem, consider the figure beow. 1
2 2 f(a) f E[f(X)] f(b) f(ex) a E[X] b Here, f is a convex function shown by the soid ine. Aso, X is a random variabe that has a 0.5 chance of taking the vaue a, and a 0.5 chance of taking the vaue b (indicated on the x-axis). Thus, the expected vaue of X is given by the midpoint between a and b. We aso see the vaues f(a), f(b) and f(e[x]) indicated on the y-axis. Moreover, the vaue E[f(X)] is now the midpoint on the y-axis between f(a) and f(b). From our exampe, we see that because f is convex, it must be the case that E[f(X)] f(ex). Incidentay, quite a ot of peope have troube remembering which way the inequaity goes, and remembering a picture ike this is a good way to quicky figure out the answer. Remark. Reca that f is [stricty] concave if and ony if f is [stricty] convex (i.e., f (x) 0 or H 0). Jensen s inequaity aso hods for concave functions f, but with the direction of a the inequaities reversed (E[f(X)] f(ex), etc.). 2 The EM agorithm Suppose we have an estimation probem in which we have a training set {x (1),..., x (m) } consisting of m independent exampes. We wish to fit the parameters of a mode p(x, z) to the data, where the ikeihood is given by m (θ) = og p(x; θ) = m og z p(x, z; θ).
3 3 But, expicity finding the maximum ikeihood estimates of the parameters θ may be hard. Here, the s are the atent random variabes; and it is often the case that if the s were observed, then maximum ikeihood estimation woud be easy. In such a setting, the EM agorithm gives an efficient method for maximum ikeihood estimation. Maximizing (θ) expicity might be difficut, and our strategy wi be to instead repeatedy construct a ower-bound on (E-step), and then optimize that ower-bound (M-step). For each i, et Q i be some distribution over the z s ( z Q i(z) = 1, Q i (z) 0). Consider the foowing: 1 og p(x (i) ; θ) = i i = i i og og p(x (i), ; θ) (1) p(x(i), ; θ) og p(x(i), z(i) ; θ) The ast step of this derivation used Jensen s inequaity. Specificay, f(x) = og x is a concave function, since f (x) = 1/x 2 < 0 over its domain x R +. Aso, the term [ ] p(x Q i ( (i), ; θ) ) in the summation is ust an expectation of the quantity [ p(x (i), ; θ)/ ] with respect to drawn according to the distribution given by Q i. By Jensen s inequaity, we have f ( [ ]) [ p(x (i), ; θ) E Q i E z(i) Qi f ( p(x (i), ; θ) )], where the Q i subscripts above indicate that the expectations are with respect to drawn from Q i. This aowed us to go from Equation (2) to Equation (3). Now, for any set of distributions Q i, the formua (3) gives a ower-bound on (θ). There re many possibe choices for the Q i s. Which shoud we choose? We, if we have some current guess θ of the parameters, it seems 1 If z were continuous, then Q i woud be a density, and the summations over z in our discussion are repaced with integras over z. (2) (3)
4 4 natura to try to make the ower-bound tight at that vaue of θ. I.e., we make the inequaity above hod with equaity at our particuar vaue of θ. (We see ater how this enabes us to prove that (θ) increases monotonicay with successsive iterations of EM.) To make the bound tight for a particuar vaue of θ, we need for the step invoving Jensen s inequaity in our derivation above to hod with equaity. For this to be true, we know it is sufficient that that the expectation be taken over a constant -vaued random variabe. I.e., we require that p(x (i), ; θ) = c for some constant c that does not depend on. This is easiy accompished by choosing p(x (i), ; θ). Actuay, since we know z Q i( ) = 1 (because it is a distribution), this further tes us that = p(x(i), ; θ) z p(x(i), z; θ) = p(x(i), ; θ) p(x (i) ; θ) = p( x (i) ; θ) Thus, we simpy set the Q i s to be the posterior distribution of the s given x (i) and the setting of the parameters θ. Now, for this choice of the Q i s, Equation (3) gives a ower-bound on the ogikeihood that we re trying to maximize. This is the E-step. In the M-step of the agorithm, we then maximize our formua in Equation (3) with respect to the parameters to obtain a new setting of the θ s. Repeatedy carrying out these two steps gives us the EM agorithm, which is as foows: Repeat unti convergence { (E-step) For each i, set (M-step) Set θ := arg max θ := p( x (i) ; θ). i og p(x(i), z(i) ; θ).
5 5 } How we we know if this agorithm wi converge? We, suppose θ (t) and θ (t+1) are the parameters from two successive iterations of EM. We wi now prove that (θ (t) ) (θ (t+1) ), which shows EM aways monotonicay improves the og-ikeihood. The key to showing this resut ies in our choice of the Q i s. Specificay, on the iteration of EM in which the parameters had started out as θ (t), we woud have chosen i ( ) := p( x (i) ; θ (t) ). We saw earier that this choice ensures that Jensen s inequaity, as appied to get Equation (3), hods with equaity, and hence (θ (t) ) = i i ( ) og p(x(i), ; θ (t) ). i ( ) The parameters θ (t+1) are then obtained by maximizing the right hand side of the equation above. Thus, (θ (t+1) ) i i i ( ) og p(x(i), ; θ (t+1) ) i ( ) i ( ) og p(x(i), ; θ (t) ) i ( ) (4) (5) = (θ (t) ) (6) This first inequaity comes from the fact that (θ) i og p(x(i), z(i) ; θ) hods for any vaues of Q i and θ, and in particuar hods for Q i = i, θ = θ (t+1). To get Equation (5), we used the fact that θ (t+1) is chosen expicity to be arg max θ i og p(x(i), z(i) ; θ) and thus this formua evauated at θ (t+1) must be equa to or arger than the same formua evauated at θ (t). Finay, the step used to get (6) was shown earier, and foows from i having been chosen to make Jensen s inequaity hod with equaity at θ (t).,
6 6 Hence, EM causes the ikeihood to converge monotonicay. In our description of the EM agorithm, we said we d run it unti convergence. Given the resut that we ust showed, one reasonabe convergence test woud be to check if the increase in (θ) between successive iterations is smaer than some toerance parameter, and to decare convergence if EM is improving (θ) too sowy. Remark. If we define J(Q, θ) = i og p(x(i), z(i) ; θ), the we know (θ) J(Q, θ) from our previous derivation. The EM can aso be viewed a coordinate ascent on J, in which the E-step maximizes it with respect to Q (check this yoursef), and the M-step maximizes it with respect to θ. 3 Mixture of Gaussians revisited Armed with our genera definition of the EM agorithm, ets go back to our od exampe of fitting the parameters φ, µ and Σ in a mixture of Gaussians. For the sake of brevity, we carry out the derivations for the M-step updates ony for φ and µ, and eave the updates for Σ as an exercise for the reader. The E-step is easy. Foowing our agorithm derivation above, we simpy cacuate = Q i ( = ) = P ( = x (i) ; φ, µ, Σ). Here, Q i ( = ) denotes the probabiity of taking the vaue under the distribution Q i. Next, in the M-step, we need to maximize, with respect to our parameters φ, µ, Σ, the quantity m og p(x(i), z(i) ; φ, µ, Σ) m k = Q i ( = ) og p(x(i) = ; µ, Σ)p( = ; φ) Q i ( = ) = m =1 k =1 og 1 (2π) n/2 Σ exp ( 1 1/2 2 (x(i) µ ) T Σ 1 (x (i) µ ) ) φ
7 7 Lets maximize this with respect to µ. If we take the derivative with respect to µ, we find µ m k =1 og = µ m = 1 2 m = m k =1 1 (2π) n/2 Σ exp ( 1 1/2 2 (x(i) µ ) T Σ (x(i) µ ) T Σ 1 (x (i) µ ) µ 2µ T Σ 1 ( ) Σ 1 x (i) Σ 1 µ x (i) µ T Σ 1 µ (x (i) µ ) ) φ Setting this to zero and soving for µ therefore yieds the update rue µ := m w(i) x (i) m w(i) which was what we had in the previous set of notes. Lets do one more exampe, and derive the M-step update for the parameters φ. Grouping together ony the terms that depend on φ, we find that we need to maximize m k =1 og φ. However, there is an additiona constraint that the φ s sum to 1, since they represent the probabiities φ = p( = ; φ). To dea with the constraint that k =1 φ = 1, we construct the Lagrangian, L(φ) = m k =1 og φ + β( k φ 1), =1 where β is the Lagrange mutipier. 2 Taking derivatives, we find φ L(φ) = m φ We don t need to worry about the constraint that φ 0, because as we shorty see, the soution we find from this derivation wi automaticay satisfy that anyway.
8 8 Setting this to zero and soving, we get φ = m w(i) β I.e., φ m w(i). Using the constraint that φ = 1, we easiy find that β = m k =1 w(i) = m 1 = m. (This used the fact that w(i) = Q i ( = ), and since probabiities sum to 1, w(i) = 1.) We therefore have our M-step updates for the parameters φ : φ := 1 m m. The derivation for the M-step updates to Σ are aso entirey straightforward.
Expectation-Maximization for Estimating Parameters for a Mixture of Poissons
Expectation-Maximization for Estimating Parameters for a Mixture of Poissons Brandon Maone Department of Computer Science University of Hesini February 18, 2014 Abstract This document derives, in excrutiating
More informationA Brief Introduction to Markov Chains and Hidden Markov Models
A Brief Introduction to Markov Chains and Hidden Markov Modes Aen B MacKenzie Notes for December 1, 3, &8, 2015 Discrete-Time Markov Chains You may reca that when we first introduced random processes,
More informationSeparation of Variables and a Spherical Shell with Surface Charge
Separation of Variabes and a Spherica She with Surface Charge In cass we worked out the eectrostatic potentia due to a spherica she of radius R with a surface charge density σθ = σ cos θ. This cacuation
More information14 Separation of Variables Method
14 Separation of Variabes Method Consider, for exampe, the Dirichet probem u t = Du xx < x u(x, ) = f(x) < x < u(, t) = = u(, t) t > Let u(x, t) = T (t)φ(x); now substitute into the equation: dt
More informationMARKOV CHAINS AND MARKOV DECISION THEORY. Contents
MARKOV CHAINS AND MARKOV DECISION THEORY ARINDRIMA DATTA Abstract. In this paper, we begin with a forma introduction to probabiity and expain the concept of random variabes and stochastic processes. After
More informationMATH 172: MOTIVATION FOR FOURIER SERIES: SEPARATION OF VARIABLES
MATH 172: MOTIVATION FOR FOURIER SERIES: SEPARATION OF VARIABLES Separation of variabes is a method to sove certain PDEs which have a warped product structure. First, on R n, a inear PDE of order m is
More informationBayesian Learning. You hear a which which could equally be Thanks or Tanks, which would you go with?
Bayesian Learning A powerfu and growing approach in machine earning We use it in our own decision making a the time You hear a which which coud equay be Thanks or Tanks, which woud you go with? Combine
More informationWeek 6 Lectures, Math 6451, Tanveer
Fourier Series Week 6 Lectures, Math 645, Tanveer In the context of separation of variabe to find soutions of PDEs, we encountered or and in other cases f(x = f(x = a 0 + f(x = a 0 + b n sin nπx { a n
More information6 Wave Equation on an Interval: Separation of Variables
6 Wave Equation on an Interva: Separation of Variabes 6.1 Dirichet Boundary Conditions Ref: Strauss, Chapter 4 We now use the separation of variabes technique to study the wave equation on a finite interva.
More information6.434J/16.391J Statistics for Engineers and Scientists May 4 MIT, Spring 2006 Handout #17. Solution 7
6.434J/16.391J Statistics for Engineers and Scientists May 4 MIT, Spring 2006 Handout #17 Soution 7 Probem 1: Generating Random Variabes Each part of this probem requires impementation in MATLAB. For the
More informationAkaike Information Criterion for ANOVA Model with a Simple Order Restriction
Akaike Information Criterion for ANOVA Mode with a Simpe Order Restriction Yu Inatsu * Department of Mathematics, Graduate Schoo of Science, Hiroshima University ABSTRACT In this paper, we consider Akaike
More information4 1-D Boundary Value Problems Heat Equation
4 -D Boundary Vaue Probems Heat Equation The main purpose of this chapter is to study boundary vaue probems for the heat equation on a finite rod a x b. u t (x, t = ku xx (x, t, a < x < b, t > u(x, = ϕ(x
More informationStrauss PDEs 2e: Section Exercise 2 Page 1 of 12. For problem (1), complete the calculation of the series in case j(t) = 0 and h(t) = e t.
Strauss PDEs e: Section 5.6 - Exercise Page 1 of 1 Exercise For probem (1, compete the cacuation of the series in case j(t = and h(t = e t. Soution With j(t = and h(t = e t, probem (1 on page 147 becomes
More informationLecture Note 3: Stationary Iterative Methods
MATH 5330: Computationa Methods of Linear Agebra Lecture Note 3: Stationary Iterative Methods Xianyi Zeng Department of Mathematica Sciences, UTEP Stationary Iterative Methods The Gaussian eimination (or
More informationC. Fourier Sine Series Overview
12 PHILIP D. LOEWEN C. Fourier Sine Series Overview Let some constant > be given. The symboic form of the FSS Eigenvaue probem combines an ordinary differentia equation (ODE) on the interva (, ) with a
More informationDavid Eigen. MA112 Final Paper. May 10, 2002
David Eigen MA112 Fina Paper May 1, 22 The Schrodinger equation describes the position of an eectron as a wave. The wave function Ψ(t, x is interpreted as a probabiity density for the position of the eectron.
More information$, (2.1) n="# #. (2.2)
Chapter. Eectrostatic II Notes: Most of the materia presented in this chapter is taken from Jackson, Chap.,, and 4, and Di Bartoo, Chap... Mathematica Considerations.. The Fourier series and the Fourier
More informationA. Distribution of the test statistic
A. Distribution of the test statistic In the sequentia test, we first compute the test statistic from a mini-batch of size m. If a decision cannot be made with this statistic, we keep increasing the mini-batch
More informationLECTURE NOTES 9 TRACELESS SYMMETRIC TENSOR APPROACH TO LEGENDRE POLYNOMIALS AND SPHERICAL HARMONICS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Physics Department Physics 8.07: Eectromagnetism II October 7, 202 Prof. Aan Guth LECTURE NOTES 9 TRACELESS SYMMETRIC TENSOR APPROACH TO LEGENDRE POLYNOMIALS AND SPHERICAL
More informationASummaryofGaussianProcesses Coryn A.L. Bailer-Jones
ASummaryofGaussianProcesses Coryn A.L. Baier-Jones Cavendish Laboratory University of Cambridge caj@mrao.cam.ac.uk Introduction A genera prediction probem can be posed as foows. We consider that the variabe
More informationAALBORG UNIVERSITY. The distribution of communication cost for a mobile service scenario. Jesper Møller and Man Lung Yiu. R June 2009
AALBORG UNIVERSITY The distribution of communication cost for a mobie service scenario by Jesper Møer and Man Lung Yiu R-29-11 June 29 Department of Mathematica Sciences Aaborg University Fredrik Bajers
More informationAST 418/518 Instrumentation and Statistics
AST 418/518 Instrumentation and Statistics Cass Website: http://ircamera.as.arizona.edu/astr_518 Cass Texts: Practica Statistics for Astronomers, J.V. Wa, and C.R. Jenkins, Second Edition. Measuring the
More informationu(x) s.t. px w x 0 Denote the solution to this problem by ˆx(p, x). In order to obtain ˆx we may simply solve the standard problem max x 0
Bocconi University PhD in Economics - Microeconomics I Prof M Messner Probem Set 4 - Soution Probem : If an individua has an endowment instead of a monetary income his weath depends on price eves In particuar,
More information221B Lecture Notes Notes on Spherical Bessel Functions
Definitions B Lecture Notes Notes on Spherica Besse Functions We woud ike to sove the free Schrödinger equation [ h d r R(r) = h k R(r). () m r dr r m R(r) is the radia wave function ψ( x) = R(r)Y m (θ,
More informationExplicit overall risk minimization transductive bound
1 Expicit overa risk minimization transductive bound Sergio Decherchi, Paoo Gastado, Sandro Ridea, Rodofo Zunino Dept. of Biophysica and Eectronic Engineering (DIBE), Genoa University Via Opera Pia 11a,
More informationThe EM Algorithm applied to determining new limit points of Mahler measures
Contro and Cybernetics vo. 39 (2010) No. 4 The EM Agorithm appied to determining new imit points of Maher measures by Souad E Otmani, Georges Rhin and Jean-Marc Sac-Épée Université Pau Veraine-Metz, LMAM,
More informationMath 124B January 31, 2012
Math 124B January 31, 212 Viktor Grigoryan 7 Inhomogeneous boundary vaue probems Having studied the theory of Fourier series, with which we successfuy soved boundary vaue probems for the homogeneous heat
More informationLECTURE NOTES 8 THE TRACELESS SYMMETRIC TENSOR EXPANSION AND STANDARD SPHERICAL HARMONICS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Physics Department Physics 8.07: Eectromagnetism II October, 202 Prof. Aan Guth LECTURE NOTES 8 THE TRACELESS SYMMETRIC TENSOR EXPANSION AND STANDARD SPHERICAL HARMONICS
More informationSrednicki Chapter 51
Srednici Chapter 51 QFT Probems & Soutions A. George September 7, 13 Srednici 51.1. Derive the fermion-oop correction to the scaar proagator by woring through equation 5., and show that it has an extra
More informationLecture Notes 4: Fourier Series and PDE s
Lecture Notes 4: Fourier Series and PDE s 1. Periodic Functions A function fx defined on R is caed a periodic function if there exists a number T > such that fx + T = fx, x R. 1.1 The smaest number T for
More informationPricing Multiple Products with the Multinomial Logit and Nested Logit Models: Concavity and Implications
Pricing Mutipe Products with the Mutinomia Logit and Nested Logit Modes: Concavity and Impications Hongmin Li Woonghee Tim Huh WP Carey Schoo of Business Arizona State University Tempe Arizona 85287 USA
More informationHaar Decomposition and Reconstruction Algorithms
Jim Lambers MAT 773 Fa Semester 018-19 Lecture 15 and 16 Notes These notes correspond to Sections 4.3 and 4.4 in the text. Haar Decomposition and Reconstruction Agorithms Decomposition Suppose we approximate
More informationMat 1501 lecture notes, penultimate installment
Mat 1501 ecture notes, penutimate instament 1. bounded variation: functions of a singe variabe optiona) I beieve that we wi not actuay use the materia in this section the point is mainy to motivate the
More information( ) is just a function of x, with
II. MULTIVARIATE CALCULUS The first ecture covered functions where a singe input goes in, and a singe output comes out. Most economic appications aren t so simpe. In most cases, a number of variabes infuence
More informationNOISE-INDUCED STABILIZATION OF STOCHASTIC DIFFERENTIAL EQUATIONS
NOISE-INDUCED STABILIZATION OF STOCHASTIC DIFFERENTIAL EQUATIONS TONY ALLEN, EMILY GEBHARDT, AND ADAM KLUBALL 3 ADVISOR: DR. TIFFANY KOLBA 4 Abstract. The phenomenon of noise-induced stabiization occurs
More informationCourse 2BA1, Section 11: Periodic Functions and Fourier Series
Course BA, 8 9 Section : Periodic Functions and Fourier Series David R. Wikins Copyright c David R. Wikins 9 Contents Periodic Functions and Fourier Series 74. Fourier Series of Even and Odd Functions...........
More informationA CLUSTERING LAW FOR SOME DISCRETE ORDER STATISTICS
J App Prob 40, 226 241 (2003) Printed in Israe Appied Probabiity Trust 2003 A CLUSTERING LAW FOR SOME DISCRETE ORDER STATISTICS SUNDER SETHURAMAN, Iowa State University Abstract Let X 1,X 2,,X n be a sequence
More informationDo Schools Matter for High Math Achievement? Evidence from the American Mathematics Competitions Glenn Ellison and Ashley Swanson Online Appendix
VOL. NO. DO SCHOOLS MATTER FOR HIGH MATH ACHIEVEMENT? 43 Do Schoos Matter for High Math Achievement? Evidence from the American Mathematics Competitions Genn Eison and Ashey Swanson Onine Appendix Appendix
More information(1 ) = 1 for some 2 (0; 1); (1 + ) = 0 for some > 0:
Answers, na. Economics 4 Fa, 2009. Christiano.. The typica househod can engage in two types of activities producing current output and studying at home. Athough time spent on studying at home sacrices
More information17 Lecture 17: Recombination and Dark Matter Production
PYS 652: Astrophysics 88 17 Lecture 17: Recombination and Dark Matter Production New ideas pass through three periods: It can t be done. It probaby can be done, but it s not worth doing. I knew it was
More informationOn the evaluation of saving-consumption plans
On the evauation of saving-consumption pans Steven Vanduffe Jan Dhaene Marc Goovaerts Juy 13, 2004 Abstract Knowedge of the distribution function of the stochasticay compounded vaue of a series of future
More informationComponentwise Determination of the Interval Hull Solution for Linear Interval Parameter Systems
Componentwise Determination of the Interva Hu Soution for Linear Interva Parameter Systems L. V. Koev Dept. of Theoretica Eectrotechnics, Facuty of Automatics, Technica University of Sofia, 1000 Sofia,
More informationXSAT of linear CNF formulas
XSAT of inear CN formuas Bernd R. Schuh Dr. Bernd Schuh, D-50968 Kön, Germany; bernd.schuh@netcoogne.de eywords: compexity, XSAT, exact inear formua, -reguarity, -uniformity, NPcompeteness Abstract. Open
More informationSome Measures for Asymmetry of Distributions
Some Measures for Asymmetry of Distributions Georgi N. Boshnakov First version: 31 January 2006 Research Report No. 5, 2006, Probabiity and Statistics Group Schoo of Mathematics, The University of Manchester
More informationLecture Notes for Math 251: ODE and PDE. Lecture 32: 10.2 Fourier Series
Lecture Notes for Math 251: ODE and PDE. Lecture 32: 1.2 Fourier Series Shawn D. Ryan Spring 212 Last Time: We studied the heat equation and the method of Separation of Variabes. We then used Separation
More informationMath 124B January 17, 2012
Math 124B January 17, 212 Viktor Grigoryan 3 Fu Fourier series We saw in previous ectures how the Dirichet and Neumann boundary conditions ead to respectivey sine and cosine Fourier series of the initia
More informationChemical Kinetics Part 2
Integrated Rate Laws Chemica Kinetics Part 2 The rate aw we have discussed thus far is the differentia rate aw. Let us consider the very simpe reaction: a A à products The differentia rate reates the rate
More informationTHE THREE POINT STEINER PROBLEM ON THE FLAT TORUS: THE MINIMAL LUNE CASE
THE THREE POINT STEINER PROBLEM ON THE FLAT TORUS: THE MINIMAL LUNE CASE KATIE L. MAY AND MELISSA A. MITCHELL Abstract. We show how to identify the minima path network connecting three fixed points on
More informationarxiv: v1 [math.co] 17 Dec 2018
On the Extrema Maximum Agreement Subtree Probem arxiv:1812.06951v1 [math.o] 17 Dec 2018 Aexey Markin Department of omputer Science, Iowa State University, USA amarkin@iastate.edu Abstract Given two phyogenetic
More information1D Heat Propagation Problems
Chapter 1 1D Heat Propagation Probems If the ambient space of the heat conduction has ony one dimension, the Fourier equation reduces to the foowing for an homogeneous body cρ T t = T λ 2 + Q, 1.1) x2
More informationb n n=1 a n cos nx (3) n=1
Fourier Anaysis The Fourier series First some terminoogy: a function f(x) is periodic if f(x ) = f(x) for a x for some, if is the smaest such number, it is caed the period of f(x). It is even if f( x)
More informationMidterm 2 Review. Drew Rollins
Midterm 2 Review Drew Roins 1 Centra Potentias and Spherica Coordinates 1.1 separation of variabes Soving centra force probems in physics (physica systems described by two objects with a force between
More informationAPPENDIX C FLEXING OF LENGTH BARS
Fexing of ength bars 83 APPENDIX C FLEXING OF LENGTH BARS C.1 FLEXING OF A LENGTH BAR DUE TO ITS OWN WEIGHT Any object ying in a horizonta pane wi sag under its own weight uness it is infinitey stiff or
More informationLearning Fully Observed Undirected Graphical Models
Learning Fuy Observed Undirected Graphica Modes Sides Credit: Matt Gormey (2016) Kayhan Batmangheich 1 Machine Learning The data inspires the structures we want to predict Inference finds {best structure,
More informationStochastic Variational Inference with Gradient Linearization
Stochastic Variationa Inference with Gradient Linearization Suppementa Materia Tobias Pötz * Anne S Wannenwetsch Stefan Roth Department of Computer Science, TU Darmstadt Preface In this suppementa materia,
More informationNotes: Most of the material presented in this chapter is taken from Jackson, Chap. 2, 3, and 4, and Di Bartolo, Chap. 2. 2π nx i a. ( ) = G n.
Chapter. Eectrostatic II Notes: Most of the materia presented in this chapter is taken from Jackson, Chap.,, and 4, and Di Bartoo, Chap... Mathematica Considerations.. The Fourier series and the Fourier
More informationPhysics 235 Chapter 8. Chapter 8 Central-Force Motion
Physics 35 Chapter 8 Chapter 8 Centra-Force Motion In this Chapter we wi use the theory we have discussed in Chapter 6 and 7 and appy it to very important probems in physics, in which we study the motion
More informationA proposed nonparametric mixture density estimation using B-spline functions
A proposed nonparametric mixture density estimation using B-spine functions Atizez Hadrich a,b, Mourad Zribi a, Afif Masmoudi b a Laboratoire d Informatique Signa et Image de a Côte d Opae (LISIC-EA 4491),
More informationCONJUGATE GRADIENT WITH SUBSPACE OPTIMIZATION
CONJUGATE GRADIENT WITH SUBSPACE OPTIMIZATION SAHAR KARIMI AND STEPHEN VAVASIS Abstract. In this paper we present a variant of the conjugate gradient (CG) agorithm in which we invoke a subspace minimization
More information2M2. Fourier Series Prof Bill Lionheart
M. Fourier Series Prof Bi Lionheart 1. The Fourier series of the periodic function f(x) with period has the form f(x) = a 0 + ( a n cos πnx + b n sin πnx ). Here the rea numbers a n, b n are caed the Fourier
More informationLegendre Polynomials - Lecture 8
Legendre Poynomias - Lecture 8 Introduction In spherica coordinates the separation of variabes for the function of the poar ange resuts in Legendre s equation when the soution is independent of the azimutha
More information4 Separation of Variables
4 Separation of Variabes In this chapter we describe a cassica technique for constructing forma soutions to inear boundary vaue probems. The soution of three cassica (paraboic, hyperboic and eiptic) PDE
More informationChemical Kinetics Part 2. Chapter 16
Chemica Kinetics Part 2 Chapter 16 Integrated Rate Laws The rate aw we have discussed thus far is the differentia rate aw. Let us consider the very simpe reaction: a A à products The differentia rate reates
More informationSVM: Terminology 1(6) SVM: Terminology 2(6)
Andrew Kusiak Inteigent Systems Laboratory 39 Seamans Center he University of Iowa Iowa City, IA 54-57 SVM he maxima margin cassifier is simiar to the perceptron: It aso assumes that the data points are
More informationMore Scattering: the Partial Wave Expansion
More Scattering: the Partia Wave Expansion Michae Fower /7/8 Pane Waves and Partia Waves We are considering the soution to Schrödinger s equation for scattering of an incoming pane wave in the z-direction
More informationc 2007 Society for Industrial and Applied Mathematics
SIAM REVIEW Vo. 49,No. 1,pp. 111 1 c 7 Society for Industria and Appied Mathematics Domino Waves C. J. Efthimiou M. D. Johnson Abstract. Motivated by a proposa of Daykin [Probem 71-19*, SIAM Rev., 13 (1971),
More informationStat 155 Game theory, Yuval Peres Fall Lectures 4,5,6
Stat 155 Game theory, Yuva Peres Fa 2004 Lectures 4,5,6 In the ast ecture, we defined N and P positions for a combinatoria game. We wi now show more formay that each starting position in a combinatoria
More informationHYDROGEN ATOM SELECTION RULES TRANSITION RATES
DOING PHYSICS WITH MATLAB QUANTUM PHYSICS Ian Cooper Schoo of Physics, University of Sydney ian.cooper@sydney.edu.au HYDROGEN ATOM SELECTION RULES TRANSITION RATES DOWNLOAD DIRECTORY FOR MATLAB SCRIPTS
More informationAn Infeasibility Result for the Multiterminal Source-Coding Problem
An Infeasibiity Resut for the Mutitermina Source-Coding Probem Aaron B. Wagner, Venkat Anantharam, November 22, 2005 Abstract We prove a new outer bound on the rate-distortion region for the mutitermina
More informationFormulas for Angular-Momentum Barrier Factors Version II
BNL PREPRINT BNL-QGS-06-101 brfactor1.tex Formuas for Anguar-Momentum Barrier Factors Version II S. U. Chung Physics Department, Brookhaven Nationa Laboratory, Upton, NY 11973 March 19, 2015 abstract A
More information8 APPENDIX. E[m M] = (n S )(1 exp( exp(s min + c M))) (19) E[m M] n exp(s min + c M) (20) 8.1 EMPIRICAL EVALUATION OF SAMPLING
8 APPENDIX 8.1 EMPIRICAL EVALUATION OF SAMPLING We wish to evauate the empirica accuracy of our samping technique on concrete exampes. We do this in two ways. First, we can sort the eements by probabiity
More informationHigher dimensional PDEs and multidimensional eigenvalue problems
Higher dimensiona PEs and mutidimensiona eigenvaue probems 1 Probems with three independent variabes Consider the prototypica equations u t = u (iffusion) u tt = u (W ave) u zz = u (Lapace) where u = u
More informationOn the estimation of multiple random integrals and U-statistics
Péter Major On the estimation of mutipe random integras and U-statistics Lecture Note January 9, 2014 Springer Contents 1 Introduction................................................... 1 2 Motivation
More informationc 2016 Georgios Rovatsos
c 2016 Georgios Rovatsos QUICKEST CHANGE DETECTION WITH APPLICATIONS TO LINE OUTAGE DETECTION BY GEORGIOS ROVATSOS THESIS Submitted in partia fufiment of the requirements for the degree of Master of Science
More informationUniformly Reweighted Belief Propagation: A Factor Graph Approach
Uniformy Reweighted Beief Propagation: A Factor Graph Approach Henk Wymeersch Chamers University of Technoogy Gothenburg, Sweden henkw@chamers.se Federico Penna Poitecnico di Torino Torino, Itay federico.penna@poito.it
More informationA NOTE ON QUASI-STATIONARY DISTRIBUTIONS OF BIRTH-DEATH PROCESSES AND THE SIS LOGISTIC EPIDEMIC
(January 8, 2003) A NOTE ON QUASI-STATIONARY DISTRIBUTIONS OF BIRTH-DEATH PROCESSES AND THE SIS LOGISTIC EPIDEMIC DAMIAN CLANCY, University of Liverpoo PHILIP K. POLLETT, University of Queensand Abstract
More informationAssignment 7 Due Tuessday, March 29, 2016
Math 45 / AMCS 55 Dr. DeTurck Assignment 7 Due Tuessday, March 9, 6 Topics for this week Convergence of Fourier series; Lapace s equation and harmonic functions: basic properties, compuations on rectanges
More informationRate-Distortion Theory of Finite Point Processes
Rate-Distortion Theory of Finite Point Processes Günther Koiander, Dominic Schuhmacher, and Franz Hawatsch, Feow, IEEE Abstract We study the compression of data in the case where the usefu information
More informationUnsupervised Learning
2018 EE448, Big Data Mining, Lecture 7 Unsupervised Learning Weinan Zhang Shanghai Jiao Tong University http://wnzhang.net http://wnzhang.net/teaching/ee448/index.html ML Problem Setting First build and
More informationPattern Frequency Sequences and Internal Zeros
Advances in Appied Mathematics 28, 395 420 (2002 doi:10.1006/aama.2001.0789, avaiabe onine at http://www.ideaibrary.com on Pattern Frequency Sequences and Interna Zeros Mikós Bóna Department of Mathematics,
More informationAutomobile Prices in Market Equilibrium. Berry, Pakes and Levinsohn
Automobie Prices in Market Equiibrium Berry, Pakes and Levinsohn Empirica Anaysis of demand and suppy in a differentiated products market: equiibrium in the U.S. automobie market. Oigopoistic Differentiated
More informationLecture 9. Stability of Elastic Structures. Lecture 10. Advanced Topic in Column Buckling
Lecture 9 Stabiity of Eastic Structures Lecture 1 Advanced Topic in Coumn Bucking robem 9-1: A camped-free coumn is oaded at its tip by a oad. The issue here is to find the itica bucking oad. a) Suggest
More informationMath 220B - Summer 2003 Homework 1 Solutions
Math 0B - Summer 003 Homework Soutions Consider the eigenvaue probem { X = λx 0 < x < X satisfies symmetric BCs x = 0, Suppose f(x)f (x) x=b x=a 0 for a rea-vaued functions f(x) which satisfy the boundary
More informationFOURIER SERIES ON ANY INTERVAL
FOURIER SERIES ON ANY INTERVAL Overview We have spent considerabe time earning how to compute Fourier series for functions that have a period of 2p on the interva (-p,p). We have aso seen how Fourier series
More informationGENERAL ASYMPTOTIC BAYESIAN THEORY OF QUICKEST CHANGE DETECTION. Alexander G. Tartakovsky and Venugopal V. Veeravalli
Fina Version Submitted to the Theory of Probabiity and Its Appications, May 1, 2004. GENERAL ASYMPTOTIC BAYESIAN THEORY OF QUICKEST CHANGE DETECTION Aexander G. Tartakovsky and Venugopa V. Veeravai ABSTRACT.
More informationSTA 216 Project: Spline Approach to Discrete Survival Analysis
: Spine Approach to Discrete Surviva Anaysis November 4, 005 1 Introduction Athough continuous surviva anaysis differs much from the discrete surviva anaysis, there is certain ink between the two modeing
More informationMONTE CARLO SIMULATIONS
MONTE CARLO SIMULATIONS Current physics research 1) Theoretica 2) Experimenta 3) Computationa Monte Caro (MC) Method (1953) used to study 1) Discrete spin systems 2) Fuids 3) Poymers, membranes, soft matter
More informationProblem set 6 The Perron Frobenius theorem.
Probem set 6 The Perron Frobenius theorem. Math 22a4 Oct 2 204, Due Oct.28 In a future probem set I want to discuss some criteria which aow us to concude that that the ground state of a sef-adjoint operator
More informationV.B The Cluster Expansion
V.B The Custer Expansion For short range interactions, speciay with a hard core, it is much better to repace the expansion parameter V( q ) by f( q ) = exp ( βv( q )), which is obtained by summing over
More informationV.B The Cluster Expansion
V.B The Custer Expansion For short range interactions, speciay with a hard core, it is much better to repace the expansion parameter V( q ) by f(q ) = exp ( βv( q )) 1, which is obtained by summing over
More informationReichenbachian Common Cause Systems
Reichenbachian Common Cause Systems G. Hofer-Szabó Department of Phiosophy Technica University of Budapest e-mai: gszabo@hps.ete.hu Mikós Rédei Department of History and Phiosophy of Science Eötvös University,
More informationPrimal and dual active-set methods for convex quadratic programming
Math. Program., Ser. A 216) 159:469 58 DOI 1.17/s117-15-966-2 FULL LENGTH PAPER Prima and dua active-set methods for convex quadratic programming Anders Forsgren 1 Phiip E. Gi 2 Eizabeth Wong 2 Received:
More informationSEMINAR 2. PENDULUMS. V = mgl cos θ. (2) L = T V = 1 2 ml2 θ2 + mgl cos θ, (3) d dt ml2 θ2 + mgl sin θ = 0, (4) θ + g l
Probem 7. Simpe Penduum SEMINAR. PENDULUMS A simpe penduum means a mass m suspended by a string weightess rigid rod of ength so that it can swing in a pane. The y-axis is directed down, x-axis is directed
More informationTheory and implementation behind: Universal surface creation - smallest unitcell
Teory and impementation beind: Universa surface creation - smaest unitce Bjare Brin Buus, Jaob Howat & Tomas Bigaard September 15, 218 1 Construction of surface sabs Te aim for tis part of te project is
More informationUniprocessor Feasibility of Sporadic Tasks with Constrained Deadlines is Strongly conp-complete
Uniprocessor Feasibiity of Sporadic Tasks with Constrained Deadines is Strongy conp-compete Pontus Ekberg and Wang Yi Uppsaa University, Sweden Emai: {pontus.ekberg yi}@it.uu.se Abstract Deciding the feasibiity
More informationHomework 5 Solutions
Stat 310B/Math 230B Theory of Probabiity Homework 5 Soutions Andrea Montanari Due on 2/19/2014 Exercise [5.3.20] 1. We caim that n 2 [ E[h F n ] = 2 n i=1 A i,n h(u)du ] I Ai,n (t). (1) Indeed, integrabiity
More informationAn explicit resolution of the equity-efficiency tradeoff in the random allocation of an indivisible good
An expicit resoution of the equity-efficiency tradeoff in the random aocation of an indivisibe good Stergios Athanassogou, Gauthier de Maere d Aertrycke January 2015 Abstract Suppose we wish to randomy
More informationarxiv: v1 [math.fa] 23 Aug 2018
An Exact Upper Bound on the L p Lebesgue Constant and The -Rényi Entropy Power Inequaity for Integer Vaued Random Variabes arxiv:808.0773v [math.fa] 3 Aug 08 Peng Xu, Mokshay Madiman, James Mebourne Abstract
More informationDiscrete Techniques. Chapter Introduction
Chapter 3 Discrete Techniques 3. Introduction In the previous two chapters we introduced Fourier transforms of continuous functions of the periodic and non-periodic (finite energy) type, we as various
More informationApproximated MLC shape matrix decomposition with interleaf collision constraint
Approximated MLC shape matrix decomposition with intereaf coision constraint Thomas Kainowski Antje Kiese Abstract Shape matrix decomposition is a subprobem in radiation therapy panning. A given fuence
More information