II.D Many Random Variables
|
|
- Cecil Houston
- 6 years ago
- Views:
Transcription
1 II.D Many Random Varables Wth more than one random varable, the set of outcomes s an -dmensonal space, S x = { < x 1, x 2,, x < }. For example, descrbng the locaton and velocty of a gas partcle requres sx coordnates. The jont PDF p(x), s the probablty densty of an outcome n a volume element d x = dx around the pont x = {x 1, x 2,, x }. The jont PDF s normalzed such that p x (S) = d x p(x) = 1. (II.32) If, and only f, the random varables are ndependent, the jont PDF s the product of ndvdual PDFs, p(x) = p (x ). (II.33) The uncondtonal PDF descrbes the behavor of a subset of random varables, ndependent of the values of the others. For example, f we are nterested only n the locaton of a gas partcle, an uncondtonal PDF can be constructed by ntegratng over all veloctes at a gven locaton, p( x) = d 3 v p( x, v); more generally p(x 1,, x m ) = =m+1 dx p(x 1,, x ). (II.34) The condtonal PDF descrbes the behavor of a subset of random varables, for specfed values of the others. For example, the PDF for the velocty of a partcle at a partcular locaton x, denoted by p( v x), s proportonal to the jont PDF p( v x) = p( x, v)/. The constant of proportonalty, obtaned by normalzng p( v x), s = d 3 v p( x, v) = p( x), (II.35) the uncondtonal PDF for a partcle at x. In general, the uncondtonal PDFs are obtaned from Bayes Theorem as p(x 1,, x m x m+1,, x ) = p(x 1,, x ) p(x m+1,, x ). (II.36) ote that f the random varables are ndependent, the uncondtonal PDF s equal to the condtonal PDF. 32
2 The expectaton value of a functon F(x), s obtaned as before from F(x) = d x p(x)f(x). (II.37) The jont characterstc functon, s obtaned from the -dmensonal Fourer transformaton of the jont PDF, p(k) = exp k j x j j=1. (II.38) The jont moments and jont cumulants are generated by p(k) and ln p(k) respectvely, as [ ] n1 [ ] n2 ] n x n 1 1 xn 2 2 xn = [ p(k = 0), ( k 1 ) ( k 2 ) ( k ) [ ] n1 [ ] n2 ] n x n 1 1 xn 2 2 xn c = [ ln p(k = 0). ( k 1 ) ( k 2 ) ( k ) (II.39) The prevously descrbed graphcal relaton between jont moments (all clusters of labelled ponts), and jont cumulant (connected clusters) s stll applcable. For example x α x β = x α c x β c + x α x β c, and x 2 α x β = xα 2 c x β c + x 2 α c x β c + 2 x α x β c x α c + x 2 αx β c. (II.40) The connected correlaton, x α x β c, s zero f x α and x β are ndependent random varables. The jont Gaussan dstrbuton s the generalzaton of eq.(ii.15) to random varables, as p(x) = [ 1 (2π) det[c] exp 1 2 ( C 1 ) ] (x mn m λ m )(x n λ n ) mn, (II.41) where C s a symmetrc matrx, and C 1 s ts nverse,. The smplest way to get the normalzaton factor s to make a lnear transformaton from the varables y j = x j λ j, usng the untary matrx that dagonalzes C. Ths reduces the normalzaton to that of the product of Gaussans whose varances are determned by the egenvalues of C. The product of the egenvalues s the determnant det[c]. (Ths also ndcates that the matrx C must be postve defnte.) The correspondng jont characterstc functon s obtaned by smlar manpulatons, and s gven by p(k) = exp [ k m λ m 12 ] C mnk m k n, (II.42) 33
3 where the summaton conventon s used. The jont cumulants of the Gaussan are then obtaned from ln p(k) as x m c = λ m, x m x n c = C mn, (II.43) wth all hgher cumulants equal to zero. In the specal case of {λ m } = 0, all odd moments of the dstrbuton are zero, whle the general rules for relatng moments to cumulants ndcate that any even moment s obtaned by summng over all ways of groupng the nvolved random varables nto pars, e.g. x a x b x c x d = C ab C cd + C ac C bd + C ad C bc. (II.44) Ths result s sometmes referred to as Wck s theorem. II.E Sums of Random Varables & the Central Lmt Theorem Consder the sum X = x, where x are random varables wth a jont PDF of p(x). The PDF for X s p X (x) = d x p(x)δ (x ) x = 1 dx p (x 1,, x 1, x x 1 x 1 ), and the correspondng characterstc functon (usng eq.(ii.38)) s gven by p X (k) = exp k j=1 x j Cumulants of the sum are obtaned by expandng ln p X (k), (II.45) = p (k 1 = k 2 = = k = k). (II.46) ln p (k 1 = k 2 = = k = k) = k 1 =1 x 1 c + ( k)2 2 1, 2 x 1 x 2 c +, (II.47) as X c = x c, X 2 = x c x j c,. (II.48) If the random varables are ndependent, p(x) = p (x ), and p X (k) = p (k). The cross cumulants n eq.(ii.48) vansh, and the n th cumulant of X s smply the sum of the ndvdual cumulants, X n c = xn c. When all the random varables 34,j
4 are ndependently taken from the same dstrbuton p(x), ths mples X n c = x n c, generalzng the result obtaned prevously for the bnomal dstrbuton. For large values of, the average value of the sum s proportonal to, whle fluctuatons around the mean, as measured by the standard devaton, grow only as. The random varable y = (X x c )/, has zero mean, and cumulants that scale as y n c 1 m/2. As, only the second cumulant survves, and the PDF for y converges to the normal dstrbuton, lm p ( ) y = x x c = ) 1 exp ( y2. (II.49) 2π x2 c 2 x 2 c (ote that the Gaussan dstrbuton s the only dstrbuton wth only frst and second cumulants.) The convergence of the PDF for the sum of many random varables to a normal dstrbuton s a most mportant result n the context of statstcal mechancs where such sums are frequently encountered. The central lmt theorem states a more general form of ths result: It s not necessary for the random varables to be ndependent, as the condton 1,, m x 1 x m c O( m/2 ), s suffcent for the valdty of eq.(ii.49). II.F Rules for Large umbers To descrbe equlbrum propertes of macroscopc bodes, statstcal mechancs has to deal wth the very large number, of mcroscopc degrees of freedom. Actually, takng the thermodynamc lmt of leads to a number of smplfcatons, some of whch are descrbed n ths secton. There are typcally three types of dependence encountered n the thermodynamc lmt: (a) Intensve quanttes, such as temperature T, and generalzed forces, e.g. pressure P, and magnetc feld B, are ndependent of,.e. O( 0 ). (b) Extensve quanttes, such as energy E, entropy S, and generalzed dsplacements, e.g. volume V, and magnetzaton M, are proportonal to,.e. O( 1 ). (c) Exponental dependence,.e. O ( exp(φ) ), s encountered n enumeratng dscrete mcro-states, or computng avalable volumes n phase space. Other asymptotc dependences are certanly not ruled out a pror. For example, the Coulomb energy of ons at fxed densty scales as Q 2 /R 5/3. Such dependences are rarely encountered n every day physcs. The Coulomb nteracton of ons s quckly 35
5 screened by counter-ons, resultng n an extensve overall energy. (Ths s not the case n astrophyscal problems snce the gravtatonal energy can not be screened. For example the entropy of a black hole s proportonal to the square of ts mass.) In statstcal mechancs we frequently encounter sums or ntegrals of exponental varables. Performng such sums n the thermodynamc lmt s consderably smplfed due to the followng results. (1) Summaton of Exponental Quanttes Consder the sum S = E, (II.50) where each term s postve, wth an exponental dependence on, 0 E O ( exp(φ ) ), (II.51) and the number of terms, s proportonal to some power of. Such a sum can be approxmated by ts largest term E max, n the followng sense. Snce for each term n the sum, 0 E E max, E max S E max. (II.52) An ntensve quantty can be constructed from ln S/, whch s bounded by ln E max ln S ln E max + ln. (II.53) For p, the rato ln / vanshes n the large lmt, and (2) Saddle Pont Integraton ln S lm = ln E max = φ max. (II.54) Smlarly, an ntegral of the form I = dx exp ( φ(x) ), (II.55) can be approxmated by the maxmum value of the ntegrand, obtaned at a pont x max whch maxmzes the exponent φ(x). Expandng around ths pont, I = { [ dx exp φ(x max ) 1 ]} 2 φ (x max ) (x x max ) 2 +. (II.56) 36
6 ote that at the maxmum, the frst dervatve φ (x max ), s zero, whle the second dervatve φ (x max ), s negatve. Termnatng the seres at the quadratc order results n I e φ(x max) dx exp [ 2 ] φ (x max ) (x x max ) 2 2π φ (x max ) eφ(x max), (II.57) where the range of ntegraton has been extended to [, ]. The latter s justfed snce the ntegrand s neglgbly small outsde the neghborhood of x max. There are two types of correcton to the above result. Frstly, there are hgher order terms n the expanson of φ(x) around x max. These correctons can be looked at perturbatvely, and lead to a seres n powers of 1/. Secondly, there may be addtonal local maxma for the functon. A maxmum at x max, leads to a smlar Gaussan ntegral that can be added to eq.(ii.57). Clearly such contrbutons are smaller by O ( exp{ [φ(x max ) φ(x max )]}). Snce all these correctons vansh n the thermodynamc lmt, [ ln I lm = lm φ(x max ) 1 2 ln( φ ] (x max ) ) 1 + O( 2π 2) = φ(x max ). (II.58) The saddle pont method for evaluatng ntegrals s the extenson of the above result to more general ntegrands, and ntegraton paths n the complex plane. (The approprate extremum n the complex plane s a saddle pont.) The smplfed verson presented above s suffcent for the purposes of ths course. Strlng s approxmaton for! at large can be obtaned by saddle pont ntegraton. In order to get an ntegral representaton of!, start wth the result 0 dxe αx = 1 α. (II.59) Repeated dfferentaton of both sdes of the above equaton wth respect to α leads to 0 dxx e αx =! α +1. (II.60) Although the above result only apples to nteger, t s possble to defne by analytcal contnuaton a functon, Γ( + 1)! = 0 dxx e x, (II.61) 37
7 for all. Whle the ntegral n eq.(ii.61) s not exactly n the form of eq.(ii.55), t can stll be evaluated by a smlar method. The ntegrand can be wrtten as exp ( φ(x) ), wth φ(x) = lnx x/. The exponent has a maxmum at x max =, wth φ(x max ) = ln 1, and φ (x max ) = 1/ 2. Expandng the ntegrand n eq.(ii.61) around ths pont yelds,! dx exp ( ln 1 2 (x )2) e 2π, (II.62) where the ntegral s evaluated by extendng ts lmts to [, ]. Strlng s formula s obtaned by takng the logarthm of eq.(ii.62) as, ln! = ln ln(2π) + O( 1 ). (II.63) II.G Informaton, Entropy, and Estmaton Informaton: Consder a random varable wth a dscrete set of outcomes S = {x }, occurrng wth probabltes {p()}, for = 1,, M. In the context of nformaton theory, there s a precse meanng to the nformaton content of a probablty dstrbuton: Let us construct a message from ndependent outcomes of the random varable. Snce there are M possbltes for each character n ths message, t has an apparent nformaton content of ln 2 M bts;.e. ths many bnary bts of nformaton have to be transmtted to convey the message precsely. On the other hand, the probabltes {p()} lmt the types of messages that are lkely. For example, f p 2 p 1, t s very unlkely to construct a message wth more x 1 than x 2. In partcular, n the lmt of large, we expect the message to contan roughly { = p } occurrences of each symbol. The number of typcal messages thus corresponds to the number of ways of rearrangng the { } occurrences of {x }, and s gven by the multnomal coeffcent g =! M!. (II.64) Ths s much smaller than the total number of messages M n. To specfy one out of g possble sequences requres ln 2 g M p ln 2 p (for ), (II.65) More precsely, the probablty of fndng any that s dfferent from p by more than ± becomes exponentally small n, as. 38
8 bts of nformaton. The last result s obtaned by applyng Strlng s approxmaton for ln!. It can also be obtaned by notng that ( ) 1 = p = M p M { }!! g p p, (II.66) where the sum has been replaced by ts largest term, as justfed n the prevous secton. Shannon s Theorem proves more rgorously that the mnmum number of bts necessary to ensure that the percentage of errors n trals vanshes n the lmt, s ln 2 g. For any non-unform dstrbuton, ths s less than the ln 2 M bts needed n the absence of any nformaton on relatve probabltes. The dfference per tral s thus attrbuted to the nformaton content of the probablty dstrbuton, and s gven by I [{p }] = ln 2 M + M p ln 2 p. (II.67) Entropy: Eq.(II.64) s encountered frequently n statstcal mechancs n the context of mxng M dstnct components; ts natural logarthm s related to the entropy of mxng. More generally, we can defne an entropy for any probablty dstrbuton as S = M p() lnp() = lnp(). (II.68) The above entropy takes a mnmum value of zero for the delta functon dstrbuton p() = δ,j, and a maxmum value of lnm for the unform dstrbuton, p() = 1/M. S s thus a measure of dspersty (dsorder) of the dstrbuton, and does not depend on the values of the random varables {x }. A one to one mappng to f = F(x ) leaves the entropy unchanged, whle a many to one mappng makes the dstrbuton more ordered and decrease S. For example, f the two values, x 1 and x 2, are mapped onto the same f, the change n entropy s [ ] p 1 p 2 S(x 1, x 2 f) = p 1 ln + p 2 ln < 0. (II.69) p 1 + p 2 p 1 + p 2 Estmaton: The entropy S, can also be used to quantfy subjectve estmates of probabltes. In the absence of any nformaton, the best unbased estmate s that all M outcomes are equally lkely. Ths s the dstrbuton of maxmum entropy. If addtonal nformaton s avalable, the unbased estmate s obtaned by maxmzng the entropy subject to the 39
9 constrants mposed by ths nformaton. For example, f t s known that F(x) = f, we can maxmze S (α, β, {p }) = ( ) ( ) p() lnp() α p() 1 β p()f(x ) f, (II.70) where the Lagrange multplers α and β are ntroduced to mpose the constrants of normalzaton, and F(x) = f, respectvely. The result of the optmzaton s a dstrbuton p exp ( βf(x ) ), where the value of β s fxed by the constrant. Ths process can be generalzed to an arbtrary number of condtons. It s easy to see that f the frst n moments (and hence n cumulants) of a dstrbuton are specfed, the unbased estmate s the exponental of an n th order polynomal. In analogy wth eq.(ii.68), we can defne an entropy for a contnuous random varable (S x = { < x < }) as S = dx p(x) lnp(x) = lnp(x). (II.71) There are, however, problems wth ths defnton, as for example S s not nvarant under a one to one mappng. (After a change of varable to f = F(x), the entropy s changed by F (x).) Snce the Jacoban of a canoncal transformaton s unty, canoncally conjugate pars offer a sutable choce of coordnates n classcal statstcal mechancs. The ambgutes are also removed f the contnuous varable s dscretzed. Ths happens qute naturally n quantum statstcal mechancs where t s usually possble to work wth a dscrete ladder of states. The approprate volume for dscretzaton of phase space s set by Planck s constant h. 40
Probability Theory. The nth coefficient of the Taylor series of f(k), expanded around k = 0, gives the nth moment of x as ( ik) n n!
8333: Statstcal Mechancs I Problem Set # 3 Solutons Fall 3 Characterstc Functons: Probablty Theory The characterstc functon s defned by fk ep k = ep kpd The nth coeffcent of the Taylor seres of fk epanded
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationSTATISTICAL MECHANICAL ENSEMBLES 1 MICROSCOPIC AND MACROSCOPIC VARIABLES PHASE SPACE ENSEMBLES. CHE 524 A. Panagiotopoulos 1
CHE 54 A. Panagotopoulos STATSTCAL MECHACAL ESEMBLES MCROSCOPC AD MACROSCOPC ARABLES The central queston n Statstcal Mechancs can be phrased as follows: f partcles (atoms, molecules, electrons, nucle,
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationConjugacy and the Exponential Family
CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationRadar Trackers. Study Guide. All chapters, problems, examples and page numbers refer to Applied Optimal Estimation, A. Gelb, Ed.
Radar rackers Study Gude All chapters, problems, examples and page numbers refer to Appled Optmal Estmaton, A. Gelb, Ed. Chapter Example.0- Problem Statement wo sensors Each has a sngle nose measurement
More informationChapter 7 Channel Capacity and Coding
Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform
More informationSTATISTICAL MECHANICS
STATISTICAL MECHANICS Thermal Energy Recall that KE can always be separated nto 2 terms: KE system = 1 2 M 2 total v CM KE nternal Rgd-body rotaton and elastc / sound waves Use smplfyng assumptons KE of
More informationUniversity of Washington Department of Chemistry Chemistry 453 Winter Quarter 2015
Lecture 2. 1/07/15-1/09/15 Unversty of Washngton Department of Chemstry Chemstry 453 Wnter Quarter 2015 We are not talkng about truth. We are talkng about somethng that seems lke truth. The truth we want
More information10.40 Appendix Connection to Thermodynamics and Derivation of Boltzmann Distribution
10.40 Appendx Connecton to Thermodynamcs Dervaton of Boltzmann Dstrbuton Bernhardt L. Trout Outlne Cannoncal ensemble Maxmumtermmethod Most probable dstrbuton Ensembles contnued: Canoncal, Mcrocanoncal,
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationLecture 4. Macrostates and Microstates (Ch. 2 )
Lecture 4. Macrostates and Mcrostates (Ch. ) The past three lectures: we have learned about thermal energy, how t s stored at the mcroscopc level, and how t can be transferred from one system to another.
More informationInductance Calculation for Conductors of Arbitrary Shape
CRYO/02/028 Aprl 5, 2002 Inductance Calculaton for Conductors of Arbtrary Shape L. Bottura Dstrbuton: Internal Summary In ths note we descrbe a method for the numercal calculaton of nductances among conductors
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationMarginal Effects in Probit Models: Interpretation and Testing. 1. Interpreting Probit Coefficients
ECON 5 -- NOE 15 Margnal Effects n Probt Models: Interpretaton and estng hs note ntroduces you to the two types of margnal effects n probt models: margnal ndex effects, and margnal probablty effects. It
More informationProf. Dr. I. Nasser Phys 630, T Aug-15 One_dimensional_Ising_Model
EXACT OE-DIMESIOAL ISIG MODEL The one-dmensonal Isng model conssts of a chan of spns, each spn nteractng only wth ts two nearest neghbors. The smple Isng problem n one dmenson can be solved drectly n several
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationPhysics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1
P. Guterrez Physcs 5153 Classcal Mechancs D Alembert s Prncple and The Lagrangan 1 Introducton The prncple of vrtual work provdes a method of solvng problems of statc equlbrum wthout havng to consder the
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationA linear imaging system with white additive Gaussian noise on the observed data is modeled as follows:
Supplementary Note Mathematcal bacground A lnear magng system wth whte addtve Gaussan nose on the observed data s modeled as follows: X = R ϕ V + G, () where X R are the expermental, two-dmensonal proecton
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationSELECTED PROOFS. DeMorgan s formulas: The first one is clear from Venn diagram, or the following truth table:
SELECTED PROOFS DeMorgan s formulas: The frst one s clear from Venn dagram, or the followng truth table: A B A B A B Ā B Ā B T T T F F F F T F T F F T F F T T F T F F F F F T T T T The second one can be
More informationChapter 7 Channel Capacity and Coding
Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationPhysics 181. Particle Systems
Physcs 181 Partcle Systems Overvew In these notes we dscuss the varables approprate to the descrpton of systems of partcles, ther defntons, ther relatons, and ther conservatons laws. We consder a system
More informationGlobal Sensitivity. Tuesday 20 th February, 2018
Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationRobert Eisberg Second edition CH 09 Multielectron atoms ground states and x-ray excitations
Quantum Physcs 量 理 Robert Esberg Second edton CH 09 Multelectron atoms ground states and x-ray exctatons 9-01 By gong through the procedure ndcated n the text, develop the tme-ndependent Schroednger equaton
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationSolution Thermodynamics
Soluton hermodynamcs usng Wagner Notaton by Stanley. Howard Department of aterals and etallurgcal Engneerng South Dakota School of nes and echnology Rapd Cty, SD 57701 January 7, 001 Soluton hermodynamcs
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More information3.1 ML and Empirical Distribution
67577 Intro. to Machne Learnng Fall semester, 2008/9 Lecture 3: Maxmum Lkelhood/ Maxmum Entropy Dualty Lecturer: Amnon Shashua Scrbe: Amnon Shashua 1 In the prevous lecture we defned the prncple of Maxmum
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationThermodynamics General
Thermodynamcs General Lecture 1 Lecture 1 s devoted to establshng buldng blocks for dscussng thermodynamcs. In addton, the equaton of state wll be establshed. I. Buldng blocks for thermodynamcs A. Dmensons,
More informationb ), which stands for uniform distribution on the interval a x< b. = 0 elsewhere
Fall Analyss of Epermental Measurements B. Esensten/rev. S. Errede Some mportant probablty dstrbutons: Unform Bnomal Posson Gaussan/ormal The Unform dstrbuton s often called U( a, b ), hch stands for unform
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationESCI 341 Atmospheric Thermodynamics Lesson 10 The Physical Meaning of Entropy
ESCI 341 Atmospherc Thermodynamcs Lesson 10 The Physcal Meanng of Entropy References: An Introducton to Statstcal Thermodynamcs, T.L. Hll An Introducton to Thermodynamcs and Thermostatstcs, H.B. Callen
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationπ e ax2 dx = x 2 e ax2 dx or x 3 e ax2 dx = 1 x 4 e ax2 dx = 3 π 8a 5/2 (a) We are considering the Maxwell velocity distribution function: 2πτ/m
Homework Solutons Problem In solvng ths problem, we wll need to calculate some moments of the Gaussan dstrbuton. The brute-force method s to ntegrate by parts but there s a nce trck. The followng ntegrals
More information1 (1 + ( )) = 1 8 ( ) = (c) Carrying out the Taylor expansion, in this case, the series truncates at second order:
68A Solutons to Exercses March 05 (a) Usng a Taylor expanson, and notng that n 0 for all n >, ( + ) ( + ( ) + ) We can t nvert / because there s no Taylor expanson around 0 Lets try to calculate the nverse
More informationDensity matrix. c α (t)φ α (q)
Densty matrx Note: ths s supplementary materal. I strongly recommend that you read t for your own nterest. I beleve t wll help wth understandng the quantum ensembles, but t s not necessary to know t n
More informationChapter 1. Probability
Chapter. Probablty Mcroscopc propertes of matter: quantum mechancs, atomc and molecular propertes Macroscopc propertes of matter: thermodynamcs, E, H, C V, C p, S, A, G How do we relate these two propertes?
More informationPoisson brackets and canonical transformations
rof O B Wrght Mechancs Notes osson brackets and canoncal transformatons osson Brackets Consder an arbtrary functon f f ( qp t) df f f f q p q p t But q p p where ( qp ) pq q df f f f p q q p t In order
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationLagrangian Field Theory
Lagrangan Feld Theory Adam Lott PHY 391 Aprl 6, 017 1 Introducton Ths paper s a summary of Chapter of Mandl and Shaw s Quantum Feld Theory [1]. The frst thng to do s to fx the notaton. For the most part,
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationPhysics 5153 Classical Mechanics. Principle of Virtual Work-1
P. Guterrez 1 Introducton Physcs 5153 Classcal Mechancs Prncple of Vrtual Work The frst varatonal prncple we encounter n mechancs s the prncple of vrtual work. It establshes the equlbrum condton of a mechancal
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More information5.62 Physical Chemistry II Spring 2008
MIT OpenCourseWare http://ocw.mt.edu 5.62 Physcal Chemstry II Sprng 2008 For nformaton about ctng these materals or our Terms of Use, vst: http://ocw.mt.edu/terms. 5.62 Lecture #8: Boltzmann, Ferm-Drac,
More informationThermodynamics and statistical mechanics in materials modelling II
Course MP3 Lecture 8/11/006 (JAE) Course MP3 Lecture 8/11/006 Thermodynamcs and statstcal mechancs n materals modellng II A bref résumé of the physcal concepts used n materals modellng Dr James Ellott.1
More informationSupplementary Notes for Chapter 9 Mixture Thermodynamics
Supplementary Notes for Chapter 9 Mxture Thermodynamcs Key ponts Nne major topcs of Chapter 9 are revewed below: 1. Notaton and operatonal equatons for mxtures 2. PVTN EOSs for mxtures 3. General effects
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors
Stat60: Bayesan Modelng and Inference Lecture Date: February, 00 Reference Prors Lecturer: Mchael I. Jordan Scrbe: Steven Troxler and Wayne Lee In ths lecture, we assume that θ R; n hgher-dmensons, reference
More informationQuantum and Classical Information Theory with Disentropy
Quantum and Classcal Informaton Theory wth Dsentropy R V Ramos rubensramos@ufcbr Lab of Quantum Informaton Technology, Department of Telenformatc Engneerng Federal Unversty of Ceara - DETI/UFC, CP 6007
More informationBezier curves. Michael S. Floater. August 25, These notes provide an introduction to Bezier curves. i=0
Bezer curves Mchael S. Floater August 25, 211 These notes provde an ntroducton to Bezer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of the
More informationThe Expectation-Maximization Algorithm
The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.
More informationTransfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system
Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng
More informationLecture 3: Shannon s Theorem
CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationLecture 7: Boltzmann distribution & Thermodynamics of mixing
Prof. Tbbtt Lecture 7 etworks & Gels Lecture 7: Boltzmann dstrbuton & Thermodynamcs of mxng 1 Suggested readng Prof. Mark W. Tbbtt ETH Zürch 13 März 018 Molecular Drvng Forces Dll and Bromberg: Chapters
More informationTHEOREMS OF QUANTUM MECHANICS
THEOREMS OF QUANTUM MECHANICS In order to develop methods to treat many-electron systems (atoms & molecules), many of the theorems of quantum mechancs are useful. Useful Notaton The matrx element A mn
More informationBézier curves. Michael S. Floater. September 10, These notes provide an introduction to Bézier curves. i=0
Bézer curves Mchael S. Floater September 1, 215 These notes provde an ntroducton to Bézer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of
More information1 Binary Response Models
Bnary and Ordered Multnomal Response Models Dscrete qualtatve response models deal wth dscrete dependent varables. bnary: yes/no, partcpaton/non-partcpaton lnear probablty model LPM, probt or logt models
More informationV.C The Niemeijer van Leeuwen Cumulant Approximation
V.C The Nemejer van Leeuwen Cumulant Approxmaton Unfortunately, the decmaton procedure cannot be performed exactly n hgher dmensons. For example, the square lattce can be dvded nto two sublattces. For
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 12 10/21/2013. Martingale Concentration Inequalities and Applications
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 1 10/1/013 Martngale Concentraton Inequaltes and Applcatons Content. 1. Exponental concentraton for martngales wth bounded ncrements.
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationStatistical Inference. 2.3 Summary Statistics Measures of Center and Spread. parameters ( population characteristics )
Ismor Fscher, 8//008 Stat 54 / -8.3 Summary Statstcs Measures of Center and Spread Dstrbuton of dscrete contnuous POPULATION Random Varable, numercal True center =??? True spread =???? parameters ( populaton
More informationWorkshop: Approximating energies and wave functions Quantum aspects of physical chemistry
Workshop: Approxmatng energes and wave functons Quantum aspects of physcal chemstry http://quantum.bu.edu/pltl/6/6.pdf Last updated Thursday, November 7, 25 7:9:5-5: Copyrght 25 Dan Dll (dan@bu.edu) Department
More informationAPPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14
APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationConvergence of random processes
DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large
More informationCalculus of Variations Basics
Chapter 1 Calculus of Varatons Bascs 1.1 Varaton of a General Functonal In ths chapter, we derve the general formula for the varaton of a functonal of the form J [y 1,y 2,,y n ] F x,y 1,y 2,,y n,y 1,y
More information8.323 Relativistic Quantum Field Theory I
MI OpenCourseWare http://ocw.mt.edu 8.323 Relatvstc Quantum Feld heory I Sprng 2008 For nformaton about ctng these materals or our erms of Use, vst: http://ocw.mt.edu/terms. MASSACHUSES INSIUE OF ECHNOLOGY
More informationComputing MLE Bias Empirically
Computng MLE Bas Emprcally Kar Wa Lm Australan atonal Unversty January 3, 27 Abstract Ths note studes the bas arses from the MLE estmate of the rate parameter and the mean parameter of an exponental dstrbuton.
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More information5.04, Principles of Inorganic Chemistry II MIT Department of Chemistry Lecture 32: Vibrational Spectroscopy and the IR
5.0, Prncples of Inorganc Chemstry II MIT Department of Chemstry Lecture 3: Vbratonal Spectroscopy and the IR Vbratonal spectroscopy s confned to the 00-5000 cm - spectral regon. The absorpton of a photon
More informationCanonical transformations
Canoncal transformatons November 23, 2014 Recall that we have defned a symplectc transformaton to be any lnear transformaton M A B leavng the symplectc form nvarant, Ω AB M A CM B DΩ CD Coordnate transformatons,
More informationThe Feynman path integral
The Feynman path ntegral Aprl 3, 205 Hesenberg and Schrödnger pctures The Schrödnger wave functon places the tme dependence of a physcal system n the state, ψ, t, where the state s a vector n Hlbert space
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More information2 More examples with details
Physcs 129b Lecture 3 Caltech, 01/15/19 2 More examples wth detals 2.3 The permutaton group n = 4 S 4 contans 4! = 24 elements. One s the dentty e. Sx of them are exchange of two objects (, j) ( to j and
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More informationPhysics 607 Exam 1. ( ) = 1, Γ( z +1) = zγ( z) x n e x2 dx = 1. e x2
Physcs 607 Exam 1 Please be well-organzed, and show all sgnfcant steps clearly n all problems. You are graded on your wor, so please do not just wrte down answers wth no explanaton! Do all your wor on
More informationLossy Compression. Compromise accuracy of reconstruction for increased compression.
Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More information