Lecture 4: Dealing with the measurement noise of a sensor
|
|
- Shawn Welch
- 5 years ago
- Views:
Transcription
1 Barrtz, March - 6, 0 Lecture 4: Dealng wth the measurement nose of a sensor S. Dlhare, D. Mallet, LOMA, Unversté Bordeaux and CNRS, Bordeaux, France E-mal: stefan.dlhare@u-bordeaux.fr LEMA, Unversté de Lorrane and CNRS, Vandoeuvre-lès-Nanc, France E-mal: dens.mallet@unv-lorrane.fr Abstract. In ths lecture, the electronc nose of a measurement sstem and ts phscal orgn s presented, as well as ts man characterstcs (see also lectures L and L5 for temperature measurements. he man propertes of random sgnals (jont probablt denst, expectaton, varance-covarance matrx are also brefl recalled. hese notons are essental for the assessment of resdual error n lnear (see lecture L3 and non-lnear parameter estmaton problems (see lecture L7 and for functon estmaton problems (see lecture L8, Introducton Measurement nose consttutes a crtcal pont that s crucal to understand n parameter estmaton, that s when comparng measurements and a model: the nowledge of nose propertes allows an assessment of the level of the estmaton errors (bas and dsperson n ths tpe of expermental nverse problems. Secton s devoted to the characterzaton of the measurement nose and a to bref descrpton of the dfferent orgns of measurement nose an nverter (that s an expermentalst combned wth a model-bulder can meet. Secton presents the vector-matrx representaton of these characterstcs, for further use n parameter estmaton problems.. Phscs of measurement nose.. Defnton Fgure Random sgnal Lecture : Gettng started page
2 Barrtz, March - 6, 0 A random sgnal (or stochastc process s a sgnal that does not reman unchanged when the experment t stems from s repeated (see Fg.. It s noted (t,ωwhere ω s a test (that s a random varable that contans the outcome of a random draw.. x (t, ω s a realzaton of (t,ωfor a specfc draw ω... Statstcal descrptors rst order descrptor Expectaton = Ensemble mean : m (t = E { (t, ω } (4. It s an ndcator of the average poston of the sgnal at tme t (see Fg. Fgure Realzatons of a random, local probablt denst functon and expectanc varaton Lecture : Gettng started page
3 Barrtz, March - 6, 0 nd order descrptors Instant power = Quadratc mean : P (t=e{ (t, ω } (4. It s a quantfcaton of the average power of the sgnal at a gven tme t Calculaton of the nstant power of sgnal at tme t : P ( t { ( t, } dt t t = E ω (4.3 0 t t + t Varance V (t=e{ l (t,ω m (tl } (4.4 It quantfes the nstant power of random fluctuatons around the mean (dsperson ndcator. Standard devaton σ ( t V ( t (4.5 = It has the same phscal meanng as the varance, but ts phscal unts are the same as the sgnal. he prevous descrptors characterze the sgnal behavor (average poston, dsperson at a tme t. he do not allow for an analss of the relatonshps (dependence that exst between samples. An ndcator s needed for quantfng the level of the «force» that maes a sample at tme t + depend on ts value at tme t. he autocorrelaton functon { ( t ( } R ( t, τ = E t + τ (4.6 It quantfes the correlaton (or the scalar product or the projecton, n the stochastc meanng between ( t + τ and (t (see Fg. 3. Lecture : Gettng started page 3
4 Barrtz, March - 6, 0 Fgure 3 Correlaton between sgnal for a gven tme lag hgh level (left and low level (rght Interpretaton of the autocorrelaton functon: A sgnal ver hghl correlated wth tself exhbts ver slow fluctuatons (wth a smooth appearance, see Fg. 4, top. A sgnal lowl correlated wth tself exhbts ver rapd fluctuatons (wth a chaotc appearance, see Fg. 4, bottom. Fgure 4 pcal slow (top and fast (bottom fluctuatons of sgnal Lecture : Gettng started page 4
5 Barrtz, March - 6, 0 Fgure 5 R ( τ / R ( τ = 0 for a a dscrete whte nose b a thermal nose and c a nose of narrow frequenc nterval.3 Propertes: statonart, ergodct he statonart propert of a sgnal Defnton: A statonart random sgnal s a sgnal whose statstcs do not depend on tme: m ( t = m ; V ( t = V ; P ( t = P (4.7a, b, c Remar : In fact, ths assumpton s nearl compulsor, n order to be able to estmate the characterstcs of the sgnal; Remar : he statonart propert of a random sgnal s analogous to the perodct condton of a determnstc sgnal. In an case, t just an dealzaton. he ergodct propert of a sgnal Defnton: A sgnal s ergodc (n the strong sense, see Fgure 6, f: Lecture : Gettng started page 5
6 Barrtz, March - 6, 0 lm / + / { ( t } ( t, ω dt = E,ω (4.8 Fgure 6 Ergodc sgnal As a partcular consequence, t elds: lm / + ( / t, ω dt = m (4.9 whch means that ts tme average s equal to the ensemble average..4. Examples Dfferent tpes of nose exst: Whte nose: constant spectrum (Fgures 7, 9 and 0 Pn nose: spectrum n /f Brown nose (brownan, sometmes called «red»: spectrum n /f ² Fgure Blue nose: spectrum n f Volet nose: spectrum n f ² Gre nose: blue + pn Lecture : Gettng started page 6
7 Barrtz, March - 6, 0 Fgure 7 Dfferent tpes of nose Other tpes of nose: hermal nose (Johnson Flcer nose Shot nose (Fgure Burst nose Avalanche nose Popcorn nose (see Fgure 8... Fgure 8 Popcorn nose Lecture : Gettng started page 7
8 Barrtz, March - 6, 0 he Popcorn nose (random telegraph sgnal corresponds to a dscrete modulaton of the channel current caused b the capture and emsson of a channel carrer Fgure 9 Whte nose: autocorrelaton (left and frequenc spectrum (rght Fgure 0 Whte nose Lecture : Gettng started page 8
9 Barrtz, March - 6, 0 Fgure Brownan nose (non-statonar Fgure Shot nose Lecture : Gettng started page 9
10 Barrtz, March - 6, 0 Fgure 3 Nose of lmted spectral nterval Fgure 4 Sne sgnal wth random phase.5 Nose n /f a hs tpe of nose les n between a whte nose and a brownan nose : - It s a process wth a long memor. - Man natural sgnals exhbt a spectrum n /f. hs s based on repeated observatons. However no clear theoretcal explanaton s et avalable. It does not stem from an dfferental equaton. - he longest seres of nformaton of data s the level of the Nle rver measured b Egptan over a 3500 ears perod: t follows a law n /f. Lecture : Gettng started page 0
11 Barrtz, March - 6, 0 hs tpe of nose s called b dfferent names : - pn nose (wth a law n /f, - Flcer nose n /f a (n the specal case where a = It s met n electroncs, see Fgures 5 and 6. Fgure 5 Electronc nose - left: spectrum (lnear scale, B. Johnson, Phs. Rev. 6 ( rght: nose spectrum of an amplfer over 3 months (log-log scale, M.A. Caloanndes J. Appl. Phs. 45 ( the heat flux denst q that flows through the wall. Other examples of pn nose are shown n Fgure 6. Flcer nose: - s a generalzaton of the pn nose - ts spectrum s n /f a, wth / < a < 3/ hermal nose of Johnson-Nqust the electronc nose has been observed frst b Johnson n 96 and t has been explaned b hs colleague, Nqust. - t results from the Brownan moton of electrons at constant temperature (thermalzaton. - n ts most general form, t s an elementar stochastc model of nose: whte nose pn nose Brownan nose Lecture : Gettng started page
12 Barrtz, March - 6, 0 Nose n a measurement chan he nose n measurement chan based on a mult-stage measurement chan results from noses at dfferent level of ths sstem, see Fgure 6. Nose for a mult-stage crcut Fgure 6 Pn nose: other examples Nose n a measurement chan he nose n measurement chan based on a mult-stage measurement chan results from noses at dfferent level of ths sstem, see Fgure 7. Fgure 7 - Nose for a mult-stage crcut Lecture : Gettng started page
13 Barrtz, March - 6, 0 Fgure 8 Current and voltage modelng of the nose of an operatonal amplfer At the local level, one has to model the behavor of each stage, see Fgure 8. he correlaton functon and the spectral denst can be used as tools for the characterzaton of the sstem. Fgure 8 Relatonshps between sgnal values at dfferent measurement tmes One has to answer the followng queston: - are an values of f (t at tmes t + t, t + t,..., t m + t, t m + t, t m + + t possble? It s possble to plot the autocorrelaton functon (see Fgure 9: C xx ( τ = lm x ( t x ( t τ dt (4.0 0 Lecture : Gettng started page 3
14 Barrtz, March - 6, 0 he value of ths functon for a zero tme lag s: C xx 0 0 ( 0 = lm x ( t x ( t dt = lm x ( t dt (4. And the sgn of the tme lag does not matter n ts defnton snce ths functon s smmetrc: τ C xx ( τ = lm x ( t x ( t τ dt = lm x ( t' + τ x ( t' dt' = C 0 τ xx (-τ (4. Fgure 9 Example of the shape of an autocorrelaton functon he energ of the sgnal s defned b: e m = lm s ( t dt (4.3 0 It can be decomposed on a frequenc bass, see Fgure 0: e m (n Fourer doman = ( f df (4.4 he quadratc mean n the tme doman s equal to the quadratc value n the frequenc doman. lm x ( t dt = 0 0 ( f df (4.4 Lecture : Gettng started page 4
15 Barrtz, March - 6, 0 Fgure 0 Measurement of the spectral denst. Mathematcal characterzaton of nose for parameter estmaton problems. A remnder of vector random varables.. Statc model and estmaton of the parameters of a probablt law We assume here that the model output s a constant µ (t does not depend on the ndependent varable, called 'tmes' her, whch means that mo = µ. hs means that ts output s 'statc' and repetton of measurements of mo corresponds too a smple random samplng process, wthout replacement here, n order to construct a sample (,,..., m. hs ver classcal procedure s used to get a statstcal estmaton of the stochastc mean µ and/or varance σ of the dstrbuton of (the nfnte number of 'potental' measurements startng from the real dscrete measurements of n the sample. Wth these assumptons, t can be easl shown that, as soon as the number of measurements m s hgh enough, hgher than 30 n practce, the samplee mean Y tends to follow a normal law, noted N here (Central lmt theorem, and the sample varance, noted s S, after a scalng b freedom : σ / m, follows a Ch - law, noted χ here, wth (m degrees of N Y : (, σ / m mo wth : Y = m m = Y and and m S S s s / σ = m m = : χ ( m (Y Y (4.5 hs allows to fnd non-based estmatons of µ and σ, usng the 'hat' notaton (^ to desgnate an estmated value of the correspondng parameter : Lecture : Gettng started page 5
16 Barrtz, March - 6, 0 ŷ mo m = et ˆ σ = s (4.6 m wth and s the observed values of Y and of correspondng measured values. S s, replacng random varables Y b the.. Measurements for a dnamcal model and random vector We consder now a dnamcal model, where measurements of = η ( t, x at m tmes t (for = to m consttute a samplng operaton but now the expectaton of the measurements vares wth tmes. hs means that ths samplng s made for a dnamcal populaton. he frst natural dea s to consder each measurement as the realzaton of a scalar perfect exact random varable Y, of expectaton E ( Y = =η ( t x that s the mo exact, exact t 'perfect' (noseless output of the model whose structure η (,. s exact as well as the set of parameters gathered n a column vector Lecture 3 of ths seres: perfect exact x (see equaton 3. n = + ε (4.7 where ε s the measurement nose, that s the dfference between the measured sgnal and the perfect output of the model that corresponds exactl to ths measurement. Let us note that t s mpossble de to dscrmnate n a measurement the contrbuton of perfect the exact model output from the nose ε. hat s wh the measurement nose s consdered as a random varable. he expectaton of the nose s equal to zero for a good measurement chan, and one sas then that the (drect perfect measurement of s unbased. So, because of (4.7, the expermental sgnal (of expectaton equal to perfect s also a random varable. One calls L the probablt law L followed b nose ε. One good measurng nstrument provdes a nose whose standard devaton σ s constant. hs probablt law s characterzed b several parameters: ts expectaton (that s ts stochastc mean, equal to zero here, ts varance σ as well as possbl other parameters requred to defne the probablt denst functon (pdf f ε of ths random varable. If one lmts oneself to the mean and varance (case of a normal law b example, one notes: perfect ε : L ( 0, σ Y : L (, σ (4.8 A captal character has been used here to desgnate the random varable Y whose Lecture : Gettng started page 6
17 Barrtz, March - 6, 05 realzaton at tme t s the meaured sgnal : so ths measurement s clearl a random varable too. perfect If the expectaton of s dfferent from, ether the model s wrong (case of a model bas, but we have excluded ths assumpton above, or the measurement s based, whch means that n average (that s wth repettons the sensor (wth ts acquston chan and ts calbraton law does not eld the exact value t s supposed to measure. he second dea s to consder the whole set of measurements (a mult-dmensonal sample as a column vector = [ L ] t, that s the realzaton of a vector random varable [ Y Y ] t vector form: where the nose vector ε s also a random vector. Y = L Y m m. So, equaton (4.7 can be gven a = perfect + ε (4.9 In the general case, vector nose ε defned n (4.9 has a probablt law noted L : ( E( ε 0, cov (,... ε : L = ε (4.0 where the (smmetrcal varance-covarance matrx cov ( ε s defned b : cov var ( ε cov ( ε, ε L cov ( ε, ε m var ( ε cov (, ( = cov ( = O ε ε m ε Y (4. O M smétrque var ( ε m Wth ths second pont of vew, the jont probablt denst functon f Y of Y can be defned as followed: Prob ( Y < + d, Y < + d, L, Y < + d n n n fy (,, L, n d d L d n = fy n ( d = d L d n (4...3 Example of a bnormal jont dstrbuton We consder here the partcular case of a Gaussan dstrbuton. hs allows a graphcal plot of the jont pdf f Y, see Fgure, whose expresson s gven below: Lecture : Gettng started page 7
18 Barrtz, March - 6, 05 Lecture : Gettng started page 8 + = = ( ( ( ( ( ( : wth ( ( σ σ ρ σ σ ρ ρ σ σ π x x x x Y, t, t, t, t, Q, Q exp, f mo mo mo mo (4.3 Fgure Bnormal law, case of correlated measurements hree parameters are present n ths law : the varances σ et σ of the two ndvdual random varables Y et Y and ther correlaton coeffcent ρ ( ρ. he varancecovarance matrx of Y s defned n able n the ver general case of a random vector Y of sze and of jont probablt denst functon, ( f Y. hs can be easl generalzed n the case m >.
19 Barrtz, March - 6, 05 able Defnton s and quanttes assocated wth two random varables Expectaton E ( Y = f Y ( dy = mo ( t, x pour = ou Expectaton of a random E ( Y E ( Y = = E ( Y mo mo vector ( t, x ( t, x Margnal dstrbutons (probablt denst functons f Y ( = Prob ( Y < + d = f Y (, d f Y ( = Prob ( Y < + d Specfc case of the bnormal law : = f Y (, d f Y ( = σ exp - π E (Y σ for = or Varances var ( Y = σ = ( E ( Y f Y (, d for = or Covarance cov ( Y, Y = ( E ( Y ( E ( Y = E Correlaton coeffcent ( [ Y E ( Y ] [ Y E ( Y ] = cov ( ε, ε ( Y, Y cov ρ = σ σ f Y (, dy dy Varance covarance matrx var (Y cov ( Y = cov ( Y, Y cov ( Y, Y var (Y σ = ρ σ σ ρ σ σ σ Lecture : Gettng started page 9
20 Barrtz, March - 6, 05 Let us note that, b defnton, the expectaton of a real nose should be equal to zero ( E ( ε = 0. hs vector nose wll be : - non correlated (or ndependent, f ts varance-covarance matrx s dagonal : ε, ε σ cov ( = δ, where δ s Kronecer smbol (null f, and equal to f =, where σ s the varance of ε. - ndependent and dentcall dstrbuted (..d. f ths nose s uncorrelated wth a constant varance : var ( ε = σ. In ths case ts varance-covarance matrx s sphercal : cov (ε = σ I, where I s the dentt matrx of dmensons (m x m. In ths case each component of ε s ndependent and follows the same probablt law.. Propertes of random vectors he noton of random vector has been ntroduced above. In the tpe of applcatons concerned b ths school, t s the column vector of measurements Y, of dmensons (m x. hs vector has an expectaton, noted E (Y, that s a vector of same sze, whose coeffcents are the expectatons of the correspondng coeffcents of Y. It has also a varance-covarance matrx, noted cov (Y, of dmensons (m x m whose coeffcents are the covarances of the coeffcents of Y : [ E ( Y ] = E Y and [ cov ( Y ] = cov (Y, Y ( (4.3 It s alwas possble to lnearl transform sgnal Y usng a lnear transformaton whose determnstc coeffcents are set n a matrx sont rangés G of sze (p x m, n order to get a transformed sgnal Z = G Y of sze (p x. Snce Y s a random vector, such s also the case for vectorel Z. he expectaton and the varance-covarance matrx of Z depend on the same properes of Y: Z = G Y E ( Z = G E ( Y et cov ( Z = G cov ( Y G ( Conclusons In ths short course, the dfferent characterstcs of an electronc nose have been detaled and the resultng effect on the sgnal, that can be consdered as the realzaton of a random vector has been gven. he stochastc propertes of ths random sgnal vector completel depend on the same propertes of the nose vector, f the model used s unbased. So ther nowledge s mportant f one wants to characterze the estmaton error n an expermental nverse problem (see lectures 3, 7 and 8 of ths seres, for example. Lecture : Gettng started page 0
The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationMulti-dimensional Central Limit Theorem
Mult-dmensonal Central Lmt heorem Outlne ( ( ( t as ( + ( + + ( ( ( Consder a sequence of ndependent random proceses t, t, dentcal to some ( t. Assume t = 0. Defne the sum process t t t t = ( t = (; t
More informationStatistics for Economics & Business
Statstcs for Economcs & Busness Smple Lnear Regresson Learnng Objectves In ths chapter, you learn: How to use regresson analyss to predct the value of a dependent varable based on an ndependent varable
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationMulti-dimensional Central Limit Argument
Mult-dmensonal Central Lmt Argument Outlne t as Consder d random proceses t, t,. Defne the sum process t t t t () t (); t () t are d to (), t () t 0 () t tme () t () t t t As, ( t) becomes a Gaussan random
More informationECONOMICS 351*-A Mid-Term Exam -- Fall Term 2000 Page 1 of 13 pages. QUEEN'S UNIVERSITY AT KINGSTON Department of Economics
ECOOMICS 35*-A Md-Term Exam -- Fall Term 000 Page of 3 pages QUEE'S UIVERSITY AT KIGSTO Department of Economcs ECOOMICS 35* - Secton A Introductory Econometrcs Fall Term 000 MID-TERM EAM ASWERS MG Abbott
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informatione i is a random error
Chapter - The Smple Lnear Regresson Model The lnear regresson equaton s: where + β + β e for,..., and are observable varables e s a random error How can an estmaton rule be constructed for the unknown
More informationEcon Statistical Properties of the OLS estimator. Sanjaya DeSilva
Econ 39 - Statstcal Propertes of the OLS estmator Sanjaya DeSlva September, 008 1 Overvew Recall that the true regresson model s Y = β 0 + β 1 X + u (1) Applyng the OLS method to a sample of data, we estmate
More informationChapter 14 Simple Linear Regression
Chapter 4 Smple Lnear Regresson Chapter 4 - Smple Lnear Regresson Manageral decsons often are based on the relatonshp between two or more varables. Regresson analss can be used to develop an equaton showng
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationChapter 2 - The Simple Linear Regression Model S =0. e i is a random error. S β2 β. This is a minimization problem. Solution is a calculus exercise.
Chapter - The Smple Lnear Regresson Model The lnear regresson equaton s: where y + = β + β e for =,..., y and are observable varables e s a random error How can an estmaton rule be constructed for the
More information4 Analysis of Variance (ANOVA) 5 ANOVA. 5.1 Introduction. 5.2 Fixed Effects ANOVA
4 Analyss of Varance (ANOVA) 5 ANOVA 51 Introducton ANOVA ANOVA s a way to estmate and test the means of multple populatons We wll start wth one-way ANOVA If the populatons ncluded n the study are selected
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationLinear Regression Analysis: Terminology and Notation
ECON 35* -- Secton : Basc Concepts of Regresson Analyss (Page ) Lnear Regresson Analyss: Termnology and Notaton Consder the generc verson of the smple (two-varable) lnear regresson model. It s represented
More informationLecture 6: Introduction to Linear Regression
Lecture 6: Introducton to Lnear Regresson An Manchakul amancha@jhsph.edu 24 Aprl 27 Lnear regresson: man dea Lnear regresson can be used to study an outcome as a lnear functon of a predctor Example: 6
More informationx = , so that calculated
Stat 4, secton Sngle Factor ANOVA notes by Tm Plachowsk n chapter 8 we conducted hypothess tests n whch we compared a sngle sample s mean or proporton to some hypotheszed value Chapter 9 expanded ths to
More informationComputation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models
Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,
More information1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands
Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of
More informationCorrelation and Regression. Correlation 9.1. Correlation. Chapter 9
Chapter 9 Correlaton and Regresson 9. Correlaton Correlaton A correlaton s a relatonshp between two varables. The data can be represented b the ordered pars (, ) where s the ndependent (or eplanator) varable,
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More informationChapter 9: Statistical Inference and the Relationship between Two Variables
Chapter 9: Statstcal Inference and the Relatonshp between Two Varables Key Words The Regresson Model The Sample Regresson Equaton The Pearson Correlaton Coeffcent Learnng Outcomes After studyng ths chapter,
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationSystems of Equations (SUR, GMM, and 3SLS)
Lecture otes on Advanced Econometrcs Takash Yamano Fall Semester 4 Lecture 4: Sstems of Equatons (SUR, MM, and 3SLS) Seemngl Unrelated Regresson (SUR) Model Consder a set of lnear equatons: $ + ɛ $ + ɛ
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationUncertainty and auto-correlation in. Measurement
Uncertanty and auto-correlaton n arxv:1707.03276v2 [physcs.data-an] 30 Dec 2017 Measurement Markus Schebl Federal Offce of Metrology and Surveyng (BEV), 1160 Venna, Austra E-mal: markus.schebl@bev.gv.at
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationResource Allocation and Decision Analysis (ECON 8010) Spring 2014 Foundations of Regression Analysis
Resource Allocaton and Decson Analss (ECON 800) Sprng 04 Foundatons of Regresson Analss Readng: Regresson Analss (ECON 800 Coursepak, Page 3) Defntons and Concepts: Regresson Analss statstcal technques
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More informationExam. Econometrics - Exam 1
Econometrcs - Exam 1 Exam Problem 1: (15 ponts) Suppose that the classcal regresson model apples but that the true value of the constant s zero. In order to answer the followng questons assume just one
More informationModule 2. Random Processes. Version 2 ECE IIT, Kharagpur
Module Random Processes Lesson 6 Functons of Random Varables After readng ths lesson, ou wll learn about cdf of functon of a random varable. Formula for determnng the pdf of a random varable. Let, X be
More informationwhere I = (n x n) diagonal identity matrix with diagonal elements = 1 and off-diagonal elements = 0; and σ 2 e = variance of (Y X).
11.4.1 Estmaton of Multple Regresson Coeffcents In multple lnear regresson, we essentally solve n equatons for the p unnown parameters. hus n must e equal to or greater than p and n practce n should e
More informationStatistics for Managers Using Microsoft Excel/SPSS Chapter 13 The Simple Linear Regression Model and Correlation
Statstcs for Managers Usng Mcrosoft Excel/SPSS Chapter 13 The Smple Lnear Regresson Model and Correlaton 1999 Prentce-Hall, Inc. Chap. 13-1 Chapter Topcs Types of Regresson Models Determnng the Smple Lnear
More informationChapter 3. Two-Variable Regression Model: The Problem of Estimation
Chapter 3. Two-Varable Regresson Model: The Problem of Estmaton Ordnary Least Squares Method (OLS) Recall that, PRF: Y = β 1 + β X + u Thus, snce PRF s not drectly observable, t s estmated by SRF; that
More informationDepartment of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6
Department of Quanttatve Methods & Informaton Systems Tme Seres and Ther Components QMIS 30 Chapter 6 Fall 00 Dr. Mohammad Zanal These sldes were modfed from ther orgnal source for educatonal purpose only.
More informationSTAT 3008 Applied Regression Analysis
STAT 3008 Appled Regresson Analyss Tutoral : Smple Lnear Regresson LAI Chun He Department of Statstcs, The Chnese Unversty of Hong Kong 1 Model Assumpton To quantfy the relatonshp between two factors,
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More informationStrong Markov property: Same assertion holds for stopping times τ.
Brownan moton Let X ={X t : t R + } be a real-valued stochastc process: a famlty of real random varables all defned on the same probablty space. Defne F t = nformaton avalable by observng the process up
More informationPsychology 282 Lecture #24 Outline Regression Diagnostics: Outliers
Psychology 282 Lecture #24 Outlne Regresson Dagnostcs: Outlers In an earler lecture we studed the statstcal assumptons underlyng the regresson model, ncludng the followng ponts: Formal statement of assumptons.
More informationDigital Modems. Lecture 2
Dgtal Modems Lecture Revew We have shown that both Bayes and eyman/pearson crtera are based on the Lkelhood Rato Test (LRT) Λ ( r ) < > η Λ r s called observaton transformaton or suffcent statstc The crtera
More informationBasic Business Statistics, 10/e
Chapter 13 13-1 Basc Busness Statstcs 11 th Edton Chapter 13 Smple Lnear Regresson Basc Busness Statstcs, 11e 009 Prentce-Hall, Inc. Chap 13-1 Learnng Objectves In ths chapter, you learn: How to use regresson
More informationInterval Estimation in the Classical Normal Linear Regression Model. 1. Introduction
ECONOMICS 35* -- NOTE 7 ECON 35* -- NOTE 7 Interval Estmaton n the Classcal Normal Lnear Regresson Model Ths note outlnes the basc elements of nterval estmaton n the Classcal Normal Lnear Regresson Model
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationMarginal Effects in Probit Models: Interpretation and Testing. 1. Interpreting Probit Coefficients
ECON 5 -- NOE 15 Margnal Effects n Probt Models: Interpretaton and estng hs note ntroduces you to the two types of margnal effects n probt models: margnal ndex effects, and margnal probablty effects. It
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationx yi In chapter 14, we want to perform inference (i.e. calculate confidence intervals and perform tests of significance) in this setting.
The Practce of Statstcs, nd ed. Chapter 14 Inference for Regresson Introducton In chapter 3 we used a least-squares regresson lne (LSRL) to represent a lnear relatonshp etween two quanttatve explanator
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationStatistics for Business and Economics
Statstcs for Busness and Economcs Chapter 11 Smple Regresson Copyrght 010 Pearson Educaton, Inc. Publshng as Prentce Hall Ch. 11-1 11.1 Overvew of Lnear Models n An equaton can be ft to show the best lnear
More informationRockefeller College University at Albany
Rockefeller College Unverst at Alban PAD 705 Handout: Maxmum Lkelhood Estmaton Orgnal b Davd A. Wse John F. Kenned School of Government, Harvard Unverst Modfcatons b R. Karl Rethemeer Up to ths pont n
More informationβ0 + β1xi. You are interested in estimating the unknown parameters β
Revsed: v3 Ordnar Least Squares (OLS): Smple Lnear Regresson (SLR) Analtcs The SLR Setup Sample Statstcs Ordnar Least Squares (OLS): FOCs and SOCs Back to OLS and Sample Statstcs Predctons (and Resduals)
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationEconometrics of Panel Data
Econometrcs of Panel Data Jakub Mućk Meetng # 8 Jakub Mućk Econometrcs of Panel Data Meetng # 8 1 / 17 Outlne 1 Heterogenety n the slope coeffcents 2 Seemngly Unrelated Regresson (SUR) 3 Swamy s random
More informationPrimer on High-Order Moment Estimators
Prmer on Hgh-Order Moment Estmators Ton M. Whted July 2007 The Errors-n-Varables Model We wll start wth the classcal EIV for one msmeasured regressor. The general case s n Erckson and Whted Econometrc
More informationIntroduction to Regression
Introducton to Regresson Dr Tom Ilvento Department of Food and Resource Economcs Overvew The last part of the course wll focus on Regresson Analyss Ths s one of the more powerful statstcal technques Provdes
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Recall: man dea of lnear regresson Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 8 Lnear regresson can be used to study an
More informationConvergence of random processes
DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large
More informationLaboratory 1c: Method of Least Squares
Lab 1c, Least Squares Laboratory 1c: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 008 Recall: man dea of lnear regresson Lnear regresson can be used to study
More informationEmpirical Methods for Corporate Finance. Identification
mprcal Methods for Corporate Fnance Identfcaton Causalt Ultmate goal of emprcal research n fnance s to establsh a causal relatonshp between varables.g. What s the mpact of tangblt on leverage?.g. What
More information/ n ) are compared. The logic is: if the two
STAT C141, Sprng 2005 Lecture 13 Two sample tests One sample tests: examples of goodness of ft tests, where we are testng whether our data supports predctons. Two sample tests: called as tests of ndependence
More informationLINEAR REGRESSION ANALYSIS. MODULE VIII Lecture Indicator Variables
LINEAR REGRESSION ANALYSIS MODULE VIII Lecture - 7 Indcator Varables Dr. Shalabh Department of Maematcs and Statstcs Indan Insttute of Technology Kanpur Indcator varables versus quanttatve explanatory
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationA REVIEW OF ERROR ANALYSIS
A REVIEW OF ERROR AALYI EEP Laborator EVE-4860 / MAE-4370 Updated 006 Error Analss In the laborator we measure phscal uanttes. All measurements are subject to some uncertantes. Error analss s the stud
More informationβ0 + β1xi and want to estimate the unknown
SLR Models Estmaton Those OLS Estmates Estmators (e ante) v. estmates (e post) The Smple Lnear Regresson (SLR) Condtons -4 An Asde: The Populaton Regresson Functon B and B are Lnear Estmators (condtonal
More informationA Comparative Study for Estimation Parameters in Panel Data Model
A Comparatve Study for Estmaton Parameters n Panel Data Model Ahmed H. Youssef and Mohamed R. Abonazel hs paper examnes the panel data models when the regresson coeffcents are fxed random and mxed and
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationDurban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications
Durban Watson for Testng the Lack-of-Ft of Polynomal Regresson Models wthout Replcatons Ruba A. Alyaf, Maha A. Omar, Abdullah A. Al-Shha ralyaf@ksu.edu.sa, maomar@ksu.edu.sa, aalshha@ksu.edu.sa Department
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationSystematic Error Illustration of Bias. Sources of Systematic Errors. Effects of Systematic Errors 9/23/2009. Instrument Errors Method Errors Personal
9/3/009 Sstematc Error Illustraton of Bas Sources of Sstematc Errors Instrument Errors Method Errors Personal Prejudce Preconceved noton of true value umber bas Prefer 0/5 Small over large Even over odd
More informationExponential Type Product Estimator for Finite Population Mean with Information on Auxiliary Attribute
Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 193-9466 Vol. 10, Issue 1 (June 015), pp. 106-113 Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) Exponental Tpe Product Estmator
More informationThis column is a continuation of our previous column
Comparson of Goodness of Ft Statstcs for Lnear Regresson, Part II The authors contnue ther dscusson of the correlaton coeffcent n developng a calbraton for quanttatve analyss. Jerome Workman Jr. and Howard
More informationLecture 4 Hypothesis Testing
Lecture 4 Hypothess Testng We may wsh to test pror hypotheses about the coeffcents we estmate. We can use the estmates to test whether the data rejects our hypothess. An example mght be that we wsh to
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the non-lnear case, and also
More informationSTAT 511 FINAL EXAM NAME Spring 2001
STAT 5 FINAL EXAM NAME Sprng Instructons: Ths s a closed book exam. No notes or books are allowed. ou may use a calculator but you are not allowed to store notes or formulas n the calculator. Please wrte
More informationStatistics Chapter 4
Statstcs Chapter 4 "There are three knds of les: les, damned les, and statstcs." Benjamn Dsrael, 1895 (Brtsh statesman) Gaussan Dstrbuton, 4-1 If a measurement s repeated many tmes a statstcal treatment
More informationLecture 20: Hypothesis testing
Lecture : Hpothess testng Much of statstcs nvolves hpothess testng compare a new nterestng hpothess, H (the Alternatve hpothess to the borng, old, well-known case, H (the Null Hpothess or, decde whether
More informationThe Ordinary Least Squares (OLS) Estimator
The Ordnary Least Squares (OLS) Estmator 1 Regresson Analyss Regresson Analyss: a statstcal technque for nvestgatng and modelng the relatonshp between varables. Applcatons: Engneerng, the physcal and chemcal
More informationb ), which stands for uniform distribution on the interval a x< b. = 0 elsewhere
Fall Analyss of Epermental Measurements B. Esensten/rev. S. Errede Some mportant probablty dstrbutons: Unform Bnomal Posson Gaussan/ormal The Unform dstrbuton s often called U( a, b ), hch stands for unform
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationISQS 6348 Final Open notes, no books. Points out of 100 in parentheses. Y 1 ε 2
ISQS 6348 Fnal Open notes, no books. Ponts out of 100 n parentheses. 1. The followng path dagram s gven: ε 1 Y 1 ε F Y 1.A. (10) Wrte down the usual model and assumptons that are mpled by ths dagram. Soluton:
More informationSupplementary Notes for Chapter 9 Mixture Thermodynamics
Supplementary Notes for Chapter 9 Mxture Thermodynamcs Key ponts Nne major topcs of Chapter 9 are revewed below: 1. Notaton and operatonal equatons for mxtures 2. PVTN EOSs for mxtures 3. General effects
More informationUNIVERSITY OF TORONTO Faculty of Arts and Science. December 2005 Examinations STA437H1F/STA1005HF. Duration - 3 hours
UNIVERSITY OF TORONTO Faculty of Arts and Scence December 005 Examnatons STA47HF/STA005HF Duraton - hours AIDS ALLOWED: (to be suppled by the student) Non-programmable calculator One handwrtten 8.5'' x
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationNegative Binomial Regression
STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...
More informationProbability and Random Variable Primer
B. Maddah ENMG 622 Smulaton 2/22/ Probablty and Random Varable Prmer Sample space and Events Suppose that an eperment wth an uncertan outcome s performed (e.g., rollng a de). Whle the outcome of the eperment
More information2016 Wiley. Study Session 2: Ethical and Professional Standards Application
6 Wley Study Sesson : Ethcal and Professonal Standards Applcaton LESSON : CORRECTION ANALYSIS Readng 9: Correlaton and Regresson LOS 9a: Calculate and nterpret a sample covarance and a sample correlaton
More informationProbability Theory (revisited)
Probablty Theory (revsted) Summary Probablty v.s. plausblty Random varables Smulaton of Random Experments Challenge The alarm of a shop rang. Soon afterwards, a man was seen runnng n the street, persecuted
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More information