The Geometry of Logit and Probit


 Leslie Thompson
 4 years ago
 Views:
Transcription
1 The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters. To recap, the normal vector s denoted as N j and ts reflecton as N j. The normal vector s perpendcular to the cuttng plane. The cuttng plane n two dmensons s defned by the equaton N j (W Z j ) + N j (W Z j ) = 0 () where N j and N j are the components of the normal vector and (Z j, Z j ) s the mdpont of the roll call outcomes (see Fgure.). Any pont, (W, W ), that satsfes the equaton above les on the cuttng plane. For example, f the normal vector s (3, ) and the roll call mdpont s (, 0), then ths produces the equaton: 3(w ) (w 0) = 3w 3 w = 3w w 3 = 0 or 3w w = 3 so that (0, 3/), (/3, ), (, 3/), etc., all le on the plane. In three dmensons the cuttng plane s defned by the equaton N j (W Z j ) + N j (W Z j ) + N j3 (W 3 Z j3 ) = 0 () Where, as above, any pont, (W, W, W 3 ), that satsfes the equaton above les on the cuttng plane. For example, f the normal vector s (, , ) and the roll call mdpont s (,, ), ths produces the equaton: (w ) (w ) + (w 3 ) = w w + w 3 = 0 or w w + w 3 = so that (0, 0, ), (, 0, ), (0,, 3), etc., all le on the plane.
2 For s dmensons the cuttng plane s defned by the vector equaton N j (W Z j ) = 0 (book,.8) (3) where N j s the s by normal vector such that N j N j =, W and Z j are s by vectors and 0 s an s by vector of zeroes. The normal vector s constraned to be of unt length for roll call votng work but t s a general vector n other applcatons. In general, f W A and W B are both ponts n the plane, N j W A = N j W B = c j, where c j s a scalar constant. Geometrcally, every pont n the plane projects onto the same pont on the lne defned by the normal vector, N j and ts reflecton N j (see Fgure.). Ths projecton pont for the general case (N j not necessarly normalzed to one; that s, N j N j = ): M j = c j s N k= N j jk (book,.9) (4) To see ths consder the example of the 3dmensonal plane above: w w = w w + w3 = w 3 [ ] (5) We need to fnd the pont on the normal vector [  ] that satsfes equaton (5); that s, the pont where the plane passes through the normal vector tself. Let k be a scalar constant. The soluton s: k k = 4k + 4k + k =, so that k= 9 k [ ]
3 And the pont s whch s gven n equaton (4). Note that, by constructon, N j M j = c j. In addton, n the roll call context, because the mdpont of the Yea and Nay polcy ponts, Z j, s on the cuttng plane, t also projects to the pont M j. The cuttng plane passes through the lne formed by the normal vector and ts reflecton (see Fgure.) at the pont M j. In the case of a smple probt analyss, the cuttng plane conssts of all possble legslator deal ponts such that the probablty of the correspondng legslator votng Yea or votng Nay s exactly.5; namely: P(legslator votes Yea) = P(legslator votes Nay) = ss Φ s = ss Φ s = Φ ( 0) =.5 Where Φ(.) s the dstrbuton functon for the normal and,,, s are legslator s coordnates on the s dmensons. Because the s and s cannot be separately dentfed, the usual assumpton s to set s =. The equaton above reduces to: s s = 0 or s s = =  0 (6) where s the slength vector of legslator coordnates: = 3 s 3
4 and s an slength vector of the coeffcents,, 3,, s ; that s: = 3 s Note that the expresson =  0 s exactly the same as N j W = c j, whch was used above. Namely, set N j = and every pont n the plane projects onto the pont: M j =  0 s k= k. (book,.0) (7) In other words, n a regular probt context the coeffcents on the ndependent varables form a normal vector to a plane that passes through the pont  0 s k= k. Note that ths pont s fxed wth regard to s. In partcular, ss 0 = + s s s So that  =  s s 0 0 s s k k k= s k= (8) When there s complete separaton that s, no error, ths plays an mportant role below. are: The smple logt case s dentcal to probt when s =. The logt probabltes 4
5 ( s s ) e = + e + e ( ) ( ) 0 s s 0 s s =.5 Cancelng out the denomnator and takng the natural log of both sdes yelds the same equaton as probt: s s = 0 In both probt and logt the coeffcents on the ndependent varables form a normal vector to a plane that passes through the pont  0 s k= k. The cosne of the angle between the normal vectors from probt and logt should be very close to one. That s: ' P L cosθ = P L (9) where θ s the angle between the two normal vectors, and. s the correspondng norm of the normal vector. Computng cosθ s a useful check on the two estmaton technques. Several examples of ths are shown n the Appendx. The Geometry of Complete Separaton (Perfect Votng) To smplfy the notaton below let the normal probablty densty functon be 0 + σ 0 + and the dstrbuton functon be Φ σ whch I wll smplfy to just and Φ, respectvely. I leave s n the expressons because t s the source of the dentfcaton problem. Namely, as a practcal matter, the cuttng plane s dentfed (up to a slght wggle, dependng upon the number of ponts) but, because there s no error, 5
6 6 s 0 and the observed coeffcent vector explodes to entres of + or  because of the mplct dvson of the coeffcents by s. In a standard presentaton the th row of the matrx of ndependent varables would be: 3 s, and the coeffcent vector would be 0 3 s. But for clarty of presentaton I want to separate the ntercept term, 0, from the other s coeffcents. In the general context my use of Yea and Nay below corresponds to the bnary dependent varable beng and 0, respectvely. The frst dervatves for the Probt Loglkelhood functon are: ln L = Yea Nay Yea Nay Yea Nay s s Yea Nay Φ Φ Φ Φ (0) Let a and b be ndces from to s. Treatng 0 separately, the second dervatves are
7 + + ( ) 0 0 +( Φ) ln L σ σ = 0 Yea Φ Nay ( ) + + ( ) 0 0 +( Φ) ln L σ σ = a 0 a Yea Φ Nay ( ) a (A) (B) And the remanng are: b + +b ( ) 0 0 +( Φ) ln L σ σ = a b ab ba bb Yea Φ Nay ( ) (C) Before turnng the pure case of complete separaton, consder the ntermedate but vexng case where one of the ndependent varables s an ndcator varable that separates wth respect to the bnary dependent varable. That s, whenever the ndcator s the correspondng value of the dependent varable s. Let the ndcator varable be. The frst partal dervatve s: ln L = + = = 0 Φ Φ Φ Φ () Yea,x= Yea,x= 0 Nay,x= 0 Yea,x= Because multplcaton by zero cancels the nd and 3 rd terms and the case Nay, x =, by defnton, does not occur. Hence, for equaton () to hold t must be the case that because =, 0 + σ + K σ = 0, where + Φ Φ + K σ σ Yea,x= Yea,x= 0 K = s s s. 7
8 The K can be treated as constants so that for equaton () to hold t must be the case that +. Note that ths means that the normal vector tself s not dentfed. Wth the K as constants so that s s a constant, then there wll be a dfferent normal vector wth every change n the value of so that the cuttng plane s not dentfed. The case of complete separaton has a dfferent geometry. By defnton, there exsts a plane that perfectly dvdes the Yeas ( s) from the Nays ( 0 s). Let N j be the s by normal vector to ths plane such that N j N j = (I do not need the j subscrpt here but I retan t for notatonal consstency). Hence, as dscussed above, f A and B are both ponts n the plane, then N j A = N j B = c j, where c j s a scalar constant. For complete separaton ether we have: f Yea, then N > c ; and f Nay, then N < c (3) j j j j or f Yea, then N < c ; and f Nay, then N > c j j j j Wthout loss of generalty I wll assume that equaton (3) s the correct polarty. Usng equaton (6) above, ths s equvalent to: 0 0 f Yea, then ; and f Nay, then > < (4) s s s s Or more smply: f Yea then > and f Nay, then < s s 0 s s 0 s s, 0; 0 However, f ths s true then the lkelhood functon forces the followng: f Yea, then Φ ; and f Nay, then Φ 0 σ σ 8
9 because σ 0. Agan, however, note that the cuttng plane s dentfed up to a slght wggle because the s are fxed constants and, by defnton, the plane perfectly dvdes the Yeas ( s) from the Nays ( 0 s). Ths plane passes through the sdmensonal space of the ndependent varables and separates those cases correspondng to Yeas ( s) from those correspondng to Nays ( 0 s). In actual estmaton, the lkelhood functon for the lnear probt (and logt) problem s globally convex the nverse of the Hessan (equaton system above) s negatve defnte. Hence, the gradent vector wll quckly clmb up the surface of the lkelhood functon to the global maxmum. In the case of perfect separaton what ths means n practce s that the gradent vector explodes as t approaches the global maxmum of zero; that s, the vector becomes nfntely long. Therefore, a straghtforward soluton to ths problem s to apply the constrant = at each teraton and then adjustng the standard devaton term, σ, to compensate. When the standard devaton term begns to vansh, stop. It s then a smple matter to take the normal vector and fnd c j wth the Jance algorthm. Note that ths produces the coeffcent vector correspondng to a perfect specfcaton for ths partcular set of ndependent varables,, and the bnary dependent varable, y. However, snce there s no error, there s no probablty, and no standard errors. 9
10 Appendx Here are two examples from the NES 000 electon study. The varables are whether or not the respondent voted (0=not voted, > 0 voted), and the ndependent varables are party (0 6), ncome ( categores), race (0 = whte; = black), sex (0 = male, = female), south (0=north, =south), educaton (=hgh school, =some college, 3=college). Frst Example: Smple Probt and Smple Logt. probt voted party ncome race sex south educaton age Iteraton 0: log lkelhood = Iteraton : log lkelhood = Iteraton : log lkelhood = Iteraton 3: log lkelhood = Iteraton 4: log lkelhood = Probt regresson Number of obs = 06 LR ch(7) = Prob > ch = Log lkelhood = Pseudo R = voted Coef. Std. Err. z P> z [95% Conf. Interval] party ncome race sex 3.3e south educaton age _cons logt voted party ncome race sex south educaton age Iteraton 0: log lkelhood = Iteraton : log lkelhood = Iteraton : log lkelhood = Iteraton 3: log lkelhood = Iteraton 4: log lkelhood = Logstc regresson Number of obs = 06 LR ch(7) = Prob > ch = Log lkelhood = Pseudo R = voted Coef. Std. Err. z P> z [95% Conf. Interval] party ncome race sex south educaton age _cons
11 Probt Normalzed (Normal Vector) Logt Normalzed (Normal Vector) Party Income Race Sex South Educaton Age The correlaton (Cosne) between the two normal vectors =
12 Second Example: Ordered Probt and Ordered Logt. oprobt party voted ncome race sex south educaton age Iteraton 0: log lkelhood = Iteraton : log lkelhood = Iteraton : log lkelhood = Iteraton 3: log lkelhood = Ordered probt regresson Number of obs = 06 LR ch(7) = 0.73 Prob > ch = Log lkelhood = Pseudo R = party Coef. Std. Err. z P> z [95% Conf. Interval] voted ncome race sex south educaton age /cut /cut /cut /cut /cut /cut ologt party voted ncome race sex south educaton age Iteraton 0: log lkelhood = Iteraton : log lkelhood = Iteraton : log lkelhood = Iteraton 3: log lkelhood = Ordered logstc regresson Number of obs = 06 LR ch(7) = 96.3 Prob > ch = Log lkelhood = Pseudo R = party Coef. Std. Err. z P> z [95% Conf. Interval] voted ncome race sex south educaton age /cut /cut /cut /cut /cut /cut
13 Probt Normalzed (Normal Vector) Logt Normalzed (Normal Vector) voted Income Race Sex South Educaton Age The correlaton (Cosne) between the two normal vectors =
Maximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452  OE 4: Probt and Logt Models ECO 452  OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452  OE 4: Probt and Logt Models ECO 452  OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationLectures  Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures  Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fntedmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the lefthand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationBIO Lab 2: TWOLEVEL NORMAL MODELS with school children popularity data
Lab : TWOLEVEL NORMAL MODELS wth school chldren popularty data Purpose: Introduce basc twolevel models for normally dstrbuted responses usng STATA. In partcular, we dscuss Random ntercept models wthout
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationMarginal Effects in Probit Models: Interpretation and Testing. 1. Interpreting Probit Coefficients
ECON 5  NOE 15 Margnal Effects n Probt Models: Interpretaton and estng hs note ntroduces you to the two types of margnal effects n probt models: margnal ndex effects, and margnal probablty effects. It
More informationInterval Estimation in the Classical Normal Linear Regression Model. 1. Introduction
ECONOMICS 35*  NOTE 7 ECON 35*  NOTE 7 Interval Estmaton n the Classcal Normal Lnear Regresson Model Ths note outlnes the basc elements of nterval estmaton n the Classcal Normal Lnear Regresson Model
More informationDO NOT OPEN THE QUESTION PAPER UNTIL INSTRUCTED TO DO SO BY THE CHIEF INVIGILATOR. Introductory Econometrics 1 hour 30 minutes
25/6 Canddates Only January Examnatons 26 Student Number: Desk Number:...... DO NOT OPEN THE QUESTION PAPER UNTIL INSTRUCTED TO DO SO BY THE CHIEF INVIGILATOR Department Module Code Module Ttle Exam Duraton
More informationLinear Regression Analysis: Terminology and Notation
ECON 35*  Secton : Basc Concepts of Regresson Analyss (Page ) Lnear Regresson Analyss: Termnology and Notaton Consder the generc verson of the smple (twovarable) lnear regresson model. It s represented
More informationBasically, if you have a dummy dependent variable you will be estimating a probability.
ECON 497: Lecture Notes 13 Page 1 of 1 Metropoltan State Unversty ECON 497: Research and Forecastng Lecture Notes 13 Dummy Dependent Varable Technques Studenmund Chapter 13 Bascally, f you have a dummy
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More information10701/ Machine Learning, Fall 2005 Homework 3
10701/15781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons10701@autonlaborg for queston Problem 1 Regresson and Crossvaldaton [40
More informationAdditional Codes using Finite Difference Method. 1 HJB Equation for ConsumptionSaving Problem Without Uncertainty
Addtonal Codes usng Fnte Dfference Method Benamn Moll 1 HJB Equaton for ConsumptonSavng Problem Wthout Uncertanty Before consderng the case wth stochastc ncome n http://www.prnceton.edu/~moll/ HACTproect/HACT_Numercal_Appendx.pdf,
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the faketest data; fxed
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5*  NOTE (Summary) ECON 5*  NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationβ0 + β1xi. You are interested in estimating the unknown parameters β
Ordnary Least Squares (OLS): Smple Lnear Regresson (SLR) Analytcs The SLR Setup Sample Statstcs Ordnary Least Squares (OLS): FOCs and SOCs Back to OLS and Sample Statstcs Predctons (and Resduals) wth OLS
More informationSupplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso
Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationLecture 6: Introduction to Linear Regression
Lecture 6: Introducton to Lnear Regresson An Manchakul amancha@jhsph.edu 24 Aprl 27 Lnear regresson: man dea Lnear regresson can be used to study an outcome as a lnear functon of a predctor Example: 6
More informationInterpreting Slope Coefficients in Multiple Linear Regression Models: An Example
CONOMICS 5*  Introducton to NOT CON 5*  Introducton to NOT : Multple Lnear Regresson Models Interpretng Slope Coeffcents n Multple Lnear Regresson Models: An xample Consder the followng smple lnear
More informationEcon Statistical Properties of the OLS estimator. Sanjaya DeSilva
Econ 39  Statstcal Propertes of the OLS estmator Sanjaya DeSlva September, 008 1 Overvew Recall that the true regresson model s Y = β 0 + β 1 X + u (1) Applyng the OLS method to a sample of data, we estmate
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationThe Second AntiMathima on Game Theory
The Second AntMathma on Game Theory Ath. Kehagas December 1 2006 1 Introducton In ths note we wll examne the noton of game equlbrum for three types of games 1. 2player 2acton zerosum games 2. 2player
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 111 Emprcal Models 112 Smple Lnear Regresson 113 Propertes of the Least Squares Estmators 114 Hypothess Test n Smple Lnear Regresson 114.1 Use of ttests
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationSystems of Equations (SUR, GMM, and 3SLS)
Lecture otes on Advanced Econometrcs Takash Yamano Fall Semester 4 Lecture 4: Sstems of Equatons (SUR, MM, and 3SLS) Seemngl Unrelated Regresson (SUR) Model Consder a set of lnear equatons: $ + ɛ $ + ɛ
More informationLogistic Regression Maximum Likelihood Estimation
HarvardMIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla OhnoMachado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationU.C. Berkeley CS294: Beyond WorstCase Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond WorstCase Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationGoodness of fit and Wilks theorem
DRAFT 0.0 Glen Cowan 3 June, 2013 Goodness of ft and Wlks theorem Suppose we model data y wth a lkelhood L(µ) that depends on a set of N parameters µ = (µ 1,...,µ N ). Defne the statstc t µ ln L(µ) L(ˆµ),
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationLaboratory 3: Method of Least Squares
Laboratory 3: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly they are correlated wth
More information3.1 Expectation of Functions of Several Random Variables. )' be a kdimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationsince [1( 0+ 1x1i+ 2x2 i)] [ 0+ 1x1i+ assumed to be a reasonable approximation
Econ 388 R. Butler 204 revsons Lecture 4 Dummy Dependent Varables I. Lnear Probablty Model: the Regresson model wth a dummy varables as the dependent varable assumpton, mplcaton regular multple regresson
More informationModule 2. Random Processes. Version 2 ECE IIT, Kharagpur
Module Random Processes Lesson 6 Functons of Random Varables After readng ths lesson, ou wll learn about cdf of functon of a random varable. Formula for determnng the pdf of a random varable. Let, X be
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenuemaxmzng assortment of products to offer when the prces of products are fxed.
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More informationLaboratory 1c: Method of Least Squares
Lab 1c, Least Squares Laboratory 1c: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly
More informationELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM
ELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM An elastc wave s a deformaton of the body that travels throughout the body n all drectons. We can examne the deformaton over a perod of tme by fxng our look
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationβ0 + β1xi. You are interested in estimating the unknown parameters β
Revsed: v3 Ordnar Least Squares (OLS): Smple Lnear Regresson (SLR) Analtcs The SLR Setup Sample Statstcs Ordnar Least Squares (OLS): FOCs and SOCs Back to OLS and Sample Statstcs Predctons (and Resduals)
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationLecture 16 Statistical Analysis in Biomaterials Research (Part II)
3.051J/0.340J 1 Lecture 16 Statstcal Analyss n Bomaterals Research (Part II) C. F Dstrbuton Allows comparson of varablty of behavor between populatons usng test of hypothess: σ x = σ x amed for Brtsh statstcan
More information8 : Learning in Fully Observed Markov Networks. 1 Why We Need to Learn Undirected Graphical Models. 2 Structural Learning for Completely Observed MRF
10708: Probablstc Graphcal Models 10708, Sprng 2014 8 : Learnng n Fully Observed Markov Networks Lecturer: Erc P. Xng Scrbes: Meng Song, L Zhou 1 Why We Need to Learn Undrected Graphcal Models In the
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3d rotaton around the xaxs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3d rotaton around the yaxs:
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of ExermentsI MODULE III LECTURE  2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationUniversity of California at Berkeley Fall Introductory Applied Econometrics Final examination
SID: EEP 118 / IAS 118 Elsabeth Sadoulet and Daley Kutzman Unversty of Calforna at Berkeley Fall 01 Introductory Appled Econometrcs Fnal examnaton Scores add up to 10 ponts Your name: SID: 1. (15 ponts)
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationBasic R Programming: Exercises
Basc R Programmng: Exercses RProgrammng John Fox ICPSR, Summer 2009 1. Logstc Regresson: Iterated weghted least squares (IWLS) s a standard method of fttng generalzed lnear models to data. As descrbed
More informationOutline. Zero Conditional mean. I. Motivation. 3. Multiple Regression Analysis: Estimation. Read Wooldridge (2013), Chapter 3.
Outlne 3. Multple Regresson Analyss: Estmaton I. Motvaton II. Mechancs and Interpretaton of OLS Read Wooldrdge (013), Chapter 3. III. Expected Values of the OLS IV. Varances of the OLS V. The Gauss Markov
More informationChapter 15  Multiple Regression
Chapter  Multple Regresson Chapter  Multple Regresson Multple Regresson Model The equaton that descrbes how the dependent varable y s related to the ndependent varables x, x,... x p and an error term
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationMaximum Likelihood Estimation (MLE)
Maxmum Lkelhood Estmaton (MLE) Ken KreutzDelgado (Nuno Vasconcelos) ECE 175A Wnter 01 UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the righthand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More information8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore
8/5/17 Data Modelng Patrce Koehl Department of Bologcal Scences atonal Unversty of Sngapore http://www.cs.ucdavs.edu/~koehl/teachng/bl59 koehl@cs.ucdavs.edu Data Modelng Ø Data Modelng: least squares Ø
More informationChapter 2  The Simple Linear Regression Model S =0. e i is a random error. S β2 β. This is a minimization problem. Solution is a calculus exercise.
Chapter  The Smple Lnear Regresson Model The lnear regresson equaton s: where y + = β + β e for =,..., y and are observable varables e s a random error How can an estmaton rule be constructed for the
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationIntroduction to Dummy Variable Regressors. 1. An Example of Dummy Variable Regressors
ECONOMICS 5*  Introducton to Dummy Varable Regressors ECON 5*  Introducton to NOTE Introducton to Dummy Varable Regressors. An Example of Dummy Varable Regressors A model of North Amercan car prces
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationSociété de Calcul Mathématique SA
Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationSTAT 3008 Applied Regression Analysis
STAT 3008 Appled Regresson Analyss Tutoral : Smple Lnear Regresson LAI Chun He Department of Statstcs, The Chnese Unversty of Hong Kong 1 Model Assumpton To quantfy the relatonshp between two factors,
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Recall: man dea of lnear regresson Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 8 Lnear regresson can be used to study an
More information9. Binary Dependent Variables
9. Bnar Dependent Varables 9. Homogeneous models Log, prob models Inference Tax preparers 9.2 Random effects models 9.3 Fxed effects models 9.4 Margnal models and GEE Appendx 9A  Lkelhood calculatons
More informationAPPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14
APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 008 Recall: man dea of lnear regresson Lnear regresson can be used to study
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationThe KMO Method for Solving Nonhomogenous, m th Order Differential Equations
The KMO Method for Solvng Nonhomogenous, m th Order Dfferental Equatons Davd Krohn Danel MarñoJohnson John Paul Ouyang March 14, 2013 Abstract Ths paper shows a smple tabular procedure for fndng the
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More informationSalmon: Lectures on partial differential equations. Consider the general linear, secondorder PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of secondorder equatons There are general methods for classfyng hgherorder partal dfferental equatons. One s very general (applyng even to
More informationOpen Systems: Chemical Potential and Partial Molar Quantities Chemical Potential
Open Systems: Chemcal Potental and Partal Molar Quanttes Chemcal Potental For closed systems, we have derved the followng relatonshps: du = TdS pdv dh = TdS + Vdp da = SdT pdv dg = VdP SdT For open systems,
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: GuoJun QI
Logstc Regresson CAP 561: achne Learnng Instructor: GuoJun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate classcondtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationHomework Notes Week 7
Homework Notes Week 7 Math 4 Sprng 4 #4 (a Complete the proof n example 5 that s an nner product (the Frobenus nner product on M n n (F In the example propertes (a and (d have already been verfed so we
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationEffects of Ignoring Correlations When Computing Sample ChiSquare. John W. Fowler February 26, 2012
Effects of Ignorng Correlatons When Computng Sample ChSquare John W. Fowler February 6, 0 It can happen that chsquare must be computed for a sample whose elements are correlated to an unknown extent.
More informationDummy variables in multiple variable regression model
WESS Econometrcs (Handout ) Dummy varables n multple varable regresson model. Addtve dummy varables In the prevous handout we consdered the followng regresson model: y x 2x2 k xk,, 2,, n and we nterpreted
More informationSupplementary Notes for Chapter 9 Mixture Thermodynamics
Supplementary Notes for Chapter 9 Mxture Thermodynamcs Key ponts Nne major topcs of Chapter 9 are revewed below: 1. Notaton and operatonal equatons for mxtures 2. PVTN EOSs for mxtures 3. General effects
More informationBézier curves. Michael S. Floater. September 10, These notes provide an introduction to Bézier curves. i=0
Bézer curves Mchael S. Floater September 1, 215 These notes provde an ntroducton to Bézer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 50757 HIKARI Ltd, wwwmhkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More informationβ0 + β1xi and want to estimate the unknown
SLR Models Estmaton Those OLS Estmates Estmators (e ante) v. estmates (e post) The Smple Lnear Regresson (SLR) Condtons 4 An Asde: The Populaton Regresson Functon B and B are Lnear Estmators (condtonal
More informationEdge Isoperimetric Inequalities
November 7, 2005 Ross M. Rchardson Edge Isopermetrc Inequaltes 1 Four Questons Recall that n the last lecture we looked at the problem of sopermetrc nequaltes n the hypercube, Q n. Our noton of boundary
More informationProf. Dr. I. Nasser Phys 630, T Aug15 One_dimensional_Ising_Model
EXACT OEDIMESIOAL ISIG MODEL The onedmensonal Isng model conssts of a chan of spns, each spn nteractng only wth ts two nearest neghbors. The smple Isng problem n one dmenson can be solved drectly n several
More information