Predictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
|
|
- Geoffrey Cameron
- 6 years ago
- Views:
Transcription
1
2 Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters. Examples: Challenger Shuttle, German Bank Credt Ratng.
3 Classfcaton Problems Classfcaton s an mportant category of problems n whch the decson maker would lke to classfy the customers nto two or more groups. Examples of Classfcaton Problems: Customer Churn. Credt Ratng (low, hgh and medum rsk) Employee attrton. Fraud (classfcaton of a transacton to fraud/non-fraud) Outcome of any bnomal and multnomal experment.
4 Always Cheerful Always Sad Logstc Regresson attempts to classfy customers nto dfferent categores
5 Dscrete Choce Models Problems nvolvng dscrete choces avalable to the decson maker. Dscrete choce models (n busness) examnes whch alternatve s chosen by a customer and why? Most dscrete choce models estmate the probablty that a customer chooses a partcular alternatve from several possble alternatves.
6 Logstc Regresson Classfcaton Problem Dscrete Choce Model Probablty of an event
7
8 Logstc Regresson - Introducton The name logstc regresson emerges from logstc functon. P( Y 1) e 1 Z e Z Mathematcally, logstc regresson attempts to estmate condtonal probablty of an event.
9 Logstc Regresson Logstc regresson models estmate how probablty of an event may be affected by one or more explanatory varables.
10 Bnomal Logstc Regresson Bnomal (or bnary) logstc regresson s a model n whch the dependent varable s dchotomous. The ndependent varables may be of any type.
11 Logstc Functon (Sgmodal functon) n n z z x x x z e e z... 1 ) (
12 Logstc Regresson wth one Explanatory Varable P( Y 1 X x) ( x) exp( x) 1 exp( x) = 0 mples that P(Y x) s same for each value of x > 0 mples that P(Y x) s ncreases as the value of x ncreases < 0 mples that P(Y x) s decreases as the value of x ncreases
13 Logt Functon The Logt functon s the logarthmc transformaton of the logstc functon. It s defned as the natural logarthm of odds. Logt of a varable s gven by: Logt( ) ln( ) 0 1x 1 1 π 1 π odds
14 Logstc regresson More robust Error terms need not be normal. No requrement for equal varance for error term (homoscedastcty). No requrement for lnear relatonshp between dependent and ndependent varables.
15
16 Estmaton of parameters n Logstc Regresson Estmaton of parameters n logstc regresson s carred out usng Maxmum Lkelhood Estmaton (MLE) technque. No closed form soluton exsts for estmaton of regresson parameters of logstc regresson.
17 Maxmum Lkelhood Estmator (MLE) MLE s a statstcal model for estmatng model parameters of a functon. For a gven data set, the MLE chooses the values of model parameters that makes the data more lkely, than other parameter values.
18 Lkelhood Functon Lkelhood functon L() represents the jont probablty or lkelhood of observng the data that have been collected. MLE chooses that estmator of the set of unknown parameters whch maxmzes the lkelhood functon L().
19 Maxmum Lkelhood Estmator Assume that x 1, x 2,, x n are some sample observatons of a dstrbuton f(x, ), where s an unknown parameter. The lkelhood functon s L() = f(x 1, x 2,, x n, ) whch s the jont probablty densty functon of the sample. The value of, *, whch maxmzes L() s called the maxmum lkelhood estmator of.
20 Example: Exponental Dstrbuton Let x1, x2,, xn be the sample observaton that follows exponental dstrbuton wth parameter. That s: f(x, θ) θe θx The lkelhood functon s gven by (assumng ndependence): L( x, ) f θe ( x 1, ) f ( x θe 2, ) f θe -θx1 -θx1 -θx ( x n n, ) n e n 1 x
21 Log-lkelhood functon Log-lkelhood functon s gven by: Ln( L( x, )) nln The optmal, * s gven by: n x 1 d (ln( L( x, )) d * n n 1 x 1 x n n 1 x 0
22 Lkelhood functon for Bnary Logstc Functon Probablty densty functon for bnary logstc regresson s gven by: f y y ( ) (1 ) y 1 n y y y n 1 1, 2,..., ) (1 ) 1 L( ) f ( y y ln( L( )) n 1 y ln n ( x ) (1 y ) ln(1 ( x )) 1
23 Lkelhood functon for Bnary Logstc Functon )) exp( ln(1 ) ( ) ( ln n n x x y L
24 Estmaton of LR parameters 0 ) exp( 1 ) exp( )), ( ln( n n x x y L 0 ) exp( 1 ) exp( )), ( ln( n n x x x y x L The above system of equatons are solved teratvely to estmate 0 and 1
25 Lmtatons of MLE Maxmum lkelhood estmator may not be unque or may not exst. Closed form soluton may not exst for many cases, one may have to use teratve procedure to estmate the parameter values.
26
27 Challenger Data Flt Temp Damage STS-1 66 No STS-2 70 Yes STS-3 69 No STS-4 80 No STS-5 68 No STS-6 67 No STS-7 72 No STS-8 73 No STS-9 70 No STS-41B 57 Yes STS-41C 63 Yes STS-41D 70 Yes Flt Temp Damage STS-41G 78 No STS-51-A 67 No STS-51-C 53 Yes STS-51-D 67 No STS-51-B 75 No STS-51-G 70 No STS-51-F 81 No STS-51-I 76 No STS-51-J 79 No STS-61-A 75 Yes STS-61-B 76 No STS-61-C 58 Yes
28 Challenger launch temperature vs damage data
29 Logstc Regresson of challenger data Let: Y = 0 denote no damage Y = 1 denote damage to the O-rng P(Y = 1) = and P(Y = 0) = 1 -. We predct P(Y = 1 x ), x = launch temperature
30 Logstc Regresson usng SPSS Dependent varable: In Bnary logstc regresson, the dependent varable can take only two values. In multnomal logstc regresson, the dependent varable can take two or more values (but not contnuous). Covarate: All ndependent (predctor) varables are entered as covarates.
31 Step 1 a LaunchTemperature Constant Varables n the Equaton a. Varable(s) entered on step 1: LaunchTemperature. B S.E. Wald df Sg. Exp(B) ln X P( Y 1) e 1 e X X
32 Challenger: Probablty of falure estmate e 1 e X X Probablty
33 Classfcaton table from SPSS
34 Accuracy Paradox Assume an example of nsurance fraud. Past data has revealed that out of 1000 clams n the past, 950 are true clams and 50 are fraudulent clams. The classfcaton table usng a logstc regresson model s gven below: Observed Predcted % accuracy % % The overall accuracy s 94.5%. Not predctng fraud wll gve 95% accuracy!
35
36 ODDS and ODDS RATIO ODDS: Rato of two probablty values. odds 1 ODDS Rato: Rato of two odds.
37 ODDS RATIO Assume that X s an ndependent varable (covarate). The odds rato, OR, s defned as the rato of the odds for X = 1 to the odds for X = 0. The odds rato s gven by: (1) OR (0) 1 (1) 1 (0)
38 Interpretaton of Beta Coeffcent n LR π(x) ln( ) β0 β1x1 1 π(x) For x 0 π(0) ln( ) β0 1 π(0) For x 1 π(1) ln( ) β0 β1 1 π(1) 1 (1) /(1 (1)) ln (0) /(1 (0)) (1) (2) (3)
39 Interpretaton of LR coeffcents 1 ( x 1) (1 ( x 1)) ln ( x) (1 ( x 1)) Change n ln odds rato e 1 ( x 1) (1 ( x 1)) ( x) (1 ( x 1)) Change n odds rato
40 Odds Rato for Bnary Logstc Regresson OR e (0) 1 (0) ( 1) 1 (1) 1 If OR = 2, then the event s twce lkely to occur when X = 1 compared to X = 0. Odds rato approxmates the relatve rsk.
41 Interpretaton of LR coeffcents 1 s the change n log-odds rato for unt change n the explanatory varable. 1 s the change n odds rato by a factor exp( 1 ).
42
43 Sesson Outlne Measurng the ftness of Logstc Regresson Model. Testng ndvdual regresson parameters (Wald s test). Omnbus test for overall model ftness Hosmer-Lemeshow Goodness of ft test. R 2 n Logstc Regresson. Confdence Intervals for parameters and probabltes.
44 Wald Test Wald test s used to check the sgnfcance of ndvdual explanatory varables (smlar to t-statstc n lnear regresson). Wald test statstc s gven by: W ( ) SE 2 W s a ch-square statstc
45 Wald test hypothess Null Hypothess H0: = 0 Alternatve Hypothess H1: 0
46 Wald Test Challenger Data The explanatory varable s sgnfcant, snce the p value s less than 0.05
47 Wald Test Challenger Data For sgnfcant varables, the CI for Exp(β) wll not contan 1
48 Model Ch-Square Omnbus test: H 0 : β 1 = β 2 = = β k = 0 H1: Not all βs are zero
49 Hosmer-Lemeshow Goodness of Ft Test for overall ftness of the model for a bnary logstc regresson (smlar to ch-square goodness of ft test). The observatons are grouped nto 10 groups based on ther predcted probabltes.
50 Hosmer-Lemeshow Test Statstc Hosmer-Lemeshow Test Statstc s gven by: C g k 1 ( O n k k n k k ) 2 k (1 k ) g Number of groups n O k k k Number of observatons n each group Sum of the values for k The avearge n k th th group group.
51 H-L test for Challenger Data Hosme r and Le meshow Test Step 1 Ch-square df Sg Snce P (=0.310) s more than 0.05, we accept the null hypothess that there s no dfference between the predcted and observed frequences (accept the model)
52
53
54 Classfcaton Table Predcton (Classfcaton) Observed 1 (Postve) 7 0 (Negatve) 17 1 (Postve) 4 [True Postve] TP 0 [False Postve] FP 0 (Negatve) 3 [False Negatve] FN 17 [True Negatve] TN Senstvty Specfcty TP TP FN TN TN FP
55 Senstvty & Specfcty Senstvty Number of No of true postves true postves Number of false negatves Senstvty s the probablty that the predcted value of y = 1 gven that the observed value s 1. Specfcty Number of No of true negatves true negatves Number of false postves Specfcty s the probablty that the predcted value of y = 0 gven that the observed value s 0.
56 Recever Operatng Characterstcs (ROC) Curve ROC curve plots the true postve rato (rght postve classfcaton) aganst the false postve rato (1- specfcty) and compares t wth random classfcaton. The hgher the area under the ROC curve, the better the predcton ablty.
57 Challenger Example: Senstvty Vs 1-Specfcty (True postve Vs False postve) Cut off Value Senstvty Specfcty 1-specfcty
58 ROC Curve Challenger Example
59 ROC Curve
60 Area Under the ROC Curve Area under the ROC curve s nterpreted as the probablty that the model wll rank a randomly chosen postve hgher than randomly chosen negatve. If n1 s the number of postves (1s) and n2 s the number of negatves (0s), then the area under the ROC curve s the proporton of cases n all possble combnatons of (n1, n2) such that n1 wll have hgher probablty than n2.
61 ROC Curve General rule for acceptance of the model: If the area under ROC s: 0.5 No dscrmnaton 0.7 ROC area < 0.8 Acceptable dscrmnaton 0.8 ROC area < 0.9 Excellent dscrmnaton ROC area 0.9 Outstandng dscrmnaton
62 Gn Coeffcent Gn coeffcent measures ndvdual mpact of the an explanatory varable. Gn coeffcent = 2 AUC 1 AUC = Area under the ROC Curve
63 Optmal Cut-off probabltes Usng classfcaton plots. Youden s Index. Cost based optmzaton.
64 Classfcaton Plots Step number: 1 Observed Groups and Predcted Probabltes I 1 I I 1 I F I 1 I R E I 1 0 I Q I 1 0 I U I 1 0 I E N I I C I I Y I I I I I I I I Predcted Prob: Group:
65 Youden s Index Youden's ndex s a measures for dagnostc accuracy. Youden's ndex s calculated by deductng 1 from the sum of test s senstvty and specfcty. Youden' s Index J(p) Senstvty(p) specfcty(p)-1
66 Cost based Model for Optmal Cut-off Observed Predcted P 00 P 01 1 P 10 P 11 R 00 = Cost of classfyng 0 as 0 C 01 = Cost of classfyng 0 as 1 C 10 = Cost of classfyng 1 as 0 R 11 = Cost of classfyng 1 as 1 Optmal cut - off Mn p P 00 R 00 P 01 C 01 P 10 C 10 P 11 R 11
STAT 405 BIOSTATISTICS (Fall 2016) Handout 15 Introduction to Logistic Regression
STAT 45 BIOSTATISTICS (Fall 26) Handout 5 Introducton to Logstc Regresson Ths handout covers materal found n Secton 3.7 of your text. You may also want to revew regresson technques n Chapter. In ths handout,
More information3/3/2014. CDS M Phil Econometrics. Vijayamohanan Pillai N. CDS Mphil Econometrics Vijayamohan. 3-Mar-14. CDS M Phil Econometrics.
Dummy varable Models an Plla N Dummy X-varables Dummy Y-varables Dummy X-varables Dummy X-varables Dummy varable: varable assumng values 0 and to ndcate some attrbutes To classfy data nto mutually exclusve
More informationStatistics for Business and Economics
Statstcs for Busness and Economcs Chapter 11 Smple Regresson Copyrght 010 Pearson Educaton, Inc. Publshng as Prentce Hall Ch. 11-1 11.1 Overvew of Lnear Models n An equaton can be ft to show the best lnear
More informationDiagnostics in Poisson Regression. Models - Residual Analysis
Dagnostcs n Posson Regresson Models - Resdual Analyss 1 Outlne Dagnostcs n Posson Regresson Models - Resdual Analyss Example 3: Recall of Stressful Events contnued 2 Resdual Analyss Resduals represent
More informationSee Book Chapter 11 2 nd Edition (Chapter 10 1 st Edition)
Count Data Models See Book Chapter 11 2 nd Edton (Chapter 10 1 st Edton) Count data consst of non-negatve nteger values Examples: number of drver route changes per week, the number of trp departure changes
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationStatistics for Economics & Business
Statstcs for Economcs & Busness Smple Lnear Regresson Learnng Objectves In ths chapter, you learn: How to use regresson analyss to predct the value of a dependent varable based on an ndependent varable
More informationLecture 6: Introduction to Linear Regression
Lecture 6: Introducton to Lnear Regresson An Manchakul amancha@jhsph.edu 24 Aprl 27 Lnear regresson: man dea Lnear regresson can be used to study an outcome as a lnear functon of a predctor Example: 6
More informationDepartment of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6
Department of Quanttatve Methods & Informaton Systems Tme Seres and Ther Components QMIS 30 Chapter 6 Fall 00 Dr. Mohammad Zanal These sldes were modfed from ther orgnal source for educatonal purpose only.
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More information1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands
Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of
More informationRegression with limited dependent variables. Professor Bernard Fingleton
Regresson wth lmted dependent varables Professor Bernard Fngleton Regresson wth lmted dependent varables Whether a mortgage applcaton s accepted or dened Decson to go on to hgher educaton Whether or not
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationStatistics for Managers Using Microsoft Excel/SPSS Chapter 13 The Simple Linear Regression Model and Correlation
Statstcs for Managers Usng Mcrosoft Excel/SPSS Chapter 13 The Smple Lnear Regresson Model and Correlaton 1999 Prentce-Hall, Inc. Chap. 13-1 Chapter Topcs Types of Regresson Models Determnng the Smple Lnear
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More informationBasically, if you have a dummy dependent variable you will be estimating a probability.
ECON 497: Lecture Notes 13 Page 1 of 1 Metropoltan State Unversty ECON 497: Research and Forecastng Lecture Notes 13 Dummy Dependent Varable Technques Studenmund Chapter 13 Bascally, f you have a dummy
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Recall: man dea of lnear regresson Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 8 Lnear regresson can be used to study an
More informationANSWERS CHAPTER 9. TIO 9.2: If the values are the same, the difference is 0, therefore the null hypothesis cannot be rejected.
ANSWERS CHAPTER 9 THINK IT OVER thnk t over TIO 9.: χ 2 k = ( f e ) = 0 e Breakng the equaton down: the test statstc for the ch-squared dstrbuton s equal to the sum over all categores of the expected frequency
More informationMaximum Likelihood Estimation
Maxmum Lkelhood Estmaton INFO-2301: Quanttatve Reasonng 2 Mchael Paul and Jordan Boyd-Graber MARCH 7, 2017 INFO-2301: Quanttatve Reasonng 2 Paul and Boyd-Graber Maxmum Lkelhood Estmaton 1 of 9 Why MLE?
More informationBasic Business Statistics, 10/e
Chapter 13 13-1 Basc Busness Statstcs 11 th Edton Chapter 13 Smple Lnear Regresson Basc Busness Statstcs, 11e 009 Prentce-Hall, Inc. Chap 13-1 Learnng Objectves In ths chapter, you learn: How to use regresson
More informationLecture 9: Linear regression: centering, hypothesis testing, multiple covariates, and confounding
Lecture 9: Lnear regresson: centerng, hypothess testng, multple covarates, and confoundng Sandy Eckel seckel@jhsph.edu 6 May 008 Recall: man dea of lnear regresson Lnear regresson can be used to study
More informationMarginal Effects in Probit Models: Interpretation and Testing. 1. Interpreting Probit Coefficients
ECON 5 -- NOE 15 Margnal Effects n Probt Models: Interpretaton and estng hs note ntroduces you to the two types of margnal effects n probt models: margnal ndex effects, and margnal probablty effects. It
More information4.1. Lecture 4: Fitting distributions: goodness of fit. Goodness of fit: the underlying principle
Lecture 4: Fttng dstrbutons: goodness of ft Goodness of ft Testng goodness of ft Testng normalty An mportant note on testng normalty! L4.1 Goodness of ft measures the extent to whch some emprcal dstrbuton
More informationJanuary Examinations 2015
24/5 Canddates Only January Examnatons 25 DO NOT OPEN THE QUESTION PAPER UNTIL INSTRUCTED TO DO SO BY THE CHIEF INVIGILATOR STUDENT CANDIDATE NO.. Department Module Code Module Ttle Exam Duraton (n words)
More informationLOGIT ANALYSIS. A.K. VASISHT Indian Agricultural Statistics Research Institute, Library Avenue, New Delhi
LOGIT ANALYSIS A.K. VASISHT Indan Agrcultural Statstcs Research Insttute, Lbrary Avenue, New Delh-0 02 amtvassht@asr.res.n. Introducton In dummy regresson varable models, t s assumed mplctly that the dependent
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationsince [1-( 0+ 1x1i+ 2x2 i)] [ 0+ 1x1i+ assumed to be a reasonable approximation
Econ 388 R. Butler 204 revsons Lecture 4 Dummy Dependent Varables I. Lnear Probablty Model: the Regresson model wth a dummy varables as the dependent varable assumpton, mplcaton regular multple regresson
More information[The following data appear in Wooldridge Q2.3.] The table below contains the ACT score and college GPA for eight college students.
PPOL 59-3 Problem Set Exercses n Smple Regresson Due n class /8/7 In ths problem set, you are asked to compute varous statstcs by hand to gve you a better sense of the mechancs of the Pearson correlaton
More informationDO NOT OPEN THE QUESTION PAPER UNTIL INSTRUCTED TO DO SO BY THE CHIEF INVIGILATOR. Introductory Econometrics 1 hour 30 minutes
25/6 Canddates Only January Examnatons 26 Student Number: Desk Number:...... DO NOT OPEN THE QUESTION PAPER UNTIL INSTRUCTED TO DO SO BY THE CHIEF INVIGILATOR Department Module Code Module Ttle Exam Duraton
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationNegative Binomial Regression
STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationBIO Lab 2: TWO-LEVEL NORMAL MODELS with school children popularity data
Lab : TWO-LEVEL NORMAL MODELS wth school chldren popularty data Purpose: Introduce basc two-level models for normally dstrbuted responses usng STATA. In partcular, we dscuss Random ntercept models wthout
More informationSDMML HT MSc Problem Sheet 4
SDMML HT 06 - MSc Problem Sheet 4. The recever operatng characterstc ROC curve plots the senstvty aganst the specfcty of a bnary classfer as the threshold for dscrmnaton s vared. Let the data space be
More informationResource Allocation and Decision Analysis (ECON 8010) Spring 2014 Foundations of Regression Analysis
Resource Allocaton and Decson Analss (ECON 800) Sprng 04 Foundatons of Regresson Analss Readng: Regresson Analss (ECON 800 Coursepak, Page 3) Defntons and Concepts: Regresson Analss statstcal technques
More information1 Binary Response Models
Bnary and Ordered Multnomal Response Models Dscrete qualtatve response models deal wth dscrete dependent varables. bnary: yes/no, partcpaton/non-partcpaton lnear probablty model LPM, probt or logt models
More informationUNIVERSITY OF TORONTO Faculty of Arts and Science. December 2005 Examinations STA437H1F/STA1005HF. Duration - 3 hours
UNIVERSITY OF TORONTO Faculty of Arts and Scence December 005 Examnatons STA47HF/STA005HF Duraton - hours AIDS ALLOWED: (to be suppled by the student) Non-programmable calculator One handwrtten 8.5'' x
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationStatistics II Final Exam 26/6/18
Statstcs II Fnal Exam 26/6/18 Academc Year 2017/18 Solutons Exam duraton: 2 h 30 mn 1. (3 ponts) A town hall s conductng a study to determne the amount of leftover food produced by the restaurants n the
More informationSTATISTICS QUESTIONS. Step by Step Solutions.
STATISTICS QUESTIONS Step by Step Solutons www.mathcracker.com 9//016 Problem 1: A researcher s nterested n the effects of famly sze on delnquency for a group of offenders and examnes famles wth one to
More informationScatter Plot x
Construct a scatter plot usng excel for the gven data. Determne whether there s a postve lnear correlaton, negatve lnear correlaton, or no lnear correlaton. Complete the table and fnd the correlaton coeffcent
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationChapter 14: Logit and Probit Models for Categorical Response Variables
Chapter 4: Logt and Probt Models for Categorcal Response Varables Sect 4. Models for Dchotomous Data We wll dscuss only ths secton of Chap 4, whch s manly about Logstc Regresson, a specal case of the famly
More informationCredit Card Pricing and Impact of Adverse Selection
Credt Card Prcng and Impact of Adverse Selecton Bo Huang and Lyn C. Thomas Unversty of Southampton Contents Background Aucton model of credt card solctaton - Errors n probablty of beng Good - Errors n
More information/ n ) are compared. The logic is: if the two
STAT C141, Sprng 2005 Lecture 13 Two sample tests One sample tests: examples of goodness of ft tests, where we are testng whether our data supports predctons. Two sample tests: called as tests of ndependence
More informationChapter 15 Student Lecture Notes 15-1
Chapter 15 Student Lecture Notes 15-1 Basc Busness Statstcs (9 th Edton) Chapter 15 Multple Regresson Model Buldng 004 Prentce-Hall, Inc. Chap 15-1 Chapter Topcs The Quadratc Regresson Model Usng Transformatons
More informationMultinomial logit regression
07/0/6 Multnomal logt regresson Introducton We now turn our attenton to regresson models for the analyss of categorcal dependent varables wth more than two response categores: Y car owned (many possble
More informationChapter 8 Multivariate Regression Analysis
Chapter 8 Multvarate Regresson Analyss 8.3 Multple Regresson wth K Independent Varables 8.4 Sgnfcance tests of Parameters Populaton Regresson Model For K ndependent varables, the populaton regresson and
More informationInterval Estimation in the Classical Normal Linear Regression Model. 1. Introduction
ECONOMICS 35* -- NOTE 7 ECON 35* -- NOTE 7 Interval Estmaton n the Classcal Normal Lnear Regresson Model Ths note outlnes the basc elements of nterval estmaton n the Classcal Normal Lnear Regresson Model
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationTests of Single Linear Coefficient Restrictions: t-tests and F-tests. 1. Basic Rules. 2. Testing Single Linear Coefficient Restrictions
ECONOMICS 35* -- NOTE ECON 35* -- NOTE Tests of Sngle Lnear Coeffcent Restrctons: t-tests and -tests Basc Rules Tests of a sngle lnear coeffcent restrcton can be performed usng ether a two-taled t-test
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationSimulated Power of the Discrete Cramér-von Mises Goodness-of-Fit Tests
Smulated of the Cramér-von Mses Goodness-of-Ft Tests Steele, M., Chaselng, J. and 3 Hurst, C. School of Mathematcal and Physcal Scences, James Cook Unversty, Australan School of Envronmental Studes, Grffth
More informationLinear regression. Regression Models. Chapter 11 Student Lecture Notes Regression Analysis is the
Chapter 11 Student Lecture Notes 11-1 Lnear regresson Wenl lu Dept. Health statstcs School of publc health Tanjn medcal unversty 1 Regresson Models 1. Answer What Is the Relatonshp Between the Varables?.
More informationDurban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications
Durban Watson for Testng the Lack-of-Ft of Polynomal Regresson Models wthout Replcatons Ruba A. Alyaf, Maha A. Omar, Abdullah A. Al-Shha ralyaf@ksu.edu.sa, maomar@ksu.edu.sa, aalshha@ksu.edu.sa Department
More informationECONOMETRICS - FINAL EXAM, 3rd YEAR (GECO & GADE)
ECONOMETRICS - FINAL EXAM, 3rd YEAR (GECO & GADE) June 7, 016 15:30 Frst famly name: Name: DNI/ID: Moble: Second famly Name: GECO/GADE: Instructor: E-mal: Queston 1 A B C Blank Queston A B C Blank Queston
More informationCorrelation and Regression. Correlation 9.1. Correlation. Chapter 9
Chapter 9 Correlaton and Regresson 9. Correlaton Correlaton A correlaton s a relatonshp between two varables. The data can be represented b the ordered pars (, ) where s the ndependent (or eplanator) varable,
More information2016 Wiley. Study Session 2: Ethical and Professional Standards Application
6 Wley Study Sesson : Ethcal and Professonal Standards Applcaton LESSON : CORRECTION ANALYSIS Readng 9: Correlaton and Regresson LOS 9a: Calculate and nterpret a sample covarance and a sample correlaton
More informationIntroduction to Regression
Introducton to Regresson Dr Tom Ilvento Department of Food and Resource Economcs Overvew The last part of the course wll focus on Regresson Analyss Ths s one of the more powerful statstcal technques Provdes
More informationSTAT 3008 Applied Regression Analysis
STAT 3008 Appled Regresson Analyss Tutoral : Smple Lnear Regresson LAI Chun He Department of Statstcs, The Chnese Unversty of Hong Kong 1 Model Assumpton To quantfy the relatonshp between two factors,
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More informationContinuous vs. Discrete Goods
CE 651 Transportaton Economcs Charsma Choudhury Lecture 3-4 Analyss of Demand Contnuous vs. Dscrete Goods Contnuous Goods Dscrete Goods x auto 1 Indfference u curves 3 u u 1 x 1 0 1 bus Outlne Data Modelng
More informationLogistic Regression Maximum Likelihood Estimation
Harvard-MIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla Ohno-Machado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall
More informationDesigning a Pseudo R-Squared Goodness-of-Fit Measure in Generalized Linear Models
Desgnng a Pseudo R-Squared Goodness-of-Ft Measure n Generalzed Lnear Models H. I. Mbachu Dept. of Mathematcs/Statstcs, Unversty of Port Harcourt, Port Harcourt E. C. Nduka Dept. of Mathematcs/Statstcs,
More informationHere is the rationale: If X and y have a strong positive relationship to one another, then ( x x) will tend to be positive when ( y y)
Secton 1.5 Correlaton In the prevous sectons, we looked at regresson and the value r was a measurement of how much of the varaton n y can be attrbuted to the lnear relatonshp between y and x. In ths secton,
More informationComputation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models
Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,
More informationASYMPTOTIC PROPERTIES OF ESTIMATES FOR THE PARAMETERS IN THE LOGISTIC REGRESSION MODEL
Asymptotc Asan-Afrcan Propertes Journal of Estmates Economcs for and the Econometrcs, Parameters n Vol. the Logstc, No., Regresson 20: 65-74 Model 65 ASYMPTOTIC PROPERTIES OF ESTIMATES FOR THE PARAMETERS
More information9. Binary Dependent Variables
9. Bnar Dependent Varables 9. Homogeneous models Log, prob models Inference Tax preparers 9.2 Random effects models 9.3 Fxed effects models 9.4 Margnal models and GEE Appendx 9A - Lkelhood calculatons
More informationLimited Dependent Variables and Panel Data. Tibor Hanappi
Lmted Dependent Varables and Panel Data Tbor Hanapp 30.06.2010 Lmted Dependent Varables Dscrete: Varables that can take onl a countable number of values Censored/Truncated: Data ponts n some specfc range
More information4.3 Poisson Regression
of teratvely reweghted least squares regressons (the IRLS algorthm). We do wthout gvng further detals, but nstead focus on the practcal applcaton. > glm(survval~log(weght)+age, famly="bnomal", data=baby)
More informationChapter 12 Analysis of Covariance
Chapter Analyss of Covarance Any scentfc experment s performed to know somethng that s unknown about a group of treatments and to test certan hypothess about the correspondng treatment effect When varablty
More informationMACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression
11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING
More informationChapter 5 Multilevel Models
Chapter 5 Multlevel Models 5.1 Cross-sectonal multlevel models 5.1.1 Two-level models 5.1.2 Multple level models 5.1.3 Multple level modelng n other felds 5.2 Longtudnal multlevel models 5.2.1 Two-level
More information28. SIMPLE LINEAR REGRESSION III
8. SIMPLE LINEAR REGRESSION III Ftted Values and Resduals US Domestc Beers: Calores vs. % Alcohol To each observed x, there corresponds a y-value on the ftted lne, y ˆ = βˆ + βˆ x. The are called ftted
More informationCHAPTER 8. Exercise Solutions
CHAPTER 8 Exercse Solutons 77 Chapter 8, Exercse Solutons, Prncples of Econometrcs, 3e 78 EXERCISE 8. When = N N N ( x x) ( x x) ( x x) = = = N = = = N N N ( x ) ( ) ( ) ( x x ) x x x x x = = = = Chapter
More informationClassification as a Regression Problem
Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class
More informationComparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method
Appled Mathematcal Scences, Vol. 7, 0, no. 47, 07-0 HIARI Ltd, www.m-hkar.com Comparson of the Populaton Varance Estmators of -Parameter Exponental Dstrbuton Based on Multple Crtera Decson Makng Method
More informationECON 351* -- Note 23: Tests for Coefficient Differences: Examples Introduction. Sample data: A random sample of 534 paid employees.
Model and Data ECON 35* -- NOTE 3 Tests for Coeffcent Dfferences: Examples. Introducton Sample data: A random sample of 534 pad employees. Varable defntons: W hourly wage rate of employee ; lnw the natural
More informationDecision Analysis (part 2 of 2) Review Linear Regression
Harvard-MIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla Ohno-Machado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationIssues To Consider when Estimating Health Care Costs with Generalized Linear Models (GLMs): To Gamma/Log Or Not To Gamma/Log? That Is The New Question
Issues To Consder when Estmatng Health Care Costs wth Generalzed Lnear Models (GLMs): To Gamma/Log Or Not To Gamma/Log? That Is The New Queston ISPOR 20th Annual Internatonal Meetng May 19, 2015 Jalpa
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationSTK4080/9080 Survival and event history analysis
SK48/98 Survval and event hstory analyss Lecture 7: Regresson modellng Relatve rsk regresson Regresson models Assume that we have a sample of n ndvduals, and let N (t) count the observed occurrences of
More informationAdvanced Statistical Methods: Beyond Linear Regression
Advanced Statstcal Methods: Beyond Lnear Regresson John R. Stevens Utah State Unversty Notes 2. Statstcal Methods I Mathematcs Educators Workshop 28 March 2009 1 http://www.stat.usu.edu/~rstevens/pcm 2
More informationLINEAR REGRESSION ANALYSIS. MODULE VIII Lecture Indicator Variables
LINEAR REGRESSION ANALYSIS MODULE VIII Lecture - 7 Indcator Varables Dr. Shalabh Department of Maematcs and Statstcs Indan Insttute of Technology Kanpur Indcator varables versus quanttatve explanatory
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationIntroduction to Econometrics (3 rd Updated Edition, Global Edition) Solutions to Odd-Numbered End-of-Chapter Exercises: Chapter 13
Introducton to Econometrcs (3 rd Updated Edton, Global Edton by James H. Stock and Mark W. Watson Solutons to Odd-Numbered End-of-Chapter Exercses: Chapter 13 (Ths verson August 17, 014 Stock/Watson -
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Winter 2017 Instructor: Victor Aguirregabiria
ECOOMETRICS II ECO 40S Unversty of Toronto Department of Economcs Wnter 07 Instructor: Vctor Agurregabra SOLUTIO TO FIAL EXAM Tuesday, Aprl 8, 07 From :00pm-5:00pm 3 hours ISTRUCTIOS: - Ths s a closed-book
More informationx = , so that calculated
Stat 4, secton Sngle Factor ANOVA notes by Tm Plachowsk n chapter 8 we conducted hypothess tests n whch we compared a sngle sample s mean or proporton to some hypotheszed value Chapter 9 expanded ths to
More informationLearning Objectives for Chapter 11
Chapter : Lnear Regresson and Correlaton Methods Hldebrand, Ott and Gray Basc Statstcal Ideas for Managers Second Edton Learnng Objectves for Chapter Usng the scatterplot n regresson analyss Usng the method
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More information