Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
|
|
- Clarissa Nichols
- 5 years ago
- Views:
Transcription
1 Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010
2 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton Smulaton study Concluson
3 2 Introducton Z: a vector of random varables wth dstrbuton F (z; θ). z 1,, z n are n ndependent realzatons of Z. Two types of parameters of nterest 1 θ: the parameter n the model F (z; θ) 2 η g = E θ {g(z)}: nduced parameter from θ. η g can be computed n two ways: 1 Maxmum lkelhood method: ˆη g = η g (ˆθ MLE ) 2 Smple method: ˆθ g = n 1 n =1 g (z )
4 3 Introducton Suppose that z s not fully observed. z = (z obs,, z ms, ): (observed, mssng) part of z Under the mssng-at-random assumpton, we can use the followng estmators: ˆθ: soluton to n E {S (θ; z ) z obs, } = 0 =1 ( ) ˆη g = n 1 n E {g (z ) z obs, } =1 where S (θ; z ) = ln f (z ; θ) / θ s the score functon of θ under complete response. The equaton n (*) s called mean score equaton.
5 4 Introducton Under some regularty condtons, the soluton ˆθ to the mean score equaton maxmzes the observed lkelhood (= the lkelhood assocated wth the margnal dstrbuton of z obs, ). Computng the condtonal expectaton can be a challengng problem. 1 Do not know θ n E {g (z ) z obs, } = E {g (z ) z obs, ; θ}. 2 Even f we know θ, computng the condtonal expectaton s numercally dffcult.
6 5 Introducton Imputaton: Monte Carlo approxmaton of the condtonal expectaton E { S (θ; z ) z obs, ; ˆθ } = 1 M { E g (z ) z obs, ; ˆθ } = 1 M ( where z (j) z ms, z obs, ; ˆθ ) M S j=1 M g j=1 ( ) θ; z obs,, z (j) ms, ( ) z obs,, z (j) ms, ms, f. By provdng mputed data, the estmates are very easy to compute and dfferent users can get consstent results. Very attractve when η g s unknown at the tme of mputaton (e.g. publc-access data n survey samplng).
7 6 Introducton To compute ˆθ, EM algorthm can be used. ˆθ (t+1) : soluton to M 1 n M S =1 j=1 ( ) where zj(t) = z obs,, z (j) ms,(t) wth ( ) θ; zj(t) = 0 z (j) ms,(t) f (z ms, z obs, ; ˆθ (t)). (1) Computatonally heavy f (1) requres MCMC for each t. Convergence s hard to be acheved (unless M s ncreased) snce the mputed values are re-generated for each teraton.
8 7 Proposed method: Fractonal mputaton Fractonal mputaton 1 More than one (say M) mputed values of z ms, : z (1) ms,,, z (M) ms, from some (ntal) densty h (y,ms ). 2 Create weghted data set {( w j, zj ) } ; j = 1, 2,, M; = 1, 2, n where M j=1 w j = 1, z j = (z obs,, z (j) ms, ) w j f (z j ; ˆθ)/h(z (j),ms ), ˆθ s the maxmum lkelhood estmator of θ, and f (z; θ) s the jont densty of z. 3 The weght wj are the normalzed mportance weghts and can be called fractonal weghts.
9 8 Proposed method: Fractonal mputaton Product: fractonally mputed data set of sze nm {( w j, zj ) } ; j = 1, 2,, M; = 1, 2, n Property: for suffcently large M, M j=1 wj g ( f (z ;ˆθ) zj ) = h(z,ms ) g(z )h(z,ms )dz,ms f (z ;ˆθ) h(z,ms ) h(z,ms)dz,ms for any g such that the expectaton exsts. = E { } g (z ) z,obs ; ˆθ If we choose h(z,ms ) = f (z,ms z,obs, ˆθ) where ˆθ s the MLE, then t s equal to the usual Monte Carlo mputaton for maxmum lkelhood estmaton.
10 9 Proposed method: Fractonal mputaton EM algorthm by fractonal mputaton 1 Intal mputaton: generate z (j) ms, h (y,ms ). 2 E-step: compute where M j=1 w j(t) = 1. 3 M-step: update w j(t) f (z j ; ˆθ (t) )/h(z (j),ms ) ˆθ (t+1) : soluton to n M =1 j=1 w j(t) S ( θ; z j ) = 0. 4 Repeat Step2 and Step 3 untl convergence.
11 10 Proposed method: Fractonal mputaton If we set h( ) ndependent of θ, then the mputed values are not changed for each teraton. Only the fractonal weghts are changed. 1 Computatonally effcent (because we use mportance samplng only once). 2 Convergence s acheved (because the mputed values are not changed). For suffcently large t, ˆθ (t) ˆθ. Also, for suffcently large M, ˆθ ˆθ MLE. Thus, we need to use bg M for satsfactory approxmaton.
12 11 Approxmaton: Calbraton Fractonal mputaton In large scale survey samplng, we prefer to have smaller M. Two-step method for fractonal mputaton: 1 Create a set of fractonally mputed data wth sze nm 1, (say M 1 = 500). 2 Use an effcent samplng and weghtng method to get a fnal set of fractonally mputed data wth sze nm 2, (say M 2 = 10). Thus, we treat the step-one mputed data as a fnte populaton and the step-two mputed data as a sample. We can use effcent samplng technque (such as systematc samplng or stratfcaton) to get a fnal mputed data and use calbraton technque for fractonal weghtng.
13 Approxmaton: Calbraton Fractonal mputaton Step-One data set (of sze nm 1 ): {( w j, zj ) ; j = 1, 2,, M1 ; = 1, 2, n } and the fractonal weghts satsfy M 1 j=1 w j = 1 and n M 1 ) wj S (ˆθ; zj = 0 =1 j=1 where ˆθ s obtaned from the EM algorthm after convergence. The fnal fractonally mputed data set can be wrtten {( w j, z j ) ; j = 1, 2,, M2 ; = 1, 2, n } and the fractonal weghts satsfy M 2 j=1 w j = 1 and n M 2 =1 j=1 ) w j S (ˆθ; z j = 0 12
14 13 Approxmaton: Calbraton Fractonal mputaton Thus, fractonal weghts can be constructed by the calbraton technques n survey samplng. If the dstrbuton belongs to the exponental famly, then the calbraton constrants can be smplfed to n M 2 =1 j=1 w j T ( z j ) = n M 1 wj T (zj ) =1 j=1 where T (z) s the complete suffcent statstc for θ.
15 14 Varance estmaton for fractonal mputaton Wrte where ˆη g,fi = ˆη g,fi (ˆθ) n 1 S(ˆθ) n M =1 j=1 n M =1 j=1 wj (ˆθ)g ( zj ) w j (ˆθ)S(ˆθ; z j ) = 0. Taylor lnearzaton { } η g,fi ) = ˆηg,FI (θ 0 ) θ ˆη g,fi (θ 0 ) (ˆθ θ0 { } 0 = S(ˆθ) = S(θ 0 ) + θ S(θ ) 0 ) (ˆθ θ 0
16 15 Varance estmaton for fractonal mputaton Combne the two η g,fi = ˆηg,FI (θ 0 ) = n 1 n M =1 j=1 n = n 1 ē (θ 0 ) =1 { } { } 1 θ ˆη g,fi (θ 0 ) θ S(θ 0 ) S(θ0 ) wj (θ 0 ) { g ( zj ) K S(θ 0 ; zj ) } where K = { θ ˆη g,fi (θ 0 ) } { S(θ θ 0 ) } 1 and ē (θ 0 ) are IID random varables. Plug-n estmator for the lnearzed varance can be used.
17 16 Multple mputaton Generate M mputed values (wth equal weghts) Features 1 Imputed values are generated from z (j),ms f (z,ms z,obs, θ ) where θ s generated from the posteror dstrbuton π (θ z,obs ). 2 Varance estmaton formula s smple. ˆV MI ( η g,m ) = 1 M ( ˆV I (m) ) 1 M ) 2 (ˆηg(m) η g,m M M M 1 m=1 m=1 where η g,m = M 1 M m=1 ˆη g(m) s the average of M mputed estmators and ˆV I (m) s the mputed verson of the varance estmator of ˆη g under complete response.
18 17 Multple mputaton The computaton for Bayesan mputaton can be mplemented by the data augmentaton (Tanner and Wong, 1987) technque, whch s a specal applcaton of the Gbb s samplng method: 1 I-step: Generate z ms f (z ms z obs, θ ) 2 P-step: Generate θ g (θ z obs, z ms ) Consstency of varance estmator s questonable (Km et al, 2006, JRSSB). 1 If η g = η g (θ), then the varance estmator wth large M s consstent. 2 If η g η g (θ), then the varance estmator s not consstent.
19 18 Smulaton study: smulaton setup B = 2, 000 smulaton samples of sze n = 200 are generated wth x N (2, 1) y x N (β 0 + β 1 x, σ ee ) z (x, y ) Bernoull(p ) where (β 0, β 1 ) = (1, 0.7), σ ee = 1, p = exp (ψ 0 + ψ 1 x + ψ 2 y ) 1 + exp (ψ 0 + ψ 1 x + ψ 2 y ) wth (ψ 0, ψ 1, ψ 2 ) = ( 3, 0.5, 0.7).
20 19 Smulaton study: smulaton setup x s always observed. y s observed f δ 1 = 1 and y s mssng f δ 1 = 0, where and wth (ϕ 0, ϕ 1 ) = (0, 0.5). δ 1 Bernoull(π 1 ) π 1 = exp (ϕ 0 + ϕ 1 x ) 1 + exp (ϕ 0 + ϕ 1 x ) z s observed f δ 2 = 1 and z s mssng f δ 2 = 0, where δ 2 Bernoull(0.7). The response mechansm s mssng at random.
21 20 Smulaton study: fractonal mputaton 1 Intal mputaton: a Ft a parametrc model f 1 (y x, θ 1 ) for the condtonal dstrbuton of y gven x among respondents of y. In ths smulaton setup, we use the followng model y (x, δ 1 = 1) N (β 0 + β 1 x, σ ee ) for some θ 1 = (β 0, β 1, σ ee ). b Estmate parameter θ 1 usng the samples wth δ 1 = 1 only. c For each unt wth δ 1 = 0, generate M mputed values of y, say y (1),, y (M), from the estmated densty f 1 (y x, ˆθ 1 ).
22 21 Smulaton study: fractonal mputaton 1 Intal mputaton (cont d): d Smlarly, ft a parametrc model f 2 (z x, y, θ 2 ) for the condtonal dstrbuton of z gven x and y among z-respondents. e Estmate parameter θ 2 usng the samples wth δ 2 = 1 only. f For each unt wth δ 2 = 0, generate M mputed values of z by f 2 (z x, y, ˆθ 2 ). z (j)
23 22 Smulaton study: fractonal mputaton 2 Fractonal weghtng (E-step): For the current parameter estmate ˆθ (t) = (ˆθ 1(t), ˆθ 2(t) ) where ˆθ 1(t) = ( ˆβ 0(t), ˆβ 1(t), ˆσ ee(t) ) and ˆθ 2(t) = ( ˆψ 0(t), ˆψ 1(t), ˆψ 2(t) ), compute the fractonal weghts assocated wth the mputed values. The ( fractonal weghts assocate wth w y (j) ), z (j) are ( ) ( ) f 1 y (j) j(t) x, ˆθ 1(t) f 2 z (j) x, y (j), ˆθ 2(t) ( ) ( ), (2) f 1 y (j) x, ˆθ 1(0) f 2 z (j) x, y (j), ˆθ 2(0) where f 1 (y x, θ 1 ) f 2 (z x, y, θ 2 ) s the jont densty of (y, z) gven x. In (2), t s understood that y (j) = y f δ 1 = 1 and z (j) = z f δ 2 = 1.
24 23 Smulaton study: fractonal mputaton 3 Update parameter estmates (M-step): Usng the current fractonal weghts, compute the maxmum lkelhood estmator to update ˆθ (t+1) by solvng the followng mputed score equatons: S (t) (θ) n =1 j=1 M ( ) wj(t) S θ; x, y (j), z (j) = 0 The soluton to the above equaton can be obtaned usng the exstng software.
25 24 Smulaton study: fractonal mputaton Usng the fnal fractonal weghts after convergence, we can estmate η g = E {g(x, y, z)} by ˆη g = 1 n n M =1 j=1 w j g(x, y (j), z (j) ) where y (j) = y f δ 1 = 1 and z (j) = z f δ 2 = 1. For fractonal mputaton wth M = 10, calbraton method s used wth an ntal mputaton of sze M 1 = 100. The followng parameters were consdered n the smulaton study: 1 η 1 = E(y) 2 η 2 = Pr (y < 3).
26 25 Smulaton study: result Table 1 Monte Carlo bas and varance of the pont estmators. Parameter Estmator Bas Varance Std Var Complete sample η 1 FI (M = 100) MI (M = 100) FI (M = 10) MI (M = 10) Complete sample η 2 FI (M = 100) MI (M = 100) FI (M = 10) MI (M = 10)
27 26 Smulaton study: result Table 2 Monte Carlo relatve bas of the varance estmator. Parameter Imputaton Relatve bas (%) FI (M = 100) 3.7 V (ˆη 1 ) MI (M = 100) 4.3 FI (M = 10) 2.7 MI (M = 10) 4.2 FI (M = 100) 3.9 V (ˆη 2 ) MI (M = 100) 15.6 FI (M = 10) 2.3 MI (M = 10) 15.5
28 27 Smulaton study: result For the estmaton of E(Y ), fractonal mputaton s more effcent. For the estmaton of proportons, multple mputaton s slghtly more effcent. However, multple mputaton provdes based varance estmates for proportons. Note that we are not usng the MLE for η 2. Under complete response, ˆη 2 = 1 n n I (y < 3) =1 s less effcent than the maxmum lkelhood estmator. The MI varance estmator s justfed only for the MLE s.
29 28 Concluson Fractonal mputaton s proposed as a tool for computng the condtonal expectaton of any functon of orgnal data gven the observed data. Fractonal mputaton can be used to mplement the Monte Carlo EM algorthm effcently n a computatonally effcent manner. Varance estmaton based on Taylor lnearzaton s also covered. Further detals can be found n Km (2010, Bometrka, tentatvely accepted).
30 29 Future Research Survey samplng applcaton (wth multvarate mssng data) Nonparametrc or sem-parametrc fractonal mputaton. Measurement error models Random effect models (wth possble applcatons n small area estmaton) More theoretcal work on nference (Wlk s theorem?)
Parametric fractional imputation for missing data analysis
Secton on Survey Research Methods JSM 2008 Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Wayne Fuller Abstract Under a parametrc model for mssng data, the EM algorthm s a popular tool
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More informationarxiv: v1 [stat.me] 27 Aug 2015
Submtted to Statstcal Scence Fractonal Imputaton n Survey Samplng: A Comparatve Revew Shu Yang and Jae Kwang Km Harvard Unversty and Iowa State Unversty arxv:1508.06945v1 [stat.me] 27 Aug 2015 Abstract.
More informationEfficient nonresponse weighting adjustment using estimated response probability
Effcent nonresponse weghtng adjustment usng estmated response probablty Jae Kwang Km Department of Appled Statstcs, Yonse Unversty, Seoul, 120-749, KOREA Key Words: Regresson estmator, Propensty score,
More informationMarkov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement
Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs
More informationOn an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems. Vahid Tadayon 1
On an Extenson of Stochastc Approxmaton EM Algorthm for Incomplete Data Problems Vahd Tadayon Abstract: The Stochastc Approxmaton EM (SAEM algorthm, a varant stochastc approxmaton of EM, s a versatle tool
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationSmall Area Interval Estimation
.. Small Area Interval Estmaton Partha Lahr Jont Program n Survey Methodology Unversty of Maryland, College Park (Based on jont work wth Masayo Yoshmor, Former JPSM Vstng PhD Student and Research Fellow
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors
Stat60: Bayesan Modelng and Inference Lecture Date: February, 00 Reference Prors Lecturer: Mchael I. Jordan Scrbe: Steven Troxler and Wayne Lee In ths lecture, we assume that θ R; n hgher-dmensons, reference
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More informationComputing MLE Bias Empirically
Computng MLE Bas Emprcally Kar Wa Lm Australan atonal Unversty January 3, 27 Abstract Ths note studes the bas arses from the MLE estmate of the rate parameter and the mean parameter of an exponental dstrbuton.
More informationMaximum Likelihood Estimation
Maxmum Lkelhood Estmaton INFO-2301: Quanttatve Reasonng 2 Mchael Paul and Jordan Boyd-Graber MARCH 7, 2017 INFO-2301: Quanttatve Reasonng 2 Paul and Boyd-Graber Maxmum Lkelhood Estmaton 1 of 9 Why MLE?
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationStat 543 Exam 2 Spring 2016
Stat 543 Exam 2 Sprng 206 I have nether gven nor receved unauthorzed assstance on ths exam. Name Sgned Date Name Prnted Ths Exam conssts of questons. Do at least 0 of the parts of the man exam. I wll score
More informationOn Outlier Robust Small Area Mean Estimate Based on Prediction of Empirical Distribution Function
On Outler Robust Small Area Mean Estmate Based on Predcton of Emprcal Dstrbuton Functon Payam Mokhtaran Natonal Insttute of Appled Statstcs Research Australa Unversty of Wollongong Small Area Estmaton
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationEM and Structure Learning
EM and Structure Learnng Le Song Machne Learnng II: Advanced Topcs CSE 8803ML, Sprng 2012 Partally observed graphcal models Mxture Models N(μ 1, Σ 1 ) Z X N N(μ 2, Σ 2 ) 2 Gaussan mxture model Consder
More informationStat 543 Exam 2 Spring 2016
Stat 543 Exam 2 Sprng 2016 I have nether gven nor receved unauthorzed assstance on ths exam. Name Sgned Date Name Prnted Ths Exam conssts of 11 questons. Do at least 10 of the 11 parts of the man exam.
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More information4 Analysis of Variance (ANOVA) 5 ANOVA. 5.1 Introduction. 5.2 Fixed Effects ANOVA
4 Analyss of Varance (ANOVA) 5 ANOVA 51 Introducton ANOVA ANOVA s a way to estmate and test the means of multple populatons We wll start wth one-way ANOVA If the populatons ncluded n the study are selected
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationSIO 224. m(r) =(ρ(r),k s (r),µ(r))
SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small
More informationECONOMICS 351*-A Mid-Term Exam -- Fall Term 2000 Page 1 of 13 pages. QUEEN'S UNIVERSITY AT KINGSTON Department of Economics
ECOOMICS 35*-A Md-Term Exam -- Fall Term 000 Page of 3 pages QUEE'S UIVERSITY AT KIGSTO Department of Economcs ECOOMICS 35* - Secton A Introductory Econometrcs Fall Term 000 MID-TERM EAM ASWERS MG Abbott
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationBias-correction under a semi-parametric model for small area estimation
Bas-correcton under a sem-parametrc model for small area estmaton Laura Dumtrescu, Vctora Unversty of Wellngton jont work wth J. N. K. Rao, Carleton Unversty ICORS 2017 Workshop on Robust Inference for
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationA note on multiple imputation for method of moments estimation
Statstcs Publcatons Statstcs 2-2016 A note on multple mputaton for method of moments estmaton Shu Yang Harvard Unversty Jae Kwang Km Iowa State Unversty, jkm@astate.edu Follow ths and addtonal works at:
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationNon-Mixture Cure Model for Interval Censored Data: Simulation Study ABSTRACT
Malaysan Journal of Mathematcal Scences 8(S): 37-44 (2014) Specal Issue: Internatonal Conference on Mathematcal Scences and Statstcs 2013 (ICMSS2013) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Journal
More informationAn adaptive SMC scheme for ABC. Bayesian Computation (ABC)
An adaptve SMC scheme for Approxmate Bayesan Computaton (ABC) (ont work wth Prof. Mke West) Department of Statstcal Scence - Duke Unversty Aprl/2011 Approxmate Bayesan Computaton (ABC) Problems n whch
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationAppendix B. The Finite Difference Scheme
140 APPENDIXES Appendx B. The Fnte Dfference Scheme In ths appendx we present numercal technques whch are used to approxmate solutons of system 3.1 3.3. A comprehensve treatment of theoretcal and mplementaton
More informationConjugacy and the Exponential Family
CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the
More informationThe EM Algorithm (Dempster, Laird, Rubin 1977) The missing data or incomplete data setting: ODL(φ;Y ) = [Y;φ] = [Y X,φ][X φ] = X
The EM Algorthm (Dempster, Lard, Rubn 1977 The mssng data or ncomplete data settng: An Observed Data Lkelhood (ODL that s a mxture or ntegral of Complete Data Lkelhoods (CDL. (1a ODL(;Y = [Y;] = [Y,][
More informationASYMPTOTIC PROPERTIES OF ESTIMATES FOR THE PARAMETERS IN THE LOGISTIC REGRESSION MODEL
Asymptotc Asan-Afrcan Propertes Journal of Estmates Economcs for and the Econometrcs, Parameters n Vol. the Logstc, No., Regresson 20: 65-74 Model 65 ASYMPTOTIC PROPERTIES OF ESTIMATES FOR THE PARAMETERS
More informationAn R implementation of bootstrap procedures for mixed models
The R User Conference 2009 July 8-10, Agrocampus-Ouest, Rennes, France An R mplementaton of bootstrap procedures for mxed models José A. Sánchez-Espgares Unverstat Poltècnca de Catalunya Jord Ocaña Unverstat
More informationStatistical registers by restricted neighbor imputation
Statstcal regsters by restrcted neghbor mputaton an applcaton to the Norwegan Agrculture Survey Abstract Nna Hagesæther 1 and L-Chun Zhang Statstcs Norway In ths paper we mplement the method of Zhang and
More informationFinite Mixture Models and Expectation Maximization. Most slides are from: Dr. Mario Figueiredo, Dr. Anil Jain and Dr. Rong Jin
Fnte Mxture Models and Expectaton Maxmzaton Most sldes are from: Dr. Maro Fgueredo, Dr. Anl Jan and Dr. Rong Jn Recall: The Supervsed Learnng Problem Gven a set of n samples X {(x, y )},,,n Chapter 3 of
More informationNonparametric model calibration estimation in survey sampling
Ames February 18, 004 Nonparametrc model calbraton estmaton n survey samplng M. Govanna Ranall Department of Statstcs, Colorado State Unversty (Jont work wth G.E. Montanar, Dpartmento d Scenze Statstche,
More informationEstimation of the Mean of Truncated Exponential Distribution
Journal of Mathematcs and Statstcs 4 (4): 84-88, 008 ISSN 549-644 008 Scence Publcatons Estmaton of the Mean of Truncated Exponental Dstrbuton Fars Muslm Al-Athar Department of Mathematcs, Faculty of Scence,
More informationA Hybrid Variational Iteration Method for Blasius Equation
Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 1932-9466 Vol. 10, Issue 1 (June 2015), pp. 223-229 Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) A Hybrd Varatonal Iteraton Method
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationEfficient estimation in missing data and survey sampling problems
Graduate Theses and Dssertatons Iowa State Unversty Capstones, Theses and Dssertatons 2012 Effcent estmaton n mssng data and survey samplng problems Sxa Chen Iowa State Unversty Follow ths and addtonal
More informationECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Winter 2017 Instructor: Victor Aguirregabiria
ECOOMETRICS II ECO 40S Unversty of Toronto Department of Economcs Wnter 07 Instructor: Vctor Agurregabra SOLUTIO TO FIAL EXAM Tuesday, Aprl 8, 07 From :00pm-5:00pm 3 hours ISTRUCTIOS: - Ths s a closed-book
More informationRepresentation Theorem for Convex Nonparametric Least Squares. Timo Kuosmanen
Representaton Theorem or Convex Nonparametrc Least Squares Tmo Kuosmanen 4th Nordc Econometrc Meetng, Tartu, Estona, 4-6 May 007 Motvaton Inerences oten depend crtcally upon the algebrac orm chosen. It
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VIII LECTURE - 34 ANALYSIS OF VARIANCE IN RANDOM-EFFECTS MODEL AND MIXED-EFFECTS EFFECTS MODEL Dr Shalabh Department of Mathematcs and Statstcs Indan
More informationPrimer on High-Order Moment Estimators
Prmer on Hgh-Order Moment Estmators Ton M. Whted July 2007 The Errors-n-Varables Model We wll start wth the classcal EIV for one msmeasured regressor. The general case s n Erckson and Whted Econometrc
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationGlobal Sensitivity. Tuesday 20 th February, 2018
Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values
More informationTesting for seasonal unit roots in heterogeneous panels
Testng for seasonal unt roots n heterogeneous panels Jesus Otero * Facultad de Economía Unversdad del Rosaro, Colomba Jeremy Smth Department of Economcs Unversty of arwck Monca Gulett Aston Busness School
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationLogistic regression models 1/12
Logstc regresson models 1/12 2/12 Example 1: dogs look lke ther owners? Some people beleve that dogs look lke ther owners. Is ths true? To test the above hypothess, The New York Tmes conducted a quz onlne.
More informationPopulation Design in Nonlinear Mixed Effects Multiple Response Models: extension of PFIM and evaluation by simulation with NONMEM and MONOLIX
Populaton Desgn n Nonlnear Mxed Effects Multple Response Models: extenson of PFIM and evaluaton by smulaton wth NONMEM and MONOLIX May 4th 007 Carolne Bazzol, Sylve Retout, France Mentré Inserm U738 Unversty
More information4.3 Poisson Regression
of teratvely reweghted least squares regressons (the IRLS algorthm). We do wthout gvng further detals, but nstead focus on the practcal applcaton. > glm(survval~log(weght)+age, famly="bnomal", data=baby)
More informationParameters Estimation of the Modified Weibull Distribution Based on Type I Censored Samples
Appled Mathematcal Scences, Vol. 5, 011, no. 59, 899-917 Parameters Estmaton of the Modfed Webull Dstrbuton Based on Type I Censored Samples Soufane Gasm École Supereure des Scences et Technques de Tuns
More informationCourse 395: Machine Learning - Lectures
Course 395: Machne Learnng - Lectures Lecture 1-2: Concept Learnng (M. Pantc Lecture 3-4: Decson Trees & CC Intro (M. Pantc Lecture 5-6: Artfcal Neural Networks (S.Zaferou Lecture 7-8: Instance ased Learnng
More informationDurban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications
Durban Watson for Testng the Lack-of-Ft of Polynomal Regresson Models wthout Replcatons Ruba A. Alyaf, Maha A. Omar, Abdullah A. Al-Shha ralyaf@ksu.edu.sa, maomar@ksu.edu.sa, aalshha@ksu.edu.sa Department
More informationβ0 + β1xi and want to estimate the unknown
SLR Models Estmaton Those OLS Estmates Estmators (e ante) v. estmates (e post) The Smple Lnear Regresson (SLR) Condtons -4 An Asde: The Populaton Regresson Functon B and B are Lnear Estmators (condtonal
More informationHidden Markov Models
Hdden Markov Models Namrata Vaswan, Iowa State Unversty Aprl 24, 204 Hdden Markov Model Defntons and Examples Defntons:. A hdden Markov model (HMM) refers to a set of hdden states X 0, X,..., X t,...,
More information8 : Learning in Fully Observed Markov Networks. 1 Why We Need to Learn Undirected Graphical Models. 2 Structural Learning for Completely Observed MRF
10-708: Probablstc Graphcal Models 10-708, Sprng 2014 8 : Learnng n Fully Observed Markov Networks Lecturer: Erc P. Xng Scrbes: Meng Song, L Zhou 1 Why We Need to Learn Undrected Graphcal Models In the
More informationSolutions Homework 4 March 5, 2018
1 Solutons Homework 4 March 5, 018 Soluton to Exercse 5.1.8: Let a IR be a translaton and c > 0 be a re-scalng. ˆb1 (cx + a) cx n + a (cx 1 + a) c x n x 1 cˆb 1 (x), whch shows ˆb 1 s locaton nvarant and
More informationHidden Markov Models
CM229S: Machne Learnng for Bonformatcs Lecture 12-05/05/2016 Hdden Markov Models Lecturer: Srram Sankararaman Scrbe: Akshay Dattatray Shnde Edted by: TBD 1 Introducton For a drected graph G we can wrte
More informationMarkov Chain Monte Carlo Lecture 6
where (x 1,..., x N ) X N, N s called the populaton sze, f(x) f (x) for at least one {1, 2,..., N}, and those dfferent from f(x) are called the tral dstrbutons n terms of mportance samplng. Dfferent ways
More informationA Robust Method for Calculating the Correlation Coefficient
A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal
More informationInterval Estimation in the Classical Normal Linear Regression Model. 1. Introduction
ECONOMICS 35* -- NOTE 7 ECON 35* -- NOTE 7 Interval Estmaton n the Classcal Normal Lnear Regresson Model Ths note outlnes the basc elements of nterval estmaton n the Classcal Normal Lnear Regresson Model
More informationPHYS 450 Spring semester Lecture 02: Dealing with Experimental Uncertainties. Ron Reifenberger Birck Nanotechnology Center Purdue University
PHYS 45 Sprng semester 7 Lecture : Dealng wth Expermental Uncertantes Ron Refenberger Brck anotechnology Center Purdue Unversty Lecture Introductory Comments Expermental errors (really expermental uncertantes)
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationRELIABILITY ASSESSMENT
CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationStatistical inference for generalized Pareto distribution based on progressive Type-II censored data with random removals
Internatonal Journal of Scentfc World, 2 1) 2014) 1-9 c Scence Publshng Corporaton www.scencepubco.com/ndex.php/ijsw do: 10.14419/jsw.v21.1780 Research Paper Statstcal nference for generalzed Pareto dstrbuton
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More informationChapter 12 Analysis of Covariance
Chapter Analyss of Covarance Any scentfc experment s performed to know somethng that s unknown about a group of treatments and to test certan hypothess about the correspondng treatment effect When varablty
More informationISQS 6348 Final Open notes, no books. Points out of 100 in parentheses. Y 1 ε 2
ISQS 6348 Fnal Open notes, no books. Ponts out of 100 n parentheses. 1. The followng path dagram s gven: ε 1 Y 1 ε F Y 1.A. (10) Wrte down the usual model and assumptons that are mpled by ths dagram. Soluton:
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationENG 8801/ Special Topics in Computer Engineering: Pattern Recognition. Memorial University of Newfoundland Pattern Recognition
EG 880/988 - Specal opcs n Computer Engneerng: Pattern Recognton Memoral Unversty of ewfoundland Pattern Recognton Lecture 7 May 3, 006 http://wwwengrmunca/~charlesr Offce Hours: uesdays hursdays 8:30-9:30
More informationT E C O L O T E R E S E A R C H, I N C.
T E C O L O T E R E S E A R C H, I N C. B rdg n g En g neern g a nd Econo mcs S nce 1973 THE MINIMUM-UNBIASED-PERCENTAGE ERROR (MUPE) METHOD IN CER DEVELOPMENT Thrd Jont Annual ISPA/SCEA Internatonal Conference
More informationWeighted Estimating Equations with Response Propensities in Terms of Covariates Observed only for Responders
Weghted Estmatng Equatons wth Response Propenstes n Terms of Covarates Observed only for Responders Erc V. Slud, U.S. Census Bureau, CSRM Unv. of Maryland, Mathematcs Dept. NISS Mssng Data Workshop, November
More informationComputation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models
Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,
More informationLecture 3 Stat102, Spring 2007
Lecture 3 Stat0, Sprng 007 Chapter 3. 3.: Introducton to regresson analyss Lnear regresson as a descrptve technque The least-squares equatons Chapter 3.3 Samplng dstrbuton of b 0, b. Contnued n net lecture
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationChapter 20 Duration Analysis
Chapter 20 Duraton Analyss Duraton: tme elapsed untl a certan event occurs (weeks unemployed, months spent on welfare). Survval analyss: duraton of nterest s survval tme of a subject, begn n an ntal state
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More information4DVAR, according to the name, is a four-dimensional variational method.
4D-Varatonal Data Assmlaton (4D-Var) 4DVAR, accordng to the name, s a four-dmensonal varatonal method. 4D-Var s actually a drect generalzaton of 3D-Var to handle observatons that are dstrbuted n tme. The
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationSTAT 3008 Applied Regression Analysis
STAT 3008 Appled Regresson Analyss Tutoral : Smple Lnear Regresson LAI Chun He Department of Statstcs, The Chnese Unversty of Hong Kong 1 Model Assumpton To quantfy the relatonshp between two factors,
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationChapter 2 - The Simple Linear Regression Model S =0. e i is a random error. S β2 β. This is a minimization problem. Solution is a calculus exercise.
Chapter - The Smple Lnear Regresson Model The lnear regresson equaton s: where y + = β + β e for =,..., y and are observable varables e s a random error How can an estmaton rule be constructed for the
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationRobust mixture modeling using multivariate skew t distributions
Robust mxture modelng usng multvarate skew t dstrbutons Tsung-I Ln Department of Appled Mathematcs and Insttute of Statstcs Natonal Chung Hsng Unversty, Tawan August, 1 T.I. Ln (NCHU Natonal Chung Hsng
More informationProbabilistic Graphical Models
School of Computer Scence robablstc Graphcal Models Appromate Inference: Markov Chan Monte Carlo 05 07 Erc Xng Lecture 7 March 9 04 X X 075 05 05 03 X 3 Erc Xng @ CMU 005-04 Recap of Monte Carlo Monte
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationSupporting Information
Supportng Informaton The neural network f n Eq. 1 s gven by: f x l = ReLU W atom x l + b atom, 2 where ReLU s the element-wse rectfed lnear unt, 21.e., ReLUx = max0, x, W atom R d d s the weght matrx to
More informationOutline. Multivariate Parametric Methods. Multivariate Data. Basic Multivariate Statistics. Steven J Zeil
Outlne Multvarate Parametrc Methods Steven J Zel Old Domnon Unv. Fall 2010 1 Multvarate Data 2 Multvarate ormal Dstrbuton 3 Multvarate Classfcaton Dscrmnants Tunng Complexty Dscrete Features 4 Multvarate
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationRockefeller College University at Albany
Rockefeller College Unverst at Alban PAD 705 Handout: Maxmum Lkelhood Estmaton Orgnal b Davd A. Wse John F. Kenned School of Government, Harvard Unverst Modfcatons b R. Karl Rethemeer Up to ths pont n
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More information