SEMIPARAMETRIC SINGLEINDEX MODELS. Joel L. Horowitz Department of Economics Northwestern University

 Jasmin Kennedy
 8 months ago
 Views:
Transcription
1 SEMIPARAMETRIC SINGLEINDEX MODELS by Joel L. Horowitz Departmet of Ecoomics Northwester Uiversity
2 INTRODUCTION Much of applied ecoometrics ad statistics ivolves estimatig a coditioal mea fuctio: E ( Y X = x) Y may be cotiuous or biary If biary, the E ( Y X = x) is P ( Y = 1 X = x) I biary respose model, Y may idicate a idividual s choice amog two alteratives, occurrece or ooccurrece of a evet, etc. Possible approaches Fully parametric Fully oparametric Semiparametric
3 FULLY PARAMETRIC MODELING I fully parametric model, E ( Y X = x) is kow up to a fiitedimesioal parameter: E ( Y = 1 X = x) = F( x, θ ) F is kow fuctio θ is ukow, fiitedimesioal parameter Example: biary probit or logit model Advatages: If F is correctly specified Maximizes estimatio efficiecy Permits extrapolatio of x beyod rage of data Ofte has atural behavioral iterpretatio Disadvatages: F rarely kow i applicatios Ca be highly misleadig if F is misspecified
4 FULLY NONPARAMETRIC MODELING E ( Y X = x) G( x) assumed to be smooth fuctio of x Nothig assumed about shape of G. G estimated by oparametric mea regressio of Y o X This miimizes a priori assumptios ad likelihood of specificatio error Disadvatages: Hard to icorporate behavioral hypotheses draw from ecoomic or other theory models Estimatio precisio is expoetially decreasig fuctio of dimesio of X Extrapolatio ot possible
5 SEMIPARAMETRIC MODELING Achieves greater precisio tha oparametric models but with weaker assumptios tha parametric models Does this by restrictig G( x ) so as to reduce effective dimesio of x. Risk of specificatio error greater tha with fully oparametric model but less tha with parametric oe Examples: Sigleidex model: Gx ( ) = Fxβ ( ), where F is ukow Additive model: Gx ( ) = H[ f( x) f( x)], 1 1 where H is kow or ukow fuctio ad f i s are ukow d d
6 IDENTIFICATION OF SINGLEINDEX MODELS E ( Y X = x) = G( x β ) β ot idetified if G is costat fuctio. Sig, scale, ad locatio ormalizatios eeded to idetify β To implemet assume X has o itercept ad β 1 = 1. X 1 must be cotiuously distributed coditioal o other compoets of X. Let X = ( X1, X 2) ad X β = X1+ β2x2. G ad β 2 ca be aythig that satisfy: (X 1,X 2) G(X 1 + β 2X 2) E(Y X) (0,0) G(0) 0 (1,0) G(1) 1 (0,1) G(β 2 ) 3 (1,1) G(1 + β 2 ) 4
7 OPTIMZATION ESTIMATORS If G kow, β ca be estimated by oliear least squares. 1 2 i i i i= 1 miimize: w( X )[ Y G( X b] b where w ( ) is a weight fuctio. Whe G ukow, replace G(X i b) with oparametric estimator of E(Y X i b) (e.g., kerel). Estimator ow solves 1 2 i i i i= 1 miimize: w( X )[ Y G ( X b)] w may be chose to b Keep deomiator of G away from 0 Achieve asymptotic efficiecy
8 ASYMPTOTIC NORMALITY Ichimura (1993) gives coditios uder which 1/2 ( b β ) N(0, V ) where b is weighted NLS estimator Proof based o stadard Taylor series methods of asymptotic distributio theory Estimator has 1/2 rate of covergece Hall ad Ichimura (1991) derived asymptotic efficiecy boud for β i Y G( X ) ( Xi ) U i = i β + σ β where the U i are iid with mea 0 Hall ad Ichimura also derived asymptotically efficiet estimator Uses estimate of σ(x i β) 1 as weight fuctio i NLS objective fuctio ad kerel estimator of G. i
9 MLE FOR BINARY RESPONSE MODEL If Y = 0 or 1, G(xβ) = P(Y=1 X=x) If G kow, log likelihood is i= 1 { i i [ i ]} log Lb ( ) = log GXb ( ) + (1 Y)log 1 GXb ( ) If G ukow, replace it with estimator G log Lb ( ) = i= 1 τ { log G ( X b) + (1 Y)log[ 1 G ( X b) ]} i i i i τ i trims away observatios for which G( Xib) is too close to 0 or 1. Klei ad Spady (1993) gave coditios uder 1/2 which semiparametric MLE estimator is  cosistet ad asymptotically ormal Chamberlai (1986) ad Cosslett (1987) derived asymptotic efficiecy boud for case i which G is a CDF Semiparametric MLE achieves boud
10 DIRECT ESTIMATORS NLS ad ML estimators are hard to compute Direct estimators avoid eed to solve optimizatio problem Direct estimators are ot asymptotically efficiet Efficiet estimator ca be obtaied easily by oestep method If X is cotiuous radom vector, β proportioal to average derivative of G β E [ wx ( ) GX ( β ) X] where w is a weight fuctio Oly weighted average derivative eeded because β idetified oly up to scale If w is idetity fuctio, get average derivative estimator of β (Härdle ad Stoker 1989) This estimator is hard to aalyze because of its radom deomiator
11 DENSITY WEIGHTED AVERAGE DERIVATIVE ESTIMATORS Radom deomiator problem ca be overcome by settig w(x) = f(x), desity of X Itegratio by parts gives [ f( X) G( Xβ ) X] δ E [ β ] = 2 EGX ( ) f( X)/ X [ X] = 2 EY f( X) Estimate δ by replacig E with sample average ad f with kerel estimator to get δ = ( 2/ ) i= 1 Y i fi( Xi) x where f i is leaveoeout kerel estimator of f(x). Powell, Stock, ad Stoker (1989) gave coditios uder which 1/2 ( δ δ ) N (0, V)
12 METHOD OF PROOF Write δ as U statistic of order 2 with badwidthdepedet kerel U statistic is asymptotically equivalet to its projectio, which gives δ 1/2 = (2/ ) r( Yi, Xi) + op( ), i= 1 where r ( Y, X ) i i = k Xi x K [ Yi E( Y X = x) ] f( x) dx h h Chagig variables i itegral shows that leadig term of r does ot deped o h or So δ is asymptotically equivalet to a sum of iid radom variables 1/2 cosistecy ad asymptotic ormality follow from LidebergLevy theorem
13 TECHNICAL DETAILS Must use higherorder K with udersmoothig to isure that asymptotic distributio of 1/2 (δ  δ) is cetered at 0. Härdle ad Tsybakov (1993) ad Powell ad Stoker (1996) describe methods for selectig applicatios. h i Horowitz ad Härdle (1996) show how to iclude discrete compoets of X i direct estimator.
14 ESTIMATOR WITH DISCRETE COVARIATES Write model as E(Y X = x, Z = z) = G(Xβ + Zα), where X is cotiuous ad Z is discrete with M poits of support. Idetificatio requires a cotiuous covariate Assume estimator of β, b is available, possibly average of average derivative estimates computed at each poit i support of Z. Suppose there are fiite umbers c 0, c 1, v 0, v 1 such that Gv ( + z α) is bouded for all v [v 0,v 1 ] ad z supp( Z). v v0 G( v zα ) c0 + for each z supp( Z) v v1 G( v zα ) c1 Defie + > for each z supp( Z) v > v J 1 ( z ) = { cigv [ ( + z α) < c ] + cigv [ ( + z α) c 1 ] 0 + Gv ( + zα) Ic [ Gv ( + zα) c]} dv 0 1
15 The for i DISCRETE COVARIATES (cot.) = 2,..., M () i (1) () i (1) = 1 0 Jz [ ] Jz [ ] ( c c)[ z z ] α. This is M  1 liear equatios i compoets of α. To solve, write (2) (1) Jz [ ] Jz [ ] J =... ; ( M ) (1) Jz [ ] Jz [ ] W (2) (1) z z =.... ( M ) (1) z z The α = ( c c ) ( WW ) W J. Obtai estimator by replacig G with oparametric regressio estimate of E( Y Xb = v, Z = z). Let J be resultig estimator of J Estimator of α is 1 1 α = ( c c ) ( WW ) W J 0 1 Horowitz ad Härdle (1996) give coditios uder 1/2 d which ( α α) N(0,Vα ).
16 1.8 J K G(V + 2) G G(V).2 E F G H 0 A B V C D ( c0, c 1) = (0.2,0.8), ( v0, v 1) = (2.85,0.85) (1) J[ z ] = ACGE+ CDHG+ GH = 2c + 1.7c + GHK 0 0 (2) J[ z ] = ABFE + BDKJ + EFJ = 1.7c + 2c + EFJ 0 1 (2) (1) (2) (1) Jz [ ] Jz [ ] = 2( c c) = ( c c)[ z z ] α K
17 HIGHDIMENSIONAL X Average derivative estimators require G ad f to have may derivatives) if X is high dimesioal. This is form of curse of dimesioality Implies that fiitesample precisio of average derivatives may be low if dim( ) X large. Hristache, Juditsky, ad Spokoiy (2001) proposed method for iteratively improvig a average derivative estimator. Method uses two badwidths: a large oe i the directio orthogoal to curret estimate ad a small oe i parallel directio. Calculate ew estimate of β usig average derivatives with the two badwidths This procedure yields estimator that is 1/2  cosistet ad asymptotically ormal regardless of dimesio of X whe G is twice differetiable. Mote Carlo evidece idicates that iterated estimator has smaller fiitesample errors tha oiterated oe.
18 OUTLINE OF ITERATIVE METHOD Iitializatio: Specify parameters ρ 1, ρ mi, a ρ, h 1, hmax, ah, k = 1, 0 ˆβ (iitial estimate of β ) Compute S = ( I + ρ ˆ β ˆ β ) 2 1/2 k k k 1 k 1 For every i = 1,...,, compute fˆ ( X ) from ˆ 1 1 = 2 f ( ) k X i SX k ij K 2 fˆ ( ) j 1 Xij Xij h k X i = k k i SX k ij Yj K 2 j= 1 X ij h k where X ij = X j Xi Compute ˆ 1 k = f ( ) j= 1 k Xi β Set hk+ 1 = ahhk, ρ = k+ 1 a ρ ρ. If k ρk+ 1 > ρm i, set k = k+1 ad retur to step 2. Otherwise, stop. ˆ
19 AN APPLICATION Model of product iovatio by Germa maufacturers of ivestmet goods Data assembled by IFO Istitute i Muich Cosist of observatios o 1100 maufacturers Model: P(Y=1 X=x) = G(Xβ), where Y = 1 if maufacturer realized a iovatio i a specific product category i 1989 ad 0 otherwise Variables: o. of employees i product category (EMPLP), o. of employees i etire firm (EMPLF), idicator of firm s productio capacity utilizatio (CAP), DEM = 1 if firm expected icreasig demad for product ad 0 otherwise
20 ESTIMATED COEFFICIENTS FOR MODEL OF PRODUCT INNOVATION EMPLP EMPLF CAP DEM Semiparametric Model (0.028) (0.078) (0.509) Probit Model (0.242) (0.163) (0.387)
21 1.8 G(V) V ESTIMATE OF G(V).1 dg/dv V
22 CONCLUSIONS Sigleidex models: Provide compromise betwee restrictios of parametric models ad imprecisio of fully oparametric models May be structural (e.g., radom utility biaryrespose model) Asymptotic efficiecy bouds available i some cases Two classes of estimators Noliear optimizatio: provides asymptotically efficiet estimator i some cases Direct: Noiterative, does ot require solvig oliear optimizatio problem Oestep estimatio from directestimate yields asymptotic efficiecy whe efficiet estimator available Example based o real data illustrates usefuless
Kernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Crosssectioal data. 2. Time series data.
More informationUnbiased Estimation. February 712, 2008
Ubiased Estimatio February 72, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationAlgorithms for Clustering
CR2: Statistical Learig & Applicatios Algorithms for Clusterig Lecturer: J. Salmo Scribe: A. Alcolei Settig: give a data set X R p where is the umber of observatio ad p is the umber of features, we wat
More information71. Chapter 4. Part I. Sampling Distributions and Confidence Intervals
71 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7 Sectio 1. Samplig Distributio 73 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More information1 Covariance Estimation
Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4.
More informationA RANK STATISTIC FOR NONPARAMETRIC KSAMPLE AND CHANGE POINT PROBLEMS
J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NONPARAMETRIC KSAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider ksample ad chage poit problems for idepedet data i a
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More information3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N.
3/3/04 CDS M Phil Old Least Squares (OLS) Vijayamohaa Pillai N CDS M Phil Vijayamoha CDS M Phil Vijayamoha Types of Relatioships Oly oe idepedet variable, Relatioship betwee ad is Liear relatioships Curviliear
More informationElement sampling: Part 2
Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig
More informationMonte Carlo Integration
Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce
More informationOn an Application of Bayesian Estimation
O a Applicatio of ayesia Estimatio KIYOHARU TANAKA School of Sciece ad Egieerig, Kiki Uiversity, Kowakae, HigashiOsaka, JAPAN Email: ktaaka@ifokidaiacjp EVGENIY GRECHNIKOV Departmet of Mathematics, auma
More informationLocal Polynomial Regression
Local Polyomial Regressio Joh Hughes October 2, 2013 Recall that the oparametric regressio model is Y i f x i ) + ε i, where f is the regressio fuctio ad the ε i are errors such that Eε i 0. The NadarayaWatso
More informationLinear Regression Models
Liear Regressio Models Dr. Joh MellorCrummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More information4 Conditional Distribution Estimation
4 Coditioal Distributio Estimatio 4. Estimators Te coditioal distributio (CDF) of y i give X i = x is F (y j x) = P (y i y j X i = x) = E ( (y i y) j X i = x) : Tis is te coditioal mea of te radom variable
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationFirst Year Quantitative Comp Exam Spring, Part I  203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I1 Part I  203A A radom variable X is distributed with the margial desity: >
More informationMonte Carlo Methods: Lecture 3 : Importance Sampling
Mote Carlo Methods: Lecture 3 : Importace Samplig Nick Whiteley 16.10.2008 Course material origially by Adam Johase ad Ludger Evers 2007 Overview of this lecture What we have see... Rejectio samplig. This
More informationTable 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab
Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationJacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3
NoParametric Techiques Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3 Parametric vs. NoParametric Parametric Based o Fuctios (e.g Normal Distributio) Uimodal Oly oe peak Ulikely real data cofies
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationIP Reference guide for integer programming formulations.
IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more
More informationA semiparametric singleindex estimator for a class of estimating equation models
A semiparametric sigleidex estimator for a class of estimatig equatio models arxiv:1608.04244v2 [math.s] 26 Apr 2017 Maria Hristache Weiyu Li Valeti Patilea Abstract We propose a twostep pseudomaximum
More informationDistribution Free Estimation of Heteroskedastic Binary Response Models Using Probit/Logit Criterion Functions
Distributio Free Estimatio of Heteroskedastic Biary Respose Models Usig Probit/Logit Criterio Fuctios Shakeeb Kha Duke Uiversity Revised: February Abstract I this paper estimators for distributio free
More informationMatrix Representation of Data in Experiment
Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y
More informationStatistical Fundamentals and Control Charts
Statistical Fudametals ad Cotrol Charts 1. Statistical Process Cotrol Basics Chace causes of variatio uavoidable causes of variatios Assigable causes of variatio large variatios related to machies, materials,
More informationThe Sampling Distribution of the Maximum. Likelihood Estimators for the Parameters of. BetaBinomial Distribution
Iteratioal Mathematical Forum, Vol. 8, 2013, o. 26, 12631277 HIKARI Ltd, www.mhikari.com http://d.doi.org/10.12988/imf.2013.3475 The Samplig Distributio of the Maimum Likelihood Estimators for the Parameters
More informationSimple Linear Regression
Simple Liear Regressio 1. Model ad Parameter Estimatio (a) Suppose our data cosist of a collectio of pairs (x i, y i ), where x i is a observed value of variable X ad y i is the correspodig observatio
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture  9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that
More informationStatisticians use the word population to refer the total number of (potential) observations under consideration
6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space
More informationSTAT331. Example of Martingale CLT with Cox s Model
STAT33 Example of Martigale CLT with Cox s Model I this uit we illustrate the Martigale Cetral Limit Theorem by applyig it to the partial likelihood score fuctio from Cox s model. For simplicity of presetatio
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 11
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract We will itroduce the otio of reproducig kerels ad associated Reproducig Kerel Hilbert Spaces (RKHS). We will cosider couple
More informationMOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.
XI1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI2 (1075) STATISTICAL DECISION MAKING Advaced
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More information5.1 A mutual information bound based on metric entropy
Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local
More informationSupport vector machine revisited
6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chisquare Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chisquare Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationIntroduction to Optimization Techniques. How to Solve Equations
Itroductio to Optimizatio Techiques How to Solve Equatios Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually
More informationARIMA Models. Dan Saunders. y t = φy t 1 + ɛ t
ARIMA Models Da Sauders I will discuss models with a depedet variable y t, a potetially edogeous error term ɛ t, ad a exogeous error term η t, each with a subscript t deotig time. With just these three
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationComputing the maximum likelihood estimates: concentrated likelihood, EMalgorithm. Dmitry Pavlyuk
Computig the maximum likelihood estimates: cocetrated likelihood, EMalgorithm Dmitry Pavlyuk The Mathematical Semiar, Trasport ad Telecommuicatio Istitute, Riga, 13.05.2016 Presetatio outlie 1. Basics
More informationPoint Estimation: properties of estimators 1 FINITESAMPLE PROPERTIES. finitesample properties (CB 7.3) largesample properties (CB 10.
Poit Estimatio: properties of estimators fiitesample properties CB 7.3) largesample properties CB 10.1) 1 FINITESAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:
More informationTopic 18: Composite Hypotheses
Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:
More informations = and t = with C ij = A i B j F. (i) Note that cs = M and so ca i µ(a i ) I E (cs) = = c a i µ(a i ) = ci E (s). (ii) Note that s + t = M and so
3 From the otes we see that the parts of Theorem 4. that cocer us are: Let s ad t be two simple oegative Fmeasurable fuctios o X, F, µ ad E, F F. The i I E cs ci E s for all c R, ii I E s + t I E s +
More informationSection A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics
X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.
More informationStat 200 Testing Summary Page 1
Stat 00 Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More informationSAMPLING LIPSCHITZ CONTINUOUS DENSITIES. 1. Introduction
SAMPLING LIPSCHITZ CONTINUOUS DENSITIES OLIVIER BINETTE Abstract. A simple ad efficiet algorithm for geeratig radom variates from the class of Lipschitz cotiuous desities is described. A MatLab implemetatio
More informationTEACHER CERTIFICATION STUDY GUIDE
COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra
More informationStatistical Intervals for a Single Sample
3/5/06 Applied Statistics ad Probability for Egieers Sixth Editio Douglas C. Motgomery George C. Ruger Chapter 8 Statistical Itervals for a Sigle Sample 8 CHAPTER OUTLINE 8 Cofidece Iterval o the Mea
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of otime jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationSome Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More information9. Simulation lect09.ppt S Introduction to Teletraffic Theory  Fall
lect09.ppt S38.145  Itroductio to Teletraffic Theory  Fall 2001 1 Cotets Itroductio Geeratio of realizatios of the traffic process Geeratio of realizatios of radom variables Collectio of data Statistical
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 5766 ON POINTWISE BINOMIAL APPROXIMATION BY wfunctions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationSection 14. Simple linear regression.
Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo
More informationEstimation of Gumbel Parameters under Ranked Set Sampling
Joural of Moder Applied Statistical Methods Volume 13 Issue 2 Article 112014 Estimatio of Gumbel Parameters uder Raked Set Samplig Omar M. Yousef Al Balqa' Applied Uiversity, Zarqa, Jorda, abuyaza_o@yahoo.com
More informationAnalysis of Experimental Data
Aalysis of Experimetal Data 6544597.0479 ± 0.000005 g Quatitative Ucertaity Accuracy vs. Precisio Whe we make a measuremet i the laboratory, we eed to kow how good it is. We wat our measuremets to be both
More informationTHE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.
THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (RegulaFalsi) Fied poit iteratio Newto Raphso Secat The root of
More informationFinal Examination Solutions 17/6/2010
The Islamic Uiversity of Gaza Faculty of Commerce epartmet of Ecoomics ad Political Scieces A Itroductio to Statistics Course (ECOE 30) Sprig Semester 00900 Fial Eamiatio Solutios 7/6/00 Name: I: Istructor:
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More information5.1 Review of Singular Value Decomposition (SVD)
MGMT 69000: Topics i Highdimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of
More informationIt should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.
Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig
More informationNCSS Statistical Software. Tolerance Intervals
Chapter 585 Itroductio This procedure calculates oe, ad two, sided tolerace itervals based o either a distributiofree (oparametric) method or a method based o a ormality assumptio (parametric). A twosided
More informationMATH 10550, EXAM 3 SOLUTIONS
MATH 155, EXAM 3 SOLUTIONS 1. I fidig a approximate solutio to the equatio x 3 +x 4 = usig Newto s method with iitial approximatio x 1 = 1, what is x? Solutio. Recall that x +1 = x f(x ) f (x ). Hece,
More informationLecture 6 Ecient estimators. RaoCramer bound.
Lecture 6 Eciet estimators. RaoCramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have
More informationTopics Machine learning: lecture 2. Review: the learning problem. Hypotheses and estimation. Estimation criterion cont d. Estimation criterion
.87 Machie learig: lecture Tommi S. Jaakkola MIT CSAIL tommi@csail.mit.edu Topics The learig problem hypothesis class, estimatio algorithm loss ad estimatio criterio samplig, empirical ad epected losses
More information6.867 Machine learning, lecture 7 (Jaakkola) 1
6.867 Machie learig, lecture 7 (Jaakkola) 1 Lecture topics: Kerel form of liear regressio Kerels, examples, costructio, properties Liear regressio ad kerels Cosider a slightly simpler model where we omit
More information17 Phonons and conduction electrons in solids (Hiroshi Matsuoka)
7 Phoos ad coductio electros i solids Hiroshi Matsuoa I this chapter we will discuss a miimal microscopic model for phoos i a solid ad a miimal microscopic model for coductio electros i a simple metal.
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationA note on selfnormalized DickeyFuller test for unit root in autoregressive time series with GARCH errors
Appl. Math. J. Chiese Uiv. 008, 3(): 970 A ote o selformalized DickeyFuller test for uit root i autoregressive time series with GARCH errors YANG Xiaorog ZHANG Lixi Abstract. I this article, the uit
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationStatistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
More informationIE 230 Probability & Statistics in Engineering I. Closed book and notes. No calculators. 120 minutes.
Closed book ad otes. No calculators. 120 miutes. Cover page, five pages of exam, ad tables for discrete ad cotiuous distributios. Score X i =1 X i / S X 2 i =1 (X i X ) 2 / ( 1) = [i =1 X i 2 X 2 ] / (
More informationIntroduction to Artificial Intelligence CAP 4601 Summer 2013 Midterm Exam
Itroductio to Artificial Itelligece CAP 601 Summer 013 Midterm Exam 1. Termiology (7 Poits). Give the followig task eviromets, eter their properties/characteristics. The properties/characteristics of the
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More informationA Distributional Approach Using Propensity Scores
A Distributioal Approach Usig Propesity Scores Zhiqiag Ta Departmet of Biostatistics Johs Hopkis School of Public Health http://www.biostat.jhsph.edu/ zta Jue 20, 2005 Outlie Itroductio Couterfactual framework
More informationMarkov Decision Processes
Markov Decisio Processes Defiitios; Statioary policies; Value improvemet algorithm, Policy improvemet algorithm, ad liear programmig for discouted cost ad average cost criteria. Markov Decisio Processes
More informationDimensionality Reduction vs. Clustering
Dimesioality Reductio vs. Clusterig Lecture 9: Cotiuous Latet Variable Models Sam Roweis Traiig such factor models (e.g. FA, PCA, ICA) is called dimesioality reductio. You ca thik of this as (o)liear regressio
More informationBasis for simulation techniques
Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios
More informationR. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State
Bayesia Cotrol Charts for the Twoparameter Expoetial Distributio if the Locatio Parameter Ca Take o Ay Value Betwee Mius Iity ad Plus Iity R. va Zyl, A.J. va der Merwe 2 Quitiles Iteratioal, ruaavz@gmail.com
More informationOutput Analysis and RunLength Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad RuLegth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationGeneral IxJ Contingency Tables
page1 Geeral x Cotigecy Tables We ow geeralize our previous results from the prospective, retrospective ad crosssectioal studies ad the Poisso samplig case to x cotigecy tables. For such tables, the test
More informationConfidence Level We want to estimate the true mean of a random variable X economically and with confidence.
Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio
More informationConfidence Intervals for the Population Proportion p
Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical:
More informationThe Ratio Test. THEOREM 9.17 Ratio Test Let a n be a series with nonzero terms. 1. a. n converges absolutely if lim. n 1
460_0906.qxd //04 :8 PM Page 69 SECTION 9.6 The Ratio ad Root Tests 69 Sectio 9.6 EXPLORATION Writig a Series Oe of the followig coditios guaratees that a series will diverge, two coditios guaratee that
More informationGoodnessOfFit For The Generalized Exponential Distribution. Abstract
GoodessOfFit For The Geeralized Expoetial Distributio By Amal S. Hassa stitute of Statistical Studies & Research Cairo Uiversity Abstract Recetly a ew distributio called geeralized expoetial or expoetiated
More information