# SEMIPARAMETRIC SINGLE-INDEX MODELS. Joel L. Horowitz Department of Economics Northwestern University

Save this PDF as:

Size: px
Start display at page:

Download "SEMIPARAMETRIC SINGLE-INDEX MODELS. Joel L. Horowitz Department of Economics Northwestern University"

## Transcription

1 SEMIPARAMETRIC SINGLE-INDEX MODELS by Joel L. Horowitz Departmet of Ecoomics Northwester Uiversity

2 INTRODUCTION Much of applied ecoometrics ad statistics ivolves estimatig a coditioal mea fuctio: E ( Y X = x) Y may be cotiuous or biary If biary, the E ( Y X = x) is P ( Y = 1 X = x) I biary respose model, Y may idicate a idividual s choice amog two alteratives, occurrece or o-occurrece of a evet, etc. Possible approaches Fully parametric Fully oparametric Semiparametric

3 FULLY PARAMETRIC MODELING I fully parametric model, E ( Y X = x) is kow up to a fiite-dimesioal parameter: E ( Y = 1 X = x) = F( x, θ ) F is kow fuctio θ is ukow, fiite-dimesioal parameter Example: biary probit or logit model Advatages: If F is correctly specified Maximizes estimatio efficiecy Permits extrapolatio of x beyod rage of data Ofte has atural behavioral iterpretatio Disadvatages: F rarely kow i applicatios Ca be highly misleadig if F is misspecified

4 FULLY NONPARAMETRIC MODELING E ( Y X = x) G( x) assumed to be smooth fuctio of x Nothig assumed about shape of G. G estimated by oparametric mea regressio of Y o X This miimizes a priori assumptios ad likelihood of specificatio error Disadvatages: Hard to icorporate behavioral hypotheses draw from ecoomic or other theory models Estimatio precisio is expoetially decreasig fuctio of dimesio of X Extrapolatio ot possible

5 SEMIPARAMETRIC MODELING Achieves greater precisio tha oparametric models but with weaker assumptios tha parametric models Does this by restrictig G( x ) so as to reduce effective dimesio of x. Risk of specificatio error greater tha with fully oparametric model but less tha with parametric oe Examples: Sigle-idex model: Gx ( ) = Fxβ ( ), where F is ukow Additive model: Gx ( ) = H[ f( x) f( x)], 1 1 where H is kow or ukow fuctio ad f i s are ukow d d

6 IDENTIFICATION OF SINGLE-INDEX MODELS E ( Y X = x) = G( x β ) β ot idetified if G is costat fuctio. Sig, scale, ad locatio ormalizatios eeded to idetify β To implemet assume X has o itercept ad β 1 = 1. X 1 must be cotiuously distributed coditioal o other compoets of X. Let X = ( X1, X 2) ad X β = X1+ β2x2. G ad β 2 ca be aythig that satisfy: (X 1,X 2) G(X 1 + β 2X 2) E(Y X) (0,0) G(0) 0 (1,0) G(1) 1 (0,1) G(β 2 ) 3 (1,1) G(1 + β 2 ) 4

7 OPTIMZATION ESTIMATORS If G kow, β ca be estimated by oliear least squares. 1 2 i i i i= 1 miimize: w( X )[ Y G( X b] b where w ( ) is a weight fuctio. Whe G ukow, replace G(X i b) with oparametric estimator of E(Y X i b) (e.g., kerel). Estimator ow solves 1 2 i i i i= 1 miimize: w( X )[ Y G ( X b)] w may be chose to b Keep deomiator of G away from 0 Achieve asymptotic efficiecy

8 ASYMPTOTIC NORMALITY Ichimura (1993) gives coditios uder which 1/2 ( b β ) N(0, V ) where b is weighted NLS estimator Proof based o stadard Taylor series methods of asymptotic distributio theory Estimator has 1/2 rate of covergece Hall ad Ichimura (1991) derived asymptotic efficiecy boud for β i Y G( X ) ( Xi ) U i = i β + σ β where the U i are iid with mea 0 Hall ad Ichimura also derived asymptotically efficiet estimator Uses estimate of σ(x i β) -1 as weight fuctio i NLS objective fuctio ad kerel estimator of G. i

9 MLE FOR BINARY RESPONSE MODEL If Y = 0 or 1, G(xβ) = P(Y=1 X=x) If G kow, log likelihood is i= 1 { i i [ i ]} log Lb ( ) = log GXb ( ) + (1 Y)log 1 GXb ( ) If G ukow, replace it with estimator G log Lb ( ) = i= 1 τ { log G ( X b) + (1 Y)log[ 1 G ( X b) ]} i i i i τ i trims away observatios for which G( Xib) is too close to 0 or 1. Klei ad Spady (1993) gave coditios uder 1/2 which semiparametric MLE estimator is - cosistet ad asymptotically ormal Chamberlai (1986) ad Cosslett (1987) derived asymptotic efficiecy boud for case i which G is a CDF Semiparametric MLE achieves boud

10 DIRECT ESTIMATORS NLS ad ML estimators are hard to compute Direct estimators avoid eed to solve optimizatio problem Direct estimators are ot asymptotically efficiet Efficiet estimator ca be obtaied easily by oe-step method If X is cotiuous radom vector, β proportioal to average derivative of G β E [ wx ( ) GX ( β ) X] where w is a weight fuctio Oly weighted average derivative eeded because β idetified oly up to scale If w is idetity fuctio, get average derivative estimator of β (Härdle ad Stoker 1989) This estimator is hard to aalyze because of its radom deomiator

11 DENSITY WEIGHTED AVERAGE DERIVATIVE ESTIMATORS Radom deomiator problem ca be overcome by settig w(x) = f(x), desity of X Itegratio by parts gives [ f( X) G( Xβ ) X] δ E [ β ] = 2 EGX ( ) f( X)/ X [ X] = 2 EY f( X) Estimate δ by replacig E with sample average ad f with kerel estimator to get δ = ( 2/ ) i= 1 Y i fi( Xi) x where f i is leave-oe-out kerel estimator of f(x). Powell, Stock, ad Stoker (1989) gave coditios uder which 1/2 ( δ δ ) N (0, V)

12 METHOD OF PROOF Write δ as U statistic of order 2 with badwidthdepedet kerel U statistic is asymptotically equivalet to its projectio, which gives δ 1/2 = (2/ ) r( Yi, Xi) + op( ), i= 1 where r ( Y, X ) i i = k Xi x K [ Yi E( Y X = x) ] f( x) dx h h Chagig variables i itegral shows that leadig term of r does ot deped o h or So δ is asymptotically equivalet to a sum of iid radom variables 1/2 -cosistecy ad asymptotic ormality follow from Lideberg-Levy theorem

13 TECHNICAL DETAILS Must use higher-order K with udersmoothig to isure that asymptotic distributio of 1/2 (δ - δ) is cetered at 0. Härdle ad Tsybakov (1993) ad Powell ad Stoker (1996) describe methods for selectig applicatios. h i Horowitz ad Härdle (1996) show how to iclude discrete compoets of X i direct estimator.

14 ESTIMATOR WITH DISCRETE COVARIATES Write model as E(Y X = x, Z = z) = G(Xβ + Zα), where X is cotiuous ad Z is discrete with M poits of support. Idetificatio requires a cotiuous covariate Assume estimator of β, b is available, possibly average of average derivative estimates computed at each poit i support of Z. Suppose there are fiite umbers c 0, c 1, v 0, v 1 such that Gv ( + z α) is bouded for all v [v 0,v 1 ] ad z supp( Z). v v0 G( v zα ) c0 + for each z supp( Z) v v1 G( v zα ) c1 Defie + > for each z supp( Z) v > v J 1 ( z ) = { cigv [ ( + z α) < c ] + cigv [ ( + z α) c 1 ] 0 + Gv ( + zα) Ic [ Gv ( + zα) c]} dv 0 1

15 The for i DISCRETE COVARIATES (cot.) = 2,..., M () i (1) () i (1) = 1 0 Jz [ ] Jz [ ] ( c c)[ z z ] α. This is M - 1 liear equatios i compoets of α. To solve, write (2) (1) Jz [ ] Jz [ ] J =... ; ( M ) (1) Jz [ ] Jz [ ] W (2) (1) z z =.... ( M ) (1) z z The α = ( c c ) ( WW ) W J. Obtai estimator by replacig G with oparametric regressio estimate of E( Y Xb = v, Z = z). Let J be resultig estimator of J Estimator of α is 1 1 α = ( c c ) ( WW ) W J 0 1 Horowitz ad Härdle (1996) give coditios uder 1/2 d which ( α α) N(0,Vα ).

16 1.8 J K G(V + 2) G G(V).2 E F G H 0 A B V C D ( c0, c 1) = (0.2,0.8), ( v0, v 1) = (2.85,0.85) (1) J[ z ] = ACGE+ CDHG+ GH = 2c + 1.7c + GHK 0 0 (2) J[ z ] = ABFE + BDKJ + EFJ = 1.7c + 2c + EFJ 0 1 (2) (1) (2) (1) Jz [ ] Jz [ ] = 2( c c) = ( c c)[ z z ] α K

17 HIGH-DIMENSIONAL X Average derivative estimators require G ad f to have may derivatives) if X is high dimesioal. This is form of curse of dimesioality Implies that fiite-sample precisio of average derivatives may be low if dim( ) X large. Hristache, Juditsky, ad Spokoiy (2001) proposed method for iteratively improvig a average derivative estimator. Method uses two badwidths: a large oe i the directio orthogoal to curret estimate ad a small oe i parallel directio. Calculate ew estimate of β usig average derivatives with the two badwidths This procedure yields estimator that is 1/2 - cosistet ad asymptotically ormal regardless of dimesio of X whe G is twice differetiable. Mote Carlo evidece idicates that iterated estimator has smaller fiite-sample errors tha oiterated oe.

18 OUTLINE OF ITERATIVE METHOD Iitializatio: Specify parameters ρ 1, ρ mi, a ρ, h 1, hmax, ah, k = 1, 0 ˆβ (iitial estimate of β ) Compute S = ( I + ρ ˆ β ˆ β ) 2 1/2 k k k 1 k 1 For every i = 1,...,, compute fˆ ( X ) from ˆ 1 1 = 2 f ( ) k X i SX k ij K 2 fˆ ( ) j 1 Xij Xij h k X i = k k i SX k ij Yj K 2 j= 1 X ij h k where X ij = X j Xi Compute ˆ 1 k = f ( ) j= 1 k Xi β Set hk+ 1 = ahhk, ρ = k+ 1 a ρ ρ. If k ρk+ 1 > ρm i, set k = k+1 ad retur to step 2. Otherwise, stop. ˆ

19 AN APPLICATION Model of product iovatio by Germa maufacturers of ivestmet goods Data assembled by IFO Istitute i Muich Cosist of observatios o 1100 maufacturers Model: P(Y=1 X=x) = G(Xβ), where Y = 1 if maufacturer realized a iovatio i a specific product category i 1989 ad 0 otherwise Variables: o. of employees i product category (EMPLP), o. of employees i etire firm (EMPLF), idicator of firm s productio capacity utilizatio (CAP), DEM = 1 if firm expected icreasig demad for product ad 0 otherwise

20 ESTIMATED COEFFICIENTS FOR MODEL OF PRODUCT INNOVATION EMPLP EMPLF CAP DEM Semiparametric Model (0.028) (0.078) (0.509) Probit Model (0.242) (0.163) (0.387)

21 1.8 G(V) V ESTIMATE OF G(V).1 dg/dv V

22 CONCLUSIONS Sigle-idex models: Provide compromise betwee restrictios of parametric models ad imprecisio of fully oparametric models May be structural (e.g., radom utility biaryrespose model) Asymptotic efficiecy bouds available i some cases Two classes of estimators Noliear optimizatio: provides asymptotically efficiet estimator i some cases Direct: No-iterative, does ot require solvig oliear optimizatio problem Oe-step estimatio from direct-estimate yields asymptotic efficiecy whe efficiet estimator available Example based o real data illustrates usefuless

### Kernel density estimator

Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I

### Properties and Hypothesis Testing

Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

### Unbiased Estimation. February 7-12, 2008

Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

### Algorithms for Clustering

CR2: Statistical Learig & Applicatios Algorithms for Clusterig Lecturer: J. Salmo Scribe: A. Alcolei Settig: give a data set X R p where is the umber of observatio ad p is the umber of features, we wat

### 7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals

7-1 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7- Sectio 1. Samplig Distributio 7-3 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses

### Chapter 6 Sampling Distributions

Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

### 1 Covariance Estimation

Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4.

### A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS

J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a

### Probability and Statistics

ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

### 3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N.

3/3/04 CDS M Phil Old Least Squares (OLS) Vijayamohaa Pillai N CDS M Phil Vijayamoha CDS M Phil Vijayamoha Types of Relatioships Oly oe idepedet variable, Relatioship betwee ad is Liear relatioships Curviliear

### Element sampling: Part 2

Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig

### Monte Carlo Integration

Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce

### On an Application of Bayesian Estimation

O a Applicatio of ayesia Estimatio KIYOHARU TANAKA School of Sciece ad Egieerig, Kiki Uiversity, Kowakae, Higashi-Osaka, JAPAN Email: ktaaka@ifokidaiacjp EVGENIY GRECHNIKOV Departmet of Mathematics, auma

### Local Polynomial Regression

Local Polyomial Regressio Joh Hughes October 2, 2013 Recall that the oparametric regressio model is Y i f x i ) + ε i, where f is the regressio fuctio ad the ε i are errors such that Eε i 0. The Nadaraya-Watso

### Linear Regression Models

Liear Regressio Models Dr. Joh Mellor-Crummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect

### Regression with an Evaporating Logarithmic Trend

Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,

### Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

### 4 Conditional Distribution Estimation

4 Coditioal Distributio Estimatio 4. Estimators Te coditioal distributio (CDF) of y i give X i = x is F (y j x) = P (y i y j X i = x) = E ( (y i y) j X i = x) : Tis is te coditioal mea of te radom variable

### 1 Inferential Methods for Correlation and Regression Analysis

1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

### First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

### Monte Carlo Methods: Lecture 3 : Importance Sampling

Mote Carlo Methods: Lecture 3 : Importace Samplig Nick Whiteley 16.10.2008 Course material origially by Adam Johase ad Ludger Evers 2007 Overview of this lecture What we have see... Rejectio samplig. This

### Table 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab

Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet

### Chapter 6 Principles of Data Reduction

Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

### Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3

No-Parametric Techiques Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3 Parametric vs. No-Parametric Parametric Based o Fuctios (e.g Normal Distributio) Uimodal Oly oe peak Ulikely real data cofies

### Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn

Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom

### IP Reference guide for integer programming formulations.

IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more

### A semiparametric single-index estimator for a class of estimating equation models

A semiparametric sigle-idex estimator for a class of estimatig equatio models arxiv:1608.04244v2 [math.s] 26 Apr 2017 Maria Hristache Weiyu Li Valeti Patilea Abstract We propose a two-step pseudo-maximum

### Distribution Free Estimation of Heteroskedastic Binary Response Models Using Probit/Logit Criterion Functions

Distributio Free Estimatio of Heteroskedastic Biary Respose Models Usig Probit/Logit Criterio Fuctios Shakeeb Kha Duke Uiversity Revised: February Abstract I this paper estimators for distributio free

### Matrix Representation of Data in Experiment

Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y

### Statistical Fundamentals and Control Charts

Statistical Fudametals ad Cotrol Charts 1. Statistical Process Cotrol Basics Chace causes of variatio uavoidable causes of variatios Assigable causes of variatio large variatios related to machies, materials,

### The Sampling Distribution of the Maximum. Likelihood Estimators for the Parameters of. Beta-Binomial Distribution

Iteratioal Mathematical Forum, Vol. 8, 2013, o. 26, 1263-1277 HIKARI Ltd, www.m-hikari.com http://d.doi.org/10.12988/imf.2013.3475 The Samplig Distributio of the Maimum Likelihood Estimators for the Parameters

### Simple Linear Regression

Simple Liear Regressio 1. Model ad Parameter Estimatio (a) Suppose our data cosist of a collectio of pairs (x i, y i ), where x i is a observed value of variable X ad y i is the correspodig observatio

### LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that

### Statisticians use the word population to refer the total number of (potential) observations under consideration

6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space

### STAT331. Example of Martingale CLT with Cox s Model

STAT33 Example of Martigale CLT with Cox s Model I this uit we illustrate the Martigale Cetral Limit Theorem by applyig it to the partial likelihood score fuctio from Cox s model. For simplicity of presetatio

### Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 11

Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract We will itroduce the otio of reproducig kerels ad associated Reproducig Kerel Hilbert Spaces (RKHS). We will cosider couple

### MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.

XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced

### Solutions: Homework 3

Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe

### 5.1 A mutual information bound based on metric entropy

Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local

### Support vector machine revisited

6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector

### Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

### Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

### The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

### Introduction to Optimization Techniques. How to Solve Equations

Itroductio to Optimizatio Techiques How to Solve Equatios Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually

### ARIMA Models. Dan Saunders. y t = φy t 1 + ɛ t

ARIMA Models Da Sauders I will discuss models with a depedet variable y t, a potetially edogeous error term ɛ t, ad a exogeous error term η t, each with a subscript t deotig time. With just these three

### 5. Likelihood Ratio Tests

1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,

### Computing the maximum likelihood estimates: concentrated likelihood, EM-algorithm. Dmitry Pavlyuk

Computig the maximum likelihood estimates: cocetrated likelihood, EM-algorithm Dmitry Pavlyuk The Mathematical Semiar, Trasport ad Telecommuicatio Istitute, Riga, 13.05.2016 Presetatio outlie 1. Basics

### Point Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10.

Poit Estimatio: properties of estimators fiite-sample properties CB 7.3) large-sample properties CB 10.1) 1 FINITE-SAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:

### Topic 18: Composite Hypotheses

Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:

### s = and t = with C ij = A i B j F. (i) Note that cs = M and so ca i µ(a i ) I E (cs) = = c a i µ(a i ) = ci E (s). (ii) Note that s + t = M and so

3 From the otes we see that the parts of Theorem 4. that cocer us are: Let s ad t be two simple o-egative F-measurable fuctios o X, F, µ ad E, F F. The i I E cs ci E s for all c R, ii I E s + t I E s +

### Section A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics

X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.

### Stat 200 -Testing Summary Page 1

Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece

### SAMPLING LIPSCHITZ CONTINUOUS DENSITIES. 1. Introduction

SAMPLING LIPSCHITZ CONTINUOUS DENSITIES OLIVIER BINETTE Abstract. A simple ad efficiet algorithm for geeratig radom variates from the class of Lipschitz cotiuous desities is described. A MatLab implemetatio

### TEACHER CERTIFICATION STUDY GUIDE

COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra

### Statistical Intervals for a Single Sample

3/5/06 Applied Statistics ad Probability for Egieers Sixth Editio Douglas C. Motgomery George C. Ruger Chapter 8 Statistical Itervals for a Sigle Sample 8 CHAPTER OUTLINE 8- Cofidece Iterval o the Mea

### Lecture 1 Probability and Statistics

Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark

REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d

### Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number

MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios

### w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.

2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For

### Statistical Theory MT 2009 Problems 1: Solution sketches

Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where

### Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;

### IIT JAM Mathematical Statistics (MS) 2006 SECTION A

IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim

### 9. Simulation lect09.ppt S Introduction to Teletraffic Theory - Fall

lect09.ppt S-38.145 - Itroductio to Teletraffic Theory - Fall 2001 1 Cotets Itroductio Geeratio of realizatios of the traffic process Geeratio of realizatios of radom variables Collectio of data Statistical

### ON POINTWISE BINOMIAL APPROXIMATION

Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece

### ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

### Section 14. Simple linear regression.

Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

### Estimation of Gumbel Parameters under Ranked Set Sampling

Joural of Moder Applied Statistical Methods Volume 13 Issue 2 Article 11-2014 Estimatio of Gumbel Parameters uder Raked Set Samplig Omar M. Yousef Al Balqa' Applied Uiversity, Zarqa, Jorda, abuyaza_o@yahoo.com

### Analysis of Experimental Data

Aalysis of Experimetal Data 6544597.0479 ± 0.000005 g Quatitative Ucertaity Accuracy vs. Precisio Whe we make a measuremet i the laboratory, we eed to kow how good it is. We wat our measuremets to be both

### THE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.

THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of

### Final Examination Solutions 17/6/2010

The Islamic Uiversity of Gaza Faculty of Commerce epartmet of Ecoomics ad Political Scieces A Itroductio to Statistics Course (ECOE 30) Sprig Semester 009-00 Fial Eamiatio Solutios 7/6/00 Name: I: Istructor:

### Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

### 5.1 Review of Singular Value Decomposition (SVD)

MGMT 69000: Topics i High-dimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of

### It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.

Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig

### NCSS Statistical Software. Tolerance Intervals

Chapter 585 Itroductio This procedure calculates oe-, ad two-, sided tolerace itervals based o either a distributio-free (oparametric) method or a method based o a ormality assumptio (parametric). A two-sided

### MATH 10550, EXAM 3 SOLUTIONS

MATH 155, EXAM 3 SOLUTIONS 1. I fidig a approximate solutio to the equatio x 3 +x 4 = usig Newto s method with iitial approximatio x 1 = 1, what is x? Solutio. Recall that x +1 = x f(x ) f (x ). Hece,

### Lecture 6 Ecient estimators. Rao-Cramer bound.

Lecture 6 Eciet estimators. Rao-Cramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have

### Topics Machine learning: lecture 2. Review: the learning problem. Hypotheses and estimation. Estimation criterion cont d. Estimation criterion

.87 Machie learig: lecture Tommi S. Jaakkola MIT CSAIL tommi@csail.mit.edu Topics The learig problem hypothesis class, estimatio algorithm loss ad estimatio criterio samplig, empirical ad epected losses

### 6.867 Machine learning, lecture 7 (Jaakkola) 1

6.867 Machie learig, lecture 7 (Jaakkola) 1 Lecture topics: Kerel form of liear regressio Kerels, examples, costructio, properties Liear regressio ad kerels Cosider a slightly simpler model where we omit

### 17 Phonons and conduction electrons in solids (Hiroshi Matsuoka)

7 Phoos ad coductio electros i solids Hiroshi Matsuoa I this chapter we will discuss a miimal microscopic model for phoos i a solid ad a miimal microscopic model for coductio electros i a simple metal.

### Binomial Distribution

0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible

### A note on self-normalized Dickey-Fuller test for unit root in autoregressive time series with GARCH errors

Appl. Math. J. Chiese Uiv. 008, 3(): 97-0 A ote o self-ormalized Dickey-Fuller test for uit root i autoregressive time series with GARCH errors YANG Xiao-rog ZHANG Li-xi Abstract. I this article, the uit

### MAT1026 Calculus II Basic Convergence Tests for Series

MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real

### A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence

Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as

### Statistical inference: example 1. Inferential Statistics

Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

### IE 230 Probability & Statistics in Engineering I. Closed book and notes. No calculators. 120 minutes.

Closed book ad otes. No calculators. 120 miutes. Cover page, five pages of exam, ad tables for discrete ad cotiuous distributios. Score X i =1 X i / S X 2 i =1 (X i X ) 2 / ( 1) = [i =1 X i 2 X 2 ] / (

### Introduction to Artificial Intelligence CAP 4601 Summer 2013 Midterm Exam

Itroductio to Artificial Itelligece CAP 601 Summer 013 Midterm Exam 1. Termiology (7 Poits). Give the followig task eviromets, eter their properties/characteristics. The properties/characteristics of the

### Infinite Sequences and Series

Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

### Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12

Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig

### A Distributional Approach Using Propensity Scores

A Distributioal Approach Usig Propesity Scores Zhiqiag Ta Departmet of Biostatistics Johs Hopkis School of Public Health http://www.biostat.jhsph.edu/ zta Jue 20, 2005 Outlie Itroductio Couterfactual framework

### Markov Decision Processes

Markov Decisio Processes Defiitios; Statioary policies; Value improvemet algorithm, Policy improvemet algorithm, ad liear programmig for discouted cost ad average cost criteria. Markov Decisio Processes

### Dimensionality Reduction vs. Clustering

Dimesioality Reductio vs. Clusterig Lecture 9: Cotiuous Latet Variable Models Sam Roweis Traiig such factor models (e.g. FA, PCA, ICA) is called dimesioality reductio. You ca thik of this as (o)liear regressio

### Basis for simulation techniques

Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios

### R. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State

Bayesia Cotrol Charts for the Two-parameter Expoetial Distributio if the Locatio Parameter Ca Take o Ay Value Betwee Mius Iity ad Plus Iity R. va Zyl, A.J. va der Merwe 2 Quitiles Iteratioal, ruaavz@gmail.com

### Output Analysis and Run-Length Control

IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

### General IxJ Contingency Tables

page1 Geeral x Cotigecy Tables We ow geeralize our previous results from the prospective, retrospective ad cross-sectioal studies ad the Poisso samplig case to x cotigecy tables. For such tables, the test

### Confidence Level We want to estimate the true mean of a random variable X economically and with confidence.

Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio

### Confidence Intervals for the Population Proportion p

Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical: