Parametric Identification of Multiplicative Exponential Heteroskedasticity Alyssa Carlson Department of Economics, Michigan State University East Lansing, MI 48824-1038, United States Dated: October 5, 2017 Abstract Harvey (1976) first proposed multiplicative exponential heteroskedasticity in the context of a linear regression. These days it is more commonly seen in non-linear models such as a binary response model where correctly modeling the heteroskedasticity is imperative for consistent parameter estimates. However, there doesn t appear to be a formal proof of point identification for the parameters in the model. This paper presents several examples that show the conditions presumed throughout the literature are not sufficient for identification. The major contribution of this paper is to provide additional conditions and show identification for such a pervasive model. Keywords: Identification, Heteroskedasticity, Binary Response Model JEL: C35 Electronic address: carls405@msu.edu 1
1 Introduction To motivate the result, consider the latent variable set up for a binary response model with heteroskedasticity. Y = 1{Xβ o + u 0} (1) where X is a 1 k X row vector and u is the idiosyncratic error with a known distribution, possibly heteroskedastic in Z (a 1 k Z row vector of random variables) which is allowed to be related with X. Unlike in a linear model where heteroskedasticity only effects the consistency of the standard error estimates, ignoring the heteroskedasticity in this nonlinear model would result in inconsistent parameter estimates. First proposed by Harvey (1976) in a regression context, assume an exponential linear index for the heteroskedasticity, then the log-likelihood is, ( ( )) ( ( )) Xβo Xβo L = Y log G + (1 Y ) log 1 G exp(zδ o ) exp(zδ o ) (2) where G( ) is the known cumulative distribution function of u. By imposing an exponential transformation, the heteroskedasticity is strictly positive, a necessary requirement for modeling the conditional variance for the idiosyncratic error u. Prior to any estimation, showing identification is necessary first step. For any Extremum Estimator, identification insures the objective function has a unique maximum at the truth (Newey and McFadden (1994)). Returning to the binary response model, since the distribution is known, estimation usually follows a Maximum Likelihood procedure. The following restates the identification definition from Newey and McFadden (1994) for Maximum Likelihood Estimation. Definition 1.1 (Identification). Let f(y X, Z, θ o ) be the conditional probability distribution of Y defined over the measures of X and Z with positive probability. If θ θ o implies P (f(y X, Z, θ) f(y X, Z, θ o )) > 0 then θ o is point identified. If G( ) is strictly monotonic (as in the Probit or Logit cases), then identification requires, For any (β, δ) (β o, δ o ) then P ( Xβo exp(zδ o ) Xβ ) > 0 (3) exp(zδ) 2
The above statement captures the basic identification requirement for models with multiplicative exponential heteroskedasticity. This paper aims to clarify when the statement holds and under what necessary or sufficient conditions. The simplest case of identification would be when X and Z are not functionally related in the sense that the elements of Z cannot be functions of the elements in X. More formally stated, in the discrete case, X and Z are not functionally related if there exists two points, (x, z 1 ) and (x, z 2 ), in the support of joint distribution of (X, Z) such that z 1 z 2. In the continuous case, X and Z are not functionally related if X/ Z is zero with positive probability. A sketch of the proof: suppose Xβ o exp(zδ o ) = Xβ exp(zδ) (4) holds for all X and Z in their support. If β o is non-zero and E(X X) is non-singular then Xβ o is non-zero with positive probability and the equation above can be rearranged to ln(xβ o /Xβ) = Z(δ o δ) (5) If Z is continuous and does not include a constant, then take the derivative with respect to Z. Since X/ Z is zero with positive probability, δ o δ = 0 and consequently, β o β = 0. When Z is discrete, if δ o δ 0, Z does not include a constant, and E(Z Z) is non-singular, then Z(δ o δ) must vary over the support Z. Since the realizations (x, z 1 ) and (x, z 2 ) occur with positive probability such that z 1 z 2, equation (5) cannot hold for δ o δ 0. As a result, δ o = δ and β o = β. Proving identification becomes more involved when one allows for an arbitrary relationship between X and Z. The reason why one would want to allow for an arbitrary relationship can be seen by returning to the latent variable set up in the binary response example. In equation (1), we would like to model the mean of Y conditional on X. Then let σ(x) 2 denote the conditional variance of u. Without entering the domain of semi-parametric identification of β o and σ(x) 1, it seems reasonable to assume a double index model such 1 The works of Klein and Vella (2009), Khan (2013) discuss identification in the semi-parametric case in which Klein and Vella (2009) use a re-indexing approach following Ichimura and Lee (1991) while Khan (2013) shows that the semi-parametric heteroskedastic probit model is observationally equivalent to a conditional median restriction. 3
that, ( ) ( ) Xβo Xβo E(Y X) = Φ = Φ σ(x) exp(zδ o ) (6) where Z consists of functions the elements in X. Therefore it would be useful to have a general statement of identification that allows for an arbitrary relationship between X and Z. If one were to assume that E(X X) is non-singular and β o is non-zero, then one may rearrange the equality in equation (4) to X(β o exp(z(δ δ o ))β) = 0 (7) Now it is easier to see how the relationship between X and Z will determine whether or not the model is identified. If there is some way that Xβ o is equal to a scaling of Xβ by exp(z(δ δ o )) for all X, then the model is not identified. This paper will look at two ways the scaling of Xβ by exp(z(δ δ o )) can be manipulated: through the joint support of (X, Z) and through the functional form of the heteroskedasticity, exp(zδ o ). The main contribution of this paper is to first show the conditions set by the literature are not sufficient for identification and to provide new and sufficient conditions for identification. 2 Main Results Of all the models with multiplicative exponential heteroskedasticity, the heteroskedastic probit (or logit) models are by far the most popular. Standard text (Greene (2011) and Wooldridge (2010)) state that models that the heteroskedastic probit model is estimable under the following conditions, Condition 1. Z does not contain a constant. Condition 2. E(X X) is nonsingular. Condition 3. E(Z Z) is nonsingular. Condition 1 is necessary since the probit model is only identified up to scale. Conditions 2 and 3 are needed in order to show Xβ o = Xβ and Zδ o = Zδ implies β o = β and δ o = δ respectively. But even under these conditions, there are contrived examples in which identification fails. 4
2.1 Example: Binary Suppose X = (1, Z) where Z is a binary variable (like a treatment) in which we are concerned there is heteroskedasticity with respect to the treatment. Then equation (7) can be decomposed to, β 1o β 1 if Z = 0 X(β o exp(z(δ δ o ))β) = β 1o + β 2o exp(δ δ o )(β 1 + β 2 ) if Z = 1 (8) The first part implies β 1 = β 1o. Plugging into the second part, equation (7) holds if β 2 = exp(δ o δ)β 2o β 1o (1 exp(δ o δ)) which does not imply δ = δ o or β 2 = β 2o. Even though conditions 1-3 are satisfied, identification is loss because under the binary support of Z, the parameters β 2o and δ o are inherently linked. There are several sufficient conditions in which the specification can be altered to obtain identification, the simplest of which is to require Z has at least three or more points in its support. This can be achieved by either expanding Z to multi-valued or to include an additional random variable in Z. When Z only has two points in its support, Z exp(z(δ o )) is linear in Z and therefore the heteroskedasticity is impacting the model in a way that is not separately identifiable from the mean component Zβ o. By requiring three or more points in its support, the effect of the heteroskedasticity is exponential while the effect of the mean component is linear which can be separately identified. 2.2 Example: Exponential Transformation Unlike the previous example, this example does not manipulate the support of (X, Z) but rather takes advantage of the functional form of the heteroskedasticity. Suppose X = (1, exp(z)) in which Z is a strictly positive random variable. Then equation (7) becomes, X(β o exp(z(δ δ o ))β) = β 1o + exp(z)β 2o exp(z(δ δ o ))β 1 exp(z(1 + δ δ o ))β 2 If δ δ o = 1 and β 1 = β 2o = 0, then any values β 1o = β 2 make the above equation equal to 0 for all values of X 1. Alternatively if δ δ o = 1 and β 1o = β 2 = 0, then any values β 1 = β 2o also make the above equation equal to 0. But this only works for the exponential transformation because the heteroskedasticity is 5
of exponential form. By imposing the same transformation in the mean term as in the heteroskedastic term, it becomes difficult to differentiate between the linear mean effect Xβ o and the exponential heteroskedastic effect exp(zδ o ). To rule out the contrived non-identified cases, one merely needs to assume that β 1o and β 2o are non-zero. This provides bite for separately identifying the heteroskedastic effect from the mean effect. These two counter examples show that Conditions 1-3 are not sufficient for identification. They manipulate the support of the random variables and the form of the heteroskedasticity to lose identification. Therefore I am proposing two additional conditions for identification. Condition 4. X includes a constant such that Xβ o = β 1o + X 2 β 2o, and the parameter space of (β 1o,β 2o ) excludes the cases where either are zero. Condition 5. The joint support of X 2 and Z must contain 3 or more points. The first condition rules out the exponential transformation example while the second condition rules out the binary example. Including a non-zero intercept is not a restrictive condition as in most empirical applications, one would expect a location normalization. It is also not that restrictive to require β 2o is non-zero as it would be strange for there to be heteroskedasticity without a mean effect. Moreover, Condition 4 is sufficient but not necessary. When a simple relationship between X and Z are define (ie: not exponential transformations or defined over restricted support of the other) then Condition 4 is unnecessary. Both of these conditions are used to insure that the exponential heteroskedastic effect can be separately identified from the linear mean effect. With the addition of Conditions 4-5, I can now show identification form multiplicative exponential heteroskedasticity. Theorem 1. In the case of multiplicative exponential heteroskedasticity, ( ) Xβ o f(y X, Z) = g Y, exp(zδ o ) (9) where g(y, ) is strictly monotonic. If Conditions 1-5 hold, then the parameters β o and δ o are identified. Proof. Suppose for some β and δ in the parameter space, equation (7) holds with positive probability. 6
This can be rearranged to: ( ) β1o + X 2 β 2o Z(δ o δ) = log β 1 + X 2 β 2 (10) If δ o δ = 0 this immediately implies β 1 = β 1o and β 2 = β 2o. So suppose δ o δ = 0. Then the left hand side of the above equation is linear in Z and can therefore only hold if the right hand side is also linear in Z. If the joint support of X 2 and Z only contains two points this is immediately satisfied. But by condition 5, the joint support of X 2 and Z must contain 3 or more points, therefore equation (10) will only hold if X 2 is some function of Z that undoes the log transformation. But since the parameter space excludes the cases of β 1o, β 1, β 2o, or β 2 being zero, there is no two linear combinations of a function whose ratio will result in an exponential function. As we saw in the simple example with the exponential transformation, X 2 = exp(z) will undo the log transformation but requires β 1o or β 2o is zero. But by Condition 4, the right hand side of equation (10) cannot be linear in Z. Consequently, δ o δ cannot be non-zero. 3 Conclusion Unlike the homoskedastic probit (or logit) model where identification was shown in Newey and McFadden (1994), identification of a heteroskedastic probit model with is not as apparent. It has been widely accepted that the model was estimable under Conditions 1-3. This paper has provided two examples in which those conditions were satisfied but point identification is not obtainable. By recognizing that the conditions previously stated in the literature are not sufficient in distinguishing a linear effect from an exponential effect, this paper provided two additional conditions and a proof of identification. 4 Acknowledgments This is a revised section of the first chapter of my MSU Ph.D. dissertation. I would like to thank Professor Kyoo Il Kim and Professor Wooldridge for their thoughtful comments and assistance. The author does not have any conflicts of interest. This research did not receive any specific grant from funding agencies in the 7
public, commercial, or not-for-profit sectors. References Greene, W.H. (2011), Econometric Analysis. Pearson Education, URL https://books.google.com/books?id=lwquaaaaqbaj. Harvey, A. C. (1976), Estimating regression models with multiplicative heteroscedasticity. Econometrica, 44, 461 465, URL http://www.jstor.org/stable/1913974. Ichimura, Hidehiko and Lung-Fei Lee (1991), Semiparametric least squares estimation of multiple index models: single equation estimation. In Nonparametric and semiparametric methods in econometrics and statistics: Proceedings of the Fifth International Symposium in Economic Theory and Econometrics. Cambridge, 3 49, Cambridge University Press, URL https://books.google.com/books?id=whtszjdi2h0c. Khan, Shakeeb (2013), Distribution free estimation of heteroskedastic binary response models using probit/logit criterion functions. Journal of Econometrics, 172, 168 182, URL http://www.sciencedirect.com/science/article/pii/s0304407612001753. Klein, Roger and Francis Vella (2009), A semiparametric model for binary response and continuous outcomes under index heteroscedasticity. Journal of Applied Econometrics, 24, 735 762, URL http://dx.doi.org/10.1002/jae.1064. Newey, Whitney K. and Daniel McFadden (1994), Chapter 36 large sample estimation and hypothesis testing. Handbook of Econometrics, 4, 2111 2245, URL http://www.sciencedirect.com/science/article/pii/s1573441205800054. Wooldridge, J.M. (2010), Econometric Analysis of Cross Section and Panel Data. Econometric Analysis of Cross Section and Panel Data, MIT Press, URL https://books.google.com/books?id=yov6aqaaqbaj. 8