A Simple Nonparametric Approach to Estimating the Distribution of Random Coefficients in Structural Models

Transcription

1 A Simple Nonpaametic Appoach to Estimating the Distibution of Random Coefficients in Stuctual Models Jeemy T. Fox Rice Univesity & NBER Kyoo il Kim Michigan State Univesity May 2016 Chenyu Yang Univesity of Rocheste Abstact We exploe least squaes and likelihood nonpaametic mixtues estimatos of the joint distibution of andom coefficients in stuctual models. The estimatos fix a gid of heteogenous paametes and estimate only the weights on the gid points, an appoach that is computationally attactive compaed to altenative nonpaametic estimatos. We povide conditions unde which the estimated distibution function conveges to the tue distibution in the weak topology on the space of distibutions. We veify most of the consistency conditions fo thee discete choice models. We also deive the convegence ates of the least squaes nonpaametic mixtues estimato unde additional estictions. We pefom a Monte Calo study on a dynamic pogamming model. Keywods: Random coefficients, mixtues, discete choices, dynamic pogamming, sieve estimation Coesponding authos: Jeemy Fox at jeemyfox@gmail.com and Kyoo il Kim at kyookim@msu.edu. 1

2 1 Intoduction Economic eseaches often wok with models whee the paametes ae heteogeneous acoss the population. A classic example is that consumes may have heteogeneous pefeences ove a set of poduct chaacteistics in an industy with diffeentiated poducts. These heteogeneous paametes ae often known as andom coefficients. When woking with coss sectional data, the goal is often to estimate the distibution of heteogeneous paametes. Ou pape establishes the consistency and ates of convegence of fixed gid nonpaametic estimatos fo a distibution of heteogeneous paametes due to Bajai, Fox and Ryan 2007), Tain 2008, Section 6), Fox, Kim, Ryan and Bajai 2011), and Koenke and Mizea 2014). These estimatos ae computationally simple than some altenatives. We use FKRB to efe to Fox, Kim, Ryan and Bajai 2011). We estimate the distibution of heteogeneous paametes F β) in the model P j x) = g j x, β) df β), 1) whee j is the index of the jth out of J finite values of the outcome y, x is a vecto of obseved explanatoy vaiables, β is the vecto of heteogeneous paametes, and g j x, β) is the pobability that the jth outcome occus fo an obsevation with heteogeneous paametes β and explanatoy vaiables x. Given this stuctue, P j x) is the coss sectional pobability of obseving the jth outcome when the explanatoy vaiables ae x. The eseache picks g j x, β) as the undelying model, has an i.i.d. sample of N obsevations y i, x i ), and wishes to estimate F β). As F is only esticted to be a valid CDF, the mixtue model 1) is nonpaametic. The unknown distibution F β) entes 1) linealy. The estimatos we analyze exploit lineaity and achieve a computationally simple estimato than some altenatives. All the fixed gid estimatos divide the suppot of the vecto β into a finite and known gid of vectos β 1,..., β R. Computationally, the unknown paametes ae the weights θ 1,..., θ R on the R gid points. These can be estimated using a least squaes o likelihood citeion with the constaints that each θ 0 and that R θ = 1. The estimato of the distibution F β) with N obsevations and R gid points becomes ˆF N β) = R ˆθ 1 [β β], whee ˆθ s denote estimated weights and 1 [β β] is equal to 1 when β β. Computationally, the least squaes and likelihood constained optimization poblems ae globally convex and concave, espectively. Paticula numeical algoithms ae guaanteed to convege to a global optimum. FKRB discuss the advantages of this estimato fo complex stuctual models, like dynamic pogamming models with heteogeneous paametes. In this espect, fixed gid estimatos shae some computational advantages with the paametic appoach in Ackebeg 2009). Ou Monte Calo study in an online appendix is to a discete choice, dynamic pogamming model. FKRB and othe pevious analyses assume that the R gid points used in a finite sample ae indeed the tue gid points that contain the finite suppot of the tue F 0 β). Thus, the tue distibution F 0 β) is assumed to be known up to a finite numbe of weights θ 1,..., θ R. As economists often lack convincing economic ationales to pick one set of gid points ove anothe, assuming that the eseache knows 2

3 the tue distibution up to finite weights is unealistic. Instead of assuming that the distibution is known up to weights θ 1,..., θ R, this pape equies the tue distibution F 0 β) to satisfy much weake estictions. In paticula, the tue F 0 β) can have any of continuous, discete and mixed continuous and discete suppots. The pio appoaches ae paametic as the tue weights θ 1,..., θ R lie in a finite-dimensional subset of a eal space. Hee, the appoach is nonpaametic as the tue F 0 β) is known to lie only in the infinite-dimensional space of multivaiate distibutions on the space of heteogeneous paametes β. In a finite sample of N obsevations, ou estimatos ae still implemented by choosing a fixed gid of points θ 1,..., θ R, ideally to tade off bias and vaiance in the estimate ˆF N β). We, howeve, ecognize that as the sample inceases, R and thus the fineness of the gid of points should also incease in ode to educe the bias in the appoximation of F β). We wite R N) to emphasize that the numbe of gid points and implicitly the gid of points itself) is now a function of the sample size. The main theoem in ou pape is that, unde estictions on the economic model and an appopiate choice of R N), ou least squaes and likelihood estimatos ˆF N β) convege to the tue F 0 β) as N, in a function space. We use the Lévy-Pokhoov metic, a common metization of the weak topology on the space of multivaiate distibutions. We ecognize that the nonpaametic vesions of ou estimatos ae special cases of sieve estimatos Chen 2007). Sieve estimatos estimate functions by inceasing the flexibility of the appoximating class used fo estimation as the sample size inceases. A sieve estimato fo a smooth function might use an appoximating class defined by a Fouie seies, fo example. As we ae motivated by pactical consideations in empiical wok, ou estimatos choice of basis, a finite gid of points, is justified by the estimatos computational simplicity. Futhe and unlike a typical sieve estimato, we need to constain ou estimated functions to be valid distibution functions. Ou constained least squaes and likelihood appoaches ae both computationally simple and ensue that the estimated CDFs satisfy the theoetical popeties of a valid CDF. Because ou estimatos ae sieve estimatos, we pove thei consistency by satisfying high-level conditions fo the consistency of sieve extemum estimatos, as given in an appendix lemma in Chen and Pouzo 2012). We epeat this lemma and its poof in ou pape so ou consistency poof is selfcontained. Ou fixed gid estimatos ae not a special case of the two-step sieve estimatos exploed using lowe-level conditions in the main text of Chen and Pouzo. 1 We pove the consistency of ou estimatos fo the distibution of heteogeneous paametes, in function space unde the weak topology. We pesent sepaate theoems fo mixtues of discete gid points and mixtues of continuous densities with a gid of points ove the paametes of each density. The theoem fo the mixtue of gid points equies the heteogeneous paametes to lie in a, not necessaily known, compact set. The theoem fo a mixtue of continuous densities allows fo unbounded suppot of the heteogeneous paametes. Ou consistency theoems ae not specific to the economic model being estimated. We povide the ate of convegences fo a subset of the models handled by ou consistency theoem, namely those that ae diffeentiable in the heteogeneous paametes, which include the andom 1 Note that unde the Lévy-Pokhoov metic on the space of multivaiate distibutions, the poblem of optimizing the population objective function ove the space of distibutions tuns out to be well posed unde the definition of Chen 2007). Thus, ou method does not ely on a sieve space to egulaize the estimation poblem to addess the ill-posed invese poblem, as much of the sieve liteatue focuses on. 3

4 coefficients logit model. The convegence ates, the asymptotic estimation eo bounds, consist of two tems: the bias and the vaiance. While obtaining the vaiance tem is athe standad in the sieve estimation liteatue, deiving the bias tem depends on the specific appoximation methods e.g., powe seies o splines). Because ou use of appoximating functions is new in the sieve estimation liteatue, deiving the bias tem is not tivial. We povide the bias tem, which is the smallest possible appoximation eo of the tue function using sieves fo the class of models we conside. Ou ate of convegence esults highlight an impotant pactical issue with any nonpaametic estimato: thee is a cuse of dimensionality in the dimension of the heteogeneous paametes. Lage sample sizes will be needed if the vecto of heteogeneous paametes has moe elements. Futhe, the ate esults indicate that ou baseline estimato is not pactical when thee is a lage numbe of heteogeneous paametes. In high dimensional settings, we suggest allowing heteogeneous paametes on only a subset of explanatoy vaiables and estimating homogenous paametes on the emaining explanatoy vaiables. We extend ou consistency esult to models whee some paametes ae homogeneous. Howeve, including homogeneous paametes equies nonlinea optimization, which loses some of the computational advantages of ou estimatos. We povide a Monte Calo study in an online appendix. We estimate a dynamic pogamming, discete choice model, adding heteogeneous paametes to the famewok of Rust 1987). The dynamic pogamming poblem must be solved once fo each ealization of the heteogeneous paametes. We pesent esults fo both the fixed gid likelihood and least squaes estimatos as well as, fo compaison, a likelihood estimato whee we estimate both the gid of points and the weights on those points. We show that ou fixed gid estimatos have supeio speed but infeio statistical accuacy compaed to the moe usual appoach of estimating a flexible gid. The outline of ou pape is as follows. Section 2 pesents thee examples of discete choice mixtue models. Section 3 intoduces the estimation pocedues. Section 4 demonstates consistency of ou estimatos in the space of multivaiate distibutions. Section 5.1 extends ou consistency esults to models with both heteogeneous paametes and homogeneous paametes and Section 5.2 consides mixtues of smooth basis densities. Section 6 veifies most of the pimitive conditions fo consistency established in Section 4 using the thee examples of mixtue models in Section 2. Section 7 deives the convegence ates of the nonpaametic estimato fo a class of models. Finally, an online appendix pesents the Monte Calo study. 2 Examples of Mixtue Models In ou famewok, the object the econometician wishes to estimate is F β), the distibution of the vecto of heteogeneous paametes β. One definition of identification is that a unique F β) solves 1) fo all x and all outcomes j = 1,..., J. This is the definition used in cetain elevant papes on identification in the statistics liteatue, fo example Teiche 1963). We will etun to these thee examples of discete choice models late in the pape. Each example consides economic models with heteogeneous paametes that play a lage ole in empiical wok. Some of the example models ae nested in othes, but veification of the conditions fo consistency in Section 6 will use additional estictions on the suppots of x and β that ae non-nested acoss models. 4

5 Example 1. logit) Let thee be a multinomial choice model such that y is one of J unodeed choices, such as types of cas fo sale. Fo j 2, the utility of choice j to consume i is u i,j = x i,j β i +ɛ i,j, whee x i,j is a vecto of obsevable poduct chaacteistics of choice j and the demogaphics of consume i, β i is a vecto of andom coefficients giving the maginal utility of each ca s chaacteistics to consume i, and ɛ i,j is an additive, consume- and choice-specific eo. Thee is an outside good 1 with utility u i,1 = ε i,1. The consume picks choice j when u i,j > u i,h h j. The andom coefficients logit model occus when ɛ i,j is known to have the type I exteme value distibution. In this example, 1) becomes, fo j 2, P j x) = g j x, β) df β) = ) exp x j β 1 + J h=2 exp β), x hβ)df whee x = x 2,..., x J ). A simila expession occus fo othe choices h j. Compaed to pio empiical wok using the andom coefficients logit, ou goal is to estimate F β) nonpaametically. Example 2. binay choice) Let J = 2 in the pevious example, so that thee is one inside good and one outside good. Thus, the utility of the inside good 2 is u i,2 = ɛ i + x i β 2,i, whee β i = ɛ i, β 2,i ) is seen as one long vecto and ɛ i supplants the logit eos in Example 1 and plays the ole of a andom intecept. The outside good 1 has utility u i,1 = 0. In this example, 1) becomes, fo j = 2, P 2 x) = g 2 x, β) df β) = 1 [ ɛ + x β 2 0 ] df β), whee 1 [ ] is the indicato function equal to 1 if the inequality in the backets is tue. Without logit eos, the joint distibution of both the intecept and the slope coefficients is estimated nonpaametically. In this example, g 2 x, β) is discontinuous in β. Example 3. multinomial choice without logit eos) Conside a multinomial choice model whee the distibution of the peviously logit eos is also estimated nonpaametically. In this case, the utility to choice j 2 is u i,j = x β i,j i + ɛ i,j and the utility of the outside good 1 is u i,1 = 0. The notation β ) i is used because the full heteogeneous paamete vecto is now β i = βi, ɛ i,2,..., ɛ i,j, which is seen as one long vecto. We will not assume that the additive eos ɛ i,j ae distibuted independently of β i o of each othe. In this example, 1) becomes, fo j 2, P j x) = g j x, β) df β) = [ 1 x β j + ɛ j max {0, x β } ] h + ɛ h h j, h 2 df β). 3 Estimato We analyze both least squaes linea pobability models) and maximum likelihood citeia. We fist discuss the least squaes citeion, fom FKRB. Recall that y i,j is equal to 1 wheneve the outcome y i fo the ith obsevation is j, and 0 othewise. Stat with the model 1) and add y i,j to both sides while 5

6 moving P j x) to the ight side. Fo the statistical obsevation i, this gives y i,j = g j x i, β) df β) + y i,j P j x i )). 2) By the definition of P j x), the expectation of the composite eo tem y i,j P j x), conditional on x, is 0. This is a linea pobability model with an infinite-dimensional paamete, the distibution F β). We could wok diectly with this equation if it was computationally simple to estimate this infinite-dimensional paamete while constaining it to be a valid CDF. Instead, we wok with a finite-dimensional sieve space appoximation to F. In paticula, we let R N) be the numbe of gid points in the gid B RN) = β 1,..., β RN)). A gid point is a vecto if β is a vecto, so R N) is the total numbe of points in all dimensions. The eseache chooses B RN). Given the choice of B RN), the eseache estimates θ = θ 1,..., θ RN)), the weights on each of the gid points. With this appoximation, 2) becomes y i,j RN) θ g j x i, β ) + y i,j P j x)). 3) We use the symbol to emphasize that 3) uses a sieve appoximation to the distibution function F β). Because each θ entes y i,j linealy, we estimate θ 1,..., θ RN)) using the linea pobability model egession of y i,j on the R egessos z i,j = g j x i, β ). To be a valid CDF, θ 0 and RN) θ = 1. Theefoe, the estimato is 1 N J θ = ag min θ NJ i=1 y i,j RN) subject to θ 0 = 1,..., R N) and θ zi,j RN) 2 θ = 1. Thee ae J egession obsevations fo each statistical obsevation y i, x i ). This minimization poblem is a quadatic pogamming poblem subject to linea inequality constaints. The minimization poblem is convex and outines like MATLAB s lsqlin guaantee finding a global optimum. One can constuct the estimated cumulative distibution function fo the heteogeneous paametes as ˆF N β) = RN) ˆθ 1 [β β]. Thus, we have a stuctual estimato fo a distibution of heteogeneous paametes in addition to a flexible method fo appoximating choice pobabilities. Following Tain, we can also use the log-likelihood citeion divided by the sample size) L = RN) N log θ zi,y i /N, i=1 4) 6

7 whee the zi,y i ae the pobabilities computed above fo the obseved outcome y i fo obsevation i. As with the least squaes citeion, we enfoce the constaints θ 0 = 1,..., R N) and RN) θ = 1. Computationally, one can use the EM algoithm in Tain s Section 6 o a nonlinea, gadient-based seach outine, which is available in most scientific packages. The pefomance of the gadient-based seach outine will be impoved if the gadient of the likelihood is povided to the solve in closed fom. The sth element of that gadient is L N θ s = zi,y s i RN) /N. i=1 θ zi,y i The log likelihood is globally concave and any local maximum found will be the global maximum. The fixed gid appoach, whethe based on a least squaes o a likelihood citeion, has two main advantages ove othe appoaches to estimating distibutions of heteogeneous paametes. Fist, the appoach is computationally simple: we can always find a global optimum and, by solving fo zi,j = g j x i, β ) befoe optimization commences, we avoid many evaluations of complex stuctual models such as dynamic pogamming poblems. Second, the appoach is nonpaametic. In the next section, we show that if the gid of points is made fine as the sample size N inceases, the estimatos ˆF N β) convege to the tue distibution F 0. We do not need to impose that F 0 lies in known paametic family. On the othe hand, a disadvantage is that the estimates may be sensitive to the choice of tuning paametes. While most nonpaametic appoaches equie choices of tuning paametes, hee the choice of a gid of points is a paticulaly high-dimensional tuning paamete. FKRB popose cossvalidation methods to pick these tuning paametes, including the numbe of gid points, the suppot of the points, and the gid points within the suppot. expx Example. 1 logit) Fo the logit example, i,j β ) 1+ J h=2 expx i,h β ) = g j x i, β ). To implement the least squaes estimato, fo each statistical obsevation i, the eseache computes R J pobabilities zi,j = expx i,j β ) 1+ J h=2 expx i,h β ). This computation is done befoe optimization commences. The outcome fo choosing the outside good 1 does not need to be included in the objective function, as J g j x i, β ) = 1. To implement the likelihood estimato, the eseache computes R pobabilities z i,y i fo each statistical obsevation i. Remak 1. FKRB discuss the case of panel data on T peiods. Let yi T = y i,1,..., y i,t ) be the actual sequence of T outcomes fo panel obsevation i. Similaly, let the explanatoy vaiables be x i,1,..., x i,t ). The likelihood citeion fo panel data is L = RN) N log θ z /N, i,yi T i=1 whee z i,y T i = T t=1 g y i,t x i,t, β ). We do not exploe panel data in ou theoetical esults. 7

8 4 Consistency in Function Space Assume that the tue distibution function F 0 lies in the space F of distibution functions on the suppot B of the heteogeneous paametes β. We wish to show that the estimated distibution function ˆF N β) = RN) ˆθ 1 [β β] conveges to the tue F 0 F as the sample size N gows lage. To pove consistency, we use the esults fo sieve estimatos developed by Chen and Pouzo 2012), heeafte efeed to as CP. We define a sieve space to appoximate F as { F R = F F β) = { R θ θ 1 [β β], θ R 1,..., θ R) θ 0, }} R θ = 1, fo a choice of gid B R = β 1,..., β R) that becomes fine as R inceases. We equie F R F S F fo S > R, o that lage sieve spaces encompass smalle sieve spaces. The choice of the gids and R N) ae up to the eseache; howeve consistency will equie conditions on these choices. Based on CP, we pove that the estimato ˆF N conveges to the tue F 0. In thei main text, CP study sieve minimum distance estimatos that involve a two-stage pocedue. Ou estimato is a one-stage sieve least squaes estimato Chen, 2007) and so we cannot poceed by veifying the conditions in the theoems in the main text of CP. Instead, we show its consistency based on CP s geneal consistency theoem in thei appendix, thei Lemma A.2, which we quote in the poof of ou consistency theoem fo completeness. As a consequence, ou consistency poof veifies CP s high-level conditions fo the consistency of a sieve extemum estimato. Fist, we conside the least squaes citeion and then the likelihood citeion follows. Let y i denote the J 1 finite vecto of binay outcomes y i,1,..., y i,j ) and let gx i, β) denote the coesponding J 1 vecto of choice pobabilities g 1 x i, β),..., g J x i, β)) given x i and the heteogeneous paamete β. Then we can define ou sample citeion function fo least squaes as ˆQ N F ) 1 N NJ i=1 y i gx i, β)df β) 2 E = 1 N NJ i=1 y i R θ gx i, β ) fo F F RN), whee E denotes the Euclidean nom. We can ewite ou estimato as ˆF N = agmin F FRN) ˆQN F ) + C ν N 6) whee we can allow fo some toleance slackness) of minimization, C ν N, that is a positive sequence tending to zeo as N gets lage, if necessay. Also let [ ] QF ) E y 2 g x, β) df β) /J be the population objective function. As a distance measue fo distibutions, we use the Lévy-Pokhoov metic, denoted by d LP ), which is a metization of the weak topology fo the space of multivaiate distibutions F. The Lévy- Pokhoov metic in the space of F is defined on a metic space B, d) with its Boel sigma algeba ΣB). We use the notation d LP F 1, F 2 ), whee the measues ae implicit. This denotes the Lévy- Pokhoov metic d LP µ 1, µ 2 ), whee µ 1 and µ 2 ae pobability measues coesponding to F 1 and F 2. E 2 E 5) 8

9 The Lévy-Pokhoov metic is defined as d LP µ 1, µ 2 ) = inf {ɛ > 0 µ 1 C) µ 2 C ɛ ) + ɛ and µ 2 C) µ 1 C ɛ ) + ɛ fo all Boel measuable C ΣB)}, whee C B and C ɛ = {b B a C, d a, b) < ɛ}. The Lévy-Pokhoov metic is a metic, so that d LP µ 1, µ 2 ) = 0 only when µ 1 = µ 2. See Hube 1981, 2004). The following assumptions ae on the economic model and data geneating pocess. We wite P x, F ) = gx, β)df β). Assumption Let F be a space of distibution functions on a finite-dimensional eal space B. F is compact in the weak topology and contains the tue F Let y i, x i )) N i=1 be i.i.d. 3. Let β be independently distibuted fom x. 4. Assume the model g x, β) is identified, meaning that fo any F 1 F 0, F 1 F, the set X X whee P x, F 0 ) P x, F 1 ) has a positive measue in X Q F ) is continuous on F in the d LP, ) metic. Assumption 1.1, the compactness of F is satisfied if B itself is compact in Euclidean space Pathasaathy 1967, Theoem 6.4). Unfotunately, the compactness of B ules out some examples such as nomal distibutions of heteogeneous paametes. In pat to addess this, Section 5.2 povides a consistency theoem fo a elated estimato, which can use mixtues of nomal distibutions, whee the suppot of the heteogeneous paametes is allowed to be R K. Assumptions 1.2 and 1.3 ae standad fo nonpaametic mixtues models with coss-sectional data. Assumption 1.4 equies that the model be identified at a set of values of x that occus with positive pobability. The assumption ules out so-called fagile identification that could occu at values of x with measue zeo such as identification at infinity). Assumptions 1.4 and 1.5 need to be veified fo each economic model g x, β). We will discuss these assumptions fo ou thee examples in Section 6. Remak 2. Assumption 1.5 is satisfied when gx, β) is continuous in β fo all x because in this case P x, F ) is also continuous on F fo all x in the Lévy-Pokhoov metic. Then by the dominated convegence theoem, the continuity of Q F ) in the d LP, ) metic follows fom the continuity of P x, F ) on F fo all x and P x, F ) 1 unifomly bounded). Hee the continuity of P x, F ) on F means fo any F 1 F and such that d LP F 1, F 2 ) 0 it must follow that g j x, β)df 1 β) g j x, β)df 2 β) 0 fo all j. This holds by the definition of weak convegence when gx, β) is continuous and bounded and because the Lévy-Pokhoov metic is a metization of the weak topology. 2 This is with espect to the pobability measue of the undelying pobability space. This pobability is well defined whethe x is continuous, discete o some elements of x ae functions of othe elements e.g. polynomials o inteactions). 9

10 Remak 3. If the suppot B is a finite set, the continuity of Q F ) holds even when g j x, β) is discontinuous, because in this case the Lévy-Pokhoov metic becomes equivalent to the total vaiation metic see Hube 1981, p.34). This implies g j x, β)df 1 β) g j x, β)df 2 β) 0 fo any F 1, F 2 F such that d LP F 1, F 2 ) 0, in pat because g j x, β) is bounded between 0 and 1. We do not have a counteexample to continuity when B is not a finite set. Note that ou consistency esult fo a mixtues of paametic densities, pesented in Section 5.2, has a continuity condition that is easie to veify fo discontinuous g j x, β), as Section 6 discusses. In addition to Assumption 1, we also equie that the gid of points be chosen so that the gid B R becomes dense in B in the usual topology on the eals. Condition 1. Let the choice of gids satisfy the following popeties: 1. Let B R become dense in B as R. 2. F R F R+1 F fo all R RN) as N and it satisfies RN) log RN) N 0 as N. The fist two pats of this condition have peviously been mentioned and ensue that the sieve spaces give inceasingly bette appoximations to the space of multivaiate distibutions. Condition 1.3 specifies a ate condition so that the convegence of the sample citeion function ˆQ N F ) to the population citeion function QF ) is unifom ove F R. Unifom convegence of the citeion function and identification ae both key conditions fo consistency. Theoem 1. Suppose Assumption 1 and Condition 1 hold. Then, d LP ˆFN, F 0 ) p 0. See Appendix B.1 fo the poof of the fothcoming Theoem 2, which nests Theoem 1. Remak 4. Appendix A poves that this estimation poblem is well posed unde the definition of Chen 2007). Next, we conside the distibution function estimato using the log-likelihood citeion. Define the population citeion function and its sample analog, espectively, as J J QF ) Q ML F ) E y i,j log P j x i, F ) = E y i,j log g j x i, β)df β) and ˆQ N F ) ML ˆQ N F ) 1 N N i=1 J y i,j log P j x i, F ) = 1 N N J i=1 y i,j log g j x i, β)df β) fo F F RN). Then we can obtain the ML estimato as in 6) and denote the esulting estimato by ˆF ML. We obtain the following coollay fo the consistency of this ML estimato N 10

11 Coollay 1. Suppose Assumption 1 and Condition 1 hold. Futhe suppose that P j x, F ) is bounded away fom zeo fo all j and F F. Then, d LP ˆF ML N, F 0) p 0. See Appendix B.2 fo the poof. Remak 5. The liteatue on sieve estimation has not established geneal esults on the asymptotic distibution of sieve estimatos, in function space. Howeve, fo ich classes of appoximating basis functions that do not include ou appoximation poblem, the liteatue has shown conditions unde which finite dimensional functionals of sieve estimatos have asymptotically nomal distibutions. In the case of nonpaametic heteogeneous paametes, we might be inteested in infeence in the mean o median of β. Fo demand estimation, say Example 3, we might be inteested in aveage esponses o elasticities) of choice pobabilities with espect to changes in paticula poduct chaacteistics. Let Π N F 0 be a sieve appoximation to F 0 in ou sieve space F RN). If we could obtain an eo bound fo d LP Π N F 0, F 0 ), we could also deive the convegence ate in the Lévy-Pokhoov metic Chen 2007). If the eo bound shinks fast enough as RN) inceases, we conjectue that we could also pove that plug-in estimatos fo functionals of F 0 ae asymptotically nomal Chen, Linton, and van Keilegom 2003). 3 Eo bounds fo discete appoximations ae available in the liteatue fo a class of paametic distibutions F, but we ae not awae of esults fo the unesticted class of multivaiate distibutions. Fo a subset of poblems, including the andom coefficients logit, we ae able to deive appoximation eo bounds. We povide convegence ates fo these cases in Section 7. 5 Extensions 5.1 Models with Homogenous Paametes In many empiical applications, it is common to have both heteogeneous paametes β and finitedimensional paametes γ Γ R dimγ). We wite the model choice pobabilities as gx, β, γ) and the aggegate choice pobabilities as P x, F, γ). Hee we conside the consistency of estimatos fo models with both homogenous paametes and heteogeneous paametes. Fo conciseness, we state a theoem fo the least squaes citeion and omit a coollay fo the likelihood citeion. Remak 6. Estimating a model allowing a paamete to be a heteogeneous paamete when in tuth the paamete is homogeneous will not affect consistency if the model with heteogeneous paametes is identified. Remak 7. Seaching ove γ as a homogeneous paamete fo the least squaes citeion equies nonlinea least squaes. The optimization poblem may also not be globally convex. The objective function may not be diffeentiable fo ou examples whee gx, β, γ) involves an indicato function. Ou estimato fo models with homogeneous paametes is defined as similaly to 6)) ˆγ N, ˆF N ) = agmin γ,f ) Γ FRN) ˆQN γ, F ) + C ν N, 3 We conjectue that we could pove an analog to Theoem 2 in Chen et al 2003) if we could veify analogs to conditions 2.4) 2.6) in that pape fo ou sieve space. 11

12 whee ˆQ N γ, F ) denotes the coesponding sample citeion function. Qγ, F ) is the population citeion function based on the model gx, β, γ). An altenative computational stategy is pofiling, as in ˆF N γ) = agmin F FRN) ˆQN γ, F ) + C ν N fo all γ Γ. Pofiling gives us ˆγ N = agmin γ Γ ˆQN γ, ˆF ) N γ) + C ν N, and theefoe ˆF N = ˆF N ˆγ N ). We make the following assumptions to pove consistency fo models with homogenous paametes. Assumption Let A Γ F whee F is compact in the weak topology and Γ is a compact subset of R dimγ) and A contains the tue γ 0, F 0 ). 2. Let y i, x i )) N i=1 be i.i.d. 3. Let β be independently distibuted fom x. 4. Assume the model g x, β, γ) is identified, meaning that fo any γ 1, F 1 ) γ 0, F 0 ), γ 1, F 1 ) Γ F, the set X X whee P x, F0, γ 0 ) P x, F 1, γ 1 ) has a positive measue in X. 5. At least one of the following popeties holds. a) g j x, β, γ) is Lipschitz continuous in γ fo each outcome j. b) i) ˆγ N, ˆF ) N is well-defined and measuable. ii) Fo each outcome j, thee exists a vecto of known functions h x, β, γ) = h 1 x, β, γ),..., h J x, β, γ)) such that, fo each j, g j x, β, γ) = 1 [A j h x, β, γ) > 0] fo some vecto of known constants A j. iii) Each of the J functions h j x, β, γ) is Lipschitz continuous in γ. 6. Q γ, F ) is lowe semicontinuous in γ, is continuous on F in the weak topology, and is continuous at γ 0, F 0 ). If homogeneous paametes wee added in the examples we conside, Assumption 2.5.a would hold fo Example 1, the logit model with andom coefficients. Assumption 2.5.b might hold fo Examples 2 and 3. See the emak below. We pesent the consistency theoem fo the estimato with homogeneous paametes. Theoem 2. Suppose Assumption 2 and Condition 1 hold. Then, ˆγ N p γ0 and ˆF N p F0. See Appendix B.1 fo the poof. Again, we omit a coollay fo the likelihood case. Remak 8. Assumption 2.5.b is designed to handle extensions of Examples 2 and 3 to include homogeneous paametes. In the extensions of these examples, the homogeneous paametes ente inside indicato functions. The non-pimitive potion, Assumption 2.5.b.i, equies that the estimato ˆγ N, ˆF ) N be well-defined and measuable, echoing a condition in Lemma A.2 of Chen and Pouzo 2012), which 12

13 ou consistency poof elies on. Remak A.1.i.a in CP states that lowe semicontinuity of the least squaes sample objective function is a sufficient condition fo the estimato to be well-defined and measuable. Even though an indicato function fo an open set is lowe semicontinuous, the least squaes sample objective o likelihood) function itself might not be lowe semicontinuous in γ even if each g j x, β, γ) is lowe semicontinuous in γ. Fo example, multiplication by a negative numbe is enough to change lowe semicontinuity into uppe semicontinuity. Theefoe, we follow the main text of CP and, fo the case of indicato functions, maintain the non-pimitive assumption that the estimato is well-defined and measuable. Assumption 2.5.b.i states in pat that the sample objective function has a unique global optimum. Although Assumption 2.5.b.i may not actually hold if the homogeneous paametes ente inside indicato functions, the sample citeion is conveging to a population citeion with a unique global optimum due to Assumptions 2.4 and 2.6. Extending the appendix lemma in Chen and Pouzo 2012) to the case whee the sample objective function has a continuum of multiple global optima just like in maximum scoe) is a mino extension that we do not pusue fo space easons. The continuity of the population citeion Qγ, F ) with espect to F can be satisfied using the sufficient condition discussed in Remak 3 fo the ealie Assumption 1.5. The lowe semicontinuity with espect to γ may equie futhe pimitive conditions, like the conditions on the suppot of the explanatoy vaiables in the esults on binay choice in Ichimua and Thompson 1998, Theoem 1). 5.2 Continuous Distibution Function Estimato A limitation of the discete appoximation estimato is that the CDF of the heteogeneous paametes will be a step function. In applied wok, it is often attactive to have a smooth distibution of heteogeneous paametes. In this subsection, we descibe one appoach to obtain a continuous distibution o density function estimato that allows fo unbounded suppots. Instead of modeling the distibution of the heteogeneous paametes as a mixtue of point masses, we instead model the density as a mixtue of paametic densities, e.g. nomal densities. Appoximating a density o distibution function using a mixtue of paametic densities o distibutions is popula e.g. Jacobs, Jodan, Nowlan, and Hinton 1991, Li and Baon 2000, McLachlan and Peel 2000, and Geweke and Keane 2007). Ou estimato s advantage is its computational simplicity. As a leading case, let a basis be a nomal distibution with mean the K 1 vecto µ and standad deviation the K 1 vecto σ. Let φ β µ, σ ) denote the joint nomal density coesponding to the th basis distibution. Unde independent nomal basis functions, the joint density fo a given is ). We can also use only a location mixtue with the basis functions φ β µ ) = K k=1 φ β k µ k ) o use a multivaiate nomal mixtue with the basis functions φ β µ, Σ ), whee Σ denotes a vaiance-covaiance matix. We can also conside mixtues of othe paametic density functions. We use the geneic notation φ β λ ) to denote the th basis function, whee λ is the th distibution paamete. Let θ denote the pobability weight given to the th basis function, φ β λ ). As in the discete appoximation estimato, the vecto of weights θ lies in the unit simplex, R. Thee ae many ways to pefom K-dimensional numeical integation, such as spase gid quadatue Heiss and Winschel 2008). We focus on simulation fo simplicity. To implement ou continuous density estimato fo a given R, make S simulation daws fom φ β λ ) independently of i.e. use independent simulation daws fo each λ ). Let a paticula daw s be denoted as β,s. We then ceate just the poduct of the maginals, o φ β µ, σ ) = K k=1 φ β k µ k, σ k 13

14 the R egessos zi,j = g j x i, β) φ β λ ) dβ 1 S S g j x i, β,s ). The emphasizes the eo possibly quite small) in numeical integation. This numeical integation step is done fist, befoe any optimization. We then appoximate P j x i ) as P j x i ) R θ zi,j. Hee, we use the to emphasize both sieve and numeical integation appoximations, although typically the sieve appoximation will be a lage souce of eo. We estimate θ using the inequality constained least squaes poblem as befoe s=1 1 N J θ S = ag min y i,j ) 2 R θ R NJ i=1 θ zi,j. 7) This is once again inequality-constained linea least squaes, a globally convex optimization poblem with an easily-computed unique solution. The esulting density estimato is ˆf N,S β) = R ˆθ S φ β λ ) and the distibution function estimato is ˆF N,S β) = R ˆθ S Φ β λ ), whee dφ ) = φ ) Consistency of the Continuous Distibution Function Estimato We show consistency fo the continuous distibution function estimato afte imposing additional estictions on the data geneating pocess. Fo this pupose, we estict the class of the tue distibution functions to F M = { F : F = Φβ λ)p λ dλ), P λ P λ, dφβ λ) G = { dφβ λ) β B R K, λ Λ R d}}, 8) such that any distibution in F M is in tuth given by a mixtue of paametic distibutions in G. Note that we allow fo B = R K in F M, thus emoving the estiction that B is compact undelying Theoem 1. Hee P λ denotes a pobability measue on Λ, the suppot of the distibution paamete λ. This means that we assume the tue distibution is in the space of possibly continuous mixtues ove some known paametic basis functions. Note, howeve, that Petesen 1983, as Lemma 3.4 of Zeevi and Mei 1997)) suggests that any density function can be appoximated to abitay accuacy by an infinitely countable convex combination of basis densities, including nomals, i.e. G can be dense in the space of continuous density functions. Fo example Zeevi and Mei 1997) show that G is dense in the space of all density functions that ae bounded away fom zeo on compact suppot. Theefoe we ague that the class F M can be abitay close to the space of any continuous distibution functions with suitable choices of G. We appoximate this possibly continuous mixtue using a finite mixtue ove the same basis functions. Accodingly we constuct ou sieve space as F M R = { F F = R θ Φβ λ ), dφβ λ) G, θ R }. 14

15 Fist conside an estimato ignoing numeical integation eo in the egessos ˆF N β) = RN) ˆθ Φβ λ ), 9) whee θ 1 = ag min N J θ RN) NJ i=1 y i,j RN) 2 θ zi,j) and z i,j = g j x i, β)dφβ λ ) with a choice of gid Λ R = λ 1,..., λ R) on Λ. We conside simulation eo late. Let F 0 = Φβ λ)p λ,0 dλ) F M. Then we can pesent the consistency esult in tems of estimating the tue P λ,0 because knowing P λ fully chaacteizes the tue F 0 given a eseache s choice of Φβ λ) e.g. nomal distibutions). Note that 9) can be also witten in tems of estimating P λ with the estimato ˆP λ,n λ) = RN) ˆθ 1 [λ λ] whee the sieve space fo P λ is given by { P λ,r = P λ P λ λ) = } R θ 1 [λ λ], θ R. Theefoe ˆF N = Φ λ)d ˆP λ,n dλ) p F 0 = Φ λ)dp λ,0 dλ) as long as ˆP λ,n p Pλ,0 because we assume F 0 F M and because Φβ λ) is continuous in λ fo all β and theefoe F = Φ λ)dp λ dλ) is also continuous on P λ in the Lévy-Pokhoov metic. This facilitates ou analysis because we can diectly apply Theoem 1 to ˆP λ,n unde assumptions below. To pesent the consistency theoem we need additional notation. Define a sample citeion function in tems of estimating P λ,0 as ˆQ N P λ ) 1 N NJ i=1 y i gx i, λ)dp λ λ) 2 E = 1 N NJ i=1 y i R θ gx i, λ ) fo P λ P λ,rn), whee gx, λ ) = gx, β)dφβ λ ). Then we can ewite the estimato ˆP λ,n as 2 E 10) ˆP λ,n = agmin Pλ P λ,rn) ˆQN P λ ) + C ν n 11) fo some positive sequence ν n tending to zeo. Also define the population citeion function as [ ] QP λ ) E y 2 g x, λ) dp λ λ) /J. We make the following assumptions, which ae simila to those used in Theoem 1 Assumption Let F M be a space of distibution functions geneated by P λ in 8), whee Λ is compact. F M contains the tue F 0. E 2. Let y i, x i )) N i=1 be i.i.d. 3. Let both β and λ be independently distibuted fom x. 15

16 4. Assume the model gx, β) is identified, meaning that fo any P λ,1 P λ,0, the set X X whee P x, F 0 ) P x, F 1 ) such that F 0 = Φ λ)p λ,0 dλ) and F 1 = Φ λ)p λ,1 dλ) has a positive measue in X. 5. Q P λ ) is continuous on P λ in the weak topology. We leave out lengthy discussion of these assumptions because most of these assumptions ae made fo simila easons as those in Assumption 1. Moe discussion is in the beginning of Section 6. We also equie that the choice of gids on Λ satisfies the following popeties. Condition Let Λ R = λ 1,..., λ R) become dense in Λ as R. 2. P λ,r P λ,r+1 P λ fo all R RN) as N and it satisfies RN) log RN) N 0 as N. Theoem 3. Suppose Assumption 3 and Condition 2 hold. Then, d LP ˆPλ,N, P λ,0 ) p 0 and dlp ˆFN, F 0 ) p 0. A bief poof is in an appendix. Howeve, most of the steps mio the poofs of Theoems 1 and 2, and so the appendix omits the unchanged steps fo conciseness. Next we account fo the simulation eo in the basis functions fom appoximating the integal with espect to Φβ λ ). Denote the esulting distibution estimato with simulated basis functions by ˆF N,S β) RN) ˆθ S Φβ λ ), whee 1 θ S = ag min θ RN) N N i=1 J y i,j RN) θ 1 S S gx i, β,s ) s= ) Note that all simulation daws ae independent and that diffeent daws ae used fo each. We obtain the mixing atios ˆθ S as in 12) using the simulated basis functions and ou distibution function estimato is still FN,S β), which belongs to FR M. Note that we only use simulation to appoximate Φβ λ ) and to obtain ˆθ S in 12). Ou distibution estimato is still FN,S β), not F N,S β) RN) ˆθ S 1 S S s=1 1 [β,s β], because the CDF Φβ λ ) is known to eseaches. In the nomal distibution case, specialized softwae calculates the nomal CDF Φβ λ ) with much less eo than typical simulation appoaches. We show ou estimato is consistent when the numbe of simulation daws tends to infinity. Theoem 4. Suppose Assumption 3 and Condition 2 hold. Let S. Then, d LP ˆFN,S, F 0 ) p 0. The poof is in the appendix. 6 Discussion of Examples We etun to the examples we intoduced in Section 2. We discuss the two key conditions fo each model g x, β): Assumption 1.4, identification of F β), and Assumption 1.5, continuity of the population objective function unde the Lévy-Pokhoov metic. Thoughout this section, we assume Assumptions 16

17 hold. Note that Matzkin 2007) is an excellent suvey of olde esults on the identification of models with heteogeneity. Fo the mixtue of continuous densities, the identification of the mixtue distibution, Assumption 3.4, will typically occu if the undelying distibution of heteogeneous paametes is identified, Assumption 1.4, and the basis functions ae chosen appopiately. We do not need to explicitly veify Assumption 3.4 once Assumption 1.4 is veified. We also do not explicitly veify Assumption 3.5, continuity of the population objective function in the continuous mixtues cases. An impotant point is that even when gx, β) is nonsmooth as in Examples 2 and 3, gx, λ) = gx, β)dφβ λ) can still be continuous and diffeentiable in λ because it is smoothed by integation with espect to the distibution Φβ λ). Theefoe, Assumption 3.5 can be satisfied by Remak 2. Example. 1 logit) The identification of F β) in the andom coefficients logit model is the main content of Fox, Kim, Ryan, and Bajai 2012, Theoem 15). 4 Assumption 14 in Fox et al states that The suppot of x, X contains x = 0, but not necessaily an open set suounding it. Futhe, the suppot ) contains a nonempty open set of points open in R dimxj) ) of the fom x 2,..., x j 1, x j, x j+1,..., x J = ) 0,..., 0, x j, 0,..., 0. 5 Fox et al also equie the suppot of X to be a poduct space, which ules out including polynomial tems in an element of x j o including inteactions of two elements of x j. Given this assumption, Assumption 1.4 holds. Assumption 1.5 holds by Remak 2 in the cuent pape. Example. 2 binay choice) Ichimua and Thompson 1998, Theoem 1) establish the identification of F β) unde the conditions that i) the coefficient on one of the the non-intecept explanatoy vaiables in x is known to eithe always be positive o eithe always be negative the sign can be identified), ii) thee is some nomalization such as the coefficient known to be positive o negative is always eithe +1 o 1 moe geneally the andom coefficients lie on an unknown hemisphee), iii) thee ae lage and poduct suppots on each of the explanatoy vaiables othe than the intecept. This ules out polynomial tems and inteactions. If we impose the scale nomalization β k = ±1, Assumption 1.4 holds if we add lage and poduct suppot conditions on each explanatoy vaiable in x. An advantage of ou estimato is the ease of imposing sign estictions if necessay. Because the eseache picks B RN) = β 1,..., β RN)), the eseache can choose the gid so that the fist element of each vecto β is always positive, fo example. Note that binay choice is a special case of multinomial choice, so the non-nested identification conditions in example 3, below, can eplace these used hee. Next, note that the continuity condition Assumption 1.5) holds by Remak 3 when the suppot B is a finite set. We have not shown that continuity holds unde only the identification assumptions of Ichimua and Thompson 1998, Theoem 1), although we know of no counteexamples. Example. 3 multinomial choice without logit eos) Fox and Gandhi 2015, Theoem 2) study the identification of the multinomial choice model without logit eos. Ou linea specification of the utility function fo each choice is a special case of what they allow. Fox and Gandhi equie a choicej-specific explanatoy vaiable with lage and poduct suppot. On othe hand, Fox and Gandhi allow 4 Theoem 15 of Fox et al also allows homogeneous, poduct-specific intecepts. 5 Fox et al discusses what x = 0 means when the means of poduct chaacteistics can be shifted. 17

18 polynomial tems and inteactions fo x s othe than the choice-j-specific lage suppot explanatoy vaiables, unlike examples 1 and 2. They do not equie lage suppot fo the x s that ae not the lage suppot, choice-specific explanatoy vaiables. The most impotant additional assumption fo identification is that Fox and Gandhi equie that F β) takes on at most a finite numbe T of suppot points, although the numbe T and suppot point identities β 1,..., β T ae leaned in identification. The numbe T in the tue F 0 is not elated in any way to the finite-sample R N) used fo estimation in this pape. So Assumption 1.4 holds unde this estiction on F. Next, the continuity Assumption 1.5 holds also by Remak 3 when the suppot B is a finite set. 7 Estimation Eo Bounds We next deive convegence ates fo the appoximation of the undelying distibution of heteogeneous paametes F β) fo a subset of the models allowed in the consistency theoems in Sections 3 and 5.2. These esults highlight, among othe issues, the cuse of dimensionality in the dimension of the heteogeneous paametes. Lage sample sizes ae needed if the dimension K of the heteogeneous paametes is lage. We pesent esults fo the least squaes citeion and do not include coollaies fo the likelihood citeion. 7.1 Discete Appoximation Estimato Ou esults apply to a naowe set of tue models g j x, β) than the ealie consistency theoems. In paticula, we equie that g j x, β) be smooth, so we allow ou Example 1, the logit case, but not the othe examples, whee g j x, β) involves an indicato function. Nevetheless, the andom coefficients logit is a leading model used in empiical wok and the esults exploiting the smoothness of g j x, β) ae illustative of issues, such as the cuse of dimensionality, that apply also to nonsmooth choices fo g j x, β). Ou esults elate to the liteatue on sieve estimations e.g., Newey 1997) and Chen 2007)). Of couse, the majo diffeence is ou choice of discete basis functions, motivated by ou estimato s computational advantages and ou desie to easily constain ou estimate to be a valid CDF. We cannot diectly apply pevious esults because of ou diffeent sieve space. Ou poof technique to deive ou eo bounds uses esults on quadatue, as we will explain. We estict the tue distibution F 0 to lie in a class of distibutions that have smooth densities on a compact space of heteogeneous paametes. We define the paamete space fo the choice pobabilities P x, F ) as the collection of those geneated by such smooth densities: H = { P x, F ) max sup 0 s s β B D s fβ) C, B = K k=1 } ] [β k, β k, f = df C s [B], 13) whee D s = s β α βα K, s = α α K with D 0 f = f, giving the collection of all deivatives of K ode s. Also, C s [B] is a space of s-times continuously diffeentiable density functions defined on B. Theefoe we assume any element of the class of density functions that geneates H is defined on a Catesian poduct B, is unifomly bounded by C <, is s-times continuously diffeentiable, and has 18

19 all own and patial deivatives unifomly bounded by C. The definition of H depends on C and s; the degee of smoothness s will show up in ou convegence ate esults. Let the space of appoximating functions be H RN) = { P x, F ) F F RN) }. 14) We equie that the gid points accumulate in F RN) such that H R H R Ou appoach uses esults fom quadatue to pick appoximation choices θ such that we can appoximate the choice pobability P x, F 0 ) abitaily well using appoximating functions in H RN) as R N). In this section and in the coesponding poofs, we let v E = v v, h 2 L 2,N = 1 N N i=1 hx i) 2 E, h 2 L 2 = h 2 E dϖ the nom in L 2), and h 2 = sup x X hx) 2 E fo any function h : X R, whee ϖ denotes a pobability measue on X. We intoduce the linea pobability model eo e i,j, as in y i,j = P j x i, F ) + e i,j and E [e i,j X 1,..., X N ] = 0. In addition to esticting the class of tue distibutions, we make the following additional assumptions. Assumption 4. i) e i = e i,1,..., e i,j ) ) N i=1 ae independently distibuted; ii) E [e i X 1,..., X N ] = 0; iii) X i ) N i=1 ae i.i.d. with a density function bounded above; iv) gx, β) is s-times continuously diffeentiable w..t. β and its all own and patial) deivatives ae unifomly bounded; v) the sieve space defined in 14) satisfies H R H R Assumptions 4 i)-iii) ae about the stuctue of the data and in paticula they allow fo heteoskedasticity fo the linea pobability eo, which is necessay fo linea pobability models. As peviewed ealie, Assumption 4 iv) assumes that the tue model gx, β) is diffeentiable. We use Assumption 4 iv) to appoximate P x, F ) using a sieve method fo F combined with a quadatue method fo the choice of weights θ. Assumption 4 iv) is satisfied by Example 1. Assumption 4 v) was mentioned peviously. Any asymptotic eo bound consists of two tems: the ode of bias and the vaiance. While obtaining the vaiance tem is athe standad in the sieve estimation liteatue, deiving the bias tem depends on the specific choice of basis function e.g., powe seies o splines in the pevious liteatue). Because ou choice of basis functions is new in the sieve liteatue, we fist state the ode of bias, meaning the appoximation eo ate of ou sieve appoximation to abitay conditional choice pobabilities P x, F ) in H. Keep in mind this bias esult is pimaily about the flexibility of a class of appoximations and has less to do with using a finite sample of data. Lemma 1. Suppose P x, F ) H and suppose Assumptions 4 iv) and v) hold. Then thee exist F F RN) such that fo all x X, P x, F ) P x, F ) 2 E = O R 2s/K). Futhe suppose Assumptions 4 i)-iii) hold. Then, P x i, F ) P x i, F ) 2 L 2,N = O P R 2s/K ). To pove this esult, we combine a quadatue appoximation with a powe seies appoximation to appoximate P x, F ) = g x, β) fβ)dβ. We fist appoximate g x, β) fβ) using a tenso poduct powe seies in β. Then we appoximate the integal of the tenso poducts appoximation with espect to β using quadatue. The complete poof is in Appendix C.3. Lemma 1 is the key ingedient that allows us to use machiney [ fom the sieve liteatue fo ou estimato. Let C be a geneic) positive constant. Define Ψ R E j g j X i, β ) g j X i, β )]) 1, R 19