True Random Effects in Stochastic Frontier Models

Transcription

1 1/76

2 True Random Effects n Stochastc Fronter Models Wllam Greene New York Unversty 2/76

3 3/76 Agenda Skew normalty Adelch Azzaln Stochastc fronter model Panel Data: Tme varyng and tme nvarant neffcency models Panel Data: True random effects models Maxmum Smulated Lkelhood Estmaton Applcatons of true random effects Persstent and transent neffcency n Swss ralroads A panel data sample selecton corrected stochastc fronter model Spatal effects n a stochastc fronter model

4 4/76 Skew Normalty

5 The Stochastc Fronter Model ln y x v u, v 2 ~ N 0, v, u U U N v u = v U 2, ~ 0, u, Convenent parameterzaton (notaton) V U = N[0,1] N[0,1] v u v u 5/76

6 Log Lkelhood 6/76 u, = v log L(,,, ) = Skew Normal Densty 2 2 u v N 1 2 y x log log ( y x) log N 2 = log 1

7 Brnbaum (1950) Wrote About Skew Normalty Effect of Lnear Truncaton on a Multnormal Populaton 7/76

8 Wensten (1964) Found f() Query 2: The Sum of Values from a Normal and a Truncated Normal Dstrbuton 8/76 See, also, Nelson (Technometrcs, 1964), Roberts (JASA, 1966)

9 O Hagan and Leonard (1976) Found Somethng Lke f() 9/76 Resembles f() Bayes Estmaton Subject to Uncertanty About Parameter Constrants

10 ALS (1977) Dscovered How to Make Great Use of f() 10/76 See, also, Forsund and Hjalmarsson (1974), Battese and Corra (1976) Porer, Tmmer, several others.

11 Azzaln (1985) Fgured Out f() And Notced the Connecton to ALS The standard skew normal dstrbuton f( ) = 2 ( ) ( ) /76

12 12/76

13 ALS 13/76

14 A Useful FAQ About the Skew Normal How to generate pseudo random draws on 1. Draw UV, from ndependent N[0,1] 2. = V + U u u 14/76

15 Random Number Generator For a partcular desred and Use and 1 1 = Then u 2 v 2 u N(0,1) N(0,1) v u 15/76

16 16/76 How Many Applcatons of SF Are There?

17 W. D. Walls (2006) On Skewness n the Moves 17/76 Ctes Azzaln. 2 ( z) ( z)

18 SNARCH Model for Fnancal Crses (2013) The skew-normal dstrbuton developed by Sahu et al. (2003) Does not know Azzaln. 18/76

19 A Skew Normal Mxed Logt Model (2010) Mxed Logt Model Prob( Choce j) Random Parameters k w k exp( x ) j1 exp( x ) Asymmetrc (Skewed) Parameter Dstrbuton w v U ~ SN(0,, ) k k k J j j Greene (2010, knows Azzaln and ALS), Bhat (2011, knows not Azzaln or ALS) 19/76

20 Skew Normal Applcatons Foundaton: An Entre Feld Stochastc Fronter Model Occasonal Modelng Strategy Culture: Skewed Dstrbuton of Move Revenues Fnance: Crss and Contagon Choce Modelng: The Mxed Logt Model How can these people fnd each other? Where else do applcatons appear? 20/76

21 21/76 Stochastc Fronter

22 The Cross Secton Departure Pont: 1977 Agner et al. (ALS) Stochastc Fronter Model y x v u v 2 ~ N[0, v] u U U N 2 and ~ [0, u] Jondrow et al. (JLMS) Ineffcency Estmator ( ) ˆ u E[ u ] 2 1 ( ) v u, u 2 2, v u, v 22/76

23 The Panel Data Models Appear: 1981 Ptt and Lee Random Effects Approach: 1981 y x v u 2 2 t ~ [0, v ], and ~ [0, u] u Counterpart to Jondrow et al. (1982) uˆ [,..., ] t t t v N u U U N t t ( / ) E u 1 T 1 ( / ) v 2 T =, u 2 u 1 T 1 T v Tme fxed 23/76

24 Renterpretng the Wthn Estmator: 1984 Schmdt and Sckles Fxed Effects Approach: 1984 y v x t t t 2 t ~ N[0, v ], v semparametcally specfed fxed mean, constant varance. Counterpart to Jondrow et al. (1982) ˆ max ( ˆ ) ˆ u (The cost of the semparametrc specfcaton s the locaton of the neffcency dstrbuton. The authors also revst Ptt and Lee to demonstrate.) Tme fxed 24/76

25 Msgvngs About Tme Fxed Ineffcency: Cornwell Schmdt and Sckles (1990) 25/76 t t Kumbhakar (1990) u bt ct U t 2 1 [1 exp( )] Battese and Coell (1992, 1995) 2 u exp[ ( t T )] U, u exp[ g( t, T, z )] U t t t Cuesta (2000) u exp[ ( t T )] U, u exp[ g ( t, T, z )] U t t t t

26 Are the systematcally tme varyng models more lke tme fxed or freely tme varyng? A Pooled Model y x v u t t t t Battese and Coell (1992) u exp[ ( t T )] U Ptt and Lee (1981) y x v U t t t t Where s Battese and Coell? Closer to the pooled model or to Ptt and Lee? Greene (2004): Much closer to the Ptt and Lee model 26/76

27 In these models wth tme varyng neffcency, y x v g ( t, z ) U t t t t v N U N 2 2 t ~ [0, v ] and t ~ [0, u], where does unobserved tme nvarant heterogenety end up? In the neffcency! Even wth the extensons. 27/76

28 Skeptcsm About Tme Varyng Ineffcency Models: Greene (2004) 28/76

29 29/76 True Random Effects

30 Tme fxed 30/76 True Random and Fxed Effects: 2004 True Random and Fxed Effects Approach: 2004 y x v u t t t t v N u U U N 2 2 t ~ [0, v ], t t and t ~ [0, u] Unobserved tme nvarant heterogenety, not unobserved tme nvarant neffcency Jondrow et al. (JLMS) Ineffcency Estmator ( ) t Eu [ t t ] 2 1 t ( ) t u 2 2 t vt ut,, v u, v Tme varyng t

31 31/76 Estmaton of TFE and TRE Models: 2004 True Fxed Effects: MLE y x v u t t t t v N u U U N 2 2 t ~ [0, v ], t t and t ~ [0, u] Unobserved tme nvarant heterogenety, not unobserved tme nvarant neffcency Just add frm dummy varables to the SF model (!) True Random Effects: Maxmum Smulated Lkelhood (RPM) y ( w ) x v u t t t t v N u U U N w N t ~ [0, v ], t t and t ~ [0, u], ~ [0, w] Unobserved tme nvarant heterogenety, not unobserved tme nvarant neffcency Random parameters stochastc fronter model

32 Log lkelhood functon for stochastc fronter model log L(,,, ) = N 1 2 y x log log ( y x) log 32/76

33 Smulated log lkelhood functon for stochastc fronter model wth a tme nvarant random constant term. (TRE model) 2 y ( ) t wwr xt 1 S N R T log L (,,,, w) = log 1 r1 t1 R ( yt ( ww ) r xt ) draws from N[0,1]. w r 33/76

34 34/76 The Most Famous Fronter Study Ever

35 The Famous WHO Model logcomp= + 1 logpercaptahealthexpendture + 2 logyearseduc + 3 Log 2 YearsEduc + = v - u Schmdt/Sckles FEM 191 Countres. 140 of them observed /76

36 The Notorous WHO Results 37 36/76

37 August 12, No, t doesn t. 37/76

38 38/76 Huffngton Post, Aprl 17, 2014

39 39/76 we are #37

40 Greene, W., Dstngushng Between Heterogenety and Ineffcency: Stochastc Fronter Analyss of the World Health Organzaton s Panel Data on Natonal Health Care Systems, Health Economcs, 13, 2004, pp /76

41 x 1,log,log,log 2 Exp Ed Ed z log PopDen,log PerCaptaGDP, GovtEff, VoxPopul, OECD, GINI 41/76

42 42/76 Three Extensons of the True Random Effects Model

43 Generalzed True Random Effects Model 43/76 Generalzed True Random Effects Stochastc Fronter Model y A B x v u t t t t Transent random components v t u t Persstent random components A B Tme varyng normal - half normal SF Tme fxed normal - half normal SF

44 A Stochastc Fronter Model wth Short-Run and Long-Run Ineffcency: Colomb, R., Kumbhakar, S., Martn, G., Vttadn, G., Unversty of Bergamo, WP, 2011, JPA 2014, forthcomng. Tsonas, G. and Kumbhakar, S. Frm Heterogenety, Persstent and Transent Techncal Ineffcency: A Generalzed True Random Effects Model Journal of Appled Econometrcs. Publshed onlne, November, Extremely nvolved Bayesan MCMC procedure. Effcency components estmated by data augmentaton. 44/76

45 45/76 Generalzed True Random Effects Stochastc Fronter Model y ( w e ) x v u t w t t t Tme varyng, transent random components v N u U U N 2 2 t ~ [0, v ], t t and t ~ [0, u], Tme nvarant random components w ~ N[0,1], e ~ N[0,1] The random constant term n ths model has a closed skew normal dstrbuton, nstead of the usual normal dstrbuton.

46 Estmatng Effcency n the CSN Model Moment Generatng Functon for the Multvarate CSN Dstrbuton ( Rr t, ) E[exp( tu ) y ] exp trr tt T1 1 2 T 1( Rr, ) (..., ) Multvarate normal cdf. Parts defned n Colomb et al. Computed usng GHK smulator. u e u , t =,,..., ut /76

47 47/76 Estmatng the GTRE Model

48 Colomb et al. Classcal Maxmum Lkelhood Estmator N log T ( y X1T, AVA) log L 1 log q ( R( y X1T, )) nq log 2 (...) T-varate normal pdf. T (..., )) ( T 1) Multvarate normal ntegral. q Very tme consumng and complcated. 48/76 From the samplng theory perspectve, the applcaton of the model s computatonally prohbtve when T s large. Ths s because the lkelhood functon depends on a (T+1)-dmensonal ntegral of the normal dstrbuton. [Tsonas and Kumbhakar (2012, p. 6)]

49 Kumbhakar, Len, Hardaker Techncal Effcency n Competng Panel Data Models: A Study of Norwegan Gran Farmng, JPA, Publshed onlne, September, Three steps based on GLS: (1) RE/FGLS to estmate (,) (2) Decompose tme varyng resduals usng MoM and SF. (3) Decompose estmates of tme nvarant resduals. 49/76

50 Maxmum Smulated Full Informaton log lkelhood functon for the "generalzed true random effects stochastc fronter model", S N 1 logl, = log R w, 2 yt ( wwr Ur ) xt R T ( yt ( wwr Ur ) xt ) draws from N[0,1] 1 r1 t1 w r U absolute values of draws from N[0,1] r 50/76

51 WHO Results: 2014 x 1,log,log,log 2 Exp Ed Ed z log PopDen,log PerCaptaGDP, GovtEff, VoxPopul, OECD, GINI A B v u t t t 51/76

52 52/76

53 Emprcal applcaton Cost Effcency of Swss Ralway Companes 53/76

54 Model Specfcaton TC = f ( Y 1, Y 2, P L, P C, P E, N, NS, d t ) C : Total costs Y 1 : Passenger-km Y 2 : Ton-km P L : Prce of labor (wage per FTE) P C : Prce of captal (captal costs / total number of P E : Prce of electrcty N : Network length NS: Number of statons Dt: tme dummes seats) 54/76

55 Data 50 ralway companes Perod 1985 to 1997 unbalanced panel wth number of perods (T) varyng from 1 to 13 and wth 45 companes wth 12 or 13 years, resultng n 605 observatons Data source: Swss federal transport offce Data set avalable at Data set used n: Fars, Flppn, Greene (2005), Effcency and measurement n network ndustres: applcaton to the Swss ralway companes, Journal of Regulatory Economcs 55/76

56 56/76

57 57/76

58 58/76 Cost Effcency Estmates

59 59/76 Correlatons

60 60/76 MSL Estmaton

61 Why s the MSL method so computatonally effcent compared to classcal FIML and Bayesan MCMC for ths model? Condtoned on the permanent effects, the group observatons are ndependent. The jont condtonal dstrbuton s smple and easy to compute, n closed form. The full lkelhood s obtaned by ntegratng over only one dmenson. (Ths was dscovered by Butler and Mofftt n 1982.) Nether of the other methods takes advantage of ths result. Both ntegrate over T+1 dmensons. 61/76

62 62/76

63 Equvalent Log Lkelhood Identcal Outcome One Dmensonal Integraton over δ T+1 Dmensonal Integraton over Re. 63/76

64 Smulated [over (w,h)] Log Lkelhood 1 log G ( r,,,, w, h) R N R S 1 r1 Very Fast wth T=13, one mnute or so 64/76

65 Also Smulated Log Lkelhood GHK smulator s used to approxmate the T+1 varate normal ntegrals. Very Slow Huge amount of unnecessary computaton. 65/76

66 Computaton of the GTRE Model s Actually Fast and Easy 247 Farms, 6 years. 100 Halton draws. Computaton tme: 35 seconds ncludng computng effcences. 66/76

67 67/76 Smulaton Varance

68 Does the smulaton chatter degrade the econometrc effcency of the MSL estmator? Hajvasslou, V., Some practcal ssues n maxmum smulated lkelhood, Smulaton-based Inference n Econometrcs: Methods and Applcatons, Marano, R., Weeks, M. and Schuerman, T., Cambrdge Unversty Press, 2008 Speculated that Asy.Var[estmator] = V + (1/R)C The contrbuton of the chatter would be of second or thrd order. R s typcally n the hundreds or thousands. No other evdence on ths subject. 68/76

69 An Experment Pooled Spansh Dary Farms Data Stochastc fronter usng FIML. Random constant term lnear regresson wth constant term equal to - w, w~ N[0,1] Ths s equvalent to the stochastc fronter model. Maxmum smulated lkelhood 500 random draws for the smulaton for the base case. Uses Mersenne Twster for the RNG 50 repettons of estmaton based on 500 random draws to suggest varaton due to smulaton chatter. 69/76

70 ˆ v ˆ u 70/76

71 Smulaton Nose n Standard Errors of Coeffcents Chatter /76

72 Quas-Monte Carlo Integraton Based on Halton Sequences Coverage of the unt nterval s the objectve, not randomness of the set of draws. Halton sequences --- Markov chan p = a prme number, r= the sequence of ntegers, decomposed as H(r p) I 0 b p 1, r = r 1,... (e.g., 10,11,12,...) I 0 For example, usng base p=5, the nteger r=37 has b 0 = 2, b 1 = 2, and b 3 = 1; (37=1x x x5 0 ). Then H(37 5) = = bp 72/76

73 Is It Really Smulaton? Halton or Sobol sequences are not random Far more stable than random draws, by a factor of about 10. There s no smulaton chatter Vew the same as numercal quadrature There may be some approxmaton error. How would we know? 73/76

74 Halton Sequences Coverage of the unt nterval s the objectve, not randomness of the set of draws. Halton sequences --- Markov chan p = a prme number, r= the sequence of ntegers, decomposed as I 1 H(r p) b p, r = r 1,... (e.g., 10,11,12,...) 0 I 0 bp 74/76

75 Haltonzed Log Lkelhood LogL(,,,,, ) 2 yt xt N T 2 log 1 t1 yt t x LogL (,,,,, ) N S 1 log R 1 r1 t1 1 1 R W W H T H r w r h r r r 2 y t xt r yt xt r Halton[prme( w), r burn n] Halton[prme( h), r burn n] 75/76

76 Summary The skew normal dstrbuton Two useful models for panel data (and one potentally useful model pendng development) Extenson of TRE model that allows both transent and persstent random varaton and neffcency Sample selecton corrected stochastc fronter Spatal autocorrelaton stochastc fronter model Methods: Maxmum smulated lkelhood as an alternatve to receved brute force methods Smpler Faster Accurate Smulaton chatter s a red herrng use Halton sequences 76/76

77 77/76 Sample Selecton

78 TECHNICAL EFFICIENCY ANALYSIS CORRECTING FOR BIASES FROM OBSERVED AND UNOBSERVED VARIABLES: AN APPLICATION TO A NATURAL RESOURCE MANAGEMENT PROJECT Emprcal Economcs: Volume 43, Issue 1 (2012), Pages Bors Bravo-Ureta Unversty of Connectcut Danel Sols Unversty of Mam 78/76 Wllam Greene New York Unversty

79 The MARENA Program n Honduras Several programs have been mplemented to address resource degradaton whle also seekng to mprove productvty, manageral performance and reduce poverty (and n some cases make up for lack of publc support). One such effort s the Programa Multfase de Manejo de Recursos Naturales en Cuencas Prortaras or MARENA n Honduras focusng on small scale hllsde farmers. 79/76

80 80/76 Expected Impact Evaluaton

81 Methods A matched group of benefcares and control farmers s determned usng Propensty Score Matchng technques to mtgate bases that would stem from selecton on observed varables. In addton, we deal wth possble self-selecton on unobservables arsng from unobserved varables usng a selectvty correcton model for stochastc fronters ntroduced by Greene (2010). 81/76

82 A Sample Selected SF Model d = 1[ z + h > 0], h ~ N[0,1 2 ] y = + x +, ~ N[0, 2 ] (y,x ) observed only when d = 1. = v - u u = u U where U ~ N[0,1 2 ] v = v V where V ~ N[0,1 2 ]. (h,v ) ~ N 2 [(0,1), (1, v, v2 )] 82/76

83 Smulated logl for the Standard SF Model f ( y x, U ) exp[ 2 ( y x u U ) / v ] v 2 f ( y ) p( U ) d U exp[ 2 ( ) / ] y x U u v x U v exp[ 2 U ] p( U ), U 0. (Half normal) 2 f( y x ) R exp[ 2 ( y x U ) / ] u r v r1 R v N 1 R exp[ 2 ( y ) / ] x u Ur v log LS (,, u, v) = log =1 r1 R v 2 Ths s smply a lnear regresson wth a random constant term, α = α - σ u U 83/76

84 Lkelhood For a Sample Selected SF Model f y ( x, d, z, U ) exp 2 ( y ) / ) x u U v v 2 d (1 ) ( d z ) ( y ) / x u U z 2 1 x z x z f y (, d, ) f y (, d,, U ) f ( U ) d U U 84/76

85 Smulated Log Lkelhood for a Selectvty Corrected Stochastc Fronter Model The smulaton s over the neffcency term. d v 2 1 log L (,,,,, ) log ( ) / y x U z 1 (1 d ) ( z) exp 2 ( y x u Ur ) / v ) N R S u v u r 1 r1 R 2 85/76

86 JLMS Estmator of u ˆ ˆ ˆ ˆ ˆ 2 ˆ v fr ˆ( y ˆ ˆ x ˆ U ) / ˆ a 2 1ˆ ˆ 1 ˆ ˆ 1 = R ( ˆ ), R A ˆ 1 u Ur fr B f r r1 r R R Aˆ uˆ Estmator of E[ u ] Bˆ exp 2 ( y x u Ur ) / v ) R r1 gˆ u r v r fˆ where, 1 r ˆ ˆ R ˆ uu r gr g R ˆ r1 r f r1 r 86/76

87 Closed Form for the Selecton Model The selecton model can be estmated wthout smulaton The stochastc fronter model wth correcton for sample selecton revsted. La, Hung-pn. Forthcomng, JPA Based on closed skew normal dstrbuton Smlar to Maddala s 1982 result for the lnear selecton model. See slde 42. Not more computatonally effcent. Statstcal propertes dentcal. Suggested possblty that smulaton chatter s an element of neffcency n the maxmum smulated lkelhood estmator. 87/76

88 Closed Form vs. Smulaton Spansh Dary Farms: Selecton based on beng farm # perods 88/76 The theory works.

89 Varables Used n the Analyss Producton Partcpaton 89/76

90 90/76 Fndngs from the Frst Wave

91 A Panel Data Model Selecton takes place only at the baselne. There s no attrton. 91/76 d 1[ z h > 0] Sample Selector y w x v u, t 0,1,... Stochastc Fronter t t t t Selecton effect s exerted on w ; Corr( h, w,) P( y, d ) P( d ) P( y d ) t 0 0 t 0 Condtoned on the selecton ( h 0 P( y, y,..., y d ) P( y d ) 0 1 T 0 t0 t T 0 0 t0 t 0 ) observatons are ndependent. I.e., the selecton s actng lke a permanent random effect. P( y, y,..., y, d ) P( d ) P( y d0) T T

92 Smulated Log Lkelhood log L (,,,, ) S, C u v exp 2 ( yt xt u Utr ) / ) v 1 R T v 2 log d 1 r1 t0 R ( yt xt u Utr ) / v a /76

93 Man Emprcal Conclusons from Waves 0 and 1 93/76 Beneft group s more effcent n both years The gap s wder n the second year Both means ncrease from year 0 to year 1 Both varances declne from year 0 to year 1

94 94/76

95 95/76 Spatal Autocorrelaton

96 True Random Spatal Effects Spatal Stochastc Fronter Models: Accountng for Unobserved Local Determnants of Ineffcency: A.M.Schmdt, A.R.B.Morrs, S.M.Helfand, T.C.O.Fonseca, Journal of Productvty Analyss, 31, 2009, pp Smply redefnes the random effect to be a regon effect. Just a renterpretaton of the group. No spatal decay wth dstance. True REM does not perform as well as several other specfcatons. ( Performance has nothng to do wth the fronter model.) 96/76