Evaluating enterprise support: state of the art and future challenges. Dirk Czarnitzki KU Leuven, Belgium, and ZEW Mannheim, Germany

Transcription

1 Evaluating enterprise supprt: state f the art and future challenges Dirk Czarnitzki KU Leuven, Belgium, and ZEW Mannheim, Germany

2 Intrductin During the last decade, mircecnmetric ecnmetric cunterfactual impact evaluatins have becme an imprtant tl in the area f enterprise supprt plicies. It became standard t use ecnmetric methds, such as Matching estimatrs (Cnditinal) Difference-in-Difference regressins Instrumental variable regressins Mre recently: Regressin Discntinuity Designs Nt standard (yet): randmized cntrl trials natural experiments

3 Intrductin Why are these methds imprtant? Firms select themselves int the prgrams Gverments pick winners Result: treated firms cannt be cmpared with nn-treated firms withut further adjustment fr deriving effects f plicies Treatment is an endgenus variable in (OLS) regressin mdels, and results wuld be biased if endgeneity is nt addressed prperly

4 Intrductin Therefre all ecnmetric evaluatin methds seek t establish a crrect cntrl grup apprach t derive e.g. the treatment effect n the treated, i.e. Hw many jbs wuld a treated firm have created if it had nt been treated? Hw much wuld have a firm invested in innvatin activities if it had nt been subsidized? Which sales with new prducts wuld a firm have achieved if it had nt gtten a start-up grant?

5 What is the current evidence? Recent general review* f the available empirical evidence by the What Wrks Centre fr Lcal Ecnmic Grwth Led by the Lndn Schl f Ecnmics and Plitical Science 1,700 wrks (academic jurnals and plicy reprts) reviewed Classied accrding t the strength f the empirical evidence Using a variant f the Scientic Maryland Scale Results: nly view achieve highest methdlgical standards Hwever, limited data availability critically determines the researchers ptins f applicatins. * nt limited t enterprise supprt within Chesin Plicy; review cvers all fields f ecnmic evaluatins.

6 Mdified Scientific Maryland Scale Randmized cntrl trials, natural experiments, n selective sample attritin Instrumental variable techniques r RDD, prper balancing (OLS, matching), attritin discussed but nt addressed Difference-in-Differences, balancing (OLS, matching), but uncntrlled differences likely remain Befre and after cmparisns, r a cmparisn grup but withut balancing f cvariates Crrelatin analysis, n cntrl grup, n attempt at establishing a cunterfactual

7 Methds are imprtant, but it is equally imprtant HOW we use the methds! What des this mean? Example: Evaluatin f Eureka s Eurstars prgramme* (ttal budget 500 millin EUR - c-financed by EC = 100 millin EUR) Dirk estimated: treatment effect f Eurstars with respect t jb creatin amunts t a 3.1% higher average annual emplyment grwth-rate when cmpared t the cunterfactual situatin f n Eurstars grant Extraplatin frm regressin sample t ttal prgramme impact yields abut 7,800 jbs created *

8 What is useful infrmatin? Is this infrmatin useful fr the plicy maker? Partially yes prgramme has an estimated psitive impact HOWEVER, prgramme might cntinue t exist anyways. Mre useful infrmatin wuld be: Hw can we make the prgramme better, i.e. mre effective and/r mre efficient? Search fr hetergeneus treatment effects See e.g. Czarnitzki/Lpes-Bent (2013)

9 Hetergeneus treatment effects within a plicy scheme

10 Hetergeneus treatment effects Mnetary value f grant Subsidy rate Firm size Hetergeneity f cnsrtia start-ups, large firms, universities Prpsal quality (Peer-review scre) Multiple grants Hünermund/Czarnitzki (2016) find that treatment effect varies with the peer-review scre. Better prpsals als yield higher treatment effects (but effect is nt linear) Nte: LATE in RDD vs. ATT btained with ther estimatrs.

11 Hetergeneus treatment effects in Eurstars accrding t peer-review scre (prpsal quality) Scre

12 Hetergeneus treatments and their effects acrss plicy instruments

13 Hetergeneus treatments Instead f explring hetergeneus treatment effects within a plicy scheme, it is als pssible t search fr hetergenus effects acrss schemes Prblem: very data hungry E.g. Czarnitzki et al. (2016): enterprise supprt in German Chesin Plicy schemes Als: dynamic treatment effects Treatment effect culd evlve ver time rather than ccuring in a single perid

14 Treatment effect ver time fr ln(emplyment) by grant type General Cnsulting Innvatin/R&D Training Netwrking Marketing Investment Labr supprt % cnfidence interval

15 Indirect effects Plicy scheme may have indirect effects Example Eurstars: even rejected applicatins may have effects Beware: cntaminated cntrl grup

16 Cnclusins The use f apprpriate ecnmetrics methds increased significantly in the last decades. Next steps: There is still rm fr imprvement with respect t identificatin Explit discntinuities, instruments, experiments apply methds in a mre useful way fr plicy making (i.e. beynd hmgenus treatment effects n the treated f a single prgramme) Design f plicy schemes Selectin f plicy schemes