A Systematic Literature Review of QCA in Spatial Sciences Session: Closing Debate

Transcription

1 Faculty of Spatial Sciences Department of Spatial Planning & Environment A Systematic Literature Review of QCA in Spatial Sciences Session: Closing Debate s.verweij@rug.nl stefanverweij International QCA Summer Workshop, University of Wuppertal, May 28-30, 2018

2 What is Spatial Planning? Spatial Planning System: the methods and approaches used by the public and private sector to influence the organization of people and activities in spaces at various scales (local, regional, national, international) Spatial Planning: the coordination of policies and practices affecting the socio-spatial organization Alexander: there is no planning, only planning practices Sources: Wikipedia (2018); Alexander (2016)

3 What is Spatial Planning? 3

4 What is Spatial Planning? Objects ( what ): people and activities in spaces and the environment at various scales Context ( where ): the social-institutional environment of spatial planners practices Tools ( how ): methods and approaches, including skills and competences, for effective practices Purpose ( what for ): much debated; intervening in the public interest? Spatial Design & Institutional Design Source: adapted from Alexander (2016)

5 Spatial Planning Research Providing input for policy change and institutional-spatial design in order to guide spatial and social developments In some way or another, this involves comparison - Either between cases - Or with a framework ultimately built on known examples and previously researched cases Booth: the desire to make comparisons [has] been fundamental to research activity in the field of spatial planning Source: Alexander (2016); Booth (2015)

6 Spatial Planning Research Comparison and transfer of knowledge (case case & case existing knowledge) is challenging: - Spatial planning practices and conditions are very context-specific - So how and to what extent do lessons learned transfer to other cases? Booth: the conditions that make a policy or practice work in one place are unlikely to obtain in another This is where QCA comes in Source: Booth (2015); Janssen-Jansen et al. (2008)

7 C I M Outcome Y Cases ZUID, NOORD, IJSS, SIJT, WEST C LENT, SCHEL PERK, DELFT WIER BROEK C WAAL, GOUW 0 0 1??? DIEF

8 Introducing QCA Yet little used; errors occurring Introducing QCA to the Spatial Sciences; educating the new disciplines How? - Systematic literature review of QCA in SPARD ( Spatial Planning and Related Disciplines ) Book: Gerrits & Verweij (2018)

9 Research Questions 1. When, where, and how is QCA used in SPARD? - Descriptive analysis of QCA-studies in SPARD 2. How does SPARD in those respects compare to other disciplines? - Comparison of QCA-studies in SPARD with larger-n of QCA-studies (e.g., Rihoux et al., 2011; 2013, Seny Kan et al., 2016) 3. What are the ways forward for QCA in spatial planning research? - Analysis of good practices in the QCAstudies in SPARD (Schneider & Wagemann, 2010) - Analysis of stated usefulness of QCA

10 Method and Data Selecting articles 1. Subject fields in Scopus of Geography, Planning and Development, Transportation and Urban Studies (427,255 documents) 2. Limiting: articles, in English, years (282,267 articles) 3. Limiting: QCA-keywords (349 articles) 4. Selecting (independent coding) only empirical QCA-articles that are actually are in SPARD (33 articles) Coding articles Independent coding using coding scheme based on previous literature reviews Keywords based on: Seny Kan et al. (2016); Gerrits & Verweij (2016; 2018)

11 RQ1: Variants of QCA

12 RQ1: Disciplines and UoA

13 RQ1: Data Sources

14 The ratio was best for the category with many cases and many conditio expected, the ratio is worst for the category with few cases and many con 0.21 cases per truth table row. On average, studies with few conditions ha RQ1: Cases and Conditions and 1.92 (many cases). 14 Table2: Diversity of cases and conditions in theqca-studies Cases Few 20 Many >20 Conditions Few 4 Many >4 8 applications (24%) 15 applications (33%) 6 applications (18%) 8 applications (17%) 7 applications (21%) 8 applications (17%) 12 applications (36%) 15 applications (33%) Total 15 (45%) 23 (50%) 18 (55%) 23 (50%) Total 14 (42%) 23 (50%) 19 (58%) 23 (50%) 33 (100%) 46 (100%) 6 For the purpose of comparability, we used the same categorization as Rihoux et al. (201 7 As a general rule of thumb, having a good ratio between the number of cases and the

15 which software was used. It is good to emphasize at this point that using software for the analysis is important and considered a good practice (Schneider & Wagemann, 2010). Although the software should not be used as a press-button technique without understanding and considering the RQ1: Software & Visualization assumptions that are fed into the analysis, using the software ensures that no manual errors occur in the process of pairwise comparison occur (Schneider & Wagemann, 2010). 15 Figure3: Softwareuseand visualization of results in theqca-studies Software Visualization Regular table XY-plot Formula Flow diagram No results Figure 3 further shows that the visualization of the results by means of a regular results table is dominant (N=27). These tables are produced by the QCA-software and are therefore easy to include

16 scientists do and QCA has thus mainly been exposed to public policy analysts in tha Furthermore, as a macro-level method, QCA is applied in the discipline of Public Policy mo to analyze countries, whereas the macro-level subcategory regio a /sub atio a is dom RQ2: Comparison Disciplines SPARD-research. As expected, given the nature of the discipline of Management Studies, th and micro-levels are the most common units of analysis there. The statistics for the d disciplines SPARD are provided largely in Table in 3. They line suggest with that larger difere tresearchdiscip i es focimay population into different choices for units of analysis. Differences regarding the units of analysis 16 Table3: Units of analysis in QCA-studies across disciplines Macro (total) Transnational Country Regional/subnational Meso (total) Organization Project Total (Rihoux et al., 2013) Public Policy (Rihoux et al., 2011) 49% 92% 6% 55% 31% 21% 6% Management Studies (Seny Kan et al., 2016) 16% 16% SPARD (this study) Micro (total) 21% 2% 29% 21% 6% 31% 31% Other 24% 52% 12% 12% 28% 27% 12% 15% Total 91% 100% 100% 100% N=313 N=143 N=95 N=33

17 In their article Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets, 17 Schneider and Wagemann (2010) formulated rules of thumb to guide researchers in conducting QCA 17 proper. Although QCA is currently subject to many methodological developments, their article RQ3: Good Practices provides a good basis for beginning QCA-researchers. We coded the 33 QCA-studies in SPARD on a selection of good practices and scored how well the articles adhered to those practices. The results are provided in Table 4. The full overview of the good practices in QCA can be found in Schneider and Wagemann (2010). We also refer the reader to QCA-handbooks that detail guidelines for conducting QCA proper (e.g., Gerrits & Verweij, 2018; Rihoux & Ragin, 2009). Often unclear which analytical choices have been made discussion: instigating good practices; how? Table4: Assessment of good practices in theqca-studies (N=33) Adhered to thegood practice? Good Practices Yes Somewhat No The case selection should be explicitly detailed and the scope conditions should be clear 67% 15% 18% The calibration of case s scores should be explicitly detailed 70% 15% 15% The appropriate terminology should be used 70% 9% 21% The analysis of necessity should be conducted separately and before the truth table analysis 39% 3% 58% The choice for consistency and coverage scores should be supported with arguments 48% 9% 42% The various solution formulas from the truth table analysis (complex, intermediate, parsimonious) should be reported 24% 3% 73% The outcome and non-outcome should be dealt with in separate truth table analyses 9 55% 45% The raw data matrix should be reported 33% 36% 30% The truth table should be reported 58% 6% 36% The consistency and coverage measures for the results should be reported 79% 21%

18 RQ3: Stated Usefulness QCA is sensitive to context (N=10) QCA enables small-n and bridging qualitativequantitative gap (N=14) QCA enables studying complex wholes (N=21): conjunctural causation, asymmetry, complexity theory

19 Discussion Getting it right: ensuring good QCA-practices in new areas of application - Good practices: which are they? - Transparency in articles with tight word limits about analytical choices - Calibration of qualitative data (recently: De Block & Vis) Addressing time and dynamics with QCA Areas of applications: applying QCA in evaluation practice (the world of policymaking)

20 References Alexander, E.R. (2016). There is no planning Only planning practices: Notes for spatial planning theories. Planning Theory, 15(1), Booth, P. (2015). What can we learn from France? Some reflections on the methodologies of cross-national research. In E.A. Silva, P. Healey, N. Harris, & P. Van den Broeck (Eds.), The Routledge Handbook of Planning Research Methods (84 96). Abingdon: Routledge. De Block, D., & Vis, B. (2018). Addressing the challenges related to transforming qualitative into quantitative data in qualitative comparative analysis. Journal of Mixed Methods Research. Gerrits, L.M., & Verweij, S. (2016). Qualitative comparative analysis as a method for evaluating complex cases: An overview of literature and a stepwise guide with empirical application. Zeitschrift fu r Evaluation, 15(1), Gerrits, L.M., & Verweij, S. (2018). The Evaluation of Complex Infrastructure Projects: A guide to Qualitative Comparative Analysis. Cheltenham: Edward Elgar. Janssen-Jansen, L., Spaans, M., & Van der Veen, M. (2008). New Instruments in Spatial Planning: An International Perspective on Non-Financial Compensation. Amsterdam: IOS Press. Rihoux, B., Rezsoḧazy, I., & Bol, D. (2011). Qualitative comparative analysis (QCA) in public policy analysis: An extensive review. German Policy Studies, 7(3), Rihoux, B., A lamos-concha, P., Bol, D., Marx, A., & Rezsoḧazy, I. (2013). From niche to mainstream method? A comprehensive mapping of QCA applications in journal articles from 1984 to Political Research Quarterly, 66(1), Schneider, C.Q., & Wagemann, C. (2010). Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comparative Sociology, 9(3), Seny Kan, A.K., Adegbite, E., El Omari, S., & Abdellatif, M. (2016). On the use of qualitative comparative analysis in management. Journal of Business Research, 69(4), Wikipedia (2018). Spatial Planning. Link: