IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-47940IDA-56370) ON A CREDIT AND ADDITIONAL FINANCING IN THE AMOUNT OF SDR8.

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-47940IDA-56370) ON A CREDIT AND ADDITIONAL FINANCING Public Disclosure Authorized IN THE AMOUNT OF SDR8.3 MILLION (US$ 12 MILLION EQUIVALENT) TO THE REPUBLIC OF MOLDOVA FOR A DISASTER AND CLIMATE RISK MANAGEMENT PROJECT Public Disclosure Authorized March 20, 2017 Social, Urban, Rural and Resilience Global Practice Ukraine, Belarus and Moldova Country Unit Europe and Central Asia Region

2 CURRENCY EQUIVALENTS (Exchange Rate Effective: January 24, 2017) Currency Unit = Leu Leu = US$ 1 US$ 1.00 = Euro 0.93 FISCAL YEAR 2017 ABBREVIATIONS AND ACRONYMS ACSA AWS BETF CPESS CPF CPS DCRMP DES DRM ECC ECMWF EIB EMP EUMETNET EUMETSAT ERR GDP GFDRR ICR IDA IIASA ITC JIT MAFI MoE MoF MoIA MoldATSA NPV NWP PAD National Rural Development Agency Automatic Weather Stations Bank-Executed Trust Fund Civil Protection and Emergency Situation Service Country Partnership Framework Country Partnership Strategy Disaster and Climate Risk Management Project Department of Exceptional Situations Disaster Risk Management Emergency Command Center European Centre for Medium-Range Weather Forecasts European Investment Bank Environmental Management Plan Network of European Meteorological Services European Organization for the Exploitation of Meteorological Satellites Economic Rate of Return Gross Domestic Product Global Facility for Disaster Reduction and Recovery Implementation Completion Report International Development Association International Institute for Applied Systems Analysis Information Technology and Communication Just-in-Time Ministry of Agriculture and Food Industry Ministry of Environment Ministry of Finance Ministry of Internal Affairs Moldavian Air Traffic Services Authority Net Present Value Numerical Weather Prediction Project Appraisal Document

3 PDO PMT PPF SEEDRMAP SEE- MHEWS-A SSP SHS UNDP UNECE UNISDR WMO Project Development Objective Project Management Team Project Preparation Facility South Eastern Europe Disaster Risk Mitigation and Adaption Program South East Europe Multi-Hazard Early Warning Advisory System Shared Socio-Economic Pathway State Hydrometeorological Service United Nations Development Program United Nations Economic Commission for Europe UN International Strategy for Disaster Reduction World Meteorological Organization Senior Global Practice Director: Ede Jorge Ijjasz-Vasquez Practice Manager: David Sislen Project Team Leader: Anatol Gobjila ICR Team Leader: Daniel Kull

4 MOLDOVA DISASTER AND CLIMATE RISK MANAGEMENT PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. Comparison of Indicators in PAD Section A and the Supplemental Letter..44 Annex 10. List of Supporting Documents... 45

5 A. Basic Information Country: Moldova Project Name: Disaster and Climate Risk Management Project Project ID: P L/C/TF Number(s): IDA-47940,IDA ICR Date: 03/20/2017 ICR Type: Core ICR Lending Instrument: SIL Borrower: MOLDOVA Original Total Commitment: Revised Amount: XDR 8.29M Environmental Category: B XDR 6.80M Disbursed Amount: XDR 8.29M Implementing Agencies: Ministry of Environment, State Hydrometeorological Service, Department of Exceptional Situations, Ministry of Agriculture Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 11/25/2009 Effectiveness: 11/10/ /10/2010 Appraisal: 06/01/2010 Restructuring(s): Approval: 08/05/2010 Mid-term Review: 12/19/ /15/2013 Closing: 09/30/ /30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Moderately Satisfactory Substantial Moderately Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i

6 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No No Moderately Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Major Sector/Sector Public Administration Public administration - Water, sanitation and flood protection Original Actual Major Theme/Theme/Sub Theme Environment and Natural Resource Management Climate change Adaptation Finance Finance for Development Disaster Risk Finance Urban and Rural Development Disaster Risk Management Disaster Preparedness Disaster Response and Recovery Disaster Risk Reduction E. Bank Staff Positions At ICR At Approval Vice President: Cyril E Muller Philippe H. Le Houerou Country Director: Satu Kristiina J. Kahkonen Martin Raiser Practice Manager: David N. Sislen Wael Zakout Project Team Leader: ICR Team Leader: ICR Primary Author: Anatol Gobjila Daniel Werner Kull Alexei Ionascu ii

7 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project development objective (PDO) is to strengthen the State Hydrometeorological Service's ability to forecast severe weather and improve Moldova's capacity to prepare for and respond to natural disasters. The PDO will be achieved through strengthened capacities to: (i) monitor weather and issue early warnings of weather-related hazards by providing timely and accurate hydrometeorological forecasts and services; (ii) manage and coordinate responses to natural and man-made disasters; and (iii) help individuals, particularly farmers, be aware of, and adapt to natural hazards and climate variability. Revised Project Development Objectives (as approved by original approving authority) In 2015, the project received additional financing of USD 2 million to finance cost overruns and a resulting financing gap in the parent project. However, neither the PDO, nor the original PDO indicators were revised. There were minor editorial and numerical adjustments in the numerical values of selected indicators. See the table below. (a) PDO Indicator(s) Indicator Indicator 1 : Value quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) More specific forecasting of weather conditions Scale of weather forecasts at 5000 sq. km. Scale of weather Forecasts reduced to 300 sq km Formally Revised Target Values Actual Value Achieved at Completion or Target Years Completed Date achieved 08/05/ /30/ /15/2016 Comments (incl. % achievement) Expanded lead-time of weather warnings to users, particularly Department Indicator 2 : of Exceptional Situations (DES) Value quantitative or Qualitative) Lead time for severe weather warnings only 10 minutes to 1 hour Lead time for severe weather warnings expanded to 12 hours iii Lead time for severe weather Completed warnings expanded to 3-6 hours

8 Date achieved 08/05/ /30/ /30/ /15/2016 Comments (incl. % achievement) Indicator 3 : Strengthened capacity to coordinate response to emergencies Value quantitative or Qualitative) No Emergency Command Center to coordinate response among relevant agencies. Emergency Emergency response drill response drill shows shows capacity capacity improvements as Completed improvement compared to the s as baseline and the compared to recent test of the baseline Date achieved 05/08/ /30/ /30/ /01/2015 Comments (incl. % achievement) (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Installation of Doppler radar which improves weather monitoring and forecasting Value (quantitative or Qualitative) No effective radar is currently installed As the radar is installed, improved forecasts are produced with warnings when appropriate Completed Date achieved 08/05/ /30/ /04/2014 Comments (incl. % achievement) Indicator 2 : Now-casting tools, data collection and dissemination improved Value (quantitative or Qualitative) The process of automated data collection or Operational now-operationacasting is now-casting realized; is realized: iv Completed

9 dissemination is not available for an early warning system automated data are collected, disseminated and integrated with radar and flash flood system resulting in timely warnings for users automated data are collected, disseminated and integrated with the radar system and properly visualized, resulting in timely warnings for users Date achieved 08/05/ /30/ /30/ /15/2016 Comments (incl. % achievement) Indicator 3 : The ECC is established, tested and operational Value (quantitative or Qualitative) An additional floor is built to accommodate the ECC ECC has been tested and is operational Completed Date achieved 08/05/ /30/ /04/2014 Comments (incl. % achievement) Indicator 4 : ECC users are trained to operate the emergency management system Value (quantitative or Qualitative) No staff currently trained in the emergency management information system Relevant Relevant personnel (at personnel (at least 90) least 80) trained trained in the in the emergency emergency Completed management management system system established in established in ECC ECC Date achieved 08/05/ /30/ /30/ /04/2016 Comments (incl. % achievement) v

10 Indicator 5 : At least 50 investment grants provided and put on the demonstration plots (farmers receive information about practical techniques for adaptation to climate risks on pilot basis) Value (quantitative or Qualitative) No activities exist to disseminate practical experience on adaptation to climate risks other than drought 50 demonstration plots are established; 60 demonstration and dissemination events are organized Completed. 53 investment grants provided. 92 demonstration and dissemination events were organized Date achieved 08/05/ /30/ /04/2014 Comments (incl. % achievement) Project supported organization(s) publish reports on the effect of Indicator 6 : collaboration with stakeholders Value (quantitative or Qualitative) No Yes Yes. Completed. The project carried out a country-wide dissemination campaign related to the dissemination of climateresilient agricultural techniques tested on the demonstration plots Date achieved 04/01/ /30/ /15/2016 Comments (incl. % achievement) vi

11 G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 12/25/2010 Satisfactory Satisfactory /19/2011 Satisfactory Satisfactory /03/2012 Satisfactory Satisfactory /31/2012 Satisfactory Moderately Satisfactory /26/2012 Satisfactory Moderately Satisfactory /26/2013 Moderately Satisfactory Moderately Satisfactory /07/2013 Moderately Satisfactory Moderately Satisfactory /21/2014 Moderately Satisfactory Satisfactory /18/2014 Moderately Satisfactory Satisfactory /16/2015 Moderately Satisfactory Satisfactory /13/2015 Moderately Satisfactory Moderately Satisfactory /07/2016 Moderately Satisfactory Satisfactory /19/2016 Moderately Satisfactory Satisfactory H. Restructuring (if any) Not Applicable I. Disbursement Profile vii

12 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Moldova is a landlocked country located between Romania and Ukraine with a surface area of 33,840 km 2 and a population of about 3.5 million. Moldova s economic comparative advantage has been its fertile soil and mild temperate continental climate that has the potential to transform the country into an important agricultural producer in the region. However, the country s comparative advantage in agriculture has been increasingly affected by global climate trends. Moldova has been experiencing more frequent and intense extreme climatic events, such as early frosts, flash floods, hailstorms, droughts and other extreme weather. These extreme events result in significant economic losses and undermine the potential for growth and poverty reduction. The 2000 drought crippled Moldovan agriculture in the spring and summer and affected about 2.6 million people. According to the UNDP estimates, the proportion of overall agricultural losses in affected areas was between 70 and 90 percent. The 2007 drought caused estimated losses of about US$1.0 billion; the 2008 floods cost the country about US$120 million. But even earlier, during the period, Moldova s average annual economic losses due to natural disasters were about US$61 million, or 2.13 percent of national GDP (EM-DAT: The CRED/OFDA International Disaster Database). Natural disasters have a disproportionate impact on the poor and roll back development gains in Moldova. With 57 percent of the population living and 84 percent of the poor concentrated in rural areas, the poorest are hardest hit by weather-related hazards. Hence, the ability to anticipate and manage climate impacts, particularly extreme hydrometeorological events, are of critical importance for sustained poverty reduction in Moldova. Rationale for Bank involvement The World Bank has been a reliable development partner for Moldova since 1993, having provided significant budget support and investment funding for the adjustment of Moldova s institutions and economy to the needs of a market economy. The Country Partnership Strategy for the relevant period, FY 09-12, stressed the need for improved disaster risk management capabilities in the context of economic vulnerability, particularly the impact of extreme weather events on agriculture. The subsequent CPS for FY highlighted the promotion of green, clean and resilient Moldova to mitigate the debilitating effects of climatic events on agriculture and improved natural resource management to ensure sustainable development. 1

13 At the time, the World Bank already had substantial institutional expertise in disaster risk management operations. During , the World Bank financed 528 projects, an estimated US$26 billion in lending, to respond to natural disasters. 1 Historically, the Bank has provided post-disaster reconstruction assistance, but the focus had already started to shift to support disaster preparedness and mitigation, and catastrophe risk financing. In addition, the Bank, in partnership with other organizations, had developed critical policy tools and knowledge products, including instruments to assess post-disaster damage, disaster area needs, hazards and vulnerability, and risk-transfer instruments. Countries in the Europe and Central Asia (ECA) Region are increasingly interested in putting in place proactive measures that will communities and decision-makers better prepare for and better manage disaster risks over time, including the impacts of climate change. The Bank had supported projects in disaster risk management in Turkey, Romania, Poland, and Albania, among others. In 2009, a major World Bank report, Adapting to Climate Change in Eastern Europe and Central Asia, presents links between climate change and disaster management, as well as a useful framework to develop climate change adaptation action plans. 2 These are examples of many similar World Bank activities to develop a body of knowledge and practice that is useful to natural-hazard vulnerable countries such as Moldova. The Government of Moldova had been receiving technical assistance for disaster risk management and climate change adaptation from other development partners, and was willing to complement this technical assistance with the combined knowledge and financing that the Bank is uniquely placed to offer, to reduce vulnerability to natural hazards and introduce climate risk management policies, practices and measures. Given the above, the World Bank was uniquely positioned to provide both technical assistance and significant investments for building institutional capacity, adopting modern approaches and procuring the relevant hydrometeorological and emergency response equipment to an extent that no other multilateral or bilateral donor could match at the time of the operation. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project development objective was to strengthen the State Hydrometeorological Service s ability to forecast severe weather and improve Moldova s capacity to prepare for and respond to natural disasters. The PDO was to be achieved through strengthened capacities to: (i) monitor weather and issue early warnings of weather-related hazards by providing timely and accurate hydro meteorological forecasts and services; (ii) manage and coordinate responses to natural and man-made disasters; and (iii) help individuals, particularly farmers, be aware of, and adapt to natural hazards and climate variability. 1 Hazards of Nature, Risks to Development An IEG Evaluation of World Bank Assistance for Natural Disasters; Independent Evaluation Group (IEG), World Bank, World Bank,

14 To monitor progress towards the achievement of the PDO, the project used three project outcome indicators: (i) (ii) (iii) More accurate and specific forecasting of weather conditions Expanded lead-time of weather warnings to users, particularly the Department of Exceptional Situations (DES) Strengthened capacity to coordinate response to emergencies In addition, the project used five intermediate outcome indicators to monitor progress in individual components: 1. Installation of Doppler radar which improves precision of forecasting severe weather. 2. Now-casting tools 3, data collection and dissemination is improved 3. The Emergency Command Center (ECC) is established, tested and operational. 4. The users of ECC are trained to operate the emergency management system 5. At least 50 investments grants are provided and introduced at demonstration plots (farmers receive information about practical techniques for adaptation to climate risks on pilot basis. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification In 2015, the project received additional financing of USD 2 million to finance cost overruns and a resulting financing gap in the parent project. However, neither the PDO, nor the original PDO indicators were revised. The PAD for the additional financing highlighted that minor adjustments to the results framework aimed at correcting the intrinsic scientific weaknesses in the definition of the indicators were made, and that changes in the Results Framework didn t significantly affect the correct measurement of the achievement of the original PDO. The numerical changes in the targets mostly affected the second PDO indicator where the lead time for severe weather warnings was changed from 12 to 3-6 hours. There was also an increase in the number of ECC staff trained from 80 to 90. Finally, given the mid-term decision to drop the purchase of the flash flood system the target for Now-casting tools, data collection and dissemination improved was revised to edit out and flash flood system. The other revisions are more editorial than substantive in nature. 3 The forecasting of the weather within the next six hours is often referred to as now-casting (Source: Glossary of Meteorology). 3

15 Box 1: Summary of the Results Framework Revisions Original PDO Indicator More accurate and specific forecasting of weather conditions Expanded lead-time of weather warnings to users, particularly DES Strengthened capacity to coordinate response to emergencies Original Intermediate Results Indicator Installation of Doppler radar which improves precision of forecasting severe weather Now-casting tools, data collection and dissemination improved ECC users are trained to operate the emergency management system Revised PDO Indicator More specific forecasting of weather conditions No change No change Revised Intermediate Outcome Indicator Installation of Doppler radar which improves weather monitoring and forecasting No change No change Original Target Scale of weather forecasts reduced to 300 sq. km Lead time for severe weather warnings expanded to 12 hours Emergency response drill shows capacity improvements as compared to the baseline and the recent test of the system Original Target As the radar is installed, improved forecasts are produced with warnings when appropriate Operational nowcasting is realized; automated data are collected, disseminated and integrated with radar and flash flood system resulting in timely warnings for users Relevant personnel (at least 80) trained in the emergency management Revised Target No change Lead time for severe weather warnings expanded to 3-6 hours Emergency response drill shows capacity improvements as compared to the baseline Revised Target No change Operational nowcasting is realized: automated data are collected, disseminated and integrated with the radar system and properly visualized, resulting in timely warnings for users Relevant personnel (at least 90) trained in the emergency 4

16 New Indicator Project supported organization(s) publish reports on the effect of collaboration with stakeholders system established in ECC Yes (reports published) management system established in ECC Not applicable The revisions in the Results Framework, including the introduction of the additional indicator were properly reflected in the Supplemental Letter #2 to the Additional Financing Agreement. 1.4 Main Beneficiaries The primary beneficiaries of the project were: (i) The Government of Moldova: through strengthened capacity to forecast and respond to severe weather. The primary beneficiary institutions were the State Hydrometeorological Service, the Ministry of Environment, the State Department for Exceptional Situations and the Ministry of Agriculture. (ii) Local: the 52 farmers who were the beneficiary of the investment grants for setting demonstration plots to pilot and test activities aimed at increasing awareness about coping and adaptation techniques necessary to make agriculture more resilient to adverse weather. Other beneficiaries include the farmers who participated in the trainings on climateresilient agricultural techniques conducted on the demonstration plots as well as the farmers who subscribed to the just-in-time communication platform. Ultimately the general public is benefitting from improved forecasting, early warning and disaster response services. 1.5 Original Components (as approved) A. Project components The Moldova DCRMP proposes four components: (A) strengthen severe weather forecasting; (B) improve disaster preparedness and emergency response; (C) initiate activities for adaptation to climate risks in agriculture; and (D) support project management. A summary of project-supported components is provided below while more detailed description of project design is included in Annex 4. 5

17 Component A: Strengthen Severe Weather Forecasting This Component aimed to strengthen the State Hydro Meteorological Service s ability to forecast severe weather and provide decision makers and other users with more effective, diverse, and timely forecasts and warnings. Sub-Component A.1 Develop Early Warning/Now-casting Capabilities. Economic losses from severe weather, flash floods and floods can be significantly reduced by establishing a now casting system. This sub-component will strengthen data, communications, and modeling technology to provide timely, accurate, and geographically precise weather hazard warnings. The sub-component will support automation of the hydrometeorological observation network to accelerate accurate data delivery to users for more effective decision-making. This sub-component will upgrade meteorological instruments and hydrologic gauges to automated sensing systems to provide users with data and will also increase computer quantity and software quality to create more useful data and effective forecast products for users. Strengthening Moldova s hydro meteorological data has potential for benefiting neighboring countries. Sharing of hydro meteorological data and information with countries in the region has potential for improving forecasting capabilities for Moldova s SHS as well as its partners. Sub-Component A.2 Dual Polarization Doppler Radar Technology for Localized Forecasts. A Dual polarization Doppler radar is now the most effective meteorological tool to predict floods, high winds, hail, and other severe weather, and issue warnings. This sub-component will improve meteorological modeling systems by providing a mesoscale model and Integrated Meteorological Workstation, and eventually a hydrologic modeling system for predicting flash floods. These will provide best practices in delivering effective warnings to the Department of Exceptional Situations (DES), Ministry of Agriculture and Food Industry (MAFI) and other users of SHS services. Sub-Component A.3 Development of Plans for Seasonal/Climate Forecasts. Building a real-time hydro meteorological data system is a first step to improving medium- to longterm forecasting. Improving and automating agro-meteorological data collection and distribution will improve drought forecasting. The next phase in strengthening hydro meteorological forecasting for drought involves expanding forecasting capability to monthly and seasonal scales. SHS is developing a monthly climate forecasting program, and starting to build a national drought forecasting center, which is part of the World Meteorological Organization Eastern Europe Regional drought forecasting program. Building the data system leads to more effective input to Global and Regional Decadal climate forecasting models to improve accuracy of model simulations to use for adaptation. This sub-component will support the SHS to develop plans for such a data system and, if resources permit, to provide investments for equipment. Capacity Building. To ensure that the proposed Project investments are implemented effectively and that the SHS upgrades its technical capacity, the Global Facility for Disaster Reduction and Recovery (GFDRR) has agreed to provide a US$100,000 technical assistance grant in support of the proposed Project. This grant is expected to finance additional staff training in using new equipment and provide opportunities for SHS staff to 6

18 benefit from international experience in weather and climate service delivery. Some portion of the grant may be used to strengthen DES capacity to coordinate disaster response. Component B: Improve Disaster Preparedness and Emergency Response This Component aimed to strengthen Government capacity to manage emergencies and coordinate disaster response among levels of government agencies by establishing and operating an Emergency Command Center (ECC), and associated capacity-building activities. The ECC would facilitate joint disaster response with agency representatives and could assemble decision makers. It would operate continuously, equipped with a decisionsupport system to enhance the flow of crucial DES data, such as hydro meteorological forecasts and SHS alerts; this would allow DES to issue timely warnings, and undertake prevention and response measures, including evacuating affected populations. Sub-Component B.1 - Feasibility Study and Design. The Project will support feasibility and design studies as a basis to establish the ECC. The two studies will include: (a) ECC architectural design within the DES headquarters that ensures seismic resilience of structural and non-structural features, and efficient space planning; and (b) information management system design to link DES headquarters with DES local offices and sectoral institutions such as SHS, Ministries of Environment and Agriculture, Institute of Geology and Geophysics, and others. The design would include voice and data transfer capacity and two-way information processing, support daily DES operations, and integrate existing legacy systems. This design will be informed by existing coordination mechanisms in the country as well as by seeking input from other agencies that participate in disaster preparedness and response. Sub-Component B.2 - Establish Emergency Command Center. When feasibility and design studies are complete, the Project will support establishment of the ECC by financing the following: (i) facility renovation and refurbishment works; (ii) ECC furniture and equipment; (iii) information technology (IT) hardware; (iv) emergency information management software; and (v) communication equipment. Sub-Component B.3 Capacity Building and Evaluation. DES employees will staff the ECC on an as-needed basis, along with staff from other agencies, particularly during emergencies. The Project will support capacity building for DES and other agencies by providing training in emergency management information system, particularly operating the IT decision-support system, for an estimated 100 staff from 15 agencies and all regional and local DES units. To ensure sustainability and facilitate knowledge transfer, capacity building will be designed as train the trainers so participants can transfer knowledge to their colleagues. Component C: Initiate Activities for Adaptation to Climate Risks in Agriculture The objective of the component was to enhance the practical application of agrometeorological information in the agriculture sector in order to increase its resilience towards adverse weather effects. This objective will be achieved through support to the following activities. 7

19 Sub-Component C.1 Development of a Just-in-Time (JIT) Communication Platform. This sub-component will support the development of the technical architecture (software and hardware) of a mobile JIT communication platform that could serve as a tool to rapidly disseminate critical, localized weather information to a large number of farmers and rural communities. Sub-component C.1 would support the design and testing of the communication platform, including free dissemination of the service to a farmer focus group. The design will focus on the information flows of severe weather alerts originating from the SHS to farmers via a content provider associated with the Ministry of Agriculture and Food Industry (MAFI), and in collaboration with mobile communication companies. Following the design and testing phases the platform could be fully rolled-out by MAFI in collaboration with the SHS and mobile phone companies. Furthermore, the platform could be expanded to evolve into an early warning system that would combine current and forecasted agro-meteorological data to simulate scenarios for crop development, and provide farmers with advice on necessary mitigation/enhancement agronomical measures. Last but not least, the platform can be modulated to include dissemination of other critical information to farmers, such as market and extension information. In this context, synergies with the e-governance project currently under preparation will also be explored. This subcomponent is linked to SHS improvements under Component A. Sub-Component C.2 Adverse Weather Adaptation Advisory Services. Sub-component C.2 would provide technical advisory services, including grant investment support to farmers, farmer groups and rural communities for piloting and testing activities aimed at increasing awareness about coping and adaptation techniques necessary to make agriculture more resilient to adverse weather. It will build on the country s existing experience in implementing demonstration plots for adaptation to drought through extension services and aim to expand the existing knowledge base and awareness in the farming community about adaptation techniques to a wider spectrum of risks, such as: (i) frost; (ii) excess snow and water cover of winter crops; (iii) hail storms; (iv) drought; (v) inundation of crops from torrential downpours and rising river waters. Investments supported will include on-farm irrigation, water harvesting basins, anti-hail net systems, protective forest belts, conservation land cultivation, as well as other activities with adaptation potential. Selection of beneficiaries for sub-component C.2 would be done through a competitive scheme with clear technical, social and financial eligibility criteria. Component D: Project Management This component will provide fiduciary support for the implementation of the three components. In essence, fiduciary services will be centralized for the proposed Project. The POPs Project Management Team (PMT) housed in the Ministry of Environment will be the Project s fiduciary entity. In particular, the support will enhance the financial management and procurement functions as well as, if necessary, the technical aspects of project implementation, including monitoring and evaluation. 1.6 Revised Components There were no substantial revisions of the project components. The scope of the project after the mid-term review, as well as with the processing of the additional funding to the parent project, essentially remained the same. However, there is one important caveat 8

20 regarding the scaling down of support for hydrological services, namely the decision not to acquire flash flood forecasting models/capabilities, in favor of a stronger focus on enhancing severe weather forecasting capacity. Originally, the intermediate outcome indicator related to now-casting tools, data collection and dissemination improved included the dissemination from three sources of weather information: automated weather stations, the radar and a flash flood forecasting system. However, this could not materialize given the unsuitability of available modeling solutions to the Moldovan context and the lack of project funds and time to develop a customized system. In addition, the European Investment Bank (EIB) committed to fund the Moldova Flood Protection project, which was to develop the SHS capacity to forecast floods, as well as develop a master plan and investment program for the sub-sector. 1.7 Other significant changes The closing date of the original parent project was extended by one year from September 30, 2014 to September 30, The parent project received additional financing of US$ 2 million on May 19, 2015 to cover the cost overruns and ensuing financing gap resulting from increased construction costs and the need to develop customized software for the Emergency Command Center under Component B. The project s closing date was thus again extended to September 30, 2016, to allow for the development of the customized software for and associated staff training of the SHS, as well as to complete some of the Component A procurements which had been delayed. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Soundness of the background analysis the background analysis for the project was generally sound. The project design was informed by the recommendations developed under the South Eastern Europe Disaster Risk Mitigation and Adaption Program (SEEDRMAP), a joint WB/GFDRR and UNISDR initiative that identified hazard monitoring, early earning, disaster management and climate change adaptation as issues requiring special attention in the region. The project design was informed by other WB supported projects in the region, e.g. Croatia, Poland, Turkey and Russia. The main lesson that emerged from those projects was that ensuring system integration was key to improved forecasting and service delivery., The multiple systems needed for hydrometeorological services delivery including observation, data management, meteorology, hydrology, climatology, visualization, forecasting and dissemination must communicate effortlessly with each other. Therefore, the project activities related to weather forecasting had the interoperability of the acquired systems as a driving theme. While early in the project there were some misunderstandings of the utility of certain equipment by the project team, for example the weather radar, the project was eventually able to 9

21 integrate the procurements into a modern forecast production approach. The project also benefited from analytical work provided by other donors, such as UNDP and the local think tank community. The rationale for Bank s intervention was sound as the WB possessed the unique combination, among other donors, of both technical knowledge and experience and the ability to provide the required investment to strengthen Moldova s ability to forecast and respond to natural disasters. Assessment of the project design the objective of the project was clear and derived from the need to address existing country capacity constraints to monitor severe weather events and increase the ability of various stakeholders to respond to them to minimize economic and human losses. While the project involved three Moldovan institutions (the State Hydrometeorological Service, the State Department of Exceptional Situations and the Ministry of Agriculture), the project design cannot be qualified as overly complex. Its components targeted limited and specific priority improvements in those institutions/sectors that were expected to contribute to the improved country capacity to identify and respond to climate risks. As an example, Component B did not purport to bring about a comprehensive institutional overhaul of the DES. Instead, it focused on the specific ability of that institution to respond quickly to emerging natural and man-made disasters by building a modern command center capable of accepting and processing incoming information with the subsequent ability to respond speedily to emergencies. The individual components activities were not on a critical path to each other. Hence, the number of beneficiary institutions did not hamper implementation progress, as they could be performed in parallel and independently of each other. At the same time there were outcome interdependences, for example timely DES action and the ability of the just-in-time platform to inform farmers with useful lead-time about weather hazards is dependent on the timeliness and quality of SHS forecasts and early warnings. To keep project management coherent, the task team made use of a project management unit that was experienced with WB fiduciary procedures. The project design probably overestimated the institutional capacity of the SHS, as well as underestimated the potential for fluidity of management and staff, to implement a project of this nature. More upfront dialogue with the Government of Moldova on the role and status of the institution, investment in capacity training, and the development of an organizational strategic plan could have served as enabling reference points for an institution that went through five changes of top management throughout implementation. Additionally, while the decision to bring the PMT on board was a sound one, their earlier involvement in the project preparation could have been beneficial in establishing a better relationship between the implementing entities, particularly with the SHS which seems to have struggled in understanding the PMT s fiduciary role. Adequacy of government commitment - the commitment of the Government of Moldova at entry was high. The country was dealing with the consequences of the 2010 floods in the north and the importance of enhanced ability to receive early warnings and manage natural hazards was on top of the government agenda at that 10

22 time. There was a high degree of commitment and active involvement from the participating institutions at the project design phase. However, that commitment varied across implementation timeframe and the implementing institutions. The SHS and its supervising Ministry of Environment were particularly affected by the frequent changes in management which hampered progress in component A (see below the sections on implementation and Government Performance) Relevance of Risks Identified and Mitigation Measures the project team identified risks that were relevant to project success: (i) incremental costs associated with the operation and maintenance of the procured equipment; (ii) the relatively low level of implementation capacity; and (iii) the political risks associated with the willingness to share improved hydrometeorological data with other agencies, neighboring countries and international organizations. The team discussed those risks with the Government of Moldova at project design and received assurances that those incremental Operations and Maintenance (O&M) costs will be covered by the budget. With the benefit of hindsight, the project underestimated the institutional strengthening required to successfully operate and utilize the acquired equipment in the case of SHS. While equipment-related O&M aspects were considered, the human side of the operation was underestimated. Due to low pay, as well as other institutional weaknesses, the ability of the SHS to retain trained forecasters and ITC (Information Technology and Communication) staff remained limited. The project team, however, took that into account at the mid-term review when it commissioned a review of the status of the SHS, which produced, among other things, recommendations to increase SHS s visibility, status, and budget, and hence improve its ability to retain qualified staff. 2.2 Implementation There were two major factors that affected project implementation: i) implementation delays related primarily to the Component A of the project; ii) costs overruns in component A and B that led to the provision of additional financing to cover those overruns. Here is a description of implementation by project component. Implementation under Component A was the most challenging, with preparation of technical specifications and terms of reference, as well as ensuing procurements chronically behind schedule. Despite trainings for staff, including participation in the Turin-based procurement courses, and continuous PMT support, the SHS s technical specifications were frequently not vendor-neutral, inconsistent, and technically questionable, which delayed the acquisition process. This is not uncommon; technical specifications for hydrometeorological equipment are notoriously difficult to develop properly, even for high income countries. Due to political instability in the country, the project was confronted with frequent changes in management. There were five changes at the top of SHS during project preparation and implementation and four changes of Environment Ministers meant that the institution was often without the required management leadership, direction and supervision. There was a natural learning curve for the incoming directors 11

23 which also led to delays and sometimes changing priorities, e.g. one of the directors questioned the need for building houses for meteorological observation stations, and consequently plans for the construction of one such house was abandoned. The implementation suffered a particularly bad period when the SHS was without a General Director or a project focal point for about 8 months in 2013, resulting in near full lack of disbursements. That resulted in an accumulated backlog of uncommitted funds of close to USD 2 million. All this had the unfortunate domino effect of delaying and reducing the capacity building phase required for bringing this equipment to full operational use. For example, the automated weather stations, a critical component for the now-casting capacity of the SHS, was delivered very close to the revised project completion end date, leaving little time within the project time-frame for integrating these tools and the Doppler radar into a local visualization and forecasting platform to deliver weather-related services as originally intended by the project. Implementation under Component B was relatively even throughout the project cycle. Although the procurement and preparation of the feasibility studies and the construction design of the Emergency Command Center took longer than originally planned, once completed, the remaining activities were generally on target for the remainder of the project with small and manageable time variances. The Emergency Command Center was commissioned in August, 2013, well within the closing date of the parent project. However, due to the additional time needed to develop a customized software program as opposed to the purchase of an off-theshelf package, as suggested by the feasibility studies, and the required subsequent staff training, the component required a time-extension. This component also produced the largest cost overrun, its total cost increasing from $2.6 million to $4.4 million, funded originally by the unused funds from the behind-the-schedule Component A and later by the additional financing agreed by IDA. While the project estimated construction needs at US$ 379,000 at appraisal, the actual costs were twice the estimate at US$ 793,052. This is accounted for by the fact that the actual costs were determined by the two project-financed feasibility studies which recommended a more complex construction solution than envisaged at appraisal and price inflation for construction works. However, the more significant overrun was caused by the determination that the originally recommended off-the-shelf software could not be adjusted to the needs of the ECC concept proposed by the feasibility studies and an original system had to be developed from scratch. Original equipment costs were also underestimated. Implementation under Component C was the least problematic of all components both in terms of implementation timing and cost overruns. The development of the just-in-time communication platform was contracted to the National Rural Development Agency (ACSA) which is the national rural extension service. The platform was designed on time and became fully operational in October The additional benefit of contracting the development of the system to ACSA was in the synergies achieved through integration with a different system developed earlier the Agriculture Marketing Information System. The idea showed a lot of promise 12

24 in the beginning, however as things evolved, the relevance of the just-in-time system to the farmers showed its limitations due to the continued limited capacity of the State Hydrometeorological Service to provide localized weather forecasts. Additionally, the ability of the platform to send mobile notifications en-masse to farmers in case of adverse weather is constrained by the technical capacity of the mobile operators present on the Moldovan market, a fact that was not considered fully at project design. That resulted in relative modest take-up of the service. The number of subscribers reached 1109 and has remained essentially stagnant after activity completion. ACSA was also contracted to implement the Adverse Weather Adaptation Advisory Services. By all accounts, the activity was a great success, and this approach is now scaled up through the upcoming Moldova Climate Adaptation Project. The interaction between the focal point at the Ministry of Food Agriculture, ACSA and the project management unit was effective and it resulted in fifty-three grants implemented to showcase various approaches to improved resilience to climate change to around 3000 farmers. More importantly, two hundred and twenty farmers replicated the practices on their own farms in the post-training and demonstration phase. Bank s Response to Implementation Challenges - The Bank team s reaction to emerging problems was adequate and timely. The team had intensive discussions with both the Steering Committee and the Ministry of Environment highlighting the risks of further delays to the development outcome. At one point, the team considered the cancellation of Component A in case of continued inaction. To improve project progress and chances of achieving the development outcomes, after the mid-term review, the project offered more support for the SHS institutional strengthening by funding the development of a strategic institutional development plan with TF resources. It also funded the technical assistance to develop the legal package to change the SHS status within the Moldovan public sector to allow improved remuneration and income-generating activities to retain qualified staff. Finally, to prevent further delays, the Bank team was wise in tactically reallocating the uncommitted funds from Component A to Component B to fund the emerging funding gap while processing the additional financing in parallel. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The key performance indicators in the PAD Section A were the same as in the Supplemental Letter for the Parent Project (please refer to Section 1.2). When the additional financing was approved, there were several minor changes in the results framework (discussed in detail in Section 1.3). Similarly, the performance indicators in the PAD for Additional Financing coincide with those in the supplemental letter. Design -The results framework is compact and generally struck a sound balance between the number of indicators and the ability to capture anticipated performance improvements in the key institutions involved in the early warning of and response 13

25 to severe weather events, without resorting to an overwhelming number of indicators. The result framework is logical with an easy-to-understand flow from intermediate outcomes to project outcomes. In fact, the WB operations department highlighted this particular framework as an exemplary results framework and the staff associated with the project made several presentations at WB-organized operational workshops. However, one area of potential future improvement is to monitor not only weather forecast resolution, but also quality (in terms of accuracy and skill verification). That said, the intermediate indicators related to the establishment of the Emergency Command Center could have been slightly more specific in capturing improvements. While the baseline was zero (or in some cases nonspecific), the project could have attempted to measure a set of more specific, global standard target values for emergency institutions, such as response time, and have them monitored and reported during the drill exercises. The results framework could have benefited from the addition of an intermediate outcome indicator related to the development of the just-in-time communication platform which was supposed to be the tool through which the farmers would receive localized weather information from SHS. This represents a critical link in the project logic which was expected to build the capability to warn farmers about adverse weather. Finally, there is no clarity whether the additional indicator related to dissemination of the project results referred to the entire project or only to Component C, the piloting of climate-resilient techniques. The PAD is vague and the PMT suggested that this indicator related only to the dissemination of results achieved under Component C related to climate-resilient techniques. In terms of baseline values and numerical values for measuring progress, the first two PDO indicators had both baseline values and end target numerical values. The value for the third outcome indicator was zero as capacity was considered nonexistent at project initiation. Similarly, the baseline value for the intermediate indicators was zero as the project built capability that was non-existent or very poorly monitored in the country prior to the intervention. The results framework did provide a timeframe for the achievement of each indicator that followed logically from the implementation of project outputs. However, given important implementation delays, some of the actions shifted naturally beyond the life of the project. For example, the SHS visualization and forecasting platform was supplied extremely close to the end date of the project, such that no package of services could be developed within the project time-frame as the beneficiary, SHS in this particular case, is still in the process of learning and adopting these tools. Hence, it will be able to develop useful, relevant localized weather products/information to potential users only after the completion of the project, also with continued Bank support under a recently approved BETF (Bank Executed Trust Fund). 14

26 Implementation - There was no dedicated M&E staff to collect and track progress in terms of indicators, which was not necessary given the relative parsimony of the project log-frame. The PMT was responsible for collecting information from implementing institutions and updating the results framework bi-annually for reporting purposes to the Steering Committee and Bank supervision teams. Utilization - The Steering Committee and the Bank teams used the updated results framework more as a bird s view snapshot of bi-annual status and progress towards the development outcome rather than a day-by-day progress monitoring tool. The tool was useful as it allowed decision-makers to focus on the critical issues and try to induce corrective actions. The results framework was developed solely for the implementation of this project and was not intended to live on beyond the project completion date. The outcome indicators remain useful, in principle, for SHS in tracking its operations as a reminder of the original purpose of the equipment and tools acquired under the project. 2.4 Safeguard and Fiduciary Compliance Financial Management The project benefited from the experience of a project management unit which implemented other World Bank-funded project prior to the Disaster and Climate Risk Management Project. The World Bank reviews rated the financial management system as moderately satisfactory throughout the project cycle. All the annual audits of the project resulted in unqualified opinions. There were two issues flagged and addressed: (i) in FY 2012, the auditors brought the issue of manual transposition of accounting records from the Statement of Expenditures into the financial reports due to lack of specialized accounting software. The PMT contracted a private IT specialist who helped them upgrade the accounting software to address the issue to the satisfaction of the IDA financial management specialist, and (ii) The second issue related to the annual inventory of the assets procured under the project. While the PMT performed annual inventory of its assets on regular basis, the assets procured for the project beneficiaries were not monitored after their transfer into beneficiaries custody. Consequently, the PMT was advised to ensure monitoring of these assets either through including them in the annual inventory performed by PMT, or asking the beneficiaries to provide the copy of their inventory lists on annual basis. The PMT decided to carry out the inventory of those assets by themselves and this was done in summer of Procurement - the PMT procurement expert supported all three components, providing day-to-day guidance to the relevant implementation agency personnel on acquisitions and the consequent contract management. All the staff from the three Moldovan implementing institutions (SHS, DES and MAFI) involved in the procurement process attended procurement training courses in Turin. The project organized study visits for SHS staff to familiarize them with various suppliers in cases where expensive and sophisticated equipment had to be purchased. The Bank team helped the SHS by providing sample specifications for more complex procurements, such as the Doppler radar. The three postreview exercises carried out by the WB procurement team raised no procurement or fraud 15

27 and corruption concerns. Compliance and performance risks have been consistently rated as low. Disbursement - The Project disbursed behind the schedule throughout its life due to implementation challenges in Component A. However, in its last implementation year, it essentially converged with and surpassed the original disbursement schedule. The balance of undisbursed funds at completion of the grace period for payments was less than 0.1% (US$10,000). Environmental Assessment - The Project was rated category B as it supported several construction activities: the installation of the tower for the weather radar, the construction of four weather and hydrological monitoring stations and finally the renovation of the building for the Emergency Command Center. There were also environmental concerns under Component C related to the construction works on some of the demonstration plots, e.g. construction of water-collection ponds. The project developed Environmental Management plans for all construction sites that covered environmental, labor safety and proper equipment for use on sites. The visiting teams regularly verified the compliance with the Environmental Plans as well as proper supervision and monitoring of the plan by the PMT. The Aide Memoires confirm that the Environmental Management Plan (EMP) recommendations have all been essentially met although the Bank team had to draw continuous attention to the need to ensure proper safety gear on construction sites. Social safeguards - No social safeguards were triggered. The construction sites were all on government-owned land which was not in use either legally or illegally. Public consultations were organized on all sites, with local communities being informed about the scope, impact and expected benefits of the project. The Bank teams reported compliance with the social safeguards during their regular supervision visits. 2.5 Post-completion Operation/Next Phase Given the delays in the implementation of Component A due to frequent changes in management at SHS and the resulting lack of institutional direction and management leadership in the intermittent phases, the component s achieved outcomes continue to suffer from lack of robustness. Despite the advances made under the project in building the SHS s technical capability, the capacity of the SHS to make full use of it to provide customized, relevant and localized weather, climate and meteorological services to various users is still fairly limited. The ability of the SHS to find, attract and retain qualified staff to use the state-of-the art equipment is still constrained as the SHS s status to improve remuneration was not changed despite the support provided by the project. The interviews with the SHS management indicate that they intend to follow up on some of the institutional recommendations in the post-completion phase. Given improvements in performance closer to the end of the project and the recent stability in the SHS management, a follow-up grant of $500,000 was secured from the Global Facility for Disaster Reduction and Recover (GFDRR) to help develop the SHS into a more 16

28 service delivery-oriented agency, guided by the World Meteorological Organization s (WMO) Strategy for Service Delivery. This is a useful and needed follow-up activity that is likely to help cement the development achievements of the project and ensure the sustainability of the investments under the original project. Interviews with the current management indicated that they do plan to pursue actions that would enable the SHS to generate income and improve pay, although it must be noted that even in the most advanced national hydrometeorological services, income from fee-based services tends to be limited to a maximum of 20% of total budget. Fee-based services are only successful if they are of high quality and reliable, and if users/clients have the resources to pay, both of which continue to be challenges in Moldova. On the other hand, the DES Emergency Command Center is fully operational, staffed and budgeted for in terms of post-completion O&M. The Ministry of Finance increased the budgetary allocation for DES for the operation and maintenance of the ECC by MDL 4.0 million annually. Further DES capacity strengthening is considered under the forthcoming Climate Adaptation Project. The proposed project is considering the establishment of one regional Emergency Command Center in Balti, as per the recommendations of the projectfunded feasibility study, to ensure better quality of services at the local level, improve efficiency of support for remote locations, better support tactical field operations. It will further build preparedness and response capacity through technical assistance and investments in equipment, such as fire and rescue vehicles, extreme winter conditions access capacity, temporary flood management modules, electricity and heat generation, etc. Similarly, the follow up Climate Adaptation Project will build upon and scale up the adoption of climate-smart agriculture technologies piloted and tested under the third component of the Disaster and Climate Risk Management project, such as minimum and no-till agriculture, drip irrigation, greenhouses, and UV/hail nets. No follow up work is currently envisaged in terms of removing the existing constraints related to the optimal use of the just-in-time communication platform built under Component C, i.e. required improvements in the technical capabilities of the Moldovan mobile operators to be able to send early warnings messages with useful lead time for users. These require relatively substantial investments and cannot be forced upon private operators. It would be useful to keep the issue in the ongoing policy dialogue with the Government if the platform is to fulfil its original purpose of transmitting just-in-time localized weather information to subscribed farmers. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The Project objectives, design, and implementation continue to remain highly relevant to Moldova s development and environmental priorities. The Government of Moldova, through the Ministry of Environment and with donor-support, developed and approved a National Climate Change Adaptation Strategy for six most vulnerable sectors of the 17

29 Moldovan economy: agriculture, water resources, forestry, human health, energy and infrastructure that envisage climate-adaptation investments in these sectors. Reflecting national priorities, the third pillar of the Country Partnership Strategy (CPS) for FY14-17 focuses on promoting a green, clean and resilient Moldova. The new Systematic Country Diagnostic (SCD, 2016), validated the hypothesis that climate risk is important for achieving the twin goals in Moldova. Consistent with the CPS and the national strategy, the WB Climate Adaptation Investment Planning Technical Assistance conducted a quantitative assessment of adaptation investment opportunities and returns across the target sectors. The capability built in the three institutions under the project continues to be of critical importance to Moldova s ability to manage vulnerabilities in those six sectors. These capabilities will be further strengthened and scaled up through the proposed Moldova Climate Adaptation Project informed by the WB Climate Adaptation Investment Planning TA. 3.2 Achievement of Project Development Objectives A review of the project development objectives, results and intermediate results indicators suggests that the project has met the project outcome indicators at completion, although in relation to SHS the robustness of the achievements leaves to be desired and the risks to the outcomes remain substantial. The project objective: is to strengthen the State Hydrometeorological Service s ability to forecast severe weather and improve Moldova s capacity to prepare for and respond to natural disasters. PDO Indicators More specific forecasting of weather conditions. Baseline: Scale of weather forecasts at 5000 sq. km. End Target: Scale of weather forecasts reduced to 300 sq.km Status: The indicator was achieved as the SHS has significantly upgraded its technical capability with state-of-the-art equipment (weather radar and automatic weather stations), as well as capacity to assess and utilize regional and global high resolution information and utilize outputs from global and regional numerical models such as those of the ECMWF 4 (visualization, data analysis and now-casting tools), to be able to provide weather forecasts at the end target scale. While the project focused on severe weather such as storms, droughts and extreme temperatures, by strengthening capacity to forecast basic weather parameters such as temperature, precipitation and wind, general weather forecasting has also improved. However, as mentioned above, the robustness of this outcome remains tenuous, due to possible loss of capacity to utilize instruments and models as SHS s institutional issues described above persist. Also, more work is necessary to 4 European Centre for Medium-Range Weather Forecasts 18

30 develop customized weather-related services, particularly for the DES and farmers, and is likely to supported be with further assistance from the GFDRR grant in the post-project phase. Finally despite improvements in resolution, quality (in terms of accuracy and skill) of forecasts needs to be better monitored and if necessary improved. Expanded lead-time of weather warnings to users, particularly the Department of Exceptional Situations (DES) Baseline: Lead time for severe weather warnings only 10 minutes to 1 hour. End target: Lead time for severe weather warnings expanded to 3-6 hours. Status: The indicator was achieved as the communications interface between the two agencies exists and the ability to provide the required lead-time exists, with data exchange happening in real time. The utility for emergency response depends on the ability of SHS to analyze and interpret the raw data coming from the radar, automated stations and other sources. This capacity is currently fragile, but will be further strengthened as the SHS will improve its capacity to use the infrastructure provided by the project, and as further support from the GFDRR grant is provided for fine-tuning existing and developing new weather information delivery mechanisms to the DES, as well as other SHS customers. SHS s recent joining and launching of MeteoAlarm under EUMETNET 5, supported by this project, UNDP and the Austrian Central Institute for Meteorology and Geodynamics (ZAMG), reflects existing improvements. Strengthened capacity to coordinate response to emergencies. Baseline: No Emergency Command Center to coordinate response among relevant agencies. End target: Emergency response drill shows capacity improvements as compared to the baseline. Status: The target is achieved as the reports on two emergency drills conducted during the project lifetime showed capacity improvements in timely mobilization of relevant authorities and emergency operational units in case of modelled exceptional situations during the drills. Following implementation completion the country has not experienced weather-related hazards that would have put the ECC to the real test. However, the DES is also responsible for responding to man-made disasters, fires and traffic accidents. The DES management report that consolidation of emergency call services under the centralized ECC helped improve the speed of response and deployment of the closest to the DES accident teams. In the future, the DES would benefit from defining a set of clear and measurable emergency response indicators for further use in its daily operations. 5 European Meteorological Services Network 19

31 Intermediate Outcome Indicators Installation of Doppler radar which improves weather monitoring and forecasting. Baseline: No effective radar is currently installed. End target: The radar system is operational. Status: the indicator was achieved, as the radar system was commissioned in March The SHS staff was trained to use the numerical software used to analyze the incoming raw data and transform it into useful weather products. SHS and Moldavian Air Traffic Services Authority (MoldATSA) are co-sharing the radar which is an important arrangement for the sustainability of the investment as MoldATSA is a more reliably funded agency. Nowcasting tools, data collection and dissemination improved. Baseline: The process of automated data collection and dissemination is not available. End target: Operational nowcasting is realized: automated data are collected and integrated with the radar system and properly visualized resulting in timely warnings for users. Status: The indicator was completed, although it must be mentioned that the automated weather stations were acquired with significant delays and delivered close to the project closing date. This, in turn, delayed the SHS ability to collect, assimilate and disseminate improved data. The supplier was providing the required training for SHS users at the time of this ICR writing. Upon completion, the SHS will be able to use these stations for improved data collection, assimilate the data from the stations, Doppler radar, global remote sensing and forecast providers (EUMETSAT 6, ECMWF, etc.) through the procured visualization and forecasting platform, to provide localized weather forecasts. The Emergency Command Center (ECC) is established, tested and operational Baseline: An additional floor is built to accommodate the ECC End target: ECC has been tested and operational. Status: The Command Center was tested and fully operational. It is a fully operational state-of-the-art facility that is used by the Committee for Exceptional Situations for its bi-annual meetings and for regular emergency drills. The users of ECC are trained to operate the emergency management system Baseline: no staff currently trained in the emergency management information system. End target: Relevant personnel (at least 90) trained in the emergency management system established in ECC. Status: completed and operational, 90 ECC users were trained and fully proficient in using the technical capabilities of the command center. 6 European Organization for the Exploitation of Meteorological Satellites 20

32 At least 50 investments grants provided and put on the demonstration plots (farmers receive information about practical techniques for adaptation to climate risks on pilot basis) Baseline: No activities exist to disseminate practical experience on adaptation to climate risks other than drought End Target: 50 demonstration plots are established; 60 demonstration and dissemination events are organized Status: The intermediate outcome was met and exceeded. 53 grants had been provided to set up demonstration plots on climate-resilient practices; 220 other farmers have replicated the practices on their farms; and 3000 farmers overall participated in the trainings conducted on the demonstration plots. Project supported organization(s) publish reports on the effect of collaboration with stakeholders Baseline: No implying that no disemination efforts were carried or planned origianlly End target: Yes implying that the results were actively disseminated via various media channels Status: The project advertised extensively the availability of the demonstration plots and the climate-resilient farming techniques by targeting the farmers through various media tools: 100 media articles; 70 radio broadcasts; 37 TV broadcasts and video clips; 6 specialized editions about the activities promoted under Component C on the nation-wide show Bastina targeted mostly to rural residents and farmers. 3.3 Efficiency The original appraisal used conservative assumptions to estimate potential project performance resulting in an economic rate of return (ERR) of 25 percent, net present value (NPV) of US$10.3 million, and a benefit-cost (B/C) ratio of 2.0. Subsequent assessment to prepare the Additional Financing indicated that the addition of US$2.0 million to the costs would not prevent the project from achieving the originally estimated ERR and B/C ratio, due to the introduction into the analysis of the mitigated damages (project benefit) for the catastrophic drought of A more detailed and more conservative approach was applied for the post-project economic performance assessment (see Annex 3 for details) which indicated that the Project with Additional Financing performed on par with or even better than expected in economic terms. Even with the most conservative assumptions, the Project is delivering more benefits than costs at a rate of return greater than the assumed opportunity cost of capital (discount rate). The ex-post economic analysis indicates solid levels of efficiency in the implementation of the project. The project was implemented efficiently. Although project implementation experienced delay, the project management cost was only 6.4 % of the total project cost. 21

33 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory The PDO s objective remains relevant to Moldova s ability to prepare for and respond to natural disasters. The national technical capability was significantly improved with stateof-the art equipment and information systems both at the SHS and DES. The Emergency Command Center is fully operational. At project completion, the SHS was not able to use the upgraded capability to the full extent while the development of the flash flood forecasting system was dropped. The SHS is poised to develop further its range of customized weather-related services in the post-implementation period with the help of a WB/GFDRR grant and support from other bilateral donors, thus making maximum use of the upgraded equipment and systems and significantly enhancing the development objective. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The project did not specifically target the poorer sections of Moldovan society. The focus of the project, particularly of the first two components, was to build the domestic capacity to provide a public good, i.e. the improved response by national emergency authorities to weather-related disasters, which benefits the entire society. However, given that such climate-related disasters disproportionately affect the rural population where most of the Moldovan poor resides, the investments are expected to have a greater positive impact on the poor. Despite almost 80% of the poor and 70% the of lowest 40% of the population by income being employed in the agricultural sector, and women being more vulnerable to unfavorable climate conditions, the project missed the opportunity to explicitly integrate poverty and gender targeting in the selection criteria for the demonstration plots that piloted climate-resilient techniques. This could have provided important lessons on what techniques/investments are affordable to the poor and which are beyond their reach. (b) Institutional Change/Strengthening The project built capacity in three Moldovan institutions. The project provided modern forecasting equipment and training on how to use it for the SHS. It strengthened the capacity of the DES by building a state-of-the-art Emergency Command Center now fully operational and used by Moldovan authorities in responses to natural and man-made disasters. Finally, it built capacity at the Ministry of Agriculture and Food Industry to run pilot schemes to test modern and climate-responsive agricultural techniques (knowledge generation). Officials at two institutions, SHS and DES, stressed the capacity improvements induced by the implementation of a WB-funded project, particularly in terms of procurement of complex works and goods and developing complex technical specifications for such acquisitions. 22

34 (c) Other Unintended Outcomes and Impacts (positive or negative) The acquisition of the Doppler radar had a positive impact on the Moldovan aviation sector, particularly the safety of air flights to and from Moldova. The WB insistence on locating the radar on the premises of MOLDATSA and its use by the agency bodes well for the sustainability of this investment in Moldova. Although not captured by the results framework, there is anecdotal evidence that the grant scheme for demonstration plots from Component C led to the emergence of a network of suppliers of climate-resilient tools and equipment that was non-existent in the country prior to the implementation the scheme. Perhaps not totally unintended or unexpected, but similarly not captured in the results framework, Component C of the project resulted in about 220 climate-resilient practices replicated on other farms during the lifetime of the project. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops No project completion workshop was held as the Moldova project proponents deemed it unnecessary due to the small number of well-defined stakeholders, i.e. essentially just the implementing agencies and their subordinated institutions. Workshops were held by ACSA upon completion of implementation of Component C, to discuss results with farmers and consult them on scaling up of adaptation technologies. 4. Assessment of Risk to Development Outcome Rating: Significant The risks to the development outcome remains high. These mainly relate to the sustainability of the investments made at SHS. Although the SHS s technical capability has been significantly upgraded by the project, the SHS capacity to fully use the available equipment on a continuous basis and provide customized weather-related services to the DES, farmers and other potential clients is still limited. The project helped develop a strategic organizational plan for SHS that foresaw a change in its status within the Moldovan public administration with the ability to increase staff salaries and generate additional income. However, again due to political turmoil, the proposed changes have not yet been passed by Moldovan authorities and translated into action. Hence, the SHS ability to retain the skilled staff required to operate sophisticated weather forecasting equipment remains constrained. Finally, the allocation of the budget required for operation and maintenance of the acquired equipment will continue to remain a concern. These risks will be, to a large extent, mitigated by the implementation of the follow-up GFDRR grant that will assist the SHS in developing weather-forecasting services targeted to public and commercial clients and aims at building the institutional capacity in line with the World Meteorological Organization s (WMO) Strategy for Service Delivery. The demand for SHS services will be further increased through engagements such as the WBfunded Climate Adaptation Project (P155968) or EIB Flood Protection Project and 23

35 Dniester Basin Adaptation Plan (OSCE 7 and UNECE 8 supported). Additionally, SHS is part of regional initiatives that are likely to further improve SHS service delivery, specifically the South East Europe Multi-Hazard Early Warning Advisory System (SEE- MHEWS-A) and inclusion of Moldova as a priority country under the Global Framework for Climate Services. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory The Bank helped identify an area of support that is of critical importance for Moldova s ability to respond to increasingly frequent climate risks, particularly for a country where half the population resides in rural areas and relies on agriculture for their livelihoods. The project design was balanced and targeted critical interventions that addressed priority technical capability and capacity gaps in responding and mitigating weather-related risks in three Moldovan institutions. The results framework was compact and captured the key outputs and outcomes well. The employment of a PMT to coordinate implementation and manage fiduciary aspects across the three implementing agencies proved a useful and valuable arrangement to keep the project on track. The Bank team could have paid more attention to the existing institutional weakness at the SHS. More upfront design elements focused on capacity building and institutional strengthening could have potentially helped prevent the implementation delays down the road and served as a useful reference point to the frequently changed SHS managers. The team could have checked its assumptions about the availability of mobile operators capacity to send multiple simultaneous messages and the suitability of the off-the-shelf software for the ECC. (b) Quality of Supervision Rating: Satisfactory The Bank was involved and pro-active in the supervision of all the aspects of the project through regular semi-annual supervision missions involving the full team, and importantly, through day-to-day dialogue conducted from the country office. The Bank team diligently reported implementation challenges in the AMs, management letters and/or ISRs and tried to address the issues expeditiously. e.g. to address implementation challenges at SHS, the Bank financed an organizational and legal review of the SHS that prepared the ground for changing its status in the public sector by allowing to increase staff remuneration and be 7 Organization for Security and Co-operation in Europe 8 United Nations Economic Commission for Europe 24

36 able to keep qualified staff. In the midst of numerous reshuffles during the project life, the Bank engaged pro-actively with the counterpart to bring the incoming officials on board with the project objective, implementation status and challenges. It must however be recognized that modernization of hydrometeorological services is a highly technical and complex niche for which the Bank team struggled to deliver sufficient internal capacity. This improved throughout the project lifetime by leveraging the development of a Bank/GFDRR service team on the topic. The Bank team regularly reviewed the safeguard and fiduciary arrangements for compliance. Recognizing the risk to the sustainability of the development outcome, particularly for the first component, the Bank team secured a post-implementation grant to strengthen the service provision side of the SHS. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The quality of the supervision was high as the Bank team responded in a timely and adequate manner to implementation challenges. The quality at entry could have been better through a more robust assessment of the capacity of mobile operators to send multiple messages to farmers. The suitability of the off-the-shelf software for the ECC could have been verified upfront. The World Bank s support in project preparation and implementation is rated as moderately satisfactory as per ICR guidelines that the overall rating should be the lower of the two. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The Government of Moldova and all the implementing agencies strongly supported and were actively involved in the preparation of the project. However, the project was affected by the political turmoil of Moldova s coalition governments and the resulting frequent reshuffles, particularly at the Ministry of Environment and SHS. Those reshuffles were not conducive to good implementation progress in Component A and resulted in lack of oversight, management leadership and direction. At some point the SHS did not have a General Director for about eight months and implementation was brought to a halt with a build-up in procurement backlog of up to $2 million. After the mid-term review, the pace picked up again and the project was completed on a revised schedule after a two-year extension. Prioritization of hydrometeorological services by the Government, while identified on paper, remains challenged in terms of actual budgeting and status of the SHS. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory The project involved four implementing institutions: the SHS and the supervisory Ministry of Environment; the Department of Exceptional Situations and the Ministry of Agriculture. 25

37 The DES was an exemplary implementing agency. After a relatively slow start related to the inexperience of working with the Bank, mainly related to the development of complex technical specifications, it got up to speed and provided the required management leadership and involvement for a successful completion of Component B on time and on a revised budget; it cooperated well with the PMT and the Steering Committee. The Ministry of Agriculture was helpful and provided the support required for the implementation of Component C through ACSA. The ministerial focal point participated actively in the development of the grant selection criteria, were active members of the evaluation panels and were heavily involved in dissemination events. The Ministry of Environment and the SHS were the most affected institutions in terms of management stability and leadership. This affected the speed and quality of implementation in Component A, but both institutions made an effort, particularly after the mid-term review when faced with the prospect of component cancellation, to accelerate implementation and managed to complete the most important procurements within the revised project completion date. The PMT held the entire project together and ensured fiduciary and safeguard compliance. Staffed with people who had prior experience in WB-funded projects, they were instrumental in monitoring progress and informing the Steering Committee and other stakeholders of the emerging problems. The Steering Committee was a useful overall coordination structure, but its ability to induce corrective action was limited by the nature of the coalition Government in Moldova where heads of institution were more accountable to their respective party leadership rather than formal government arrangements. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory Overall borrower performance is rated as moderately satisfactory taking into account the government s commitment in preparing the project, the good progress in two components and the mobilization to complete the most essential procurements under the first component of the project after the mid-term review. 6. Lessons Learned It would be useful to integrate upfront into project design institutional management capacity building activities, such as development of institutional strategic plans, as well as operational and investment plans. These serve as useful guidance references for these agencies, particularly in volatile political environments of coalition governments where frequent reshuffles can be expected. Such plans, grounded in thorough contextual analysis, help provide institutional stability and continuity. This was confirmed by the feedback provided by the current SHS management. 26

38 Similarly, investments to strengthen/modernize hydrometeorological services should upfront assess if the involved agency(ies) have modern forecast/service production processes and standard operating procedures in place. If not, the project should first support development of such procedures such that subsequent procurements and capacity building investments are nested within and support a service producing process. The bigger picture must always be kept in mind; such processes and procurements must ultimately be part of a service-oriented plan to respond to hydrometeorological information needs. It should be recognized that a single 5-6 year project will generally not be able to elevate a severely resourcechallenged National Hydrometeorological Service to capacity levels seen in high income countries. Technical support provided through Bank-executed TA resources (or other similar instruments) may be required on a continuous basis to assist implementation by National Hydrometeorological Services, which may be inexperienced with implementing large-scale modernization processes and developing complex technical specifications. Such TA is also helpful in bringing staff up to speed in operating the new equipment and related software beyond the training provided by suppliers. Overall, such Bank TA is likely to significantly enhance the likelihood of achieving project outcomes and their sustainability. The use of PPF funds and potentially complementary Bank-executed TA resources for the preparation of feasibility studies before project implementation starts could inform projects better and save valuable implementation time. The use of project management teams for fiduciary and compliance purposes as well as for overall coordination is an extremely practical and valuable arrangement, particularly when the project involves several implementing agencies. However, it would be better to involve such units early in project preparation so that: a) the PMTs have an organically developed understanding of the project; b) other implementing agencies quickly gain a clear understanding of the PMT s role in implementation; c) enable early detection of potential misconceptions about roles and responsibilities of PMTs verses implementing agencies, thereby allowing corrective action before the project begins implementation. The ability of overall project coordination structures, such as Steering Committees, may have a limited power to induce corrective action in implementing agencies under coalition Governments, particularly if the heads of such supervisory and implementing institutions do not share the same party affiliation. To mitigate such risks, it is useful to map actual accountability lines (rather than formal ones) and resort more pro-actively to those informal influencers to effect corrective action and avoid implementation delays that jeopardize project outcomes. Early and continuous engagement with the Ministry of Finance is required to ensure that sufficient budget is allocated for the maintenance and operation of the acquired equipment, or at least that the issue is constantly highlighted. In Moldova, the DES 27

39 management pursued those discussions more vigorously and secured the required budget for the ECC while the combination of existing low visibility/marginalization of the institution and continuous management changes at SHS prevented them from ensuring the required budget is allocated immediately in the post-project phase. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 28

40 (b) Cofinanciers Not applicable (c) Other partners and stakeholders A number of other development partners have been involved in providing complementary technical assistance and support to the SHS during the life-time of the project. UNDP and the Austrian Development Agency through the UNDP-supported project Supporting Moldova s National Climate Change Adaptation Planning Process supported the partnership between the Austrian Central Institute for Meteorology and Geodynamics (ZAMG) and SHS to share best practices on forecasting and early warning, and explore ways to leverage their partnership s strength to benefit communities prone to climate -related disasters. One outcome of this partnership was SHS becoming a member of the MeteoAlarm, an initiative by EUMETNET, the Network of European Meteorological Services, active in 35 countries in Europe and strongly supported by the World Meteorological Organization (WMO). The Czech Development Agency supported the establishment of 11 automated hydrological posts on the river Prut. MCC supported the establishment of further 8 automated hydrological posts on the Nistru and its tributaries. The EIB supported the Moldova Flood Protection Project with the objective of developing a flood protection master plan and an associated investment plan along with the SHS capacity building in flood management, including in hydraulic modelling. 29