(1) Verification Report: Johnson, C., (2010). Simandou Hills: Verification Study of high Resolution Modelling. Exeter: UK Meteorological Office.

Transcription

1 9B Model Validation As a subset of Phase 3 of the study additional validation of the United Kingdom Meteorological Office (UKMO) Unified Model (the model) was undertaken to test the applicability to the Simandou Range. This was achieved by comparing the control outputs against observed data, including satellite imagery. The Verification Report (1) has been reproduced to form this annex. (1) Verification Report: Johnson, C., (2010). Simandou Hills: Verification Study of high Resolution Modelling. Exeter: UK Meteorological Office. Simandou SEIA Volume I Mine Annex 9B 9B-1

2 Simandou Hills : Verification Study of High Resolution Modelling For: Rio Tinto Iron Ore Atlantic Ltd (RTIOAL) Date: 15 July 2010 Author: Catrina Johnson Model verification study v2.0-1 Crown copyright 2008

3 Prepared by: Catrina Johnson (Manager, Scientific Consultancy) Reviewed by: Matthew Perry (Scientific Consultant) John Fullwood (Senior Scientific Consultant) Authorised for issue by: Philip Beauvais (International Mining Business)

4 Contents 1. Introduction Model configuration Verification Windspeed Temperature Pressure Conclusions References Crown copyright

5 1. Introduction The Met Office was commissioned to carry out a climatological study on behalf of Rio Tinto Iron Ore Atlantic Ltd (RTIOAL). The overall objective of the study was to assess the potential impact on local climatology of mining excavations in the Pic de Fon area of the Simandou Hills in south east Guinea. The Simandou Hills form a steep-sided ridge aligned north-south which crosses the border between the wet Guinée Forestière to the south and the drier Haute Guinée to the north. The ridge is approximately 110 km in length and of average elevation 1.1 km above sea level (ASL), though this is only around 400m above the surrounding plateau which lies roughly 700 m ASL. Towards the southern end of the ridge, in the vicinity of Pic de Fon, the land rises to 1650 m ASL, some m above the surrounding plain. Phase 3 of the study comprised of high-resolution numerical simulations of local weather events across the Simandou region and comparing results obtained using control (unmodified) orography with those obtained using mined orography. These results were presented at the IFC workshop held in June The IFC raised concerns related to the use of this innovative concept to model potential climatic impact of mining activity and requested that more verification of the model results be carried out. This report sets out to verify model results against data recorded at monitoring sites installed and maintained by Schlumberger Water Consultants on behalf of RTIOAL. Crown copyright

6 2. Model configuration The Met Office Unified Model (the MetUM), the forecast model used operationally by the Met Office to produce numerical weather forecasts both regionally and across the entire globe, was used in Phase 3 of the climatological study. The numerical simulations were performed on nested grids, the outermost of which was that of the global configuration of the MetUM. The inner limited area domains, which progressively focus in on the Simandou region, have horizontal resolutions of approximately 12 km, 4 km, 1 km and 333 m. The location and extent of the inner model domains are shown in Figure 1. Figure 1: The extent and location of the inner model domains. Starting with the outermost domain the rectangles show the boundaries of the 12 km, 4 km, 1 km and 333 m resolution domains. The model was initialised using the ECMWF global forecast model analysis which has a horizontal resolution of about 30 km. It is therefore important to validate the model outputs to ensure that it provides a realistic representation of the meteorology local to Simandou. The model was run twice for each case study day, the first with unmodified orography and a second time with mined orography. This study assesses the accuracy of the results from the unmodified run to increase confidence in the model s ability to adequately simulate meteorological conditions across the area of interest. A full description of the modelling technique employed in Phase 3 is given in Vosper and Webster (2009). Crown copyright

7 3. Verification Figure 2 shows the locations of meteorological stations in the Simandou Hills area during 2007 and Of these, the main stations which recorded a range of weather parameters are Dabatini (1650m), Fokou West (810m), Mafindou (830m), Mandou (710m) and Pic de Oueleba (1330m). In addition, we have data from Kerouane (1450m) which is located off the map to the northwest. Observations from these stations are used to verify the accuracy of outputs obtained from the high resolution modelling study. Figure 2: Map of meteorological monitoring stations locations Crown copyright

8 The elements to be assessed are pressure, windspeed and temperature. 15 case study days were modelled in total, however not all elements were available on each day at each of the stations. Table 1 gives details of data availability. Station/Element Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Date Pres Wind Temp Pres Wind Temp Pres Wind Temp Pres Wind Temp Pres Wind Temp Pres Wind Temp 06/05/07 x x x x x x y x x y y x y y y x x x 05/08/07 x y y y y y y y y x y x x y x y y y 22/09/07 x y y y y y y y y x y x x y y y y y 01/12/07 x y y y y y y y y y y x x y x y y y 08/02/08 x x x x x x x x x x x x x x x x x x 26/02/08 x y y y y y y y y y y x x y y y y y 27/02/08 x y y y y y y y y y y x x y y y y y 28/02/08 x y y y y y y y y y y x x y y y y y 20/10/07 x y y y y y x x x x x x x x x x y y 29/07/07 x y y y y y y y y x y x x y x y y y 01/02/07 x x x x x x x x x y y x y y y x x x 02/07/07 x x x x x x y y y y y x y y y x x x 06/06/07 x x x x x x y x x y y x y y y x x x 08/08/07 x y y y y y y y y x y x x y x y y y 27/10/07 x y y y y y y y y x y x x y y y y y Table 1: Meteorological data availability Crown copyright

9 The bias (or difference) was calculated for each parameter, namely windspeed, air temperature and pressure, by subtracting the observed value from the modelled value. These values were then averaged over all available time periods at each station. This gives a measure of the difference between the observed and modelled values across the whole time period. The standard deviation (St. Dev.) was then calculated for the differences between observed and modelled values for each of the three parameters. It shows how much variation there is from the "average" (mean or expected/budgeted) difference. A low standard deviation indicates that the data points tend to be consistently close to the mean, whereas high standard deviation indicates that the data highly variable and is spread out over a large range of values. The root mean square error (RMSE) was calculated from the differences between the observed and modelled values for each of the three parameters. The RMSE is the square root of the mean squared difference (or error). Lower values of RMSE indicate a better fit. RMSE is a good measure of how accurately the model predicts the response, and is the most important criterion for fit if the main purpose of the model is prediction which is the case in this study. 3.1 Windspeed Overall, the bias varies across the six sites, as shown in Table 2. The higher altitude sites (Dabatini, Kerouane and Pic de Oueleba) have a negative bias and lower altitude sites have a positive one. There is however, good agreement with the highest bias being at Dabatini where the average windspeed is 1.6m/s higher in the model than from observations. Observations were available at 30 minute intervals. Location Count Bias (m/s) St. Dev. (m/s) RMSE (m/s) Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Table 2: Summary statistics of the differences in half hourly wind speed between observations and high resolution modelling output. Figure 3 illustrates observations and model output at three higher altitude sites on an individual day. It can be seen that there is a tendency for the model to underestimate windspeeds at Pic de Ouelaba and Dabatini although there is closer alignment towards the end of the period. This is in contrast to Kerouane where the model overestimates windspeeds during most of the time period. At the lower altitude stations for the same time period (Figure 4) it can be seen that the model generally overestimates windspeed although the differences are much smaller. This is highlighted in the standard deviation and root mean square errors values being higher for the higher altitude sites. Crown copyright

10 Windspeed at high altitude sites on 29th to 30th July Windspeed (m/s) Dabatini Obs Dabatini Model Kerouane Obs Kerouane Model Pic de Oueleba Obs Pic de Oueleba Model :30 16:00 17:30 19:00 20:30 22:00 23:30 01:00 02:30 04:00 05:30 07:00 08:30 10:00 11:30 Time Figure 3: Comparison of model and observed windspeed at high altitude sites on 29 th to 30 th July Windspeed at lower altitude sites on 29th to 30th July Windspeed (m/s) :30 15:30 16:30 17:30 18:30 19:30 20:30 21:30 22:30 23:30 00:30 01:30 02:30 03:30 04:30 05:30 06:30 07:30 08:30 09:30 10:30 11:30 Fokou Obs Fokou Model Mafindou Obs Mafindou Model Mandou Obs Mandou Model Time Figure 4: Comparison of model and observed windspeed at lower altitude sites on 29 th to 30 th July Crown copyright

11 Windspeed at high altitude sites on 26th to 27th February Windspeed (m/s) Dabatini Obs Dabatini Model Kerouane Obs Kerouane Model Pic de Oueleba Obs Pic de Oueleba Model 0 08:30 10:00 11:30 13:00 14:30 16:00 17:30 19:00 20:30 22:00 23:30 01:00 02:30 04:00 05:30 Time Figure 5: Comparison of model and observed windspeed at high altitude sites on 26 th to 27 th February Windspeed at lower altitude sites on 26th to 27th February :30 09:30 10:30 11:30 12:30 13:30 14:30 15:30 16:30 17:30 18:30 19:30 20:30 21:30 22:30 23:30 00:30 01:30 02:30 03:30 04:30 05:30 Windspeed (m/s) Fokou Obs Fokou Model Mafindou Obs Mafindou Model Mandou Obs Mandou Model Time Figure 6: Comparison of model and observed windspeed at lower altitude sites 26 th to 27 th February Figures 5 and 6 display the modelled and observed windspeed values again at high and lower altitude sites for a different time period. On this occasion the modelled windspeeds are much closer to the observed values, although the very high peak windspeed at Kerouane is not found in the model output. Crown copyright

12 Windspeed range (m/s) Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Obs Model Obs Model Obs Model Obs Model Obs Model Obs Model > Table 3: Windspeed frequencies (%), comparing model and observations from the six sites. From Table 3 it can be seen that the distribution of windspeeds are in broad agreement. The model has a tendency to under-represent high windspeeds at Dabatini and Pic de Oueleba which are higher altitude sites. At lower altitude sites, at Fokou West for instance, there are high percentages of low windspeeds in both the observations and model output. This increases confidence in the model s ability to simulate local features well, due to the variance of frequency distributions across the six sites. Crown copyright

13 3.2 Temperature There is a slight bias in air temperatures from the model output compared with observations at all six sites, with slightly higher model values (Table 4). Location Count Bias ( C) St. Dev. ( C) RMSE ( C) Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Table 4: Summary statistics of the differences in half hourly air temperature between observations and high resolution modelling output. Temperature at four sites on 29th to 30th July :30 16:00 17:30 19:00 20:30 22:00 23:30 01:00 02:30 04:00 05:30 07:00 Temperature (o C) 08:30 10:00 11:30 Dabatini Obs Dabatini Model Fokou Obs Fokou Model Kerouane Obs Kerouane Model Pic de Oueleba Obs Pic de Oueleba Model Time Figure 7: Comparison of model and observed temperature at four sites on 29 th to 30th July Crown copyright

14 Temperature at four sites on 8th to 9th August Temperature (oc) Dabatini_Obs Dabatini_Model Fokou Obs Fokou Model Kerouane Obs Kerouane Model Pic de Oueleba Obs Pic de Oueleba Model 14 20:30 21:30 22:30 23:30 00:30 01:30 02:30 03:30 04:30 05:30 06:30 07:30 08:30 09:30 10:30 11:30 Time Figure 8: Comparison of model and observed temperature at four sites on 8 th to 9 th August Temperature at five sites on 27th to 28th October Temperature (oc) :30 09:30 10:30 11:30 12:30 13:30 14:30 15:30 16:30 17:30 18:30 19:30 20:30 21:30 22:30 23:30 00:30 01:30 02:30 03:30 04:30 05:30 Dabatini Obs Dabatini Model Fokou Obs Fokou Model Kerouane Obs Kerouane Model Mandou Obs Mandou Model Pic de Oueleba Obs Pic de Oueleba Model Time Figure 9: Comparison of model and observed temperature at five sites on 27 th to 28 th October Crown copyright

15 Temperature at five sites on 26th to 27th February :30 10:00 11:30 13:00 14:30 16:00 17:30 19:00 20:30 22:00 23:30 01:00 02:30 04:00 05:30 Temperature (oc) Dabatini Obs Dabatini Model Fokou Obs Fokou Model Kerouane Obs Kerouane Model Mandou Obs Mandou Model Pic de Oueleba Obs Pic de Oueleba Model Time Figure 10: Comparison of model and observed temperature at five sites on 26 th to 27 th February Hourly series of temperature for four one day cases are shown in Figures 7 to 10. During 29 th to 30 th July 2007 (Figure 7) observed and modelled temperature values are in close agreement at all four sites throughout the period. The model has the tendency to slightly overestimate the temperature during 8 th to 9 th August 2007 (Figure 8). The model emulates the rise in temperature well in the morning of 27 th October 2007, as shown in Figure 9. However the modelled values only fall gradually when compared with the sharp fall in observed values, therefore overestimating the values for the remainder of the period. The peak and sharp fall in temperature is better modelled for 26 th to 27 th February 2008 as shown in Figure 10. The model therefore tends to overestimate temperature as is reflected in the positive biases displayed in Table 4, but is able to emulate peaks in temperature. Crown copyright

16 Temperature Range ( C) Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Obs Model Obs Model Obs Model Obs Model Obs Model Obs Model < > Table 5: Temperature frequencies (%), comparing model and observations from the six sites. Temperature distributions of both the observed and model output are shown to follow very similar patterns at all six sites in Table 5. The main differences are slight under-representations of low temperatures and over-representations of high temperatures in the model, which are consistent with the positive bias. Stations with a high frequency of lower observed temperatures e.g. Dabatini have a similar distribution in modelled data. This again confirms the model s ability to simulate spatial variations across the area which takes differences in orography into account. This was the main difference modelled between the unmined and mined simulations in the Phase 3 study. Crown copyright

17 3.3 Pressure Pressure model output was at 20 minute intervals which coincided with observations only once during each hour; therefore the number of verification points were lower than for temperature and windspeed. A conversion factor had to be applied to the model output data as it related to the pressure at mean sea level. A factor of 1 mbar decrease in pressure per increase of 30 feet in height was applied (Meteorological Glossary, 1991). There are small biases in pressure at the majority of sites (Table 6) with model output consistently lower than observed data. The slightly larger differences seen at Kerouane and Pic de Oueleba may be due to the correction factor rather than a true difference. This is indicated by the small standard deviation at both sites which also suggests that temporal variations in pressure are well modelled. Location Count Bias (mbar) Std. Dev RMSE (mbar) (mbar) Dabatini Fokou West Kerouane Mafindou Mandou Pic de Oueleba Table 6: Summary statistics of the differences in pressure between observations and high resolution modelling output. Pressure at four sites on 1st to 2nd December Pressure (mbar) :00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 00:00 02:00 04:00 Fokou Obs Fokou Model Kerouane Obs Kerouane Model Mafindou Obs Mafindou Model Pic de Oueleba Obs Pic de Oueleba Model Time Figure 11: Comparison of model and observed pressure at four sites on 1 st to 2 nd December 2007 Crown copyright

18 Pressure at four sites on 26th to 27th February :00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 00:00 02:00 04:00 06:00 Pressure (mbar) Fokou Obs Fokou Model Kerouane Obs Kerouane Model Mafindou Obs Mafindou Model Pic de Oueleba Obs Pic de Oueleba Model Time Figure 12: Comparison of model and observed pressure at four sites on 26 th to 27 th February The model s ability to simulate pressure is demonstrated in Figures 11 and 12 where both modelled and observed pressures are very closely matched at Fokou and Mafindou. The pattern of change is simulated well at both Pic de Oueleba and Kerouane which supports the view that the differences may be due to the correction factor rather than a true difference. Crown copyright

19 4. Conclusions One of the actions to come out of the IFC workshop was for the IFC and Rio Tinto to agree on a statement regarding the accuracy of the model and calibration status. Rio Tinto was tasked with providing a draft statement. This report will form the basis of this statement. The results presented in this report illustrate the power of high resolution modelling for gaining an insight into the climatology of a local area. Spatial variations caused by the unique shape of the orography can clearly be seen in the model results. The high resolution 333m model is able to resolve local features such as differences in height between the six monitoring locations. Lower resolution models such as those at 1km and 4km are not able to do this as accurately. The results are in broad agreement with a verification study carried out for a high resolution study of Weymouth Bay, UK, Perry et al. (2010). This demonstrated the MetUM s ability to simulate local features at a fine resolution scale in the UK. Verification of the model results was carried out using hourly and half hourly observations from six stations in the Simandou Hills area. The purpose of the main study (Phase 3) was to simulate both the current and post mining meteorological conditions to assess the impact that mining activity may have on the area. This study has shown that the model is able to simulate the current conditions well, with only a small bias in the mean for most of the climate variables. Frequency distributions of both temperature and windspeed were in broad agreement at all six sites. This increases confidence in the model s ability to simulate possible changes due to mining activity in the area, as the largest change will be in the height of the ridge itself. Crown copyright

20 References Meteorological Glossary (1991), Meteorological Office, Met O 985 Perry, M., Wang, C. and Kitchen, K. (2010): Local Climatology Tool pilot study. High resolution modelling of Weymouth Bay. Met Office, Exeter. Vosper, S. and Webster, S. (2009): Climatological Study of the Simandou Hill area of Guinea Phase 3: Extended Numerical Modelling Study of Potential Impacts of Mining. Met Office report prepared for Water Management Consultants. Crown copyright

21 Met Office Tel: FitzRoy Road, Exeter Fax: Devon Crown EX1 copyright 3PB United Kingdom