Quality Control of a surface wind observations database for North Eastern North America

Transcription

1 Control of a surface wind observations database for North Eastern North America E.E. Lucio-Eceiza UCM, IGEO (eelucio@fis.ucm.es) J.F. González-Rouco UCM, IGEO J. Navarro CIEMAT A. Hidalgo Global Forecasters S.L. P.A. Jiménez CIEMAT E. García-Bustamante U. Murcia N. Casabella CIEMAT J.L. Conte Global Forecasters S.L. H. Beltrami CASI (St. Francis Xavier U.) UCM Paleoclimate Modeling & Analysis (PalMA) Dpto. Física de la Tierra, Astronomía y Astrofísica II Universidad Complutense de Madrid Instituto de Geociencias CSIC-UCM 8th Seminar for Homogenization and 3rd Conference on Spatial Interpolation May 2014 Budapest, Hungary

2 Control Procedures, suited for different purposes: Operational data A single station Historical s Several Stations One variable Wind Speed & Direction Several Variables DeGaetano, A. T., 1997: A quality-control routine for hourly wind observations. Journal of Atmospheric and Oceanic Technology, 14, Jimenez, P., J. Gonzalez-Rouco, J. Navarro, J. Montavez, and E. García-Bustamante, 20: assurance of surface wind observations from automated weather stations. Journal of Atmospheric and Oceanic Technology, 27,

3 41 automatic stations 13 years of data min/30min resolution Jiménez et al., JOAT, 27, 20 Location of Navarra and spatial distribution of the 41 stations (Jimenez et al. 20). ï90 ï75 ï60 ï45 ï Ý EC FOC ï Ý ï Ý ï Ý ï Ý ï Ý Ý Ý Ý Ý Ý Ý Ý Ý NCAR ï Ý ï Ý ï Ý ï Ý ï Ý 45 ï Ý ï Ý Ý Ý Ý Ý ï Ý 40 ï90 ï Ý 50 ï75 ï60 ï Ý Size comparison between Navarra (Jimenez et al. 20, circled) and the area of North 40 Eastern North America. ï45 2

4 ï90 ï75 ï60 ï EC 60 EC FOC ï90 ï75 ï60 NCAR 1960 NCAR 55 FOC 1980 Lifetime of the stations The vertical lines correspond to the lifetime of each station. 40 ï45 The database: 527 stations: 344 from Environment Canada (red), 40 Fisheries and Oceans (orange) and 143 from NCAR (green). Time resolution: hourly, 3hourly and synoptic, sometimes within the same station. Time span: Over 54x6 data pair values. 3

5 1 2 SPEED Unification of measurement units (m/s) WIND Unification of direction criteria (deg) Standardization of Observation times (UTC) Intra-site Repetitions Inter-site Repetitions DIRECTION Compilation Adjustments Duplication Errors 3 Consistency in Mean & Standard Deviation Consistency in Criteria: Calm & True North 4 Consistency in Limits: [0,112] m/s [0,360]º 1 m/s > 0.1 º Consistency in Values Abnormally Low Variations Blip Check + Spatial Check Abnormally High Variations 5 Detection of Long of Long Term Term Biases and Scale Scale Shifts Shifts in in Mean and/or Variance Controled Data-Base Temporal Consistency Detection Detection Homogenization of Long of Rotations Term Biases in Mean between of shifted and/or Wind wind Variance Roses roses Bias Correction Detection 4

6 5

7 Standardization of Observation Times (UTC) mislocated >1 timezone CETZ (UTC+1) UTC Spatial distribution of the time zones that the stations belong to. Most follow their geographical timezones. Three are located in Central European Time Zone (CETZ) despite belonging to Quebec. FOC & NCAR record in UTC. EC does it in Local Standard Time (LST). All stations are transformed to UTC. Example of a station that moves from being recorded in ATZ to ETZ. 6

8 7

9 Intra-site repetitions: Duplications of data periods within the same series Wind speed (m/s) Wind direction (deg) Wind speed (m/s) 1997/02/ /03/ /03/ /03/ /03/ /04/ /12/ /01/ /01/ /01/ /01/ /02/ Inter-site repetitions: Time (years) ~0.5 km ~17 km CWVY ~17 km 701Q009 Example of a station with ~1 month of duplicated data. The month of March (blue) is overwritten over January (red) for both wind speed & direction. Data transfer from one serie to another. Wind direction (deg) /07/ /08/ 1999/08/ /09/19 Time (years) Example of a station (in blue) that shares data with two others (red and orange) for both wind speed & direction. A total of ~7 years of duplicated data were found. 8

10 Both duplication cases are analyzed separately but in a similar manner. Once the repeated chains are identified, we distribute them according to their length. The absolute frequency of the chains diminishes with their length. A threshold is set when the distribution tends to zero. Intra-site repetitions. Case of wind speed. Inter-site repetitions. Case of wind direction. 9

11

12 It identifies whole stations with values that are clearly unrealistic. Consistency in Mean & Std. Dev. 40 Wind speed (m/s) Comparison between an erroneous station and good one: Station in red: mean of 16 m/s Station in bue: mean of 7.6 m/s Time (years) 8 wind speed distribution, database max. wind in tropical cyclonic events, EC webdatabase Upper limits of different anemometers 7 6 Consistency in limits. Wind speed: [0,112] m/s 0 m/s m/s m/s 44.7 m/s m/s m/s m/s Graybeal et al (112 m/s) Frequency 0 Wind speed distribution for the whole database (blue bars). Anemometer limits (vertical lines). Hurricane speed distribution (orange bars). Shaded area: unrealistic values. The limit was imposed based on Graybeal et al Wind speed (m/s)

13 12

14 Abnormally Low variations Wind speeds 1m/s Wind directions 0.1º Absolute frequency The procedure is similar to that of the duplication errors. The method is only able to identify erroneous periods longer than certain threshold. The threshold is specific for each time resolution and instrument precission Erroneous data Length of periods. Synoptic, 8 points of vane Example of a detected error with 2 consecutive constant periods for both wind speed (red) and direction (green). Wind speed (m/s) Distribution example for synoptic constant periods with 8 points of vane: Wind direction constant periods (blue). The periods of the shaded area are considered erroneous Constant Period 1 Constant Period /12/ /12/ /12/ /12/ /12/28 Time (years) wind speed wind direction Wind direction (degrees) 13

15 Abnormally Low variations Wind speeds < 1m/s (calm/low wind periods) Wind speed (m/s) The procedure is based on the comparison with neighboring stations. The method is able to identify erroneous periods of any length low wind period target series regional series 2000/11/ /11/ /11/ /12/01 Realistic calm Time (years) Partially erroneous Wind speed (m/s) erroneous data 5 Absolute freq. distribution of low wind constant periods regarded as realistic (blue). Distribution of non-analyzed periods, those with no neighboring regional stations, (red). All the periods in the shaded area are regarded as suspect /04/ // /04/ //01 Time (years) Example of an extremely long calm period of almost 2 years. 14

16 15

17 Abnormally High Low variations Variations (blip check + spatial check) The procedure evaluates the jump differences between 2 values in the context of the whole time serie (blip check). Is supplemented with an spatial evaluation of each data value (spatial check). Wind Speed (m/s) Spike Longer episode Dip Step Step 0 20 Observation Time Example of a detected erroneous long episode (in red). Different typologies in abnormally high variations: Spikes (detected with blip check) Dips (blip check) Steps (blip check) Longer episodes (blip check + spatial check) (figure based on Fiebrich et al. 20). Wind speed (m/s) /04/ /04/ /04/ /04/ /04/ /05/02 Time (years) erroneous data valid data 16

18 17

19 Detection of Long Term Scale Shifts in Mean and/or Variance The procedure works with deseasonalized and 1 month running averaged 3 variables: mean wind speed, standard deviation and coefficient of variation. For each of these variables a lower and upper interval are stablished. The outliers are thoroughtfully analyzed and if erroneous, the corresponding wind speed data is erased (in the original time resolution). Example of an analyzed station. ~1 year of data was erased. 18

20 Detection of Rotations between Wind Roses Assumption: annual wind roses remain constant throughout time or vary slightly year to year. Limitation: rotations only identified with annual resolution. Consecutive annual wind roses (or periods) are compared in search for rotations. The roses are veered relative to each other, and RMSE values are calculated between them for each angle spin. The minimum RMSE gives the angle between the roses. Only rotations corresponding to angles of at least 20º are considered erroneous. station 242, change of -20 degrees RMSE value Absolute Frequency on the wind rose _ _ Angle (degrees). RMSE min: -20 deg. 36 sectors Angle shift (deg), station Minimum:

21 Detection of Rotations between Wind Roses All the rotated periods are corrected (rotated) to match the most recent one. One possible, the topography is also checked in search of accordance. BEFORE (%) 15 5 AFTER Period Shift º º º º º ref. (%) 15 5 Example of a station with 5 rotations. The topographic channeling makes the various roses unlikely to happen. 20

22 Overview of the most important errors per station: EC FOC NCAR REMOVED MODULE intra_dup EC inter_dup FOC limit NCAR low_var1 REMOVED low_var2 high_var DIRECTION bias intra_dup inter_dup limit low_var1 bias Commonest error (by far): false calms. In buoys: false calms and bias. Two stations erased due to unrealistic mean. Stations with high var errors (NCAR, mostly) had few errors in total. Commonest errors: constant periods, then bias. In buoys: intra duplications (systematic failure) One station was erased due to inter duplication. 21

23 Impact on the Controlled : Removed Data EC FOC NCAR removed rm dire (%): < > 30 Removed values in module (%) 527 stations at the beginning: Erased module data: 506 st. (900,000 records or 1.7%) Erased direction data: 318 st. (180,000 records or 0,3%) In TOTAL: 1,000,000 records (1.8%) 4 stations removed Not included: Modified due to compilation adjusments: ~98% of the database Direction wind rose correction: 1,300,000 records (2.4%)

24 Impact on the Controlled : Changes in Mean and Skewness EC FOC NCAR >200 Wind Module Mean Difference (m/s) Wind direction bias ( º ) Wind Module Skewness. Before Wind Module Skewness. After 23

25 Thank you for your attention! For further questions we can meet after the talk or you can me at