COMPARISON OF CLEAR-SKY MODELS FOR EVALUATING SOLAR FORECASTING SKILL

Transcription

1 COMPARISON OF CLEAR-SKY MODELS FOR EVALUATING SOLAR FORECASTING SKILL Ricardo Marquez Mechanical Engineering Applied Mechanics School of Engineering University of California Merced Merced, California Carlos F. M. Coimbra Mechanical Aerospace Engineering Jacobs School of Engineering University of California San Diego La Jolla, California, ABSTRACT This work evaluates different solar forecasting models for short time horizons. The distribution of forecasting errors is generally highly dependent on the cloud conditions, consequently, a persistence model can be very effective, especially under uniformly cloudy or clear days. Otherwise, more sophisticate forecasting algorithms should be considered for operational purposes. Unless the same data sets are used, it is not trivial to compare different existing forecasting models unless the proper forecasting skill metric is used. The forecasting skill metric that we propose relies on either a defined persistence model or on a properly defined clear-sky model. Here we study the sensitivity of the forecast skill metric with the different persistence/clearsky reference models. 1 INTRODUCTION One of the leading impediments for achieving higher market penetration power grid connectivity of solar wind technologies is the variable nature of such resources. Recent studies on higher penetration impact of renewables have emphasized the need for accurate forecasts if large variable capacities are to be achieved [5, 12, 11]. Historically, most of the power grid variability has been on the load (dem) side because fossil nuclear generation are designed to operate in stable, dispatchable controllable fashion. In order to increase renewable power capacity penetration, Independent System Operators (ISOs) must now cope with generation variability. Therefore load forecasting errors have economic consequences on electricity markets, as well as other operational impacts [12]. Current efforts in developing forecast models include the development of artificial neural networks (ANN) for time series predictions, semi-empirical models based on satellite images predictions based on national or regional weather predictions (see, e.g., [7, 10]). Many of the existing forecast models have been shown to predict the 1-hour integrated solar irradiance 1 or 2 hours ahead to good accuracy [3, 9, 8] forecast models for same-day [7, 10], 1-5 days ahead have also been developed with good accuracy reliability. Effective forecasting methodologies relevant inputs considered can vary widely depending on the time horizons considered. However, quantification of the relative advantages of different forecasting methodologies is not straightforward because different authors use different evaluation criteria, also because the solar radiation data sets are dependent on geographic location climate. The second issue is particularly significant as it is relatively easier to forecast the solar irradiance during clear day periods, therefore an oversimplistic forecast model can yield very good conventional statistical metrics for those conditions when solar irradiation is highly predictable. The same oversimplistic forecast model would certainly fail under different geoclimatic conditions. The purpose of this work is to present an alternative approach to evaluate the quality of forecast models by defining comparing in a quantitative way the solar resource variability the forecast uncertainty. The observation that leads to the relationship between is that forecast model errors are typically higher during wet (cloudy) days than during dry (clear) days, as clearly demonstrated in a few recent studies [7, 10]. In this paper, we analyze forecast models quantify the relationship between `variability' `forecastability' in order to produce a consistent metric that

2 is independent of the time horizon under consideration. Since the forecast quality measures depend on using clearsky estimates of solar irradiation to compute a persistence model to calibrate the evaluation procedure to compute, we consider a few clear-sky persistence models. The clear-sky persistence models are described in Section 2 3, respectively. The goal is to evaluate the forecasting using a more universal metrics. Two forecasting models are developed in Section 4, in Section 5, we describe the forecasting model evaluation procedure. In Section 6, we evaluate the forecasting models the persistence models with respect to the different solar irradiance clear-sky estimates. 2 CLEAR-SKY MODELS 2.1 CS Model 1: Polynomial Fit The first clear-sky model is the one used in [6], where was modeled using a 3-rd degree polynomial function of cosine of the solar zenith angle, where the coefficients where determined as:,,, is in. This model was generated using selected clearsky days in the data set collected in the year 2010 at the University of California Merced solar observatory station. It was modeled on 1-minute values. In this study we only focus on 1-hour forecasting horizons so hourly averaged values of are used with no loss in accuracy. 2.2 CS Model 2: ESRA As a second clear-sky model, GHI is computed using the ESRA model adopted from [?, 1]. The algorithms used are developed by [?] were obtained from the accompanying c.d. to [1]. The ESRA model only depends on a site dependent Linke-Turbidity factor. Here we used since this value, qualitatively, fit the data for the already mentioned 2009 data set sufficiently well. A detailed analysis error analysis is not presented here. (1) Fig. 1: Comparison of ESRA polynomial-fit clearsky models. The coefficient of determination between the two models is the root-meansquared error is. 3 PERSISTENCE MODELS 3.1 Clear Sky Persistence Models The clear-sky persistence models are defined by having the clear-sky conditions persist for the next time-step, (2) From computed,, the prediction for the next value of GHI is This simple persistence model only relies on. Estimating comes for free since it only depends on time location, as a result so does the persistence model. Since the persistence model only relies on, their can be several persistence models. Here we consider two in the next section we will consider a clearness index persistent model. The two persistence models are (3) (4) The polynomial ESRA clear-sky models are compared in Fig. 1. Although their correlation is high there is a spread of. The difference among the two clear-sky models is intentional as we are interested in the resulting sensitivity to the subsequent evaluations of the developed forecasting models. where Eqn. (3) was used after substituting Eqn. (2). 3.2 Clearness Persistence Model (5)

3 We also apply a clearness persistence model which relies on extraterrestrial solar radiation, computed in as, where is the solar constant. This persistence model is defined similarly to the clear-sky persistence models with the exception that is used rather than. The resulting clearness persistence model is therefore, 4 FORECAST MODELS: NAR AND NARX The forecast model including only the hourly averaged GHI time-series as an input is referred to as the Nonlinear AutoRegressive (NAR) forecasting model, the model including additional inputs is referred to as the Nonlinear AutoRegressive with exogenous inputs (NARX) forecasting model. The NAR model for 1-hour ahead predictions can be mathematically expressed as (6) (7) stard deviations (SD) of clearness index values which are calculated from 30-second interval data. For example, the 30-minute MA SD is calculated as (10) (11) where. The 6-minute MA SD is similarly defined. Again, in Eq. 9 is also a feedforward ANN which contains more input neurons than the NAR model the number of time delays are set to 2 for each signal. The number of hidden neurons are set to 10, the early-stopping method is used for adjusting the weights. 5 FORECASTING EVALUATION METRIC We first quantitatively define the variability of solar irradiance, as motivated (8) (12) where is the number of time delays of the time series which are included as inputs to predict. The number of time delays is set to 2 (e.g.,, are used to predict ). The function is based on a feed-forward ANN structure where the number of hidden neurons is set to 10. The values of the network weights are determined by the ``early-stopping'' method for ANN training where the data is split into three sets -- a training set for computing directional derivatives of the errors in weight space, a testing set for determining when to stop training, a validation set which is not used at all during the ANN training [2, 1]. Data from October , 2010 is used for validation the rest of the data from January 1, October 14, 2010 is split romly into 80% for the training set 20% for the testing sets. The ANNs are implemented using the Matlab Neural Network Toolbox Version 7.0. This quantification of solar power variability was motivated by [4]. In the above definition for, is either the clear-sky or clearness index depending on whether we use the, or. We define the uncertainty as the stard deviation of a model forecast error divided by the estimated clear sky value of the solar irradiance over a time window from to : The NARX model is similar to the NAR model except that more time-series signals are utilized in the forecast scheme, where is the number of exogenous inputs. In this case, the 's are 30-minute 6-minute moving average (MA) Fig 2: Time series of solar irradiance. The figure illustrates the partition of the time series into window sizes of (hours). Each dashed vertical line represents the boundaries of the 500-hour time windows. (9)

4 (14) The uncertainty of solar availability is the forecasting error of a given forecasting algorithm. This definition is related to the commonly used RMSE [10, 9, 7, 8]. In some recent works on solar forecast models, a relative or normalized RMSE is used [7, 10, 8], so the present definition of solar resource uncertainty is closely related to relative model quality metric used in those studies. The variable also depends on a normalization by for which we will use, or in the evaluations. [Polynomial] [ESRA] [Extraterrestrial] Fig. 3: Scatter plot of using various clear-sky models including a polynomial-based, the ESRA-based, the clearness index model which uses Extraterrestrial irradiance for normalization.

5 The following metric directly evaluates the variability that is effectively reduced by the forecasting models by taking the difference between normalizing it with respect to. The metric for evaluating the quality of forecast models is more simply computed by considering the ratio between uncertainty,, variability,, such that: (15) (16) The forecast quality measure is defined above is such that when it means that the solar irradiance is perfectly forecastable, when the solar irradiance variability is not forecastable at all. If is negative, the forecast model performs worse than a pure chance forecast. Typical forecast models are characterized by values between 0 1, with higher values indicating better forecasts. Since are rom variables, it follows that is also a rom variable. To obtain a proper value of, we seek to compute an average value of. This is done by calculating for various time-windows. If a particular time window contains a large number of clear days then both (forecasting error) (variability) will be small. The time windows are selected by fixing (the window size) computing over each window of size in the time series. This calculation is made more clear in Fig. 2, which shows solar irradiance calculated values of for the data set used (the data set spans the dates between January 1 through October 31 of 2010 for Merced, CA). Low confidence experimental values that occurred in May July are removed from the calculations. In the case shown in Fig. 2, the window size is where each are of the dashed vertical bars are located. As mentioned above, night values are not included in Fig. 2, nor are they used in the calculations below. [Polynomial] [ESRA] 6 EVALUATIONS The forecast quality evaluations were performed for the data set collected from Jan Oct. 31 of Figure 3, shows scatter plots of versus computed for each time-window partitioning of the data set for. The difference between each of the evaluations is the normalization factor in the calculations of, where we used on separate occasions,,. These plots show that the persistence models (also distinctive in terms of the normalization variable) all result in, since for any window. Note that, for the [Extraterrestrial] Fig 4: Evaluation of versus (Timewindow sizes) after modifying algorithm with different clear-sky persistence models. extraterrestrial case, have a smaller numerical range than the cases for using. This is because. In all

6 TABLE 1: FORECASTING QUALITY METRICS FOR THE PERSISTENT, NAR, AND NARX MODELS. Training Validation Model RMSE (W/m ) RMSE (W/m ) NAR % % NARX % % NAR % % NARX % % NAR % % NARX % % cases, the general trends conclusions are the same. The NAR model forecast quality seems to be no better than persistence, while the NARX model does show some forecasting benefit since many of the scatter points fall below the 1:1 line. The results are further evaluated by computing an approximation of. As mentioned earlier, is a rom variable which depends on the ratio. The mean of this ratio is approximated by computing the slope of the scatter points as in Figure 3, except for this time we use computing the slope for each. These results are plotted in Figure 4. Again, there is very little difference in the resulting approximation of amongst which normalization variable is used in the evaluation procedures. Numerical values of obtained using are given in Table 1, along with the more common forecast quality metrics, the coefficient of determination the rootmean-squared error. Although these conventional metrics are suitable for comparing models within the same data set, they are useless for comparing forecasting models applied on different data sets are thus not generally informative. Nevertheless, we compare the forecasting metric with. Considering first, the forecasting quality would seem very accurate even for the persistence models. In turns out that each persistence model, yields a slightly different result, with the being the most accurate. The slightly worst model is probably because the rates of change of are slightly different than the actual. The also show a difference but it is not clear either how much those difference actually translate to differences in forecast quality. In terms of the metric, it is clear that each persistence has no forecasting quality. Comparing the values for the NAR NARX models the values are slightly different depending on the normalization factor. When the are used, the difference of the computation of leads us within, there there is some sensitivity on forecasting quality when using different normalization factors. 7 CONCLUSIONS Integration of variable renewable energy resources into the power grid requires a continuous reassessment of the forecasting skills provided by different methodologies. The variable energy community often expresses the need for a consensus on how to compare solar forecasting models. Here we performed a detailed evaluation of forecasting models using different solar irradiance clear-sky models as part of the normalization of the forecast evaluation procedure. The forecasting models were also compared with the persistence of the clear-sky index clearness index, where the clear-sky indices were based on. As discussed in [6], the evaluation method used here is also applicable to comparisons using methodologies applied to different data sets. ACKNOWLEDGEMENTS CFMC RM gratefully acknowledge the financial support given by the California Energy Commission (CEC) under the PIER RESCO Project PIR , by the National Science Foundation (NSF) CNS division grant N , also by the Eugene Cotta-Robles (ECR) Fellowship program of the University of California the Southern California Edison Fellowship program. Seed continued support by the Center for Information Technology Research in the Interest of Society (CITRIS) is gratefully appreciated.

7 REFERENCES (1) Badescu, V. Modeling Solar Radiation at the Earth Surface. Springer-Verlag, Berlin Heidelberg, 2008 (2) C.M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Greate Clarendon Street, Oxford, 1995 (3) Cao, J. Lin, X. Study of hourly daily solar irradiation forecast using diagonal recurrent wavelet neural networks. Energy Conversion Management, 49(6): , 2008 (4) Hoff, T. E. Perez, R. Quantifying pv power output variability. Solar Energy, 84(10): , 2010 (5) Lew, D. Piwko, R. Western Wind Solar Integration Study. Technical report, National Renewable Energy Laboratories, 2010 (6) Marquez, R. Coimbra, C. F. M. A novel metric for evaluation of solar forecasting models, under review (7) Marquez, R. Coimbra, C. F. M. Forecasting of global direct solar irradiance using stochastic learning methods, ground experiments the NWS database. Solar Energy, 85(5): , 2011 (8) Martin, L. Zarzalejo, L. F. Polo, J. Navarro, A. Marchante, R. Cony, M. Prediction of global solar irradiance based on time series analysis: application to solar thermal power plants energy production planning. Solar Energy, 84(10): , 2010 (9) Mellit, A. Artificial intelligence technique for modelling forecasting of solar radiation data: a review. Int. J. Artif. Intell. Soft Comput., 1:52--76, 2008 (10) Perez, R. Kivalov, S. Schlemmer, J. Hemker, K. Renne, D. Hoff, T. E. Validation of short medium term operational solar radiation forecasts in the US. Solar Energy, 84(5): , 2010 (11) Rodriguez, G. D. A Utility Perspective of the Role of Energy Storage in the Smart Grid. Power Energy Society General Meeting, 2010 IEEE, pages 1--2, 2010 (12) Sundar, V. Integration of Renewable Resources: Operational Requirements Generation Fleet Capability at 20 percent RPS. Technical report, California Independent System Operator (CAISO), 2010