The Pennsylvania State University. The Graduate School. Department of Meteorology ARTIFICIAL INTELLIGENCE TECHNIQUES FOR SHORT-RANGE SOLAR

Transcription

1 The Pennsylvania State University The Graduate School Department of Meteorology ARTIFICIAL INTELLIGENCE TECHNIQUES FOR SHORT-RANGE SOLAR IRRADIANCE PREDICTION A Dissertation in Meteorology by Tyler C. McCandless 2015 Tyler C. McCandless Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy December 2015

2 The dissertation of Tyler McCandless was reviewed and approved* by the following: Sue Ellen Haupt Adjunct Professor of Meteorology Dissertation Advisor Co-Chair of Committee George S. Young Professor of Meteorology and Geo-Environmental Engineering Co-Chair of Committee Jerry Y. Harrington Professor of Meteorology Jeffrey R. S. Brownson Associate Professor of Energy & Mineral Engineering Johannes Verlinde Professor of Meteorology Associate Head, Graduate Program in Meteorology *Signatures are on file in the Graduate School

3 iii ABSTRACT The world s energy system will increasingly depend upon renewable energy sources, including solar power, due to the limitation of fossil fuel resources and their influence on global pollution and climate change. Solar power can provide substantial power supply to the grid; however, it is also a highly variable energy source. Changes in weather conditions, i.e. clouds, can cause rapid changes in solar power output, thus creating a challenge for utility companies to effectively use these renewable energy resources. The energy grid, which manages and distributes the energy, requires energy generation to meet the energy demand for an efficient system. Independent systems operators (ISOs) and regional transmission organizations (RTOs) monitor the energy load, direct power generation from utilities, define operating limits and create contingency plans. ISOs, RTOs and utilities will require solar irradiance forecasts to effectively and efficiently balance the energy grid as the penetration of solar power increases. This study presents multiple nonlinear forecasting techniques to predict both the magnitude of the solar irradiance and its expected variability. The temporal irradiance variability is forecast for the temporal standard deviation of the Global Horizontal Irradiance (GHI) at eight sites in the Sacramento Valley of California and the spatial irradiance variability is forecast for the standard deviation across those same sites. A model tree with a nearest neighbor option was trained to predict the irradiance variability. The resulting artificial intelligence model reduces the mean absolute error between 10% and 55% compared to using climatological average values of the temporal and spatial GHI standard deviation. A data denial experiment shows including surface weather observations improves forecasting skill by approximately 10%. These results indicate the model tree technique can be applied in real time to produce solar variability forecasts.

4 iv Next, a cloud regime-dependent short-range solar irradiance forecasting system is developed to provide 15-min average clearness index forecasts for 15-min, 60-min, 120-min and 180-min lead-times. A k-means algorithm identifies the cloud regime based on surface weather observations and irradiance observations. Then, Artificial Neural Networks (ANNs) are trained to predict the clearness index. This regime-dependent system makes a more accurate deterministic forecast than a global ANN or clearness index persistence and produces more accurate predictions of expected irradiance variability than assuming climatological average variability. Lastly, regime-identification methods that also incorporate GOES-East satellite data both as inputs to the k-means regime algorithm and as predictors to the ANNs are explored. Several cloud-regime dependent short-range solar irradiance forecasting systems (RD-ANN) are tested to make 15-min average clearness index predictions for 15-min, 60-min, 120-min and 180-min forecast lead-times. The RD-ANN system that shows the lowest forecast error on independent test data classifies cloud regimes with a k-means algorithm based on a combination of surface weather observations, irradiance observations and GOES-East satellite data. The ANNs are then trained on each cloud regime to predict the clearness index. This RD-ANN system improves over the mean absolute error of the baseline clearness index persistence predictions by 1.0%, 21.0%, 26.4% and 27.4% at the 15-min, 60-min, 120-min and 180-min forecast lead-times. Additionally, a version of this method configured to predict the irradiance variability predicts irradiance variability more accurately than a smart persistence technique. Using statistical techniques allows for improved deterministic solar irradiance predictions as well as improved spatial and temporal solar irradiance variability predictions. The combination of deterministic predictions of irradiance and irradiance variability may offer utility

5 v companies and systems operators the necessary information to deliver services to clients on the evolving power grid.

6 vi TABLE OF CONTENTS LIST OF FIGURES... viii LIST OF TABLES... xi ACKNOWLEDGEMENTS... xii Chapter 1 Introduction: The Need for a Short- Range Solar Irradiance Forecasting System 1 References... 4 Chapter 2 A Model Tree Approach to Forecasting Solar Irradiance Variability Introduction... 5 DATA Irradiance Observation Network Meteorological Observation Network Training Data Forecasting Techniques Baseline Technique Climatology Model Tree Cubist Results Temporal Variability Prediction Results Spatial Variability Prediction Results Evaluation of Observed Surface Weather Data as Predictors Case Study Temporal Variability Prediction Case Study Spatial Variability Prediction Case Study Discussion and Conclusion References Chapter 3 A Regime- Dependent Artificial Neural Network Technique for Short- Range Solar Irradiance Forecasting Introduction Approach Data Irradiance Data Weather Data Additional Derived Variables... 43

7 3.4. Prediction Techniques Artificial Neural Network Clearness Index Persistence Cloud Regime Classification Results Average Results Regime- Dependent Results Conclusions and Future Work References Chapter 4 A Regime- Dependent Neural Network Approach to Short- Range Solar Irradiance Prediction using Surface Observations and Satellite Data Introduction Data Process Overview Prediction Methods Baseline: Clearness Index Persistence Artificial Neural Network Regime- Dependent Artificial Neural Network Regime- Dependent ANN Configuration RD- ANN- KtCC RD- ANN- GKtCC RD- ANN- GCT Results SMUD BNL Variability Prediction Discussion and Conclusions References Chapter 5 Conclusion: The Value of a Short- Range Irradiance Forecasting System Appendix K- Means Clustering vii

8 viii LIST OF FIGURES Figure 2-1. SMUD observation locations shown as green X's with METAR observation locations shown as red stars Figure 2-2. Prediction flowchart that includes GHI and METAR Observations as well as time and date information Figure 2-3. Flowchart and description of the model tree. For this arbitrary instance, the subtree in red is used to make the final prediction via the equation at the bottom Figure 2-4. Mean Absolute Error (MAE) for the model tree on the spatial standard deviation of the GHI (green - triangles) and temporal standard deviation of the GHI (red squares) Figure 2-5. Relative error for the model tree on the spatial standard deviation of the GHI (green - triangles) and temporal standard deviation of the GHI (red squares) Figure 2-6. Percent improvement with the model tree using the observed meteorological data as input compared to the cubist model without the observed meteorological data Figure 2-7. Case study for the prediction of the temporal GHI standard deviation at a lead time of 15-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the actual temporal GHI standard deviation values shown in blue and the predictions shown in red Figure 2-8. Case study for the prediction of the temporal GHI standard deviation at a lead time of 60-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the actual temporal GHI standard deviation values shown in blue and the predictions shown in red Figure 2-9. Case study for the prediction of the temporal GHI standard deviation at a lead time of 180-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed temporal GHI standard deviation values shown in blue and the predictions shown in red Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 15-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 60-min. The forecast valid time for the daylight hours of April 13, 2014 is

9 ix plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 180-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red Figure 3-1. Process design: first classify cloud regimes on the optimal set of potential inputs shown in the red rectangles outlines in the black box, then apply ANN models to predict the clearness index on each regime independently. An ANN is also applied on all data (i.e. without regime identification), and compared to the clearness index persistence prediction Figure 3-2. Map of the SMUD sites (blue triangles) and METAR/DICast predictor sites (red X s) Figure 3-3. Schematic of a feed-forward Artificial Neural Network used in this study Figure 3-4. Analysis of the regime classification (top subplots) for the Input Set 1 (left) and Input Set 5 (right). The analysis for Input Set 1 compares the regime classification for input Kt_Prev15 and input Kt_Prev30. The analysis for Input Set 5 compares the regime classification for input Kt_Prev15 and the Kt_Temporal STDEV (standard deviation previous hour). The bottom subplots are histograms of the Kt_Temporal STDEV for each regime with colors representing the different classification of regimes Figure 3-5. Sensitivity study of the Regime-Dependent ANNs averaged over all regimes for the 60-min forecast lead time. The regime-dependent ANN on Input Set 5 has the lowest MAE of all input sets tested Figure 3-6. Comparison of errors for clearness index persistence, ANN, and regimedependent ANN method at 15-min, 60-min, 120-min and 180-min forecast leadtimes. The clearness index persistence is best at 15-min, otherwise the regimedependent ANN method performs best Figure 3-7. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 15- min forecast lead-time Figure 3-8. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 60- min forecast lead-time Figure 3-9. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 120- min forecast lead-time

10 x Figure Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 180- min forecast lead-time Figure Comparison of the forecast MAE (blue columns) and the standard deviation of the Absolute Error (red columns) for the regime-dependent ANNs. The regimes with the largest errors correlate with the regimes with the largest standard deviation of the forecast error Figure Comparison of the forecast MAE and the standard deviation of the Absolute Error for the regime-dependent ANN predictions at all forecast lead times. The regimes with the largest errors correlate with the regimes with the largest standard deviation of the forecast error Figure 4-1. Locations of SMUD irradiance observations, shown in blue triangles, and the three nearest METAR surface weather observations, shown in red X's Figure 4-2. Locations of BNL irradiance observation site, shown as a blue triangle, and the three nearest METAR surface weather observations, shown in red X's Figure 4-3. Overall process design for our regime dependent prediction technique and the comparison techniques Figure 4-4. Sensitivity study results for the optimal number of training epochs of the ANN for the RDANN at SMUD sites for the 180-min lead-time Figure 4-5. Results for all methods on the satellite determined cloudy instances. The method that performs best in the majority of the forecast lead-times in the RD-ANN- GKtCC method Figure 4-6. Percent improvement over the clearness index persistence forecasts for all methods on the satellite determined cloudy instances Figure 4-7. Results for all methods on the satellite determined cloudy instances for the BNL forecast site. The method that performs best in the majority of the forecast lead-times is the clearness index persistence method

11 xi LIST OF TABLES Table 2-1. Sensitivity study for the number of nearest neighbors used in the model tree prediction. The values shown are the MAEs of the model predicting GHI temporal standard deviation (W/m 2 ). Using one nearest neighbor results in the lowest MAE for all lead times Table 3-1. List of all the observed and derived predictors for the ANN Table 3-2. Test of input sets for the k-means classification of cloud regimes Table 3-3. Comparison of the clearness index persistence MAE for each regime to the standard deviation of the absolute error for each regime. The MAEs for each regime are correlated with the variability of the errors, as expected Table 4-1. List of instances in each training, testing and validation datasets for both BNL and SMUD Table 4-2. List of predictors for the ANN model. The Kt Nearby Mean and Variability are marked with an asterisk because they are only available for the SMUD sites Table 4-3. List of inputs for the k-means algorithm in the RDANN-GKtCC configuration. The Kt Nearby Mean and Variability are marked with an asterisk because they are only available for the SMUD sites Table 4-4. Best number of regimes, K, and number of neurons in the hidden layer for all forecast lead-times at both SMUD and BNL as determined by the lowest error on the sensitivity test set Table 4-5. Comparison of MAE for the clearness index persistence and the ANN, CLEAR model for all forecast lead-times Table 4-6. List of the MAEs for predicting the clearness index variability with the clearness index persistence, ANN-ALL, and RD-ANN-GKtCC methods trained to predict the variability

12 xii ACKNOWLEDGEMENTS This material is based upon work supported by the U.S. Department of Energy under SunCast Award Number [DE-EE ] and the National Center for Atmospheric Research. I gratefully acknowledge all of the collaborators on the SunCast project for insightful discussions and ideas, including Seth Linden, Sheldon Drobot, Jared Lee, Julia Pearson and Tara Jensen. This project would not have been possible without the data from the Sacramento Municipal Utility District and Brookhaven National Labs; and the help from Thomas Brummet at NCAR for the data quality control and processing. Thanks go to Laura Hinkelman for GOES-East data acquisition and quality control; and for intellectual conversations that led to innovative applications of satellite data in this dissertation. I would like to thank Dr. Sue Ellen Haupt and Dr. George Young for their collaborative efforts in advising, guiding and mentoring me in this Ph.D. journey. I have learned many valuable lessons not only in the science but also many valuable life lessons under their mentorship. I would like to thank Jerry Harrington and Jeffrey Brownson for their insightful input and guidance as well. I would be remised not to thank my incredible friends and family who have shared many literal and figurative miles with me. I always felt supported by friends and family, even during the inevitable challenges that a Ph.D. program presents. My parents, Brenda and Ralph, have been, and continue to be, incredibly positive influences in my career and in my life. I d like to thank my girlfriend, Kristin McCormick, who was incredibly supportive during the dissertation writing process. Finally, I d like to thank everyone who shared some miles running with me

13 during this academic adventure as my athletic pursuits with good training partners, coaches and competitors helped keep me balanced and happy. xiii

14 1 Chapter 1 Introduction: The Need for a Short-Range Solar Irradiance Forecasting System The future of the world s energy system will increasingly depend upon renewable energy sources because fossil fuel resources are finite and have an influence on global pollution and climate change. On a regional level, independent systems operators (ISOs) and regional transmission organizations (RTOs) monitor the energy load, direct power generation from utilities, define operating limits and create contingency plans (Energy.gov). Renewable energy sources, including solar energy, can provide significant power supply to the grid; however, they are also highly variable sources of energy (Lew et al. 2012). Changes in weather conditions can cause rapid changes in power output, thus creating a challenge for utility companies to effectively use these renewable energy resources. With limited quantities of fossil fuels and increased penetration of renewable sources of energy into the power grid, ISOs, RTOs and utilities will require solar irradiance forecasts to effectively and efficiently balance the energy grid as the penetration of solar power increases in order to achieve a balanced and economical energy grid (Ela et al. 2013). Utility companies and system operators need accurate deterministic short-range solar irradiance forecasts, with variability estimates, so that they can adequately balance the rapid changes as well as supply and demand peaks in the electrical grid. The focus of this dissertation is the region encompassing the Sacramento Municipal Utility District (SMUD) in California, which is part of the Western Interconnection and Long Island Solar Farm in New York, which is part of the Eastern Interconnection.. In this dissertation I show that artificial intelligence forecasting techniques can be used to obtain more accurate short-range solar irradiance forecasts from 15-min

15 2 to 180-min lead times and artificial intelligence techniques can also be used to predict spatial and temporal solar irradiance variability over the same short-range forecast lead-times. The specific forecast lead-time; the type, quality and flow of observations; as well as the choice of forecasting method impact short-range solar irradiance forecasting accuracy. There are multiple methods for short-term solar irradiance forecasting, including ground-based sky image advection techniques, satellite-based cloud advection models and fast running NWP models; however, each of these have strengths and limitations depending on the data sources and forecast lead-time. In this dissertation, I utilize the strengths of multiple data sources by blending surface weather observations, irradiance observations and satellite data via statistical learning algorithms. I focus on the forecast lead-times of 15 minutes to three hours where both the deterministic irradiance prediction and the forecasted variability of the irradiance are essential for utility companies and systems operators. I also show that a model tree statistical technique can predict both the temporal and the spatial irradiance variability more accurately than assuming climatology. To achieve short-term temporal and spatial solar irradiance predictions, in Chapter 2 I show that a model tree with a nearest neighbor option technique can be used for the Sacramento Valley in California. Additionally, the value of surface weather observations is quantified with a data denial experiment that shows that the addition of surface weather observations reduces the model tree forecast error by approximately 10%. To achieve short-term solar irradiance predictions, in Chapter 3, I use surface weather observations and solar irradiance observations as inputs and predictors for a regime-dependent forecasting system. The clearness index, which is the surface GHI normalized by the top of atmosphere expected GHI, is predicted for 15-min, 60-min, 120-min and 180-min forecast leadtimes. I show that a regime-dependent system not only produces more accurate deterministic forecasts than a global ANN or a clearness index persistence, but also that the regime

16 3 identification produces more accurate predictions of expected irradiance variability than assuming climatological average variability. To test the benefit of including GOES-East satellite data, Chapter 4, includes surface weather observations, solar irradiance observations, and GOES-East data as inputs and predictors into a regime-dependent forecasting system. Several cloud-regime dependent short-range solar irradiance forecasting systems (RD-ANN) are tested to make 15-min average clearness index predictions for 15-min, 60-min, 120-min and 180-min forecast lead-times. The RD-ANN- GKtCC system that utilizes a combination of surface weather observations, irradiance observations and GOES-East satellite data shows the lowest forecast error on independent test data. This complex model is tested in both a data rich environment (Sacramento, California) and a data poor environment (Long Island, New York) and the results highlight the value of having a substantially larger dataset for a large number of predictors and complex non-linear model configuration. Another version of the RD-ANN-GKtCC model trained to predict the irradiance variability produces more accurate variability predictions than a smart persistence technique. This dissertation shows that using multiple inputs into statistical techniques improve deterministic solar irradiance predictions as well as improve spatial and temporal solar irradiance variability predictions. These predictions have been necessitated by the growth of solar energy penetration where ISOs and TSOs have to efficiently plan and maintain a balanced energy grid and this dissertation shows that statistical techniques can provide the necessary irradiance predictions that can be converted to power and used to efficiently manage the grid for the next several hours.

17 4 References Ela, E, Diakov, V., Ilbanex, E., and M. Heaney, 2013: Impacts of Variability and Uncertainty in Solar Photovoltaic Generation at Multiple Timescales. Technical Report: NREL/TP Lew, D. G. Brinkman, A. Florita, M. Heaney, B-M. Hodge, M. Hummon, and E. Ibanez, 2012: Sub-Hourly Impacts of High Solar Penetrations in the Western United States. 2 nd Annual International Workshop on Integration of Solar Power into Power Systems Conference, Lisbon, Portugal Nov United States Department of Energy. (n.d.). Office of Electricity Delivery & Energy Reliability. Retrieved from

18 5 Chapter 2 A Model Tree Approach to Forecasting Solar Irradiance Variability 2.1 Introduction A major concern of ISOs and TSOs is that the power generation is variable from renewable energy sources like wind and solar. Solar energy, in particular, varies over a broad range of space and time scales because of the complex dynamic evolution of clouds. Lew et al. (2012) provided evidence of the challenge of solar power integration with the results showing that the variability of power output was higher with high penetrations of solar than with high penetrations of wind. An important issue with generation of power from solar energy is the lack of storage for photovoltaic (PV) systems, meaning that the energy must be used as it is produced and cannot be effectively stored under current technology constraints. Concentrated Solar Power (CSP) systems, on the other hand, use mirrors or lenses to concentrate sunlight, often used to heat a substance (liquids, salts, etc.) in order to thermally store energy for some period of time, damping the power fluctuations. Rapid changes in weather conditions, especially cloud growth, advection, and dissipation, cause variability in solar power, particularly from PV systems. Because sparse meteorological data do not fully resolve the cloud field, deterministic forecasts do not capture all of this variability. Therefore, direct forecasts of solar temporal and spatial variability from the available weather data is valuable for electricity production and transmission decision-makers to manage the power grid efficiently. In an effort to maximize solar PV power as an energy resource, utility companies require back-up energy sources to balance the power supply with the expected demand. Any difference

19 6 between the supply and demand is made up with a company s operating reserves. The response speed (ramp rate and start time), response duration, frequency of use (continuously or only during rare events), direction of use (up or down), and type of control characterize these operating reserves (Ela et al. 2013). This balancing becomes more challenging when the energy resource is variable. Curtright and Apt (2008) have shown that the cost of energy can be strategically minimized with knowledge of the short and long term PV variations; thus, accurate solar power forecasts provide information to balance operating reserves. While direct forecasting of spatial and temporal variability of solar irradiance is relatively new, there have been numerous studies aimed at providing deterministic forecasts of the expected value of solar irradiance. Because the time and space dependent field of solar irradiance results from the complex evolution of clouds in the atmosphere, many of these studies have tested nonlinear methods for deterministic solar irradiance prediction (Mellit 2008, Martin et al. 2010, Pedro and Coimbra 2012, Bhardwaj et al. 2013, Bouzerdoum et al. 2013, Diagne et al. 2013, Fu and Cheng 2013, Marquez et al. 2013a, Inman et al. 2013, Fernandez et al. 2014, Chu et al. 2014, Almonacid et al. 2014, Quesada-Ruiz et al. 2015, among others). While successful, these deterministic forecasts cannot directly capture all of the spatial and temporal variability of solar irradiance, because the available weather data does not resolve all of the cloud scales. While offering the potential to resolve this problem, direct forecasts of the spatial and temporal variability of the irradiance has been limited. Lave et al. (2013) used a wavelet-based model to predict a power plant s output given a spatio-temporal correlation function and to estimate the variability ratio over the plant s spatial coverage. A more easily generalized approach based on correlation analysis was used by Hoff and Perez (2012) to predict the shortterm maximum power output variability based on satellite derived irradiances. This current study addresses this problem using the model-tree non-linear forecasting method (Witten and Frank 2005).

20 7 The temporal scale over which to compute variability depends on the needs of the electric grid management entities. United States utility companies are concerned with minute fluctuations that achieve power balance with stand-by generators or storage/load management for short range solar irradiance predictions for large cities and dense transmission networks (Perez et al. 2015). Specifically, the Sacramento Municipal Utility District (SMUD) is concerned with maximum and minimum values over a 5-minute interval (Bartholomy et al. 2014). We therefore use 15 minute windows in order to meet the needs of SMUD while still providing an adequate sample size for computation of our variability metric, standard deviation (see the Data section). The quantification of temporal solar irradiance variability caused by the dynamic evolution of clouds has been extensively studied. Mills et al showed a passing cloud at a point produces solar insolation variation exceeding 60% of peak insolation in a matter of seconds. Hinkelman (2007) found that not only are the irradiances themselves larger in the middle of the day but also the fractional change in irradiance from one time to another is larger. Hinkelman (2013) also determined that cloud optical depth and cloud height are the best predictors of irradiance variability at one minute time resolution. Kuszmaul et al analyzed 1-sec PV output data and showed that it is linearly proportional to the spatial average of irradiance. Reikard (2009) examined data at resolutions of 5, 15, 30, and 60 minutes and found that the data exhibits nonlinear variability, due to variations in weather and cloud cover. These studies have examined the variability of measured solar irradiance due to changes in cloud cover. In addition to temporal variability, several studies have examined the spatial variability of solar irradiance. Zagouras et al. (2014) used cluster analysis to determine coherent zones of Global Horizontal Irradiance (GHI) for utility scale territory in California and used step changes of the daily average clear-sky index at each location to characterize the fluctuation of GHI. Gueymard and Wilcox (2011) analyzed solar power s regional dependence and showed greater variability tends to occur in coastal areas, particularly the California coast, and mountainous areas

21 8 because of the micro-climate effects of topography. Rayl et al. (2013) and Kumpf et al. (2015) performed an irradiance co-spectrum analysis and concluded that solar power site aggregation could greatly reduce power variability on short time scales depending on the distance between sites. They found that revenue variance of a photovoltaic asset depends more on variance in power than price and power site correlations were more influential than price volatility ratios. The goal of our study is to use observed meteorological data together with a network of irradiance observing sites to predict both the temporal variability and the spatial variability, both measured in terms of a standard deviation from the mean value of the GHI, and to test those prediction methodologies within the Sacramento, California region. The focus is on short range predictions, which as Nguyen and Kleissl (2014) state, intra-hour solar forecasting for power production and ramp events has become an important need in the solar industry as the inevitable variability of solar power will have a greater impact on energy resource management as solar penetration increases. Short range predictions are defined here as the forecasts for up to three hours lead-time provided at 15-min intervals. An artificial intelligence technique called a model tree, or Cubist model, is used to predict the temporal and spatial variability, here defined as the standard deviation, of the GHI. While this same approach could also be used to make a deterministic prediction of the expected value of solar irradiance, we focus exclusively on variability in this study. Parallel work on the deterministic forecast problem is presented in McCandless et al. (2015). Section 2 describes the observational data and section 3 explains the forecasting techniques. The results for temporal and spatial standard deviation predictions are presented in section 4 while section 5 illustrates the improvements of the model tree output by inclusion of surface weather observations. The application of the model tree predictions to four specific days is described in Section 6 while the final section (7) provides discussions and conclusions.

22 9 DATA 2.2. Irradiance Observation Network The irradiance observation network used in this study is that of the Sacramento Municipal Utility District (SMUD) in Sacramento, California. We consider the data from the eight solar power forecast sites provided in the SMUD dataset, which are plotted in Figure 2-1 as green X s. The eight utility scale PV array sites that measure GHI are geographically diverse across the SMUD service region, which is approximately 900 square miles and includes nearly 100 MW of solar power PV capacity (Bartholomy et al. 2014).

23 10 Figure 2-1. SMUD observation locations shown as green X's with METAR observation locations shown as red stars. The GHI observations are available from January 25 th, 2014 through May 27 th, 2014 for a total of 122 days. The resolution of the raw data is one minute and the solar variability is computed as a standard deviation from the mean value predicted over each 15 minute interval. This interval length was selected after communication with several US utility companies and agrees with the shortest time range for which a forecast is useable to the utility. Defining the

24 11 solar variability as the standard deviation over a 15-min interval means that both the diurnal change in irradiance and the cloud cover resulting change in irradiance are captured in the irradiance variability calculated. This total variability in a 15-min period is the variable most important for utility companies to plan how to allocate the energy reserves and balances this need with the need to know the 5-min maximum and minimum irradiance estimates. The 15-min temporal GHI standard deviation is calculated independently for each SMUD site and only daylight hours are retained in the dataset and is defined as; STDEV temporal t =!!!"#!i!!"#t_mean!i!, (1) where the GHI t_mean is an average over the 15-min interval, t i is the 1-min resolution time-step from t-15 minutes to t, and n is the number of times (15). To compute the spatial variability across the eight SMUD sites, we first compute the 15-min average of the GHI for each site. Then, the standard deviation of the 15-min GHI averages across the eight sites is calculated as STDEV spatial t =!!!!"!i!!"#s_mean!i!, (2) where GHI s_mean is the average for all SMUD sites at the forecast lead time t, x is the notation for a SMUD site at forecast lead time t, and n is the total number of SMUD sites (8). Only instances when the observed 15-min average GHI is greater than 20 watts per square meter are included in the dataset because the goal is to predict only during the daylight hours. Hereafter, the GHI interval is designated by the ending time; thus, a GHI temporal standard deviation prediction at 180 minutes is actually the temporal standard deviation for the time period from 165 to 180 minutes.

25 Meteorological Observation Network It is important to augment the utility irradiance observation network with the nearest meteorological observations. The weather observation network used here is the Meteorological Aviation Report (METAR) network, which are hourly surface weather observation stations typically located at airports across the United States. The METAR observations are quality controlled and processed for ingestion into the National Center for Atmospheric Research (NCAR) Dynamic Integrated forecast (DICast) System (Mahoney et al. 2012). The closest METAR sites to the Sacramento area are the four locations plotted as red stars in Figure 2-1. Twelve weather variables are either recorded directly by the METAR stations or are derived as probability values by the NCAR processing system: cloud cover, dewpoint temperature, probability of fog, probability of precipitation in the last hour, probability of precipitation in the last three hours, quantitative precipitation in the last hour, quantitative precipitation in the last three hours, temperature, visibility, wind speed, north-south wind component, and east-west wind component. Therefore, the entire weather observation predictor dataset includes 12 observed weather variables for each hour at four stations for a total of 48 observations Training Data To create a dataset of SMUD GHI observations matched with the hourly METAR weather observations, each 15-min SMUD GHI interval is matched with the corresponding METAR observations at the top of the next hour. For example, the irradiance observation at 1/26/ :15, 15:30, 15:45, and 16:00 would all be matched with the meteorological data from 1/26/ :00 to form a combined dataset. In addition to the GHI temporal or spatial standard deviations in the last 15-min interval, the GHI temporal or spatial standard deviations

26 13 from the previous three 15-min intervals are also included in the predictor dataset. These four GHI standard deviations for the 15-min intervals are provided as predictors so that the forecasting technique can model the recent trend in variability. Using the prior four 15-min intervals is appropriate because SMUD reports data on an hourly basis, so this study matches what will be implemented operationally. Figure 2-2 shows a diagram of the predictors, which are the previous observations, observed weather data, and time information, that are fed into the model tree to predict the variability. The temporal standard deviation datasets include 40,127 instances combined for all eight SMUD sites. The spatial standard deviation dataset consists of 4057 instances aggregated at all eight sites. Instances where one or more location had missing data were omitted from the spatial standard deviation training and testing datasets.

27 14 Figure 2-2. Prediction flowchart that includes GHI and METAR Observations as well as time and date information. Ten fold cross-validation randomly partitions the data into ten subsets to be used for training and testing the model and provide an assessment of how the model tree will generalize to an independent set of data. The training of the model tree is performed on nine of ten subsets and

28 the remaining subset is used as validation. This process is repeated for all of the ten subsets and the errors are averaged over the ten repetitions to reduce variability in the results Forecasting Techniques Baseline Technique Climatology The rationale for forecasting the temporal and spatial GHI standard deviation is to quantify the expected solar variability for utility companies and system operator s situational awareness of the expected irradiance variability over a 15-min timeframe. Thus, we seek to improve upon the climatological mean values of the temporal and spatial GHI standard deviation at each forecast lead time. The spatial standard deviation mean value is calculated in a similar way by computing the training dataset s average for each 15-min forecast time interval out to 180 minutes, STDEV spatial_prediction t =!!!!!"#!i!!"#s_mean!i! D. (3) The n is equal to eight for the number of SMUD sites at forecast lead time t, the summation is over the number of training dataset instances (D). For example, to make a prediction for the 180-min spatial standard deviation, the climatological mean value of the 180- min spatial standard deviation in the training data is used as the prediction Model Tree Cubist The objective of a forecasting system is to model the actual relationships between the predictors and the predictand. In the case of weather forecasting, the relationship between the

29 16 predictors and the predictand is frequently non-linear. Thus, a non-linear artificial intelligence prediction technique is often used. The artificial intelligence technique used in this study is the model tree, or Cubist model, which is Quinlan s (1992) M5 model tree formatted as a set of rules (Kuhn et al. 2012). The model tree uses a separate-and-conquer algorithm to search for a rule that explains part of the training instances, separates these instances, and continues this process until no instances remain (Quinlan 1993). The algorithm reformulates the tree into a set of rules and places a multivariate linear model at each leaf in order to predict our continuous predictands of solar irradiance variability. See Quinlan (1987a, 1987b, and 1992) for a detailed explanation of this process. The process grows a tree that has multivariate linear regression models at its nodes and leaves. The final prediction is a weighted average of the multivariate linear regression equations at each node in the tree down to the final leaf (Kuhn et al. 2012). This weighted averaging is accomplished by a smoothing process that adjusts the predicted value from the leaf up to the root via, PV S =!!!"!"!!!!!!!!, (4) where n is the number of instances, i, in the node S, PV(Si) is the predicted value at node S and instance i Si, k is a smoothing factor set equal to 15, and M(S) is the model prediction at the leaf of the subtree. This smoothing is done to capture the skill in the predicted values at nodes along the tree down to the final leaf. Figure 2-3 displays an example description of the model tree with the red branch highlighting the subtree used in this example prediction. Thus, this model tree is a set of rules that are paths from the top to bottom of the tree with each node s multivariate linear model output used in the final prediction.

30 17 Figure 2-3. Flowchart and description of the model tree. For this arbitrary instance, the subtree in red is used to make the final prediction via the equation at the bottom. We use an additional model option that combines the model tree s prediction with a prediction given the training dataset s nearest neighbor to further reduce the model tree s error. The nearest neighbor option first finds the training cases that are more similar to the current instance. Then the model tree is used to make predictions for all of the nearest neighbor instances and the current instance. The value of the current instance prediction is adjusted based on the difference between the current instance prediction and the prediction for the nearest neighbor instances. A sensitivity study, shown in Table 2-1, indicates that optimal configuration of the model tree includes one nearest neighbor, which results in the lowest mean absolute error for all four forecast lead times tested.

31 18 Table 2-1. Sensitivity study for the number of nearest neighbors used in the model tree prediction. The values shown are the MAEs of the model predicting GHI temporal standard deviation (W/m 2 ). Using one nearest neighbor results in the lowest MAE for all lead times. Separate model trees are configured for the temporal standard deviation and the spatial standard deviation separately. In addition, model trees are built for each lead-time for a total of 24 model tree configurations. The configurations of the model trees in this study have 100 rules. A sensitivity study (not shown) had minor and inconsistent differences in model error when the rules varied between 50 and Results Temporal Variability Prediction Results To better understand the utility of our irradiance variability forecasts, we compute the Mean Absolute Error (MAE) for the prediction of the GHI temporal standard deviation and then compare this value to the error from assuming climatological averages. The MAE is computed as the average of the absolute differences between the forecast standard deviation of the GHI and the actual standard deviation of the GHI. Figure 2-4 plots both the MAE of the spatial standard deviation of the GHI and the temporal standard deviation of the GHI for all forecast lead times. The model tree MAE for the GHI temporal standard deviation prediction increases from 16 W/m 2 to 18 W/m 2 as the forecast lead time increases from 15-mins to 180-mins. The results are similar

32 for the spatial standard deviation prediction with errors ranging from approximately 15 W/m 2 at 15-min lead-time to 20 W/m 2 at 180-min lead-time. 19 Figure 2-4. Mean Absolute Error (MAE) for the model tree on the spatial standard deviation of the GHI (green - triangles) and temporal standard deviation of the GHI (red squares). In order to quantify the model tree forecast performance versus a baseline, the relative error is plotted in Figure 2-5. The relative error is the error for the model tree divided by the error from climatology. Climatology is computed as the training dataset s mean value of the GHI temporal standard deviation at that forecast lead time (t). Specifically, this is RelativeError(t) =!"#$!"#$%&'(!""#!!"#$%& (!)!"#$!"#$%&'(!""#!!"#$_!"#$% (!). (4)

33 20 The degree to which the relative error is less than 1.0 quantifies the forecasting skill improvement by the model tree compared to the climatological prediction. A value less than 1.0 indicates the model improves upon the baseline method of climatology. The relative error for the model tree begins at approximately 0.57 for the 15-min forecast lead time and increases slightly to a maximum value around 0.62 at 180-min forecast lead time. These results provide evidence that the model tree is approximately twice as accurate as using the climatological average value, thus providing utility companies with substantially more accurate forecasts of variability for resource management decision making. Figure 2-5. Relative error for the model tree on the spatial standard deviation of the GHI (green - triangles) and temporal standard deviation of the GHI (red squares).

34 Spatial Variability Prediction Results Next, we analyze the model tree s predictive ability for GHI spatial variability by examining the predictive skill of the standard deviation of the GHI among the SMUD observation sites. The MAE for the GHI spatial standard deviation prediction increases with forecast lead time as did that of the GHI temporal standard deviation prediction (Figure 2-4). However, the error range over the forecast lead times is greater than that for the temporal data. Values of the MAE range from approximately 15 W/m2 at 15 min to 21 W/m2 at 180 min. The relative error of the model tree compared to climatology (the mean GHI spatial standard deviation computed on the training dataset) is plotted in Figure 2-5. The relative error for the model tree begins at approximately 0.35 for the 15-min forecast lead time and levels off at about 0.50 for forecast lead times longer than 75-min. This relative error provides evidence that the model tree is able to provide utility companies with at least twice the accuracy as assuming climatological average variability. This is a meaningful result for utility companies that have regional coverage with a range of distributed rooftop solar and solar power farms because the model tree is able to provide a substantial increase in the accuracy of predicting short range solar radiation variability across a region Evaluation of Observed Surface Weather Data as Predictors It is important to quantify the value of the surface weather observations as input into the model tree for utility companies to understand the value surface weather observation sites add to forecasting at solar power arrays. Therefore, an analysis was performed to compare the model tree without observed surface weather as predictors to including the observed surface weather variables as predictors. This is similar methodology to the data denial experiments of Kelly et al.

35 22 (2007). The model tree trained without observed weather data is given six predictors: the last four 15-min GHI spatial or temporal standard deviation observations, the forecast hour, and the day of the year. The comparison between model tree results when trained with versus without observed weather as predictors is plotted in Figure 2-6. The percent improvement is the percentage of error reduction when the model tree includes observed surface weather observations as predictors. With the exception of the first two lead times for the spatial standard deviation prediction, all other lead times show an improvement in skill when the observed weather data are included as predictors. The improvement steadily increases with lead time for the temporal standard deviation prediction until the 60-min lead time where the improvement levels off at around an 8% increase. After the first two forecast lead times, the improvement in GHI spatial standard deviation prediction with the observed weather used as predictors varies between 3% and 15% with an average percent improvement of about 10%. The negative percent improvement for the first two lead times may be due to the model tree over-fitting the most recent GHI spatial standard deviation values at the short range forecast lead times. The day of the year and hour have a higher contribution to the regression equations at longer forecast lead-times while the 15-min and 30-min forecast lead-times rely primarily on the last observed spatial GHI, which leads to large errors when the variability is highest. These results indicate that the model tree gains substantial predictive skill when observed weather is included as predictors.

36 23 Figure 2-6. Percent improvement with the model tree using the observed meteorological data as input compared to the cubist model without the observed meteorological data Case Study We evaluate the model tree under weather conditions that are challenging to forecast. We examine the prediction on a day with morning clouds and afternoon sun for the temporal GHI standard deviation prediction (April 13th, 2014), and a mostly cloudy day across the SMUD irradiance observation sites for the spatial GHI standard deviation prediction (April 26th, 2014).

37 Temporal Variability Prediction Case Study There are several forecasting challenges and successes shown when we examine a day of morning clouds followed by mostly clear conditions for predicting the GHI temporal variability at forecast lead-times of 15-min, 60-min, and 180-min. The predictions (blue lines) and the observed values (red lines) are plotted for the 15-min, 60-min, and 180-min forecast lead-times in Figures 2-7, 2-8, and 2-9. The x-axis is the forecast valid time in Universal Time Coordinate (UTC), which correspond to the daylight hours for April 13th in Sacramento, California. At the 15-min lead-time (Figure 2-7), the predictions follow closely to the actual observation; however, the predictions have a time lag. As shown previously, the observed surface weather from the METAR sites on the model tree improved prediction by only approximately ten percent. Therefore, one expects that the forecast would have a time lag because the model depends more on the most recent changes in GHI temporal standard deviations more than the observed meteorological data. When the irradiance variability changes quickly, the model needs the most recent information to update the forecast. The plots of the predictions for 60-min and 180-min lead-time exhibit a loss of skill compared to the 15-min lead-time. This loss of skill is similarly due to the error from the time lag between the variability measurement to the forecast valid time. However, when the irradiance variability is not rapidly changing, the model tree predictions at all lead times are close to the clear day GHI values after the morning cloudiness dissipates.

38 Figure 2-7. Case study for the prediction of the temporal GHI standard deviation at a lead time of 15-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the actual temporal GHI standard deviation values shown in blue and the predictions shown in red. 25

39 Figure 2-8. Case study for the prediction of the temporal GHI standard deviation at a lead time of 60-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the actual temporal GHI standard deviation values shown in blue and the predictions shown in red. 26

40 27 Figure 2-9. Case study for the prediction of the temporal GHI standard deviation at a lead time of 180-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed temporal GHI standard deviation values shown in blue and the predictions shown in red Spatial Variability Prediction Case Study We examine the ability of the model tree technique to predict the spatial variability of the GHI on a mostly cloudy day in the Sacramento area (April 26th, 2014). Similar to the temporal variability prediction plots, Figures 2-10, 2-11, and 2-12 are the spatial variability prediction plots for the 15-min, 60-min, and 180-min lead-times respectively. The 15-min spatial GHI standard deviation forecasts experience a similar time lag to the 15-min temporal GHI standard deviation forecasts. This again highlights the greater relative importance of the recent spatial GHI standard

41 28 deviation observations compared to the surface weather observations as input to the model tree. The 60-min spatial GHI standard deviation forecasts show an average under-prediction of the spatial variability over the entire day. This is potentially due to having too few overcast days to have a reasonable sample for the model tree to accurately predict the overcast forecast lead times. The 180-min spatial GHI standard deviation forecasts are only able to forecast the overall trend of the GHI spatial variability in the last two hours of the forecast period. Once again, this is likely due to lacking a reasonable sample of similar training cases with cloudy conditions across the Sacramento area. Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 15-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red.

42 Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 60-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red. 29

43 30 Figure Case study for the prediction of the spatial GHI standard deviation at a lead time of 180-min. The forecast valid time for the daylight hours of April 13, 2014 is plotted on the x-axis with the observed spatial GHI standard deviation values shown in blue and the predictions shown in red Discussion and Conclusion A significant challenge with utilization of solar energy is its variable nature; therefore, the focus of this study is evaluating whether we can accurately predict the temporal and spatial solar irradiance variability for the Sacramento area. The variability was quantified via the 15-min temporal standard deviation of the GHI and the spatial standard deviation of the GHI across irradiance observation sites. The model tree artificial intelligence algorithm with a nearest neighbor option was trained on data both from the METAR weather observations and from

44 31 Sacramento area irradiance measurements. Short range predictions were made at 15-min intervals out to 180 minutes for both the temporal and spatial standard deviations of the GHI. The predictive ability of the model tree was assessed using the MAE of the model and the relative error compared to using the mean, over the training dataset, of the temporal or spatial standard deviation of GHI (i.e. variability climatology). This mean value is computed separately for each forecast lead time. The relative error showed that for both spatial and temporal variability, the model tree technique is able to produce forecasts with approximately half the error of the climatological variability forecast. The case studies of the spatial and temporal variability highlighted the importance of the most recent GHI variability observations, and the need for GHI variability observations at multiple times in order to capture trends. Because solar energy is inherently a highly variable renewable energy resource while stability of the energy distribution network is essential, the added value of accurate GHI variability prediction is significant for utility companies and system operators. As solar energy penetration continues to grow in many markets across the United States, these entities will require the estimation of near term solar resource variability. The results for the model tree technique indicate that could be a beneficial technique for utility companies to implement in real-time forecasting of short range solar irradiance variability. Future work will test other, probabilistic, approaches to forecasting of the GHI and its variability. A longer dataset and data from additional regions in the United States will provide a thorough evaluation of the GHI temporal and spatial variance prediction with the model tree. 2.8 References Almonacid, F., Pérez-Higueras, P.J., Fernández, E., and Hontoria, L, 2014: A methodology based on dynamic artificial neural network for short-term forecasting of the power output of a PV generator, Energy Conversion and Management, 85,

45 32 Bartholomy, O., Vargas, T., Simone, M., Hansen, C., Fitchett, S., and Pohl, A., 2014: Benchmarking Solar Power and Irradiance Forecasting Accuracy at Sacramento Municipal Utility District. IEEE Photovoltaic Specialist Conference, June 8-13, Bhardwaj, S., Sharma, V., Srivastava, S., Sastry, O.S., Bandyopadhyay, B., Chandel, S.S., and Gupta, J.R.P., 2013: Estimation of solar radiation using a combination of Hidden Markov Model and generalized Fuzzy model. Solar Energy, 93, Bouzerdoum, M., Mellit, A., and Pavan, A, 2013: A hybrid model (SARIMA-SVM) for shortterm power forecasting of a small-scale grid-connected photovoltaic plant. Solar Energy, 98, Chu, Y., Pedro, H., and Coimbra, C, 2013: Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning. Solar Energy, 98, Curtright, A. E., Apt, K., The character of power output from utility-scale photovoltaic systems. Progress in Photovoltaics: Research and Applications 16, Diagne, M., David, M., Lauret, P., Boland, J., and Schmutz, N., 2013 Review of solar irradiance forecasting methods and a proposition for small-scale insular grids, Renewable and Sustainable Energy Reviews, 27, November 2013, Pages Ela, E, Diakov, V., Ilbanex, E., and M. Heaney, 2013: Impacts of Variability and Uncertainty in Solar Photovoltaic Generation at Multiple Timescales. Technical Report: NREL/TP Fernandez, E., Almonacid, F., Sarmah, N., Rodrigo, P., Mallick, T.K., and Perez-Higueras, P, 2014: A model based on artificial neuronal network for the prediction of the maximum power of a low concentration photovoltaic module for building integration. Solar Energy, 100, Fu, C-L., and H-Y. Cheng, 2013: Predicting solar irradiance with all-sky image features via regression. Solar Energy. 97, Gueymard, C. A., and S. M. Wilcox, 2011: Assessment of spatial and temporal variability in the US solar resource from radiometric measurements and predictions from models using ground-based or satellite data. Solar Energy. 85, Hinkelman, L.M., Evans, K.F., Clothiaux, E.E., Ackerman, T.P., and P.W. Stackhouse JR, 2007: The effect of cumulus cloud field anisotrophy on domain-averaged solar fluxes and atmospheric heating rates. Journal of Atmospheric Sciences, Hinkelman, L. M. 2013: Relating Solar Resource Variability to Cloud Type. NREL Progress Report. Hoff, T. E., and Perez, R., 2012: Modeling PV fleet output variability. Solar Energy, 86,

46 33 Inman, R.H., Pedro, H.T.C., and C.F.M. Coimbra, 2013: Solar forecast methods for renewable energy integration. Progress in Energy and Combustion Science, 39, Kelly, G., Thepaut, J.-N., Buizza, R. and C. Cardinali, 2007: The value of observations. I: Data denial experiments for the Atlantic and the Pacific. Q.J.R. Meteorol. Soc., 133: Kuhn, M., Weston, S., Keefer, C., and N. Coulter, 2012: Cubist Models for Regression. Kumpf, K., Blumsack, S. A., Young G. S., and J.R.S. Brownson, 2015: Portfolio Analysis of Solar Photovoltaics: Quantifying the Contributions of Locational Marginal Pricing and Power on Revenue Variability. Solar Energy. Kuszmaul, S., A. Ellis, Ellis, A., Stein, J., and Johnson, L., 2010: Lanai High-Density Irradiance Senor Network for Characterizing Solar Resource Variability of MW-Scale PV System. 35 th IEEE PVSC, Honolulu HI. Lave, Matthew, Jan Kleissl, and Joshua S. Stein. "A wavelet-based variability model (WVM) for solar PV power plants." Sustainable Energy, IEEE Transactions on 4.2 (2013): Lave, Matthew, and Jan Kleissl. "Cloud speed impact on solar variability scaling Application to the wavelet variability model." Solar Energy 91 (2013): Lew, D. G. Brinkman, A. Florita, M. Heaney, B-M. Hodge, M. Hummon, and E. Ibanez, 2012: Sub-Hourly Impacts of High Solar Penetrations in the Western United States. 2 nd Annual International Workshop on Integration of Solar Power into Power Systems Conference, Lisbon, Portugal Nov Mahoney, W.P., K. Parks, G. Wiener, L. Yubao, W. L. Myers, S. Juanzhen, L. Delle Monache, T. Hopeson, D. Johnson, and S.E. Haupt, 2012: A Wind Power Forecasting System to Optimize Grid Integration, IEEE Transaction on Sustainable Energy, 3:4, , Oct Marquez, R., Gueorguiev, V. G., and C.F.M. Coimbra, 2011: Forecasting of Global Horizontal Irradiance using sky cover indices. Proceedings of the ASME th International Conference on Energy Sustainability. Washington DC, Aug Martin, L., Zarzalejo, L.F., Polo, J., Navarro, A., Marchante, R., and M. Cony, 2010: Prediction of global solar irradiance based on time series analysis: Application to solar thermal power plants energy production planning. Solar Energy, 84, McCandless, T. C., Haupt, S.E., and G.S. Young, 2015: A Regime-Dependent Artificial Neural Network Technique for Short-Range Solar Irradiance Forecasting. Submitted to: IEEE, Transactions on Sustainable Energy. Mellit, A., 2008: Artificial Intelligence Technique for Modeling and Forecasting of Solar Radiation Data: A Review. Int. Journal Artificial Intelligence and Soft Computing, 1:1,

47 34 Mills, A., Ahlstrom, M., Brower, M., Ellis, A., George, R., Hoff, T., Kroposki, B., Lenox, C., Miller, N., Stein, J., and Y. Wan, 2009: Understanding Variability and Uncertainty of Photovoltaics for Integration with the Electric Power System. The Electricity Journal, LBNL-2855E. Nguyen, D., and J. Kleissl, 2014: Stereographic methods for cloud base height determination using two sky imagers. Solar Energy, 107, Pedro, H.T.C., and C.F.M. Coimbra, 2012: Assessment of forecasting techniques for solar power production with no exogenous inputs, Solar Energy, 86, Perez, R., David, M., Hoff, T., Kivalov. S., Kleissl, J., Lauret, P., and M. Perez. Spatial and Temporal Variability of Solar Energy. To be published Foundations and Trends in Renewable Energy. Quesada-Ruiz, S., Linares-Rodríguez, A., Ruiz-Arias, J.A., Pozo-Vázquez, D., Tovar-Pescador, J., 2015: An advanced ANN-based method to estimate hourly solar radiation from multispectral MSG imagery. Solar Energy, 115, Quinlan, J. R., 1987a: Generating production rules from decision trees, Proceedings of the Tenth International Joint Conference of Artificial Intelligence, Milan, Italy, Quinlan, J. R., 1987b: Simplifying decision trees, International Journal of Man-Machine Studies, 27, Quinlan, J. R., 1992: Learning with Continuous Classes. World Scientific. Quinlan, J. R., 1996: Improved use of continuous attributes in C4.5, Journal of Artificial Intellgence Research, 4, Reikard, G., 2009: Predicting solar radiation at high resolutions: A comparison of time series forecasts. Solar Energy, 83, Rayl, J., G. S. Young, and J. R. S. Brownson, 2013: Irradiance co-spectrum analysis: Tools for decision support and technological planning. Solar Energy: 95, Zagouras, A., Inman, R., and C.F.M. Coimbra, 2014: On the determination of coherent solar microclimates for utility planning and operations. Solar Energy, 102,

48 35 Chapter 3 A Regime-Dependent Artificial Neural Network Technique for Short- Range Solar Irradiance Forecasting 3.1. Introduction The proliferation of photovoltaic (PV) power production has made accurate short-range solar irradiance forecasts a necessity for utility companies to ensure reliable integration of solar power into the energy grid. An accurate forecast for irradiance is necessary; however, a prediction of the variability of the irradiance is also helpful in maintaining reliable energy production with increased levels of solar power integration. The amount of solar irradiance reaching the PV panels depends on both the diurnal cycle and on the atmospheric state. While the diurnal cycle is easily forecast, the stochastic element of cloud formation makes that component of irradiance forecasting a challenge. This problem can be mitigated to some extent if one can forecast the cloud type. The identification of cloud types, i.e. cloud regimes, is a valuable tool in short-range solar irradiance forecasting because each cloud regime is associated with particular cloud properties such as cloud optical depth, cloud growth rate, and cloud dissipation rate; and therefore have various degrees of irradiance attenuation. The cloud type will also have an effect on the temporal and spatial irradiance variability and on the corresponding irradiance forecast uncertainty. The best method for solar irradiance prediction depends on forecast lead time, with statistical techniques and cloud advection techniques most effective for short-range irradiance forecasting. Short-range forecasting is defined here as solar irradiance predictions from 15 minutes out to 180 minutes. Predicting solar power through statistical techniques has gained the attention of researchers in recent years. Sharma et al found that a Support Vector Machine

49 36 approach to post-processing Numerical Weather Prediction (NWP) Models forecasts produced the lower Global Horizontal Irradiance (GHI) forecast error compared to linear regression postprocessing techniques. Hassanzadeh et al. (2010) and Yang et al. (2012) found that AutoRegressive Integrated Moving Average (ARIMA) models produced the lower solar irradiance and solar power errors compared to other time-series short-range prediction techniques while Morf et al. (2014) used a Markov process to predict sunshine and cloud cover. Mellit (2008) reported that Artificial Neural Networks (ANNs) have been used in modeling and predicting solar radiation more than any other non-linear technique. More recently, several studies determined that models based on ANNs improve solar irradiance or solar power forecast accuracy compared to various baseline techniques (Martin et al. 2010, Hall et al. 2011, Marquez and Coimbra 2011, Wang et al. 2012, Chu et al. 2013, Cornaro et al. 2013). Several studies have examined the performance of these statistical forecast models in various weather conditions. Pedro and Coimbra (2012) found the accuracy of an ANN optimized with a Genetic Algorithm had a strong seasonal dependence. Marquez et al. (2013) correlated total sky images, infrared data, and solar radiation observations at the surface to use as input to an ANN and found the variability of solar radiation to be strongly dependent on the amount of cloud cover. Each day was classified as sunny, partly sunny or cloudy and an ANN was used to forecast the daily profile of the power produced by a PV plant in Mellit et al. (2014). Fernandez et al. (2014) concluded that the ANN model has lower errors for days characterized by direct irradiance (clear days) and for days characterized by diffuse irradiance (cloudy days) than for days characterized by a mix of direct and diffuse irradiance (partly cloudy days). This current work seeks to improve two major facets of short-range solar irradiance forecasting via regime-dependent statistical forecasting: deterministic irradiance forecast accuracy and irradiance variability (uncertainty) estimates. We first classify cloud regimes with a k-means algorithm and then apply ANNs to each regime to produce a more accurate GHI forecast

50 37 with variability estimates. The k-means algorithm statistically classifies the cloud regime based on surface weather and irradiance observations. This approach parallels that of Greybush et al. (2008), who classified weather regimes with Principal Component Analysis (PCA) in order to apply regime-dependent optimal weights to ensemble temperature forecasts. After k-means clustering, ANNs are implemented for each weather regime independently with the intention of modeling each weather regime s inherent predictability, and thus, each regime s different causal relationships between predictors and predictand. Predictions are made for the clearness index (Kt), which is the ratio of the observed GHI at the surface to the Top Of Atmosphere (TOA) expected GHI. The prediction of Kt is important for utility companies because it quantifies the amount of attenuation from aerosols and clouds at a particular location (Marquez et al. 2013). We wish to make short-range predictions for multiple sites near Sacramento, California for 15-minute intervals out to 180 minutes. In operational forecasting, these short-range predictions are blended with forecasts from NWP models and a satellite based cloud advection technique in the National Center for Atmospheric Research SunCast System that predicts solar power out to 168-hours (Haupt 2015). Section 2 provides an overview of our approach. In section 3, we discuss the data and the additional predictors derived from the initial datasets, which are the Sacramento Municipal Utility District (SMUD) irradiance network and the METAR network. In section 4, the prediction techniques of the ANNs and the baseline clearness index persistence forecast are described. In section 5, we summarize the k-means algorithm for cloud regime classification and the selection of optimal inputs. We describe the prediction methods before the regime classification method because we use the ANN prediction method to inform our decision on the best selection of inputs for the k-means regime classification. In section 6, we present and discuss the prediction results. The final section, 7, summarizes and poses potential future work.

51 Approach The goal of this work is to develop a cloud regime-dependent short-range solar irradiance forecast system in order to not only improve the deterministic forecast accuracy, but also to provide a quantification of the expected solar irradiance variability and corresponding forecast uncertainty. This section outlines our classify-then-predict process; the details are described in the following sections. Our methodology begins by classifying the cloud regime with the k- means algorithm. We then train a separate ANN to make predictions for each individual regime as depicted in Figure 3-1. This novel work goes beyond Mellit et al. (2014) and Marquez and Coimbra (2011), and others in the sophistication and automation in identifying regimes with the k-means algorithm and in the regime-dependent configuration of the ANNs. The process begins by selecting the optimal set of inputs for cloud regime classification. The selected set of inputs is then used by the k-means algorithm to classify and partition the datasets into an optimal number of cloud regime subsets. Finally, ANNs are constructed on each of the cloud regime datasets independently. This classify-then-predict) process (with k-means then ANN) is repeated for each forecast lead-time.

52 39 Figure 3-1. Process design: first classify cloud regimes on the optimal set of potential inputs shown in the red rectangles outlines in the black box, then apply ANN models to predict the clearness index on each regime independently. An ANN is also applied on all data (i.e. without regime identification), and compared to the clearness index persistence prediction. A cloud regime-dependent Artificial Intelligence (AI) system requires dividing the cases into distinct regimes for which the fundamental relationship between predictors and predictand is expected to differ; therefore, careful sensitivity studies determined the optimal configurations of the AI models in order to match the complexity of the regimes. After all data are quality controlled and additional variables are derived, the datasets are randomly split 2/3 for training and 1/3 for testing. All of the results are shown are from the testing datasets; however, the sensitivity tests conducted to determine the optimal configurations of the system were performed on the training datasets. The ANN and k-means sensitivity studies similarly split the training dataset into 2/3 for training and 1/3 for testing and the optimal configuration was determined based on

53 40 this 1/3 independent test set. This approach avoids compromising the independence of the initial test dataset. We show results in this study for four forecast lead-times: 15 min, 60 min, 120 min, and 180 min. These predictions are for the 15-minute average clearness index ending at each lead-time. In a real-time forecasting environment, predictions are made in a three-step process. In the preprocessing step, the data are collected, quality controlled, and the additional predictors are computed. In the regime classification step, the trained k-means algorithm is applied to the current data. Then, the ANN trained for the currently classified cloud regime and forecast leadtime is used to predict the solar irradiance (clearness index). The ANNs use numerical weather prediction analysis data and irradiance observations as input to predict clearness index at multiple locations in the vicinity of Sacramento, California Data Irradiance Data The network of irradiance observing sites used in this study is that of the Sacramento Municipal Utility District (SMUD) in California. We use data from eight solar power forecast sites that measure irradiance, shown in Figure 3-2 as blue triangles. The GHI observations are available for a period of 367 days from January 25th, 2014 through January 26th, The temporal resolution of the raw data is one minute and averages are computed over 15-minute intervals ending at :00, :15, :30, and :45 for each hour. The 15-minute averaged GHI data is then converted to clearness index values. The clearness index is the ratio of the observed GHI at the surface to the Top Of Atmosphere (TOA) expected GHI, which is computed via a series of geometric calculations for a given location and time ( This averaging

54 41 interval was selected after communication with several utility companies and agrees with the shortest time range for which a forecast is currently useful for dispatch decision-making. The solar irradiance data from all eight solar power sites is aggregated and all instances with missing data or nighttime observations are excluded from the final dataset. There are a total of 71,184 instances in the final dataset. In order to evaluate the prediction techniques, this study follows the same procedure as the planned real-time operational implementation. The data are provided from the utility company every hour with the one minute raw data averaged over 15-min intervals ending at :00, :15, :30, and :45 for each hour. Therefore, our prediction techniques ingest four predictors from the irradiance data, each converted to Kt: the average Kt from minutes, from minutes, from minutes, and the past 15 minutes prior to forecast initialization time, which is the start of every hour. Hereafter, these predictors are named Kt_Prev60, Kt_Prev45, Kt_Prev30, and Kt_Prev15.

55 42 Figure 3-2. Map of the SMUD sites (blue triangles) and METAR/DICast predictor sites (red X s) Weather Data The meteorological dataset used here is from the Meteorological Aviation Report (METAR) network, which represent hourly surface weather observations from stations typically located at airports across the United States. The METAR observations are quality controlled by the National Center for Atmospheric Research (NCAR) for ingest to the Dynamic Integrated forecast (DICast) System (Mahoney et al. 2012). The closest METAR sites to the SMUD irradiance observations sites are the three locations plotted as red X s in Figure 3-2. We use six weather variables: cloud cover, dewpoint temperature, categorical precipitation in the last hour (1 = precipitation occurred, 0 = precipitation did not occur), precipitation amount, temperature, and wind speed.

56 Additional Derived Variables In training AI methods, it is often useful to employ derived variables that emphasize important physical processes. Here, we derive inputs specific to the k-means classification system, as well predictors specific to the ANN prediction system. In particular, we leverage our meteorological knowledge to provide the k-means algorithm with the inputs that are specific to the goal of identifying cloud regimes and to provide the ANNs with predictors that are specific to the goal of solar irradiance prediction, i.e. the predictors most important for forecasting the evolution of clouds. We first assess whether averaging the predictor values from all three METAR sites improves the forecast accuracy of the ANNs. A sensitivity study (not shown) revealed that providing the forecasting technique with data from each of the three sites produced higher overall skill of the prediction; thus, averaging the data was not used. This result highlights the ability of the ANN to capture atmospheric relationships among predictors at the different locations that provide insight into the spatio-temporal nature of the evolving atmospheric state. Dewpoint depression, defined as the difference between the temperature and the dewpoint temperature, quantifies the atmosphere s nearness to saturation at the surface. This derived predictor is averaged over the three METAR sites after a sensitivity study showed no improvement by including the dewpoint depression for each site. Preliminary testing indicated that the cloud cover predictors have the highest importance in the GHI prediction, unsurprisingly (McCandless et al. 2015). Therefore, two predictors are derived from the cloud cover at the three sites. First, the variability is quantified by taking the standard deviation of the cloud cover across the three sites. Second, the mean of the cloud cover averaged at those sites is squared to emphasize the importance of thick, unbroken cloudiness in the region.

57 44 Predictors were also derived from the irradiance observed at the SMUD sites and the corresponding clearness index values. For instance, we computed the standard deviation of the four 15-minute averages at the forecast site to quantify the irradiance variability rather than providing all four observations. Another predictor was derived by fitting a linear equation to these four data points and using its slope to capture the trend of the clearness index. Additionally, the most recent trend was quantified by computing the most recent change in clearness index by subtracting the Kt_Prev30 from the Kt_Prev15. The final derived predictors from the irradiance observations characterize the spatial distribution of the clearness index over the Sacramento Valley in the past 15 minutes. The mean and standard deviation of the nearby SMUD sites, i.e. the observations at the remaining seven sites not including the site being forecast for, were computed and used as predictors. The last step in deriving predictors for the ANN was to transform time variables. The time was converted from UTC to local time to move the 24 to 0 discontinuity into the nighttime hours. The Julian Day was converted to the sine of the Julian Day to normalize the value corresponding to the day of the year between -1 and 1. Thus, the ANN was provided 32 predictors, including the NWP analysis predictors provided for all three DICast sites (column three), as shown in Table 3-1. Some of these derived predictors for the ANN model were also tested as derived inputs for the k-means regime classification. The optimal selection of those inputs, including derived inputs, is described in detail in section 5. Table 3-1. List of all the observed and derived predictors for the ANN.

58 Prediction Techniques Artificial Neural Network We use the ANN as the non-linear Artificial Intelligence (AI) prediction technique for our forecasts. ANN s advantages include their ability to model non-linear processes without the assumption of the form of the relationship between input and output variables. In the review by Mellit (2008), AI models have been successfully developed to forecast solar radiation, clearness index, and insolation. Sfetos and Coonick (2000) found that AI approaches significantly outperform traditional linear models in uni- and multi-variate studies, with the ANN feed-forward approach showing the best results. The ANN used here (Figure 3-3) is a feed-forward neural network trained by a backpropagation algorithm (Reed and Marks 1998) also known as a multi-layer perceptron (Rosenblatt 1958). The ANN, which is an algorithm that functions analogous to the human nervous system, is constructed of interconnected signal processing units (i.e. neurons) that calculate output values based on inputs and a set of weights and biases (i.e. multiplicative and additive scaling factors) which are tuned during the training process. The feed-forward neural network permits only forward connections. Figure 3-3 is an example neural network where all the predictors are connected to each neuron and then each neuron is connected to the output layer

59 46 that computes the final prediction. In our configuration, there are more predictors and typically more neurons than shown in Figure 3-3; however, this diagram serves to show the ANN s information flow, from left (predictors) to right (output). The specific neural network module used in this study is the newff model in the Neurolab python library ( This ANN configuration has several tunable parameters and the optimal configuration was determined from multiple sensitivity studies on subsets of the training data (Witten and Frank 2005). The optimal ANN configuration was determined to have one hidden layer, a learning rate of 0.01, and a weight decay of 0.5. The ANNs are trained for 200 epochs in order to adjust the weights and biases that minimize the error between the ANN outputs and the predictands without over-fitting the data. However, the number of neurons in the hidden layer is allowed to vary with regime because the regimes differ in the number of cases and in different levels of complexity in relationships between the predictors and the predictand. Therefore, the ANNs are trained with 5, 10, 15, 20, 25, or 30 hidden layer neurons and the configuration with the lowest error on a subset of the training data held out for an independent verification was the configuration chosen for that regime. The ANN is also used on the dataset without regime separation to provide the basis from which to quantify the improvement in forecast skill resulting from regime identification.

60 47 Figure 3-3. Schematic of a feed-forward Artificial Neural Network used in this study Clearness Index Persistence We wish to compare all of our ANN-based prediction techniques to a baseline prediction method, for which we use clearness index persistence. The clearness index persistence (or smart persistence ) forecast uses the last available observation of Kt as the next forecast. This forecast is difficult to improve upon when the cloud cover stays constant or when skies are clear. Considering our forecast sites are in the Sacramento Valley of California where the majority of the time it is clear, this simple method is expected to perform relatively well and will be difficult to improve upon. When multiplied by the TOA GHI to convert back to GHI if operations require, it inherently corrects for changes in solar elevation with time.

61 Cloud Regime Classification To test our hypothesis that splitting the data into subsets based on cloud regimes can improve overall GHI forecast accuracy, we classify the cloud regime with the k-means algorithm before separate ANNs are trained and tested for each cloud regime subset. The k-means algorithm clusters data by separating samples into k groups by minimizing the within-cluster sum-of-squared departures from the cluster mean, hereafter referred to as sum-of-squares. The process begins by dividing a set of samples (N) into k clusters, each of which are described by the mean (centroid) of the cluster s instances. The k-means clustering algorithm selects the optimal centroid for each cluster by finding the centroid with the minimum within-cluster-sum of squares, i.e. to find the minimum of,!!!!!!!!! x! µμ!, (1) where the minimization is computed over each instance i in cluster k. All predictors are normalized before being clustered to avoid having the sum-of-squares dominated by the inputs with the largest magnitudes. We test five different input subsets to determine the best inputs for the k-means algorithm. The goal is to provide the k-means algorithm with inputs that can physically represent the current cloud cover characteristics. The first input set tested only included the past four 15- min clearness index observations. The second input set tested additionally included spatial clearness index information: the previous 15-min average and standard deviation across the other seven sites. The third input set included the previous inputs as well as both the most recent change in clearness index (Kt_Prev15-Kt_Prev30, which is named Kt15 Kt30) and the slope of Kt over the past hour. In the fourth input set, we used almost entirely derived variables. We still included the last observation (Kt = 15 min), but added to the derived variables from the previous input set with the variability (standard deviation) of the past four 15-min averages. In our final

62 (fifth) input set, we added the spatial cloud cover variability (standard deviation) and the squared mean cloud cover as inputs. These input sets are summarized in Table Table 3-2. Test of input sets for the k-means classification of cloud regimes. The goal is to find the best set of inputs and the value of k that balances the accuracy of assigning each data set to a cluster without over-fitting the number of clusters to the training data. We inspected plots (not shown) of the sum-of-squares by the k-means algorithm. From this analysis, the exact best value of k was unclear; however, these plots indicated that somewhere between three and seven was likely optimal. Therefore, the next step tested the predictive ability of the regime subsets to determine the optimal number of cloud regimes. The results of the sensitivity studies indicated that error decreased as the number of regimes increased from three to seven. Therefore, seven is selected as the optimal number of regimes (k) in the k-means regime classification. We analyzed the regime classification and the corresponding irradiance variability within each regime to examine the physical representation of the regimes by the k-means algorithm. Figure 3-4 shows four plots for this analysis. The top left subplot compares the regime classification relationship between two inputs, Kt_Prev15 and Kt_Prev30 for k-means on Input Set 1. This subplot shows distinct relationships in the phase space of two of the inputs, Kt_Prev15 and Kt_Prev30, with cases having similar values of these parameters being assigned to the same

63 50 regime. This plot also indicates that there is greater spread in the predictor values within each regime for the middle range of Kt values, i.e. the partly cloudy conditions, than when it is either mostly clear (black) or mostly cloudy (purple). The top right subplot compares the regime classification relationship in the phase space of Kt_Prev15 and Kt Variability (Standard Deviation Previous Hour) for k-means on Input Set 5 (the one selected). The cluster relationship is less obvious in this phase space; however, the less interpretable regime classification patterns for higher numbers of inputs increase can be largely attributed to the curse of dimensionality (Houle et al. 2010). This implies that as more predictors are added to the k-means algorithm, the mapping of clusters in a higher dimensional space results in a loss of interpretability when they are projected onto a two dimensional plane. The bottom left subplot is the 1-hr temporal variability (standard deviation) of the clearness index for each regime as classified by k-means on Input Set 1. The bottom right subplot is the 1-hr temporal variability (standard deviation) of the clearness index for each regime as classified by k-means on Input Set 5. The bottom two subplots highlight an important feature of the k-means regime classification: each regime has a different irradiance variability distribution. This assures us that the k-means algorithm classifies regimes with different underlying irradiance variability distributions because these differing distributions are expected for the various cloud types.

64 51 Figure 3-4. Analysis of the regime classification (top subplots) for the Input Set 1 (left) and Input Set 5 (right). The analysis for Input Set 1 compares the regime classification for input Kt_Prev15 and input Kt_Prev30. The analysis for Input Set 5 compares the regime classification for input Kt_Prev15 and the Kt_Temporal STDEV (standard deviation previous hour). The bottom subplots are histograms of the Kt_Temporal STDEV for each regime with colors representing the different classification of regimes. The optimal input subset among those studied for the k-means algorithm was determined to be Input Set 5 based on a sensitivity study evaluated by the MAE for the regime-dependent ANNs averaged over all regimes for the 60-min forecast lead-time (Figure 3-5).

65 52 Figure 3-5. Sensitivity study of the Regime-Dependent ANNs averaged over all regimes for the 60-min forecast lead time. The regime-dependent ANN on Input Set 5 has the lowest MAE of all input sets tested. The results indicate that Input Set 5 has lower error than the other input sets. This is a physically plausible result because Input Set 5 includes the most inputs derived to describe the current cloud pattern. Therefore, the regime identification via k-means is trained and tested with Input Set 5 and seven regimes Results Average Results To analyze the performance of the ANNs trained separately on each of the regimes, the Mean Absolute Error (MAE) for the (independent) testing datasets is computed. The MAE of the ANN prediction for each regime is compared to the MAE for forecasts given by clearness index

66 53 persistence as well as the MAE given by the ANN trained on all data without regime identification. The MAE is calculated as, MAE =!!!!!! (obs i pred i, (2) where n is the number of instances in the testing data. The overall results for the clearness index persistence, ANN, and the regime-dependent ANNs show the clearness index persistence method has the lowest error for the 15-minute forecast lead time, but the regime-dependent ANN method is best for 60-min, 120-min, and 180- min forecast lead times (Figure 3-6). These results highlight the benefit of the regime classification because the regime-dependent ANN method has lower forecast error than the ANN without regime classification at all forecast lead times. It is also important to note that the forecast accuracy improvement over the clearness index persistence increases as the forecast lead-time increases, which is expected since cloud growth and dissipation will lead to larger errors in the clearness index persistence method as lead time increases and also demonstrates the ability of the ANN to predict some of the cloud growth and dissipation.

67 54 Figure 3-6. Comparison of errors for clearness index persistence, ANN, and regime-dependent ANN method at 15-min, 60-min, 120-min and 180-min forecast lead-times. The clearness index persistence is best at 15-min, otherwise the regime-dependent ANN method performs best Regime-Dependent Results The percent improvement of the regime-dependent ANN MAE compared to that for the clearness index persistence forecast varies from regime to regime and lead-time to lead-time. The ANN trained without regime classification is also compared to clearness index persistence. For the 15-min lead-time (Figure 3-7), neither the ANN nor the regime-dependent ANN is more skillful than clearness index persistence except on the most variable cloud regime (6). In addition

68 to the MAE, we compute the standard deviation of the absolute error to quantify the variability of the forecast error for each regime. This metric is computed via, 55 σ AE =!!!!!!! (obs n pred n MAE!, (3) where N is the number of cases in the jth regime. The clearness index persistence MAE for each regime is shown in the second column of Table 3-3 and the clearness index persistence standard deviation of the absolute error is shown in the third column of Table 3-3. For regime 6, the MAE of the clearness index persistence is 0.12 and the standard deviation of the absolute error for the clearness index persistence forecast is This variability is nearly four times greater than that for Regime 4, which the standard deviation of the absolute error is 0.03 while the MAE is Table 3-3. Comparison of the clearness index persistence MAE for each regime to the standard deviation of the absolute error for each regime. The MAEs for each regime are correlated with the variability of the errors, as expected. Such forecast improvements during the most variable regime, i.e. partly cloudy conditions, can aid utility companies and ISOs in planning their units to dispatch. For the forecast lead-times of 60-min (Figure 3-8), the regime-dependent ANN reduces MAE compared to that of clearness index persistence in six of the seven regimes. Only for Regime 6 does the ANN trained

69 56 on all data out-perform the ANN trained on data for the specific regime. For the 120-min forecast lead-time (Figure 3-9), the regime-dependent ANN has highest percent improvement in MAE over clearness index persistence in five of the seven regimes. At the 180-min (Figure 3-10) forecast lead-time, the regime-dependent ANN is always best. When averaged over all seven regimes, the regime-dependent ANN method reduces the MAE from that of clearness index persistence forecast by 5.9%, 21.1%, and 29.3% for the 60-min, 120-min, and 180-min forecast lead-times. Figure 3-7. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 15-min forecast leadtime.

70 Figure 3-8. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 60-min forecast leadtime. 57

71 Figure 3-9. Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 120-min forecast leadtime. 58

72 59 Figure Percent improvement of the MAE for the ANN and the regime-dependent ANN compared to the clearness index persistence for all seven regimes at the 180-min forecast leadtime. Regimes that are more difficult to predict (i.e. those with variable cloudiness) are expected to have larger forecast uncertainty, which we quantify with the standard deviation of the absolute error. To assess this, the MAE and the standard deviation of the absolute error for the regime-dependent ANNs at the 180-min lead-time are shown in Figure The plot for the seven regimes demonstrates that different regimes have different average forecast errors and different forecast error variability. The comparison between MAE and standard deviation of the absolute error for all forecast lead times is plotted in Figure These results show a direct relationship between the magnitude of the forecast error and the variability of the forecast error for all forecast lead times. Specifically, the regimes that are more difficult to predict, as identified by the larger MAEs, also exhibit larger error variability. Therefore, by identifying

73 60 regimes before applying the ANNs, not only do we increase forecast skill for lead-times of 60- min or more, but also provide a refined estimate of the expected forecast variability. Figure Comparison of the forecast MAE (blue columns) and the standard deviation of the Absolute Error (red columns) for the regime-dependent ANNs. The regimes with the largest errors correlate with the regimes with the largest standard deviation of the forecast error.

74 61 Figure Comparison of the forecast MAE and the standard deviation of the Absolute Error for the regime-dependent ANN predictions at all forecast lead times. The regimes with the largest errors correlate with the regimes with the largest standard deviation of the forecast error Conclusions and Future Work We have tested a regime-dependent solar irradiance short-range forecasting system. The system uses k-means clustering to classify cloud regimes between which the relationship between inputs and solar irradiance is expected to vary. An ANN is then developed for each regime. The results for regime-dependent 15-min average clearness index forecasts, the shortest time frame, show that clearness index persistence forecast nonetheless is more skillful than the new system, with the exception of the regime that has the most variability. For longer lead times (60-min, 120-min, and 180-min), however, the regime-dependent ANNs yield substantial improvement

75 62 over clearness index persistence. The regime-dependent ANNs also have, on average, lower forecast errors than an ANN trained without regime identification. In addition to the improvement in forecast accuracy at lead-times of 60-min, 120-min, and 180-min, the regime classification provides value in identifying regimes with greater forecast error variability and thus higher forecast uncertainty. Knowing the forecast error uncertainty aids utility companies in effectively and efficiently managing the power grid. 3.8 References Chu, Y., H. Pedro, and C.F.M. Coimbra, 2013: Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning. Solar Energy, 98, Cornaro, C., F. Bucci, M. Pierro, F. Del Frate, S. Peronaci, and A. Taravat, 2013: Solar Radiation Forecast Using Neural Networks for the Prediction of Grid Connected PV Plants Energy Production (DSP Project). Proceedings of 28th European Photovoltaic Solar Energy Conference and Exhibition, Sept 30 - Oct 4, Fernandez, E., F. Almonacid, N. Sarmah, P. Rodrigo, T.K. Mallick, and P Perez Higueras, 2014: A model based on artificial neuronal network for the prediction of the maximum power of a low concentration photovoltaic module for building integration. Solar Energy, 100, Greybush, S.J., S.E. Haupt, and G.S. Young, 2008: The Regime Dependence of Optimally Weighted Ensemble Model Consensus Forecasts of Surface Temperature. Wea. Forecasting, 23, Hall, T. J., C. N. Mutchler, G.J. Bloy, R.N. Thessin, S.K. Gaffney, and J.J. Lareau, 2011: Performance of observation-based prediction algorithms for very short-range, probabilistic clear-sky condition forecasting. J. Appl. Meteor. Climatol., 50, Haupt, S. E., 2015: The SunCast Solar Power Forecasting System. 13th Conference on Artificial Intelligence, Phoenix, AZ, Amer. Meteor. Soc. J6.2. Hassanzadeh, M., M. Etezadi-Amoli, and M.S. Fadali, 2010: Practical approach for sub-hourly and hourly prediction of PV power output. North American Power Symposium (NAPS), 1-5, Sept Houle, M. E., Kriegel, H. P., Kröger, P., Schubert, E., and A. Zimek, 2010: Can Shared- Neighbor Distances Defeat the Curse of Dimensionality? Scientific and Statistical Database Management. Lecture Notes in Computer Science, 6187, 482.

76 63 Mahoney, W.P., K. Parks, G. Wiener, L. Yubao, W. L. Myers, S. Juanzhen, L. Delle Monache, T. Hopson, D. Johnson, and S. E. Haupt, 2012: A Wind Power Forecasting System to Optimize Grid Integration, IEEE Transaction on Sustainable Energy, 3:4, , Oct McCandless, T.C., Haupt, S. E., and G. S. Young, 2015: A Model Tree Approach to Forecasting Solar Irradiance Variability. Journal of Solar Energy, In Process. Marquez, R., and C.F.M. Coimbra, 2011: Forecasting of Global and Direct Solar Irradiance Using Stochastic Learning Methods, Ground Experiments and the NWS Database, Solar Energy, 85:5, Marquez, R., V. Gueorguiev, and C.F.M. Coimbra, 2013: Forecasting of Global Horizontal Irradiance Using Sky Cover Indices, ASME Journal of Solar Energy Engineering, 135, Martin, L., Zarzalejo, L., Polo, J., Navarro, A., Marchante, R., and Cony, M, 2010: Prediction of global solar irradiance based on time series analysis: Application to solar thermal power plants energy production planning. Solar Energy, 84, Mellit, A., 2008: Artificial Intelligence Technique for Modeling and Forecasting of Solar Radiation Data: A Review. Int. Journal Artificial Intelligence and Soft Computing, 1:1, Mellit, A., Massi Pavan, A., and V. Lughi, 2014: Short-Term Forecasting of Power Production in a Large-Scale Photovoltaic Plant, Solar Energy, 105, Morf, H., 2014: Sunshine and Cloud Cover Prediction Based on Markov Processes, Solar Energy, 110, National Oceanic and Atmospheric Administration. (n.d.). Earth Systems Research Laboratory. Retrieved from Pedro, H. T., and C.F.M. Coimbra, 2012: Assessment of forecasting techniques for solar power production with no exogenous inputs. Solar Energy, 86:7, Reed, D. R., and R. J. Marks, 1998: Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks. MIT Press, Cambridge, MA, USA. Rosenblatt, F., 1958: The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. In. Psychological Review, 65:6, Sfetsos, A. and A. H. Coonick, 2000: Univariate and Multivariate Forecasting of Hourly Solar Radiation with Artificial Intelligence Techniques. Solar Energy, 68:2, Sharma, N., P. Sharma, D. Irwin, and P. Shenoy, 2011: Predicting Solar Generation from Weather Forecasts Using Machine Learning: Proceedings of the 2nd IEEE International Conference on Smart Grid Communications, Brussels, October, pp

77 64 Wang, F., Z. Mi, S. Su, and H. Zhao, 2012: Short-Term Solar Irradiance Forecasting Model Based on Artificial Neural Network using Statistical Feature Parameters, Energies, 5, Witten, I. H., and E. Frank, 2005: Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. Yang, D., P. Jirutitijaroen, and W. M. Walsh, 2012: Hourly Solar Irradiance Time Series Forecasting Using Cloud Cover Index. Solar Energy, 86,

78 Chapter 4 A Regime-Dependent Neural Network Approach to Short-Range Solar Irradiance Prediction using Surface Observations and Satellite Data 4.1. Introduction Utility companies and independent system operators (ISOs) require accurate short range forecasting of variable renewable energy sources, such as solar energy, in order to maintain power grid load balance (IRENA and CEM 2014). Cloud cover is the most important variable in forecasting solar energy power generation because clouds cause near instantaneous changes in photovoltaic power generation as they move over the solar power plant. Forecasts of the change in cloud cover, and thus the amount of solar irradiance reaching the surface of the earth, provide necessary information for utility companies and system operators to maximize solar energy penetration while finding an optimal balance between energy generation and energy consumption. Therefore, the deterministic forecast of the solar irradiance reaching the ground is essential as well as the expected variability of solar irradiance so that the optimal balance can be found. Our goal in this study is to leverage a statistical classification of cloud regimes in order to better tune artificial intelligence prediction algorithms so as to improve the skill of deterministic global horizontal irradiance (GHI) predictions. The forecast lead-time substantially impacts the optimal predictors and forecast methodology for irradiance prediction. Day-ahead and longer forecasts are necessary in planning conventional and variable power generation and for these lead-times, Numerical Weather Prediction (NWP) forecasts are generally used (Lorenz et al. 2012, Kleissl 2013). Intraday irradiance forecasts are used by utility companies and ISOs for load following and planning for

79 66 dispatch. At these lead-times, a combination of methods: empirical models, satellite-based techniques, statistical methods, and NWP models works best (Bouzerdoum et al. 2013, Voyant et al. 2013, 2014), with the combination producing the lowest forecast error depending on the specific lead-time and available predictors. At the shortest time scales of less than 15 minutes, sky image data can be used as input to cloud-based advection techniques (Chow et al. 2011, Marquez and Coimbra 2013a, Huang et al. 2013, Quesada-Ruiz et al. 2014, Chu et al. 2015); however the number of sky imagers deployed is generally limited. We focus on the forecast leadtimes of 15 minutes to three hours, which is sufficiently short range for statistical methods to outperform NWP but beyond the range where persistence or sky imager forecasts are difficult to beat. At forecast lead-times of 15 minutes to three hours, historically satellite-based cloud advection techniques have been used. These techniques use Cloud-Motion Vectors (CMVs) that are computed from consecutive satellite images and then used to advect the satellite observed clouds into the future. The use of CMVs for solar irradiance and solar power prediction was proposed by Beyer et al. (1996) with Hammer et al. (1999) and Lorenz et al. (2004) developing more advanced advection schemes. A forecasting method that uses a phase correlation between consecutive Meteosat-9 images has been used to predict 30-min cloud index values out to four hours lead-time and on average showed 21% improvement in Root Mean Square Error (RMSE) compared to cloud index persistence (Cros et al. 2014). Bilionis et al. (2014) extend the cloud advection technique to a probabilistic prediction by using Principal Component Analysis (PCA) prior to applying a Gaussian process model. To address the errors due to assuming steady clouds during advection, Miller et al. (2014) group cloud pixels into cohesive cloud structures and then employs an appropriate steering flow that uses cloud group properties to forecast their downstream development and sheering characteristics. Their intermediate position in the lead-

80 67 time spectrum makes satellite-based techniques prime candidates for blending with other forecast techniques. Statistical methods are well suited to combining multiple predictors in such blended forecast systems. Statistical models of appropriate complexity for the GHI forecast problem maximize the predictive value from the available predictors (e.g. satellite and ground observations). Any regression method can be applied to GHI forecasting; however, the Artificial Neural Network (ANN) is one of the most powerful, general, and therefore most widely used (Mellit 2008, Martin et al. 2010, Pedro and Coimbra 2012, Notton et al. 2012, Bhardwaj et al. 2013, Bouzerdoum et al. 2013, Diagne et al. 2013, Fu and Cheng 2013, Marquez et al. 2013b, Inman et al. 2013, Chu et al. 2014, Fernandez et al. 2014, Almonacid et al. 2014, Quesada-Ruiz et al. 2015, among others). The relevant predictors for estimating Direct Normal Irradiance (DNI) with a Bayesian ANN method were found to be the clearness index and the relative air mass in Lopez et al. (2005). Pedro and Coimbra (2012) found that an ANN time series model outperformed persistence, AutoRegressive Integrated Moving Average (ARIMA), and k-nearest Neighbors (knn) models for 1-2hr solar power predictions. Marquez et al. (2013b) used processed satellite images as input into ANNs to predict GHI from 30 minutes to 120 minutes and found between 5% and 25% reduction in RMSE compared to that of persistence. A challenge with ANNs, however, is the large number of tunable parameters, which is O(Number of predictors multiplied by number of neurons). This requires a large quantity of training data to prevent overfitting and the consequent loss of skill on independent data (i.e. operational use). Another concern with using ANNs in operational forecasting is the lack of physical interpretability that could directly provide the user with forecast variability information. We partition the data into subsets based on cloud regimes in order to forecast variability and to more accurately tune the ANN model for the peculiarities and consequent forecast challenges of each specific cloud regime. This solar irradiance variability was shown to differ

81 68 among satellite data derived cloud types in Hinkelman (2014). Regime-based prediction has been used in several different solar irradiance and solar power applications. Tapakis and Charalambides (2013) provide a review of various methodologies for both supervised and unsupervised cloud classification. The unsupervised techniques classify based on the pixels of an image. The supervised techniques, which are divided into simple, statistical and artificial subgroups, classify based on available training datasets and arithmetic complexity of the technique. A one-step stochastic prediction process of cloud cover or clearness index with transition matrices dependent on the relative sunshine amount is presented in McCandless et al. (2014) and Morf (2014). Zagouras et al. (2014) used a k-means algorithm with a stable initialization method to identify regimes based on step-changes of the average daily clear sky index in the San Diego, California region. A simple approach based on the daily total solar irradiance identified clear, partly cloudy, and cloudy regimes with separate ANN models developed on each regime in Mellit et al. (2014) and showed that particularly for the cloudy days, the ANN model trained on only those days improved on the ANN model trained on all days. The separation into cloud regimes allows an AI model to identify repeatable patterns in surface solar irradiance; however, there is a lack of research into 1) what are the most important inputs for cloud regime classification and 2) what are the most important predictors for an AI method to most efficiently and accurately predict short range solar irradiance. Rather than burden the ANN with the task of both identifying cloud regimes and responding to them correctly, a separate statistical model can be used to identify regimes before fitting the ANN. This approach allows the ANN to focus on the forecast mission for a specific cloud type. This simplification of each ANN s mission allows it to be implemented with a simpler configuration (fewer neurons and tunable parameters). Thus, better tuning can be achieved for a given amount of training data. However, the accurate classification of cloud regime is necessary for the ANN to focus on each cloud regime s peculiarities. To do so, we

82 69 utilize a combination of inputs that are specific to the goal of identifying cloud regimes in a k- means regime classification method. Because training data are always limited, this new approach offers the potential for improving the skill of ANNs in solar irradiance prediction. Section 2 describes the datasets and the derived predictors. Section 3 provides an overview of the process and section 4 explains the clearness index persistence baseline prediction method and the artificial intelligence prediction techniques. We illustrate the various regimedependent ANNs used in this study in Section 5 with Appendix describing the k-means regime classification. Section 6 presents the results and Section 7 provides discussion and conclusions Data We wish to determine the optimal set of inputs for the k-means algorithm and predictors for the artificial neural network in order to create the best configuration for the regime dependent artificial neural network (RD-ANN) forecasting system. To do so, we use data from three types of sources; irradiance observation systems, surface weather observation networks, and satellite observations. We use two irradiance observation systems located in different regions of the United States in order to test the prediction system in different climatologies with different training data sizes. We use approximately one year of data from the Sacramento Utility District (SMUD) located in the Sacramento Valley of California. We use data from eight solar power forecast sites that measure irradiance, shown in Figure 4-1 as blue triangles. The GHI observations are available for a period of 367 days from January 25, 2014 through January 26, The temporal resolution of the raw data is one minute and averages are computed over 15-min intervals ending at :00, :15, :30, and :45. The 15-min averaged GHI data are then converted to clearness index values. The clearness index is the ratio of the GHI observed at the surface to the

83 70 Top of Atmosphere (TOA) expected GHI, which is computed via a series of geometric calculations for a given location and time. This averaging interval was selected after communication with several utility companies and agrees with the shortest time range for which a forecast is currently useful for dispatch decision-making in the United States. All instances with missing data or nighttime observations are excluded from the final dataset. Figure 4-1. Locations of SMUD irradiance observations, shown in blue triangles, and the three nearest METAR surface weather observations, shown in red X's. Brookhaven National Laboratory (BNL), located on Long Island in New York, is our second irradiance measurement system. We use data from one solar power forecast site that measures irradiance, shown in Figure 4-2 as a blue triangle. The dataset includes one year of data, from May 20, 2014 to May 19, All instances with missing data or nighttime observations are excluded from the final dataset.

84 71 Figure 4-2. Locations of BNL irradiance observation site, shown as a blue triangle, and the three nearest METAR surface weather observations, shown in red X's. The two locations of irradiance observations; Long Island, NY, and Sacramento, CA, have different climates and therefore have different irradiance variability characteristics. This allows a test of our method s robustness in predicting irradiance under different weather conditions and different number of training instances. For the BNL site on Long Island, NY, the climate is characterized by more variable cloud cover due to higher humidity resulting from its close proximity to the Atlantic Ocean. Monthly average precipitation for Long Island is relatively consistent, in contrast to Sacramento that typically experiences rainy winters and dry summers. In addition, Long Island can experience multiple snowstorms each winter that produce enough snow to cover the pyranometers and photovoltaic power arrays.

85 72 Surface weather observations are not available at the irradiance observation sites; thus, the three nearest METAR sites are used to characterize the local weather. The three closest METAR sites are shown as red X s in Figure 4-1 for the SMUD region and in Figure 4-2 for the BNL region. These observations are recorded at the top of every hour. We use six weather variables: cloud cover, dewpoint temperature, precipitation occurrence in the last hour (1 = precipitation occurred, 0 = precipitation did not occur), precipitation amount, temperature and wind speed. We use data from the Geostationary Operational Environmental Satellite (GOES)-East as our satellite derived predictors because the satellite coverage includes both of our prediction regions and the data arrives 15 minutes prior to the forecast initialization time so it can be used in real-time operations. The GOES system is operated by National Environmental Satellite, Data, and Information Service (NESDIS) and is used by the National Weather Service. GOES-East, which is also named GOES 13, is situated at 75 W. The GOES-East dataset was selected rather than the GOES-West dataset because the GOES-East field of view covers both the California and New York forecast sites. We use the data at :45 for our forecast initialized at the top of every hour to allow a necessary 15-min latency in real-time forecasting, which is another reason GOES- East was selected over GOES-West because the latter is available at :30 and :00. The GOES-East data consists of directly measured variables and derived variables. The directly measured variables are the temperature and reflectivity values at 650 nanometer wavelength (visible) and 3.75 micrometer wavelength (infrared). The derived variables are the cloud top temperature, cloud type, cloud fraction, solar zenith angle, cloud optical depth, and hydrometeor radius. The data are archived on a 4-km grid and the data are computed for each forecast location by averaging the nearest nine points on the grid at :45 after each hour. In addition to the observed irradiance and weather predictors, it is often useful to derive additional variables in order to emphasize important physical processes. Based on our previous

86 73 work (McCandless et al. 2015, Chapter 3), we derive inputs specific to the k-means classification system as well predictors specific to the ANN prediction system. In particular, we leverage our meteorological knowledge to provide the k-means algorithm with inputs in order to identify cloud regimes and to provide the ANNs with predictors for predicting solar irradiance. Based on that previous work (McCandless et al. 2015, Chapter 3), we variables as inputs for the k-means algorithm that are the cloud cover squared averaged over the three nearest METAR sites and the standard deviation of the cloud cover for the three nearest METAR sites so as to weight higher regional cloud cover values and to quantify the regional solar irradiance variability. Another predictor, dewpoint depression that is defined as the difference between the temperature and the dewpoint temperature, quantifies the atmosphere s nearness to saturation at the surface. This derived predictor, and the cloud cover squared predictor, are averaged over the three METAR sites based on a sensitivity study that showed no improvement by including the predictor for each site independently. For the SMUD region, we derive two additional predictors by computing the spatial average and standard deviation of the clearness index at the previous 15-min interval over the remaining sites. These predictors are computed so as to quantify the regional distribution of cloud cover as measured by the eight solar irradiance observation sites Process Overview Our complex prediction process requires sensitivity studies to determine the best configuration before applying the final prediction models to an independent validation dataset. Therefore, we create separate training datasets, sensitivity test datasets and validation datasets, which are labeled Train, Sensitivity Test, and Validation in Table 4-1. The validation datasets are used as an independent verification of our final models. Table 4-1 lists the number of instances in each of the datasets for both SMUD and BNL. The SMUD datasets have substantially more

87 instances because there are eight prediction sites within the SMUD region and there were fewer missing observations compared to the BNL datasets. 74 Table 4-1. List of instances in each training, testing and validation datasets for both BNL and SMUD. SMUD Dataset Satellite Derived Cloudy Instances Satellite Derived Clear Instances Train Sensitivity Test Validation BNL Dataset Satellite Derived Cloudy Instances Satellite Derived Clear Instances Train Sensitivity Test Validation We test multiple regime-dependent prediction methods for solar irradiance prediction given various inputs and predictors; therefore, we use a dataflow diagram (Figure 4-3) to describe the relationships between the various techniques. The top tier represents the data sources: irradiance observations, METAR surface weather observations, derived predictors, and satellite data, which are split into two boxes for the measured and the derived variables. The GOES-East satellite derived variables are included only in the instances that are not defined as clear. The second tier illustrates this separation into the satellite determined clear instances and satellite determined cloudy instances. This is inherently the first regime separation in our prediction process. The third tier of Figure 4-3 describes the prediction methods for all other instances.

88 75 From left to right, the first prediction technique is the ANN applied on the clear dataset. The next prediction technique is an ANN without additional regime classification. The final three are the Regime-Dependent ANNs, which are hereafter given the name RD-ANN. The first RD-ANN method is based on regimes determined explicitly from the cloud type variable in the GOES- East data, which is labeled RD-ANN-GCT where GCT stands for GOES Cloud Type. The next RD-ANN technique is the k-means cloud regime classification that includes inputs from all of our data sources, which we give the name RD-ANN-GKtCC because it includes GOES-East data, Kt observations and cloud cover from the METAR observations. The final prediction technique does not include the satellite measurements and is a direct comparison to previous work (McCandless et al. 2015, Chapter 3). This method is named RD-ANN-KtCC because it includes the Kt observations and the cloud cover. The fourth tier elements are the final predictions from all of the prediction techniques, including the baseline technique of the clearness index persistence. The validation dataset results from these predictions are shown in the Results Section.

89 76 Figure 4-3. Overall process design for our regime dependent prediction technique and the comparison techniques Prediction Methods Baseline: Clearness Index Persistence We use clearness index persistence as our baseline prediction technique for comparison. Clearness index persistence is commonly referred to as smart persistence. It inherently corrects for changes in solar elevation with time and can be easily converted back to GHI for operations if the clearness index forecast is multiplied by the TOA GHI.

90 77 This baseline technique uses the last available observation of the clearness index (i.e. 15-min average) as the prediction for subsequent times. For locations with either generally clear conditions or steady cloud cover, this technique is difficult to improve on. In contrast, when the sky condition is characterized by mixed or variable clouds, the clearness index persistence technique performs poorly Artificial Neural Network The ANN is our choice for nonlinear Artificial Intelligence (AI) prediction technique because an ANN does not require a priori knowledge of potentially complex relationships between the predictors and the predictand. ANNs replicate how the human learning process works and when given a set of training data, ANNs can model complex, i.e. nonlinear, relationships between the predictors and the predictand (Lippmann 1987). The ANN used here is a feed-forward neural network trained by a backpropagation algorithm (Reed 1998), which is commonly referred to as a multi-layer perceptron (Rosenblatt 1958). The specific neural network module used in this study is the newff model in the Neurolab python library ( The ANN used here has three layers: the input layer that consists of the predictors, the hidden layer that consists of tunable neurons, and the output layer that is the prediction. The actual processing is done by the neurons in the hidden layer, each of which is a linear regression post-processed by a nonlinear sigmoid function so that all outputs are on a common finite scale. These neuron outputs are then merged by a final linear regression neuron to yield the ANN s forecast. Each predictor of the input layer is connected to all neurons within the hidden layer, but the iterative training results in special weights for each neuron that together solve the different aspects of the problem.

91 78 Thus, varying the number of neurons in the hidden layer changes the complexity of the model. As more neurons are added, more complex nonlinear relationships between the predictors and the predictand can be modeled. As more neurons are added, however, the risk of overfitting the training data and decreasing the performance of the model on the independent data increases. Similarly, as the number of training epochs (i.e. iterations) is increased, the ANN may begin to tune to the random noise in the training data as well as to the real relationships. Therefore, both the number of neurons of the hidden layer and the number of training epochs determine the ANN s fit to the training data. The goal of configuring the ANN is to find the best level of complexity, i.e. the number of hidden layer neurons, and the number of training epochs that model the true relationships in the training data and thus yield the lowest error on independent data. We held the learning rate (0.01) and weight decay (0.5) constant as sensitivity studies (not shown) found these values to be best. We have a total of 42 predictors for the SMUD sites, which includes data from SMUD irradiance observation sites, METAR weather observation sites, GOES-East satellite data, and several derived predictors. A list of all predictors for the ANN is provided in Table 4-2. For the BNL locations, the predictors, Kt Nearby Mean and Kt Nearby Variability (Stdev) are not available because, unlike SMUD, the BNL data come from a single location.

92 Table 4-2. List of predictors for the ANN model. The Kt Nearby Mean and Variability are marked with an asterisk because they are only available for the SMUD sites Regime-Dependent Artificial Neural Network The ultimate goal of the ANN is to find the true relationship between the predictors and the predictand; therefore, we partition the dataset into cloud regime subsets in order to allow the ANN to find the simpler relationships applicable to each cloud regime rather than having to model both these relationships and regime identification with a single complex network. In order to improve the deterministic forecast, the regime identification technique must split regimes with different underlying forecast problems, each with different physical and, thus, statistical relationships between predictors and predictand. Thus, the regime classification method must capture differences that are directly related to short term irradiance forecasting, given the predictors available.

93 80 The three methods we use to classify regimes before applying the ANNs to each subset separately are discussed in detail in section 5. Two regime-identification methods, which are named after the input data, RD-ANN-KtCC and RD-ANN-GKtCC, use a k-means clustering algorithm. The k-means clustering algorithm is explained in detail in Appendix. For the RD- ANN-KtCC method described in section 5.1, the inputs to the k-means clustering algorithm are the past irradiance (converted to Kt) observations and cloud cover observations from the METAR data. This method is tested to determine the predictive skill of an RD-ANN method using only surface observations. For the RD-ANN-GKtCC method described in section 5.2, the inputs to the k-means clustering algorithm are the past irradiance (converted to Kt) observations, cloud cover observations from the METAR data and variables from the GOES-East data. This method is tested to determine the predictive skill of an RD-ANN method using both surface observations and satellite data. In contrast, the RD-ANN-GCT method, explained in section 5.3, does not use the k-means algorithm to classify regimes, but rather uses the derived cloud type variable in the GOES-East data to separate regimes. This test will determine if off-the-shelf cloud typing can compete with mission specific cloud regime typing in solar forecasting Regime-Dependent ANN Configuration RD-ANN-KtCC The first regime-dependent method tested uses the original configuration of the regimedependent ANN of (McCandless et al. 2015, Chapter 3), hereafter referred to as RD-ANN-KtCC. This technique does not include any GOES-East data as either inputs to the k-means regime classification or as predictors for the ANN. Sensitivity studies in McCandless et al. (2015), Chapter 3, showed that the best inputs to the k-means clustering algorithm were the following: Kt

94 81 average in the previous 15-min, nearby Kt in the previous 15-min, standard deviation of the Kt in the previous 15-min among the nearby sites, the most recent change in the Kt (Kt previous 15- min Kt previous 30-min), the slope of the Kt in the past hour, the standard deviation of the Kt over the previous hour, and standard deviation of the cloud cover. Because there are seven inputs into the k-means algorithm, there are therefore seven dimensions in the phase space of the k- means distance computation. These seven inputs provide the k-means algorithm with information that capture the meteorological state based on surface observations. Sensitivity studies indicate that the optimal number of regimes, k, was also seven. For the BNL site, only a single irradiance observation site was available; therefore, the RD-ANN-KtCC method does not include either the nearby Kt in the previous 15-min or the standard deviation of the Kt in the previous 15-min among the nearby sites RD-ANN-GKtCC The RD-ANN-GKtCC method uses 16 inputs into the k-means clustering algorithm for the SMUD sites, which are shown in Table 4-3. Again, the multi-site inputs are unavailable for BNL; thus, the RD-ANN-GKtCC method does not include either the nearby Kt in the previous 15-min or the standard deviation of the Kt in the previous 15-min among the nearby sites. Because there are 16 inputs into the k-means algorithm, there are 16 dimensions in the phase space of the k-means distance computation. These 16 inputs provide the k-means algorithm with information to capture the meteorological state given both surface irradiance and weather observations as well as satellite-based data with careful consideration given to avoiding colinearity. The inputs include all inputs used in RD-ANN-KtCC as well as additional variables from the GOES-East observations: cloud fraction, cloud top height, cloud optical depth,

95 hydrometeor radius, reflectance at 6.5 um (i.e. wavelength for shortwave IR), reflectance at 3.75 um (i.e. wavelength for water vapor), temperature at 6.5 um and temperature at 3.75 um. 82 Table 4-3. List of inputs for the k-means algorithm in the RDANN-GKtCC configuration. The Kt Nearby Mean and Variability are marked with an asterisk because they are only available for the SMUD sites. In order to match the level of complexity of the ANN with the number of training cases and complexity of relationships within each regime, we perform multiple sensitivity studies to determine the best number of training epochs and the best number of hidden layer neurons. We examine the mean absolute error (MAE) of the RD-ANN-GKtCC method on the sensitivity test cases for each lead-time. The MAE is calculated as, MAE =!!!!!! (obs i pred i, (1)

96 83 where n is the number of instances in the testing data. We varied the number of training epochs (100, 250, 500 or 1000) and averaged the error over the regimes. The test was conducted separately for each lead-time with the result for 180 minutes appearing in Figure 4-4. The results indicate that the lowest error, and thus the best number of training epochs for the ANN is 500. The same result (not shown) was obtained for the other lead-times. Figure 4-4. Sensitivity study results for the optimal number of training epochs of the ANN for the RDANN at SMUD sites for the 180-min lead-time. After the sensitivity study determined the number of training epochs, the next step in configuring the RD-ANN-GKtCC model was to determine the best number of neurons and the best number of regimes for each forecast lead-time and forecast location. We performed a sensitivity study with 5, 10, 15 and 20 neurons in the hidden layer and k ranging from two to nine for each forecast lead-time. The best combinations (in terms of the lowest MAE on the sensitivity test datasets) are shown in Table 4-4. For the SMUD sites, the best k is two for the two shorter lead-times and three for the two longer lead-times. For the BNL location, the best k

97 84 is two for all forecast lead-times. The best number of neurons varies among the different locations and lead-times; however, the results showed relatively minor differences between different number of neurons, which indicates that the increase in forecast power nearly balances the increase in overfitting for a range of model complexities around the best configuration. Table 4-4. Best number of regimes, K, and number of neurons in the hidden layer for all forecast lead-times at both SMUD and BNL as determined by the lowest error on the sensitivity test set RD-ANN-GCT The third method of regime-dependent prediction uses the cloud type variable in the GOES-East data to determine regimes; therefore, this technique is named RD-ANN-GCT. Each cloud type has a separate ANN trained for that cloud type. There are seven cloud types present in the data: fog, liquid water clouds, supercooled water clouds, opaque ice clouds, cirrus clouds, overlapping clouds and overshooting clouds.

98 Results SMUD Once the best configurations are determined, the true test of skill is the comparison of the forecast techniques on the independent test datasets. The data is initially split based on whether there are derived data in the GOES-East observations. Derived data are only available when the measured temperature and reflectance data indicate clouds are present. If an instance is identified as clear based on the GOES-East data, than an ANN trained on only those cases is used to predict the clearness index. Otherwise, the RD-ANN models and an ANN without regime identification are used to predict the clearness index. Clearness index persistence is used in both cases as our baseline technique. The results for the GOES-East defined clear cases are shown in Table 4-5 for all forecast lead-times. They indicate that the ANN improves upon the clearness index persistence method at the 60-min, 120-min and 180-min forecast lead-times. At the 15-min forecast lead-time, however, the error is nearly double that of the clearness index persistence forecast and this is likely a case of overfitting the training data. At this forecast lead-time, the magnitude of the irradiance is relatively consistent unless a cloud advects or develops over the observation site. Because these instances are rare when GOES-East data determines it to be clear, the ANN likely overfits those uncommon cases and thus hurts the overall performance of the model. We had kept the configuration of the ANN consistent throughout the forecast lead-times and across the clear and cloudy data subsets; however, future work will examine how to adjust the parameters of the ANN so that the model performs well on the test dataset for the clear data subset.

99 Table 4-5. Comparison of MAE for the clearness index persistence and the ANN, CLEAR model for all forecast lead-times. 86 Next, all of the RD-ANN methods were compared to both the ANN without regime identification (ANN-ALL) and the clearness index persistence for all the cases labeled other than clear by the GOES-East data. These results are plotted in Figure 4-5 for all forecast lead-times. As expected, the forecast error increases as the forecast lead-time increases. The only method that generally performs worse than clearness index persistence is the RD-ANN-GCT method that uses the GOES-East derived cloud types as the regime classification method. At the 15-min lead-time, the RD-ANN-KtCC; RD-ANN-GKtCC; ANN-ALL, and clearness index persistence all show similar errors. However, at the 60-min and longer lead-times, the RD-ANN-KtCC; RD- ANN-GKtCC; ANN-ALL, all show improvement over the clearness index persistence as shown by the larger MAE of the clearness index forecasts. The method that generally performs best is RD-ANN-GKtCC method, which exploits the GOES-East data in both the k-means clustering and ANN.