Regional Flood Frequency Analysis in the Range of Small to Large Floods: Development and Testing of Bayesian Regression-based Approaches

Transcription

1 Regional Flood Frequency Analysis in the Range of Small to Large Floods: Development and Testing of Bayesian Regression-based Approaches Khaled Haddad A thesis submitted for the degree of Doctor of Philosophy at the University of Western Sydney, Sydney, Australia June 013

2 PRELIMINARIES ABSTRACT Design flood estimation in the range of frequent to medium ( 100 years) and large to rare (greater than 100 and up to 000 years) average recurrence intervals (ARI) is frequently required in the design of many engineering works such as design of culverts, bridges, farm dams, spill ways, land use planning and flood insurance studies. These sorts of infrastructure works and investigations are of notable economic significance. Design flood estimation is ideally made adopting a flood frequency analysis technique; however, this needs a relatively longer period of recorded streamflow data. In many cases, recorded streamflow data is quite short or completely absent (i.e. ungauged catchment situation). In such cases, regional flood frequency analysis (RFFA) techniques are usually adopted, which attempts to utilise spatial data to compensate for temporal data on the assumption of regional homogeneity. This thesis focuses on RFFA techniques, in particular how the RFFA techniques can be enhanced by adopting an ensemble of advanced statistical techniques as well as by minimising the error and noise often found in the flood data. This thesis uses data from 68 catchments from the continent of Australia to (i) develop prediction equations involving readily obtainable catchment characteristics data for floods in the frequent to medium range ARIs ( 100 years) (ii) investigate the validation of the developed prediction equations using the most commonly used leave-one-out validation (LOO) and to compare it with the more recent Monte Carlo cross validation (MCCV) technique and (iii) to develop a large flood regionalisation model (LFRM) that corrects for spatial dependence in the annual maximum flood series data (AMFS) for flood estimation in the large to rare flood range ( years ARI). The first part of this thesis advocates the use of regression-based RFFA methods under the Bayesian generalised least squares regression (BGLSR) framework. Here, the BGLSR has been developed and tested with the quantile regression technique (QRT) and the parameter regression technique (PRT) using 45 catchments from the east coast of Australia (namely New South Wales (NSW), Victoria, Queensland and Tasmania). In forming the regions, both the fixed region and region of influence (ROI) approaches have been examined in the range of frequent to medium ARI floods. ii

3 PRELIMINARIES A LOO validation indicated that the ROI based on the minimisation of the predictive uncertainty leads to more efficient and accurate flood quantiles estimates in both the QRT and PRT regional frameworks. The regression diagnostics reveal that the catchment characteristics variables alone may not pick up all the heterogeneity in the regional model and formation of ROI sub-regions can reduce the heterogeneity level to an acceptable limit. Both the BGLSR based QRT-ROI and PRT-ROI methods show improvements in regional heterogeneity with an increase in the average pseudo coefficient of determination and a decrease in the model error variance, average variance of prediction and the average standard error of prediction. Based on the evaluation statistics, overall it has been found that there are only modest differences between the QRT-ROI and PRT-ROI regional frameworks. The developed RFFA methods based on the QRT-ROI and PRT-ROI allow design flood estimation along with its associated uncertainty (in the form of confidence limits) to be made with a relatively high degree of accuracy. The second part of this thesis looks at the detailed validation of regional hydrological regression models by investigating the popular LOO validation and the relatively new MCCV procedures using 96 catchments from the state of NSW. In this regard, both the ordinary least squares regression (OLSR) and GLSR have been tested for the estimation of flood quantiles using simulated and observed regional flood data. From the simulation and real data examples, it has been found that when developing regional hydrologic regression models, application of GLSR based MCCV validation procedure is likely to result in a more parsimonious model than the OLSR based LOO, OLSR based MCCV and GLSR based LOO validation procedures. The third part of this thesis proposes a simple LFRM that accounts for spatial dependence in the AMFS data for estimating large to rare floods. To carry this out a comprehensive dataset from all over the Australian continent has been used that consists of 654 stations. The new LFRM is easy to use and offers an alternative to the traditional rainfall-based methods. The development and application of the simplified LFRM for the Australian continent consists of three major steps (i) pooling the top 1 to 5 annual maximum flood values from member sites in a region (ii) developing a new spatial dependence model to correct for spatially correlated data (iii) application of the LFRM to ungauged catchments iii

4 PRELIMINARIES by coupling it with the BGLSR ROI technique to estimate the mean and coefficient of variation (CV) of AMFS data. To this end a simple model for the effective number of independent stations (N e ) has been developed that ignores possible variation with ARI. Meaningful results regarding spatial dependence have been established by undertaking the analysis on simulated datasets to counteract sampling and homogeneity issues. Overall, the experimental results of the analysis show that, in general, spatial dependence decreases with larger network size and that some Australian states exhibit more spatial dependence than others. While there are some limitations with this analysis, a reasonable indication of the behaviour of N e has been established. The derived generalised spatial dependence model has then been used with the LFRM to correct for the spatial dependence by adjusting the plotting position points of the LFRM frequency distribution curve. An independent validation has showed that the developed LFRM is able to estimate design floods for 100 to 1000 years ARIs with reasonable confidence as compared to at-site flood frequency analysis results, other regional flood models and the world model. Overall, the newly developed LFRM that corrects for spatially correlated data and coupled with BGLSR - ROI approach offers a powerful yet simple method of regional flood estimation for floods in the large to rare ARI range. iv

5 PRELIMINARIES COPYRIGHT STATEMENT I hereby grant the University of Western Sydney or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation. Signed Khaled Haddad v

6 PRELIMINARIES STATEMENT OF AUTHENTICATION I hereby declare that the work presented in this thesis is solely my own work and that to the best of my knowledge the work is original except where otherwise indicated by references to other authors or works. No part of this thesis has been submitted for any other degree or diploma. Signed Khaled Haddad vi

7 PRELIMINARIES ACKNOWLEDGMENTS Firstly I would like to acknowledge the contribution of my supervisor Dr Ataur Rahman in providing direction, advice and encouragement over the course of the last three and a half years. I really appreciate your support. I also appreciate the advice and friendship of other academics and researchers in the School of Computing, Engineering and Mathematics at UWS. In particular, thanks to Associate Professor Surendra Shrestha and Associate Professor Chin Leo. The advice and friendship from other universities and industry is also gratefully acknowledged. In particular Mr Erwin Weinmann of Monash University for his constructive comments, valuable guidance, advice and encouragement throughout this research, Professor George Kuczera, Associate Professor James Ball, Mr Mark Babister, Mr Robert French and Dr William Weeks for their suggestions and input to the research. A special thanks goes to Dr Nanda Nandakumar for his helpful advice on various aspects of the issues relating to spatial dependence and large flood estimation. I would also like to acknowledge various government departments throughout Australia for their help and contribution in providing the streamflow data for this study. Without their timely support this research would have not been completed on time. To my fellow PhD students, thanks for all your help, fun times and the support of knowing we re not alone through the ups and downs. Thank you to my parents and family for teaching me the value of education and hard work, which gave me the confidence to embark on this mission. vii

8 PRELIMINARIES TABLE OF CONTENTS ABSTRACT... II COPYRIGHT STATEMENT... V STATEMENT OF AUTHENTICATION...VI ACKNOWLEDGMENTS... VII TABLE OF CONTENTS...VIII LIST OF FIGURES... XVI LIST OF TABLES... XXII COMMON NOTATIONS...XXV ABBREVIATIONS...XXVII CHAPTER 1: INTRODUCTION GENERAL BACKGROUND THE NEED FOR THIS RESEARCH RESEARCH QUESTIONS MAJOR TASKS CONTRIBUTIONS OF THIS RESEARCH TO THE UNDERSTANDING OF THE RFFA PROBLEM OUTLINE OF THE THESIS AND CHAPTER INTRODUCTIONS...10 CHAPTER : REVIEW OF REGIONAL FLOOD FREQUENCY ANALYSIS TECHNIQUES, MODEL VALIDATION AND LARGE FLOODS GENERAL BASIC ISSUES REGIONAL FLOOD FREQUENCY ANALYSIS REGIONAL HOMOGENEITY INTER SITE DEPENDENCE DISTRIBUTIONAL CHOICES METHODS FOR IDENTIFICATION OF HOMOGENEOUS REGIONS...19 viii

9 PRELIMINARIES.4 REGIONAL FLOOD FREQUENCY ANALYSIS METHODS DIFFERENT APPROACHES INDEX FLOOD METHOD STATION YEAR METHOD BAYESIAN ANALYSIS AND MONTE CARLO METHODS PROBABILISTIC RATIONAL METHOD AS USED IN AUSTRALIA QUANTILE AND PARAMETER REGRESSION TECHNIQUES INTRODUCTION GENERALISED LEAST SQUARES AND WEIGHTED LEAST SQUARES REGRESSION PREVIOUS APPLICATION OF GENERALISED LEAST SQUARES AND BAYESIAN GENERALISED LEAST SQUARES REGRESSION FIXED REGIONS AND THE REGION OF INFLUENCE IN REGIONAL FLOOD FREQUENCY ANANALYS FORMATION OF REGIONS REGION OF INFLUENCE VS FLEXIBLE REGION MODEL VALIDATION IN HYDROLOGICAL REGRESSION ANALYSIS HISTORY OF MODEL VALIDATION PREVIOUS APPLICATIONS OF LEAVE-ONE-OUT VALIDATION IN HYDROLOGY38.8 REGIONAL FLOOD FREQUENCY FOR LARGE TO RARE FLOODS BRIEF REVIEW OF LARGE FLOOD ESTIMATION AND PREVIOIUS APPLICATIONS IMPACT OF CLIMATE CHANGE ON FLOOD FREQUENCY ANALYSIS SUMMARY...45 CHAPTER 3: ADOPTED STATISTICAL TECHNIQUES FOR REGIONAL FLOOD FREQUENCY ANALYSIS AND MODEL VALIDATION GENERAL AT-SITE FLOOD FREQUENCY ANALYSIS BASICS OF AT-SITE FLOOD FREQUENCY ANALYSIS FLIKE SOFTWARE FOR AT-SITE FFA LOG PEARSON TYPE 3 (LP3) DISTRIBUTION THE CLASSICAL GLS REGRESSION PROBLEM GLSR, THE STEDINGER AND TASKER MODEL BAYESIAN METHODOLOGY CLASSICAL BAYESIAN INFERENCE BAYESIAN GLS REGRESSION...56 ix

10 PRELIMINARIES APPROACH ADOPTED IN THIS STUDY FOR THE QUANTILE AND PARAMETER REGRESSION TECHNIQUES ADOPTED BAYESIAN REGRESSION APPROACH PRIOR FOR THE β COEFFICIENTS ANALYTICAL SOLUTION TO BAYESIAN APPROACH FOR THE POSTERIOR OF THE MODEL ERROR VARIANCE PRIORS FOR THE PARAMETERS AND THE QUANTILES OF THE LP3 DISTRIBUTION SELECTING PREDICTOR VARIABLES AVERAGE VARIANCE OF PREDICTION BAYESIAN AND AKAIKE INFORMATION CRITERIA BAYESIAN PLAUSIBILTY VALUE COEFFICIENT OF DETERMINATION OTHER MODEL SELECTION CRITERIA FORMATION OF REGIONS REGRESSION DIAGNOSTICS STANDARD ERROR OF PREDICTION RESIDUAL ANALYSIS COOK S DISTANCE EVALUATION STATISITCS REGIONAL UNCERTAINTY WITH FLOOD QUANTILE ESTIMATION THE MULTIVARIATE NORMAL DISTRIBUTION VALIDATION OF REGIONAL HYDROLOGICAL REGRESSION MODELS METHODOLOGY THE HYDROLOGICAL REGRESSION PROBLEM MODEL SELECTION BY MONTE CARLO CROSS VALIDATION ESTIMATING MSEP APPLICATION USING SIMULATED DATA OBSERVED REGIONAL FLOOD DATA FROM NSW, AUSTRALIA SUMMARY...84 CHAPTER 4: STUDY AREA AND PREPARATION OF STREAMFLOW AND CATCHMENT CHARACTERISITICS DATA GENERAL PUBLICATIONS STUDY AREA SELECTION OF CANDIDATE CATCHMENTS...87 x

11 PRELIMINARIES 4.4 STREAMFLOW DATA PREPARATION FILLING MISSING RECORDS IN ANNUAL MAXIMUM FLOOD SERIES TREND ANALYSIS RATING CURVE ERROR AND IDENTIFICATION SENSIVITY ANALYSIS AND IMPACT OF RATING CURVE EXTRAPOLATION ON FLOOD QUANTILE ESTIMATES TESTS FOR OUTLIERS RESULTS OF STREAMFLOW DATA PREPARATION PROCESS DATA PREPARATION FOR VICTORIA DATA PREPARATION FOR NSW AND ACT SENSITIVITY ANALYSIS - IMPACT OF RATING CURVE ERROR ON FLOOD QUANTILE ESTIMATES SUMMARY RESULTS OF STREAMFLOW DATA PREPARATION FOR THE OTHER STATES TASMANIA QUEENSLAND SOUTH AUSTRALIA NORTHERN TERRITORY WESTERN AUSTRALIA SUMMARY OF STREAMFLOW DATA AUSTRALIA WIDE SELECTION AND ABSTRACTION OF CATCHMENT CHARACTERISITCS SUMMARY...11 CHAPTER 5: RESULTS RFFA BASED ON FIXED REGIONS AND REGION OF INFLUENCE APPROACHES UNDER THE QUANTILE AND PARAMETER REGRESSION FRAMEWORKS GENERAL PUBLICATIONS RESULTS FOR TASMANIA SELECTING PREDICTOR VARIABLES WITH QRT AND PRT PSUEDO ANOVA WITH QRT AND PRT MODELS FOR THE FIXED AND ROI REGIONS ASSESMENT OF MODEL ASSUMPTIONS AND REGRESSION DIAGNOSTICS POSSIBLE SUBREGIONS IN TASMANIA EVALUATION STATISTICS SECTION SUMMARY RESULTS FOR NEW SOUTH WALES, VICTORIA AND QUEENSLAND xi

12 PRELIMINARIES SELECTING PREDICTOR VARIABLES WITH QRT AND PRT REGION OF INFLUENCE VS. FIXED REGIONS FOR PARAMETER AND QUANTILE REGRESSION TECHNIQUES REGRESSION DIAGNOSTICS PSEUDO ANALYSIS OF VARIANCE REGRESSION DIAGNOSTICS MODEL ADEQUACY AND OUTLIER ANANLYSIS DIAGNOSTIC STATISTICS EVALUATION STATISTICS SECTION SUMMARY UNCERTAINTY ESTIMATION FOR NEW SOUTH WALES, VICTORIA, QUEENSLAND AND TASMANIA IN A ROI-PRT FRAMEWORK SUMMARY CHAPTER 6: RESULTS - MODEL VALIDATION USING LOO AND MCCV GENERAL PUBLICATIONS RESULTS PREDICTORS USED SIMULATED DATA APPLICATION WITH OBSERVED REGIONAL FLOOD DATA IN NSW SUMMARY CHAPTER 7: BACKGROUND AND DEVELOPMENT OF THE LARGE FLOOD REGIONALISATION MODEL AND ISSUES RELATING TO SPATIAL DEPENDENCE GENERAL PUBLICATIONS LFRM CONCEPT INTER-SITE DEPENDENCE IN GENERAL FOR THE LFRM ANNUAL MAXIMUM DATA SET USED IN THE LFRM QUALITY CHECK OF THE LARGEST ANNUAL MAXIMA DATA IDENTIFICATION OF AN APPROPRIATE PROBABILITYY DISTRIBUTION AND TESTING FOR HOMOGENITY OF ANNUAL MAXIMA FLOOD DATA SEARCHING FOR AN APPROPRIATE PROBABILITY DISTRIBUTION GOODNESS-OF-FIT TEST RESULTS HOMOGENEITY HOMOGENEITY TEST OF HOSKING AND WALLIS xii

13 PRELIMINARIES 7.6. THE BOOTSTRAP ANDERSON-DARLING HOMOGENEITY TEST TESTING FOR HOMOGENEITY RESULTS DEVELOPMENT OF THE LFRM MODEL FOR AUSTRALIAN FLOOD DATA DEVELOPMENT AND CALIBRATION OF THE LFRM MODEL EFFECTS OF INTER-SITE DEPENDENCE ON THE LFRM MODEL EFFECTIVE NUMBER OF INDEPENDENT STATIONS REGIONAL MAXIMUM FLOOD AT A NETWORK OF SITES - REGIONAL MAXIMUM AND TYPICAL CURVES FACTORS INFLUENCING THE REGIONAL MAXIMUM NUMBER OF SITES, N CROSS CORRELATION DEFINITION OF A REGION FOR ANALYSIS METHODS OF SAMPLING REGIONAL MAXIMA ROI AND RANDOM ROI NETWORK METHODS THE TOTAL RANDOM NETWORK METHOD COMPARING SAMPLING METHODS MEASURES OF N e EFFECTIVE NUMBER OF INDEPENDENT STATIONS EFFECTIVE NUMBER OF INDEPENDENT STATIONS, N e A SIMPLE MODEL FOR N e FITTING N e BY THE MEAN SIMULATED DATASETS SYNTHETIC DATA GENERATION SUMMARY...15 CHAPTER 8: APPLICATION OF LFRM IN THE LIGHT OF SPATIAL DEPENDENCE RESULTS AND DISCUSSION GENERAL RESULTS FOR N e A CLOSER LOOK AT THE BEHAVIOUR OF N e GENERALISING THE N e MODEL CONSTANT N e MODEL AN EMPIRICAL RELATIONSHIP FOR N e BASED ON AVERAGE CORRELATION COEFFICENT (ρ) FURTHER DISCUSSION COMPARISON OF THE EFFECTIVE RECORD LENGTH ESTIMATES USING THE CONSTANT N e MODEL FOR THE REAL AND SIMULATED DATASETS REVISITING THE LFRM IN THE LIGHT OF SPATIAL DEPENDENCE APPLICATION OF THE LFRM MODEL TO UNGAUGED CATCHMENTS...40 xiii

14 PRELIMINARIES DERIVATION OF PRIORS FOR THE MEAN FLOOD AND CV ESTIMATION OF THE ERROR COVARIANCE MATRIX ESTIMATION OF THE SAMPLING ERROR VARIANCE ESTIMATION OF THE SAMPLING ERROR INTER-SITE CORRELATION SOME ISSUES ASSOCIATED WITH REGIONAL ESTIMATION OF CV SELECTION OF PREDICTOR VARIABLES BGLSR RESULTS FOR MEAN AND CV BGLSR RESULTS FOR MEAN AND CV MODELS USING ROI VALIDATION SUMMARY...57 CHAPTER 9: CONCLUSIONS INTRODUCTION OVERVIEW OF THE STUDY DATA SELECTION (CHAPTER 4) RFFA IN THE FREQUENT TO MEDIUM ARI RANGE (CHAPTER 5) MCCV VS LOO (CHAPTER 6) LARGE TO RARE FLOOD ESTIMATION (CHAPTERS 7 and 8) CONCLUSIONS DESIGN FLOOD ESTIMATION IN THE FREQUENT TO MEDIUM ARI RANGE VALIDATION OF REGIONAL HYDROLOGICAL REGRESSION MODELS LARGE TO RARE FLOOD ESTIMATION LIMITATIONS AND SUGGESTIONS FOR FUTURE RESEARCH...64 REFFRENCES APPENDIX A A.1 PUBLISHED PAPERS FROM THIS RESEARCH...88 APPENDIX B B.1 FURTHER RESULTS ASSOCIATED WITH VICTORIA AND QUEENSLAND (FROM CHAPTER 5)...89 APPENDIX C C.1 FURTHER RESULTS ASSOCIATED WITH THE LFRM (FROM CHAPTERS 7 AND 8)...95 APPENDIX D D.1 L-MOMENT RATIO DIAGRAMS AND GOODNESS-OF-FIT TEST xiv

15 PRELIMINARIES D. ANDERSON-DARLING MONTE CARLO SIMULATION GOODNESS-OF-FIT TEST D.3 HOMOGENEITY TEST OF HOSKING AND WALLIS D.4 THE BOOTSTRAP ANDERSON-DARLING HOMOGENEITY TEST D.5 GUMBEL VARIATES CORRESPONDING TO ARI xv

16 PRELIMINARIES LIST OF FIGURES Figure 1 Flash flooding in Emerald Central Queensland (Oncirculation, 011)... Figure Flow chart showing statistical techniques/ methods adopted in this thesis Figure 3 Example of ROI techniques applied in this study Figure 4 Use of multivariate normal distribution to develop confidence limits by Monte Carlo simulation Figure 5 Plot of the selected study area (i.e. NSW, VIC, QLD and TAS) Figure 6 Plot of rating ratios (RR) for station Figure 7 Rating curve extension error Figure 8 (a) Time series plot showing significant trends after 1995 and (b) CUSUM test plot showing significant trends after Here Vk is CUSUM test statistic defined in McGilchrist and Wodyer (1975) Figure 9 Histogram of rating ratios of annual maximum flood data in Victoria (stations with record lengths > 5 years) Figure 10 Distributions of streamflow record lengths of the selected 131 stations from Victoria Figure 11 Distributions of catchment areas of the 131 catchments from Victoria Figure 1 Histogram of rating ratios for 106 stations from NSW Figure 13 Distributions of streamflow record lengths of the selected 96 stations from NSW Figure 14 Distributions of catchment areas of the 96 catchments from NSW Figure 15 (a) Distribution of annual maximum flood record lengths of 68 stations from all over Australia (b) Distribution of catchment areas of 68 stations from all over Australia Figure 16 Geographical distributions of the selected 68 stations from all over Australia107 Figure 17 Selection of predictor variables for the BGLSR model for Q 10 (QRT, fixed region Tasmania), MEV = model error variance, AVPO = average variance of prediction (old), AVPN = average variance of prediction (new), AIC = Akaike information criterion, BIC = Bayesian information criterion, note R GLS uses right hand axis Figure 18 Selection of predictor variables for the BGLSR model for skew xvi

17 PRELIMINARIES Figure 19 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region, Tasmania) Figure 0 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI, Tasmania) Figure 1 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, Tasmania) Figure QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, Tasmania) Figure 3 Cook s distance (D i ) for locating outlier sites for skew model based on variable combination Figure 4 Spatial variations of the grouped minimum model error variances for Tasmania (a) mean flood model and (b) skew model Figure 5 Selection of predictor variables for the BGLSR model for the skew (note that R GLS uses the right-hand axis) Figure 6 Selection of predictor variables for the BGLSR model for Q10 model (note that uses the right-hand axis), (QRT, fixed region NSW), MEV = model error variance, AVPO = average variance of prediction (old), AVPN = average variance of prediction (new) AIC = Akaike information criteria, BIC = Bayesian information criteria Figure 7 Plots of the standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region and ROI, NSW) Figure 8 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, ROI, NSW) Figure 9 Plots of the standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI and PRT-ROI with weighted average standard deviation and skew, NSW) Figure 30 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, and PRT ROI with weighted average standard deviation and skew, NSW) Figure 31 Spatial variations of the grouped minimum model error variances for (a) mean flood model and (b) number of sites which produced the lowest predictive variance for the mean flood model Figure 3 Boxplots of Q pred /Q obs ratios for NSW for QRT and PRT, with fixed and ROI regions xvii

18 PRELIMINARIES Figure 33 Design flood quantile estimation and confidence limits curves for ARIs of to 100 years Figure 34 The mean squared error of prediction (MSEP) associated with LOO and MCCV for OLSR and GLSR simulations Figure 35 Prediction error plot for Q 10 results (models selected by OLSR and GLSR LOO and models selected by OLSR and GLSR MCCV) Figure 36 Prediction error plot for Q 100 results (models selected by OLSR and GLSR LOO and models selected by OLSR and GLSR MCCV) Figure 37 Occurrences of the highest floods data from NSW, QLD, VIC and TAS are combined (only the highest value from each station s AMFS data is taken to form the LFRM data series) Figure 38 Cross-correlation between two nearby Victorian Stations 101 and 107(Considering all concurrent AMFS data over the period of records only 1 data points are concurrent for the pair of stations) Figure 39 Relationship between the cross-correlations among AMFS data and distance between pairs of stations in Victoria Figure 40 Geographical distribution of the 8 validation catchments for the LFRM Figure 41 L-moment ratio diagrams of annual maximum flood data for NSW and QLD 186 Figure 4 Visual inspection of distributional fit for GEV, GPA and P3 distributions for WA and TAS Figure 43 Scatter of Q max /mean data in the (CV(Q), Q max /mean) plane and non linear interpolation function Figure 44 Scattering of Y max data in the (CV(Q), Y max ) plane and linear interpolation function for the pooling of 1 (1 max) and 5 (5 max) top maxima Figure 45 Frequency distribution of the standardised Y max values Figure 46 Average concurrent record lengths for different network sizes Figure 47 Example plot of regional maximum and typical growth curves and the effective number of independent stations on a Gumbel plot for a random network of and 4 gauging sites in Tasmania Figure 48 Example plot of generated data with different constant correlation coefficients for the state of Tasmania Figure 49 Variation of N e with different network methods and experiment number for NSW+QLD+VIC region (top panel for real data and bottom panel for simulated data)... 1 xviii

19 PRELIMINARIES Figure 50 Frequency of N e with different network methods for NSW+QLD+VIC region (top panel for real data and bottom panel for simulated data)... 1 Figure 51 Regression results of the N = network combining the lnn e /lnn ratio values for all the Australian states/regions and experiments... 5 Figure 5 Regression results of the N = 4 network combining the lnn e /lnn ratio values for all the Australian states/regions and experiments... 6 Figure 53 Regression results of the N = 8 network combining the lnn e /lnn ratio values for all the Australian states/regions and experiments... 7 Figure 54 Comparison of directly computed N e from the AMFS data and N e by the constant N e model... 9 Figure 55 Variation with number of sites: effective record lengths estimated using real and simulated N e models as a function of average correlation coefficient Figure 56 Frequency distribution of standardised Y max values using N and N e stations Figure 57 Various Q max /mean quantiles derived from the LFRM_N e model and PM (World) model Figure 58 Empirical frequency distributions of Q/mean quantiles derived from the LFRM_N and LFRM_N e for different ranges of CV Figure 59 Relationship between CV and catchment area Figure 60 Selection of predictor variables for the BGLSR model for CV Figure 61 Selection of predictor variables for the BGLSR model for CV using AVPO, AVPN, AIC and BIC Figure 6 Selection of predictor variables for the BGLSR model for the mean flood Figure 63 Selection of predictor variables for the BGLSR model for the mean flood using AVPO, AVPN, AIC and BIC Figure 64 Prior and posterior pdf's for the model error variance for CV (right) and the mean flood (left) models for NSW state Figure 65 Confidence interval plot of BIAS r values with the LFRM_N e and PM (world) models for the 8 test catchments Figure 66 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region, VIC) Figure 67 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI, VIC) Figure 68 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, VIC)... 9 xix

20 PRELIMINARIES Figure 69 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, VIC)... 9 Figure 70 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region, QLD) Figure 71 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI, QLD) Figure 7 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, QLD) Figure 73 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, QLD) Figure 74 L-moment ratio diagram of annual maximum flood series data for VIC Figure 75 L-moment ratio diagram of annual maximum flood series data for WA Figure 76 L-moment ratio diagram of annual maximum flood series data for SA Figure 77 L-moment ratio diagram of annual maximum flood series data for TAS Figure 78 L-moment ratio diagram of annual maximum flood series data for NT Figure 79 Visual inspection of distributional fit for GEV, GPA and P3 distributions for NSW Figure 80 Visual inspection of distributional fit for GEV, GPA and P3 distributions for VIC Figure 81 Variation of N e with different network methods and experiment number for TAS region (top panel for real data and bottom panel for simulated data) Figure 8 Frequency of N e with different network methods for TAS region (top panel for real data and bottom panel for simulated data) Figure 83 Variation of N e with different network methods and experiment number for NT region (top panel for real data and bottom panel for simulated data) Figure 84 Frequency of N e with different network methods for NT region (top panel for real data and bottom panel for simulated data) Figure 85 Variation of N e with different network methods and experiment number for WA region (top panel for real data and bottom panel for simulated data) Figure 86 Frequency of N e with different network methods for WA region (top panel for real data and bottom panel for simulated data) Figure 87 Variation of N e with different network methods and experiment number for SA region (top panel for real data and bottom panel for simulated data) Figure 88 Selection of predictor variables for the BGLSR model for CV - WA xx

21 PRELIMINARIES Figure 89 Selection of predictor variables for the BGLSR model for CV using AVPO, AVPN, AIC and BIC - WA Figure 90 Selection of predictor variables for the BGLSR model for the mean flood WA Figure 91 Selection of predictor variables for the BGLSR model for the mean flood using AVPO, AVPN, AIC and BIC - WA Figure 9 Selection of predictor variables for the BGLSR model for CV TAS Figure 93 Selection of predictor variables for the BGLSR model for CV using AVPO, AVPN, AIC and BIC - TAS Figure 94 Selection of predictor variables for the BGLSR model for the mean flood TAS Figure 95 Selection of predictor variables for the BGLSR model for the mean flood using AVPO, AVPN, AIC and BIC - TAS xxi

22 PRELIMINARIES LIST OF TABLES Table 1 Flood quantile estimates and associated errors using ARR FLIKE with and without consideration of rating curve error Table Summary of selected stations Australia wide Table 3 Catchment characteristics variables used in the study Table 4 Different combinations of predictor variables considered for the QRT models and the parameters of the LP3 distribution (QRT and PRT fixed region Tasmania) Table 5 Pseudo ANOVA table for Q 0 model for Tasmania (QRT, fixed region and ROI) Table 6 Pseudo ANOVA table for Q 100 model for Tasmania (QRT, fixed region and ROI) Table 7 Pseudo ANOVA table for the mean flood model for Tasmania (PRT, fixed region and ROI) Table 8 Pseudo ANOVA table for the standard deviation model for Tasmania (PRT, fixed region and ROI) Table 9 Pseudo ANOVA table for the skew model for Tasmania (PRT, fixed region and ROI)... 1 Table 10 Regression diagnostics for fixed region and ROI for Tasmania Table 11 Model error variances associated with fixed region and ROI for Tasmania (n = number of sites in the region) Table 1 Evaluation statistics (RMSE r and RE r ) from leave-one-out (LOO) validation for Tasmania Table 13 Summary of counts/percentages based on the rr values for QRT and PRT for Tasmania (fixed region). U = gross underestimation, D = desirable range and O = gross overestimation Table 14 Summary of counts/percentages based on the rr values for QRT and PRT for Tasmania (ROI). U = gross underestimation, D = desirable range and O = gross overestimation Table 15 Summary of the final BGLSR results for NSW Table 16 Summary of the catchment characteristics and statistical measures used in the stepwise regression for the parameters of the LP3 distribution for NSW xxii

23 PRELIMINARIES Table 17 Summary of the catchment characteristics and statistical measures used in the forward stepwise regression for the flood quantiles of the LP3 distribution (ARIs =, 10 and 100 years) for NSW Table 18 Pseudo ANOVA table for the mean flood model (PRT, fixed region and ROI, NSW, VIC and QLD states) (Here n = number of sites in the region, k = number of predictors in the regression equation, EVR = error variance ratio, = model error 0 variance when no predictor variable is used in the regression model, = model error variance when predictor variable is used in the regression model and tr[ ( yˆ )] = sum of the diagonals of the sampling covariance matrix) Table 19 Pseudo ANOVA table for the skew model (PRT, fixed region and ROI, NSW, VIC and QLD states) (variables are explained in Table 18 caption) Table 0 Pseudo ANOVA table for Q 0 model (QRT, fixed region and ROI for NSW, VIC and QLD states) (variables are explained in Table 18 caption) Table 1 Regression diagnostics for the fixed region and ROI for NSW, VIC and QLD. 149 Table Model error variances associated with the fixed region and ROI for NSW, VIC and QLD (n = number of sites needed for the LP3 parameters and flood quantiles) 151 Table 3 Evaluation statistics (RMSE r and RE r ) from LOO validation for NSW (Results NSW for PRT using the weighted regional average standard deviation and skew models, i.e. no predictor variables given in brackets), VIC and QLD Table 4 Summary of predictor variables (here log 10 is used) Table 5 Correlation between the log 10 predictor variables used in the analysis Table 6 Results from simulated data, OLSR when = Table 7 Results from simulated data, OLSR when = Table 8 Results from simulated data, GLSR when = and ˆ ( yˆ i, yˆ ) = Table 9 Results from simulated data, GLSR when = and ˆ ( yˆ i, yˆ ) = Table 30 OLSR analysis, MSEP values for calibration and validation data set (observed data from NSW). Here log 10 is used Table 31 GLSR analysis, MSEP values for calibration and validation data set (observed data from NSW). Here log 10 is used Table 3 OLSR and GLSR analysis for LOO and MCCV for Q 10, optimal models shown along with summary statistics Table 33 MSEP for ARI = 100-year j j xxiii

24 PRELIMINARIES Table 34 OLSR and GLSR analysis for LOO and MCCV for Q 100, optimal models shown along with summary statistics Table 35 Summary of goodness-of-fit tests for determining parent distribution Table 36 Summary of MRE associated with the GEV and P3 distributions Table 37 Summary of heterogeneity measures for the Australia states Table 38 Coefficients of non linear interpolation from Figure Table 39 Coefficients and R values of Y max polynomial interpolating from Figure Table 40 Comparison of the parameters of the parent distribution and the distribution for the generated data (distribution: F(x)=exp[-{1-(x-)/} 1/ ]) and correlation coefficient, ρ Table 41 Experimental values of N e for different networks and regions using the real data (average N e over the experiment reported) Table 4 Experimental values of N e for different networks and regions using the simulated data (average N e over the experiment reported) Table 43 Experimental results in which N e exceeds N at a particular ARI for different regions using the real data set... Table 44 Properties of the Constant N e Spatial dependence model... 8 Table 45 for each pair of sites for the different states/region... 3 Table 46 Total record length (L) and effective record length (L e ) for the all Australian dataset... 3 Table 47 Coefficients and R values of Y max polynomial interpolation from Figure 56 for N and N e sites Table 48 CV values for study catchments in Australia Table 49 Summary of the finally selected BGLSR models for all the Australian states used in the validation of LFRM Table 50 Regression diagnostics for the ROI approach for the various Australian states and test catchments Table 51 Summary of error statistics obtained from independent testing associated with the LFRM model Table 5 Summary of the final BGLSR results for VIC Table 53 Summary of the final BGLSR results for QLD Table 54 Values of Y T corresponding to ARI xxiv

25 PRELIMINARIES COMMON NOTATIONS A catchment area a, b constants CS coefficient of skewness CV coefficient of variation e error term in regression analysis G regional mean skewness coefficient H heterogeneity measure I N k L L e l LSK LCV LKT N N e n n a N sim p Q T R T t c X identity matrix number of parameters in regression model total record lengths of all sites in a group record lengths after correcting for spatial dependence sample l moment L coefficient of skewness L coefficient of variation L coefficient of kurtosis total number of streamflow records used in flood frequency analysis and also used to define number of simulations undertaken number of independent stations after correcting for spatial dependence total number of datasets in regression analysis average number of stations in a region number of simulated regions for homogeneity testing probability flood quantile having return period of T years coefficient of determination used in OLSR return period time of concentration nxk matrix of basin characteristics kx1 vector of regression coefficients ŷ vector of dependant variables in regression model x o row vector of basin characteristics at site 0 xxv

26 PRELIMINARIES ˆ ˆ Q i i qˆ K T i ˆ ij ˆ ij model error variance sample estimate of model error variance covariance matrix of regression errors data based estimate of residual variance from OLSR prior mean of the model error variance used in BGLSR mean annual flood population mean at site i population standard deviation i sample mean of logs of annual maxima at station i parameter of probability distribution frequency factor for return period T in log Pearson type 3 flood frequency analysis correlation between sites used in large flood model regionalisation residual error term associated with regression of mean sampling error covariance matrix correlation/distance relationship between stations estimated correlation/distance relationship between stations R GLSR pseudo coefficient of determination used in GLS regression xxvi

27 PRELIMINARIES ABBREVIATIONS AD Anderson Darling Statistic for Homogeneity ARR-FLIKE Australian Rainfall and Runoff flood frequency analysis software AUSIFD Australian Intensity Frequency Duration software AEP Annual Exceedance Probability AIC Akaike Information Criteria AMFS Annual Maximum Flood Series data ARR Australian Rainfall and Runoff ARI Average Recurrence Interval ASPE Average Squared Prediction Error AVPO Average Variance of Prediction for old site AVPN Average Variance of Prediction for new site BGLSR Bayesian Generalised Least Squares Regression BIC Bayesian Information Criteria BOM Bureau of Meteorology BPV Bayesian Plausibility Value Area Catchment area (km ) CD Compact disk CMCCV Corrected Monte Carlo Cross validation FFA Flood Frequency Analysis forest Fraction of basin covered by medium to dense forest qsa Fraction quaternary sediment area GEV Generalised Extreme Value distribution GLSR Generalised Least Squares Regression GPA Generalised Pareto distribution IFM Index Flood Method ID Instantaneous Discharge IFD Intensity Frequency Duration IM Monthly Instantaneous Maximum Data MMD Monthly Maximum Mean Daily Data LFRM Large Flood Regionalisation Model LOO Leave-One-Out Validation xxvii

28 PRELIMINARIES LP3 log Pearson type 3 distribution MCCV Monte Carlo Cross validation MCMC Markov Chain Monte Carlo MEV Model Error Variance MOM Method of Moments Estimator ML Maximum Likelihood Estimator MSEP Mean Squared Error of Prediction MVN Multivariate Normal Distribution evap Mean annual evapotranspiration (mm) rain Mean annual rainfall (mm) NERC Natural Environment Research Council (UK) NCWE National Committee on Water Engineering NSW New South Wales OLSR Ordinary Least Squares Regression P3 Pearson type 3 distribution PM Probabilstic Model PRM Probabilistic Rational Method PRT Parameter Regression Technique PD, POT Peaks over threshold QLD Queensland QRT Quantile Regression technique RR Rating Ratio T I D (mm/hr) RFFA ROI C Rainfall intensity of D-hour duration and T-year average recurrence interval Regional Flood Frequency Analysis Region of Influence Runoff Coefficient used in rational method Sden Stream Density (km/ km ) Slope Slope of central 75% of mainstream S1085 (m/km) SEP Standard error of prediction USGS United States Geological Survey VIC Victoria WLSR Weighted Least Squares Regression xxviii

29 CHAPTER 1 CHAPTER 1: INTRODUCTION 1.1 GENERAL This thesis focuses on design flood estimation problem in ungauged catchments in the range of frequent to rare average recurrence intervals (ARIs) ( to 000 years) using regional flood frequency analysis (RFFA) approaches. The RFFA attempts to transfer flood characteristics information from gauged to ungauged catchments using the concept of homogeneous regions. This thesis, in particular, investigates the research question how flood quantile estimation in ungauged catchments can be enhanced by adopting an ensemble of advanced statistical techniques. The RFFA approaches developed in this thesis attempt to minimise errors in design flood estimation through a stringent data preparation scheme, use of sophisticated statistical techniques and an in-depth validation of the techniques applied. This chapter provides the background, need and objectives of this research and an overview of the thesis. 1. BACKGROUND The flood phenomenon is a part of the natural disturbance regime, and an intrinsic component of the natural climate system. It can also be one of the most destructive hydrometeorological phenomena in terms of its impacts on human well-being and socioeconomic activities. One just has to see the considerable damage caused by flooding that has taken place in the north-eastern state of Australia (namely Queensland) in 010, 011 and 01 to really understand the significance of this issue. The death toll is estimated to be and the people affected by the flooding reached.1 million. The damage caused is estimated to be around $0 billion. Other losses that are worth noting are the disruption in trade such as the loss in agriculture and mining, which are both important revenue incomes for Queensland and Australia. Figure 1 shows the damage caused by flash flooding in Emerald Central Queensland in January

30 CHAPTER 1 Figure 1 Flash flooding in Emerald Central Queensland (Oncirculation, 011) To estimate the frequency and magnitude of floods for design purposes, the availability of streamflow data is a fundamental requirement. Flood frequency analysis is often used by practitioners to support the design of river engineering works, flood mitigation measures and civil protection strategies. It is generally carried out by fitting peak flow observations to a suitable probability distribution (Baratti et al., 01). The estimation of probability of exceedance for frequent to rare floods is essentially an extrapolation exercise based on limited observed flood data. Thus the larger the database the more accurate the estimate should be. From a statistical point of view, estimation from a small sample is likely to give unreasonable or physically unrealistic parameter sets, especially for the probability distributions with a large number of parameters (i.e. three or more). In practice, however, recorded flood data may be quite limited. In many cases, these data may be completely absent (i.e. ungauged catchment cases). In such situations, RFFA is adopted. The RFFA serves two purposes, for sites where streamflow data are not available the analysis is based on regional data (Cunnane, 1989). For sites with available data, the joint use of data measured at a site, called at-site data, and regional data from a number of stations in the region provides sufficient information to enable a probability distribution to be used with greater confidence (Dawdy et al., 01). This type of analysis represents a

31 CHAPTER 1 substitution of space for time where data from different locations in a region are used to compensate for short records at a single site (Stedinger et al., 1993). RFFA consists of three major steps: (a) identification of homogeneous regions; (b) development of estimation models; and (c) validation of the estimation models. To form the homogeneous regions, traditional approaches such as geographical and administrative regions have often been adopted (I. E. Aust., 1987; Acreman and Sinclair, 1986; Acreman, 1987; Tasker et al., 1996 and Eng et al., 005); however, these regions often lack in hydrological similarity (Burn, 1990a and 1990b; Hosking and Wallis, 1993; Merz and Blöschl, 005 and Chebana and Ouarda, 008). Regions based on climatic and physical catchment characteristics have been proposed (Tasker et al. 1996; Bates et al., 1998 and Rahman et al., 1999a). Moreover, to avoid problems associated with fixed boundaries, the region of influence (ROI) approach has been adopted (Burn, 1990a and 1990b; Tasker et al., 1996; Zringi and Burn, 1996; Merz and Blöschl, 005; Eng et al., 007a, b and Gaál et al., 008). One critical issue here is how to assign an ungauged catchment to the appropriate region when there is more than one possible region (Bates et al., 1998). In relation to the estimation model, several approaches have been proposed. They include the probabilistic rational method (PRM), the index flood method (IFM) and the quantile regression technique (QRT). In south east Australia, the PRM is recommended for general use in Australian Rainfall and Runoff (ARR) mainly due to its simplistic nature (I. E. Aust., 1987). The essential component of this method is a dimensionless runoff coefficient, which ARR assumes to vary smoothly over geographical space. This assumption may not be satisfied in many cases, because two nearby catchments can exhibit quite different physical features. Also, values associated with these runoff coefficients are estimated using conventional moment estimates with flow records of limited length (some sites had only 10 years of record in the analysis with the ARR1987 RFFA methods). This means that these runoff coefficient values are affected by severe sampling variability, which can then introduce significant bias and uncertainty into the final design flood estimates. Criticism has also been linked to the way the runoff coefficients are mapped; this can be attributed to the assumption of geographical contiguity as a surrogate to hydrological similarity, an assumption open to wide criticism. It is also worth mentioning the lack of independent validation with the PRM in ARR

32 CHAPTER 1 The IFM or index frequency approach (being applicable to both flood and rainfall estimation) (e.g. Fill and Stedinger, 1998; Madsen et al., 00; Bocchiola, et al., 003; DiBaldassarre et al., 006 and Lim and Voeller, 009) has been a popular approach for estimating flood quantiles since 1960 (Dalrymple, 1960). ARR (I. E Aust., 1987) did not favour the IFM as a design flood estimation technique for Australia. The IFM had been criticised on the grounds that the coefficient of variation of the flood series may vary inversely approximately with catchment area, thus resulting in flatter flood frequency curves for larger catchments. In the United Kingdom (UK), an index flood method is currently recommended in the Flood Estimation Handbook (FEH) where the index flood is taken as the median annual maximum flood. The growth curve for any site is estimated using a pooling group, which is formed using catchments considered to be hydrologically similar to the site of interest. The FEH recommends the generalised logisitic (GLO) distribution combined with the method of L-moments for growth curve estimation. The FEH RFFA approach was upgraded again by Kjeldsen et al. (008) and as documented in a series of papers (e.g. Kjeldsen and Jones, 009a, 009b and 010). The United States Geological Survey (USGS) proposed a QRT where a large number of gauged catchments are selected from a region and flood quantiles are estimated from recorded streamflow data, which are then regressed against catchment variables that are most likely to govern the flood generation process (Benson, 196). It has been noted that the method can give design flood estimates that do not vary smoothly with ARI; however, hydrological judgment can be exercised in situations such as these where flood frequency curves can be adjusted to increase smoothly with ARI. As an alternative to the QRT, the parameters of a probability distribution can be regressed against the explanatory variables (Tasker and Stedinger, 1989; Madsen et al., 00; Reis et al., 005; Overeem et al., 009). In the case of the LP3 distribution, regression equations can be developed for the first three moments i.e. the mean, standard deviation and skewness for logarithms of the annual maximum flood series. This method here is referred to as the parameter regression technique (PRT). There has been no detailed comparison of the QRT and PRT for ungauged catchments. The ordinary least squares regression (OLSR) estimator has traditionally been used by hydrologists to estimate the regression coefficients () in regional hydrological models (for 4

33 CHAPTER 1 both the QRT and PRT). But in order for the OLSR model to be statistically efficient and robust, the annual maximum flood series in the region must be uncorrelated, all the sites in the region should have equal record length and all estimates of ARI year events have equal variance. Since the annual maximum flow data in a region do not generally satisfy these criteria, the assumption that the model residual errors in OLSR are homoscedastic is violated. Stedinger and Tasker (1985, 1986) developed a generalised least squares regression (GLSR) model for regional hydrologic regression. The important difference in the GLSR from the OLSR models lies in the development and partitioning of the covariance matrix of the errors. The GLSR model of Stedinger and Tasker (1985) assumes that the total error results from two sources: model errors and sampling errors (Tasker and Stedinger, 1989; Pandey and Nguyen, 1999; Griffis and Stedinger, 007; Gruber and Stedinger, 008 and Micevski and Kuczera, 009). This is due to the fact that record lengths vary significantly from site to site and that the flood data are cross correlated spatially. The GLSR procedure can result in notable improvements in the precision with which the coefficients of regional hydrologic regression models can be estimated. Furthermore, Reis et al. (003 and 005) introduced a Bayesian approach to the coefficients estimation for the GLSR (BGLSR) regional regression model developed by Stedinger and Tasker (1985) for hydrological analysis. The results presented in Reis et al. (005) show that for cases in which the model error variance is small compared to sampling error of the at site estimates, which is often the case for regionalisation of a shape parameter, the Bayesian estimator provides a more reasonable and generally less biased estimates of the model error variance than the method of moments and maximum likelihood estimators. The Bayesian approach can also provide a realistic description of the possible values of the model error variance (Reis et al., 005; Micevski and Kuczera, 009; Haddad et al., 01 and Haddad and Rahman, 01). It is advantageous to provide a full posterior distribution of the quantity of interest (flood statistic, e.g. mean flood and flood quantile) which is done by the Bayesian approach as compared to classical methods which usually give a point estimate of the quantity of interest (Congdon, 001). Given the above advantages, the BGLSR is applied in this thesis using both the QRT and PRT estimation techniques. This is carried out in a ROI framework. 5

34 CHAPTER 1 Validation is generally used to assess a model s performance in hydrologic regression analyses (Sun et al., 011 and Tsakiris et al., 011). The validation procedure has some appealing and important properties, for instance, it assists in the selection of an appropriate model according to its prediction ability for the gauged sites, while at the same time it evaluates the prediction ability of the model for possible ungauged catchments. In the most commonly adopted validation approach, a fixed percentage of the data (e.g. 10% or 0%) is left out while building the model, and then the developed model is tested on the left out data (Stone, 1974; Michaelsen, 1987 and Xu et al., 005). This type of split sample validation approach has limitations, as it often provides inadequate validation when the full data set is not used in the validation following a random and unbiased fashion. To make use of all the available sites in the validation using a more efficient and random manner, two validation approaches are tested in this thesis, which are the leave-one-out (LOO) and Monte Carlo Cross validation (MCCV) techniques (Xu and Liang, 001; Xu et al., 005 and Sun et al., 011). Both the LOO and MCCV validation techniques are used with real and simulated flood datasets in the frameworks of the commonly applied OLSR approach and the more powerful GLSR approach. Large to rare flood frequency analysis is a remarkably challenging task. One often needs to estimate floods with an annual exceedance probability (AEP) much smaller than 1%, while the streamflow record lengths are usually much shorter, being between 0 and 100 years in most places in Australia. For example, average record lengths in the Australian RFFA database is around 33 years, as reported in Rahman et al. (009). Hence, it is generally the case that the required flood magnitudes in the rare range have hardly been recorded, meaning significant extrapolations from the available flood data are needed. It is therefore not surprising that a suitable statistical approach is required to estimate large to rare floods with a reasonable degree of consistency. This thesis therefore proposes a new large flood regionalisation model, which also takes into account inter-site dependence (i.e. the number of independent sites (N e ), which reduces the net information available for regional analysis). 6

35 CHAPTER THE NEED FOR THIS RESEARCH Australia is a large continent with many streams; many of which are ungauged or have little recorded flood data. For example, out of the 1 drainage divisions in Australia, seven do not have a single stream with 0 or more years of recorded flood data (Vogel et al., 1993). Therefore, RFFA techniques are quite important for Australia, as they can provide reasonably accurate design flood estimation in these ungauged or poorly gauged catchments. The sizing of minor hydraulic structures such as culverts, farm dam and embankments in small ungauged catchments is a common task faced by practising engineers. The average amount spent on these projects per year was estimated at approximately $50 million as at 1985 (Flavell, 1985; Pilgrim, 1986); this is equivalent to about $750 million per annum in 01 (based on long term CPI series for Australian capital cities, ABS, 01). Australian Rainfall Runoff, Book 4 (I. E. Aust., 1987) states that almost 50% of Australia s annual expenditure on projects requiring design flood estimation is on small to medium sized ungauged catchments. These small catchments typically have an upper limit of 5 km, while medium sized catchments have an upper limit of 1000 km (I. E. Aust., 1987). Given the economic significance, the design flood estimates in small to medium ungauged catchments need to be as accurate as possible, since under and over-estimation is associated with higher flood damage costs and increased construction costs, respectively. Both of these situations are undesirable. There have been many RFFA techniques which have been proposed and used over the years. It is well understood amongst the researchers in hydrology that some of these approaches (such as the PRM in ARR 1987) is not based on hydrologically and statistically meaningful rationale. Most of these methods are likely to introduce significant error in flood quantile estimates. As there are flaws in these empirical approaches, further research is needed to develop more reliable alternatives that can provide more accurate design flood estimates along with estimation uncertainty. Currently within Australia there is no one universally accepted RFFA method that can be applied with confidence; instead, there are many local approaches, which have hardly been 7

36 CHAPTER 1 rigorously validated. For example, ARR (I. E. Aust., 1987) have made recommendations that vary from state to state; these being the PRM, IFM and a variety of other empirical approaches such as the synthetic unit hydrograph and the Main Road s methods for the State of Queensland. Since ARR came out in 1987, there have been some notable advancements in at site and RFFA (see Chapter for more details) in Australia and internationally (e.g. Stedinger and Tasker, 1985 and 1986, Tasker and Stedinger, 1989; Kuczera, 1999a; Bates et al., 1998; Rahman, 005 and Micevski and Kuczera, 009). There have been a number of studies that have dealt with different forms of RFFA and ways of reducing errors in quantile estimates (e.g. Stedinger and Tasker, 1985 and 1986, Tasker et al., 1996; Burn 1990 and 1990b; Zringi and Burn, 1996, Reis et al., 005 and Griffis and Stedinger, 007). These methods essentially deal with ways to increase sample size (by pooling hydrological data), reduce heterogeneity and sampling errors. In Australia, there is also the extra benefit of having over 0 years of additional streamflow data (since the publication of ARR1987) at many gauged sites that can be incorporated into the new RFFA techniques. This is likely to reduce uncertainty in design flood estimates for the ungauged catchments. It can therefore be stated that there is a need for the development and testing of new RFFA methods using the most updated flood data for Australia. This thesis embarks on these tasks, which is likely to form the scientific basis of recommending new RFFA methods in the upcoming revision of the ARR. 1.4 RESEARCH QUESTIONS This thesis, in particular, examines the following research questions in the context of RFFA: 1. How to reduce the uncertainty in flood data to be used in RFFA modelling by employing rigorous data preparation and checking techniques?. How to deal with a high level of regional heterogeneity in RFFA (found by other researchers in Australia)? 3. How to form acceptable regions in Australia where the degree of heterogeneity has been found to be quite high? 8

37 CHAPTER 1 4. Whether regression-based approaches can be adopted in Australia to develop statistically sound regional flood estimation models? 5. Whether the use of sophisticated statistical techniques such as BGLSR combined with ROI can help to reduce the uncertainty in design flood estimates and thereby to form the basis of uncertainty estimation in RFFA? 6. How a more rigorous validation approach (LOO or MCCV) can be applied with RFFA methods? 7. How a new RFFA method can be developed for design flood estimation in the large to rare flood ranges that explicitly accounts for spatial dependence in the annual maximum flood series data and that can also be applied relatively easily in practice. 1.5 MAJOR TASKS The research questions presented in Section 1.4 are answered/ investigated in this thesis by undertaking the following major tasks: (i) Prepare a critical literature review to ascertain the current state of knowledge in RFFA techniques with a focus to identify gaps and limitations in the current research and thereby to formulate research questions to be investigated in this thesis. (ii) Prepare an Australian national flood and catchment database that can be used in the proposed research which mostly satisfies the principal assumptions in RFFA. (iii) Develop the regression based RFFA techniques such as BGLSR-QRT and BGLSR-PRT for the design flood estimation in the small to frequent flood range (ARIs of to 100 years). Also, compare the BGLSR-QRT and BGLSR- PRT methods in the fixed region and ROI frameworks. (iv) Compare two validation approaches, LOO and MCCV, thereby assessing their applicability to RFFA. (v) Develop the LFRM for regional flood estimation in the large to rare flood ranges (ARIs of 100 to 000 years) using a comprehensive Australian dataset. 9

38 CHAPTER 1 (vi) Develop a generalised spatial dependence model to account for the inter-site dependence (also known as spatial dependence) of annual maximum flood series data in the application of the LFRM. Benchmark the developed LFRM using a split-sample validation and by comparing it with the results from alternative RFFA methods. 1.6 CONTRIBUTIONS OF THIS RESEARCH TO THE UNDERSTANDING OF THE RFFA PROBLEM This thesis attempts to make best use of the available streamflow data by developing efficient regional data pooling methods and high-end statistical techniques. This focuses on building a quality-controlled database as well as development of appropriate model validation techniques. It develops a Bayesian GLS regression procedure with ROI approach to tackle the excessive regional heterogeneity and to deliver more efficient parameter estimation techniques of the adopted regional prediction equations. This thesis covers the frequent and rare flood estimation problems ranging from to 1000 years ARI, which can possibly be extended to 000 years ARI. The thesis has made a notable contribution in the regional flood frequency analysis research field as evidenced by the publication of 9 refereed journal papers (two in former ERA ranked A*, two in A, 5 in B category journals). These published papers are listed in Appendix A. 1.7 OUTLINE OF THE THESIS AND CHAPTER INTRODUCTIONS The investigations carried out in this research are presented across 9 chapters, as described below. Chapter 1 gives a brief introduction to the overall study, highlighting the background and need for this research. The research questions to be investigated and major tasks to be undertaken to answer the research questions are also identified. Chapter provides a critical literature review on the various aspects of RFFA. On the onset of this chapter, the basic issues related to assumptions in flood frequency analysis, 10

39 CHAPTER 1 distributional choices, regional homogeneity and spatial dependence are discussed. Different RFFA methods which include the IFM, Station-year approach, Bayesian and Monte Carlo methods and the PRM are presented. The QRT and PRT are discussed in more detail with more emphasis on GLSR and BGLSR. The ROI approach is also critically reviewed in relation to its use in previous applications and its relevance to this study. The second part of this chapter discusses model validation in hydrological regression analysis. A brief history of model validation is presented from a wide range of statistical applications along with previous applications in the hydrological field. Finally, a brief history of large flood estimation is given with examples of some of the methods currently used in Australia and internationally. Overall, this chapter gives a summary of the merits and disadvantages of each approach, thereby laying the foundation for the proposed research. Chapter 3 describes the statistical techniques adopted in this thesis for the estimation of design floods in the small to medium (frequent) ARI range and for the validation of regional hydrological regression models. On the onset of this chapter a flow chart is provided which illustrates the statistical techniques used in the thesis. Estimation of at-site flood frequency is outlined using the LP3 distribution in a Bayesian framework. The classical formulation of the GLSR problem found in the Econometrics field is presented to provide an overview of the method. The chapter then goes on to provide the formulations of the GLSR model by Stedinger and Tasker (1985 and 1986) for use in hydrological regression analysis. The Bayesian methodology is outlined in greater detail for use with the GLSR approach, hence the classical Bayesian formulation is also summarised. The quasi-analytic Bayesian approach as outlined by Reis et al. (005) for the regionalisation of shape parameters is expanded on by developing a BGLSR model for the regionalisation of quantiles and parameters of the LP3 distribution (QRT and PRT, respectively). This methodology includes formulation of the likelihood function, the prior distributions of the β coefficients and the model error variance of the regression model for the QRT and PRT. Setting up the error covariance matrices, which are vital for the solution of the BGLSR equations, are also presented. The steps and formulation involved in selecting the best predictor variables for use with the BGLSR are outlined. The ROI framework is then described in the light of its application for regionalising the parameters and quantiles of the LP3 distribution. All the statistical diagnostics and formulation regarding the residual analysis are also outlined in sufficient detail, along with the 11

40 CHAPTER 1 statistical measures of model performance. Also, a step by step framework for regional uncertainty analysis is presented for obtaining confidence limits with regional flood estimates. In the second part of Chapter 3 the mathematical and statistical techniques related to model validation for use with regional hydrological regression is outlined. Firstly the hydrological regression problem is defined. The formulations regarding the LOO and MCCV validation techniques are derived. Finally, the details regarding the statistical techniques for generating the simulated data for testing with the LOO and MCCV are discussed in detail. The assembly of streamflow data is an important step in any RFFA study. Chapter 4 describes various aspects of streamflow data collation such as selection of the study catchments, filling of gaps in the annual maximum flood data series, testing the data for any suspected trends (as one of the assumptions of flood frequency analysis is that the data must exhibit stationarity and be homogenous), exploring rating curve errors associated with the annual maximum flood data (flood data often has notable error associated with it, hence identification of this is important) and checking for outliers (both low and high outliers may be present in annual maximum flood data, these should be identified and treated accordingly). This chapter also presents the final set of catchments to be used in this thesis. Chapter 4 also covers the selection of the climatic and physical catchment characteristics variables that govern flood generation process and can be used in RFFA models. Chapter 5 integrates the techniques provided in Chapter 3 into a practical BGLSR regional hydrologic regression framework, which is able to address the issues relevant to the estimation of flood quantiles and statistics in an efficient manner. Chapter 5 also presents the results associated with the RFFA for small to medium range ARIs looking at the differences between fixed region and ROI frameworks for both the BGLSR-QRT and PRT methods. The results are illustrated for the states of Tasmania, New South Wales (NSW), Victoria and Queensland. The advantages of the BGLSR-ROI are outlined in sufficient detail. 1

41 CHAPTER 1 Chapter 6 presents the results of the comparison of two model validation techniques, the LOO and MCCV in a hydrological regression framework for the state of NSW. Both the OLSR and GLSR are applied to simulated and real datasets. This chapter also illustrates through detailed examples the overall advantages and disadvantages of the proposed methods for model selection and validation in RFFA. Chapter 7 presents the estimation of floods in the large to rare flood range. The methodology, detailed investigation and results associated with the LFRM are discussed in detial. Chapter 7 begins with a brief discussion on the LFRM concept which is based on the Station-year approach. The issue of inter-site dependence in general is discussed in the light of the application of the LFRM. The chapter also discusses the comprehensive Australian annual maximum dataset used for the analysis. The issues of identification of a probability distribution and homogeneity in the context of LFRM are investigated and discussed. The theory and development of the LFRM is outlined assuming spatial independence initially. This chapter also outlines a methodology for deriving simulated data which is used for estimating the effective number of independent sites, as it was recognised that the observed data had limitations relating to sampling variability and homogeneity issues. Chapter 8 illustrates how the effect of inter-site dependence is tackled by introducing the effective number of sites (N e ) concept. The steps and formulation needed for determining the typical degree of spatial dependence in a network or region is discussed in detail. The estimation of N e is then derived assuming a generalised extreme value (GEV) distribution with a simple model that ignores possible variation with ARI. The results are then discussed and compared in detail for N e for both the real and simulated data sets. Both the derived results helped to establish the behaviour of N e in a network and region for the analysis. The procedure for generalising the spatial dependence is provided along with the comprehensive results from this investigation. As such the LFRM was revisited using the newly developed spatial dependence model and applied to ungauged catchments by developing prediction equations using BGLSR for the mean flood and coefficient of variation of annual maximum floods. 13

42 CHAPTER 1 A summary, conclusion and recommendations for further research are presented in Chapter 9. There are four (4) appendices, as follows. Appendix A presents the refereed journal papers that have been published or that are under review based on the research presented in this thesis. Appendix B presents additional results associated with Chapter 5. Appendix C presents additional results associated with Chapter 7 and 8 while Appendix D provides some extra details on the homogeneity tests used in Chapter 7. 14

43 CHAPTER CHAPTER : REVIEW OF REGIONAL FLOOD FREQUENCY ANALYSIS TECHNIQUES, MODEL VALIDATION AND LARGE FLOODS.1 GENERAL The aim of this chapter is to review previous studies on regional flood frequency analysis (RFFA) techniques with a particular emphasis on the estimation of flood quantiles in the range of average recurrence intervals (ARIs) of 100 years in relation to the quantile and parameter regression techniques. The concepts of fixed region and region of influence (ROI) approaches are discussed and past applications are presented. This chapter also reviews previous studies on the validation of regression models especially in the area of hydrology, with an emphasis in the area of hydrological regression. Finally this chapter also reviews past studies in the area of large to rare flood estimation. Both the advantages and limitations of the methods presented are also outlined. At the beginning, the basic issues on RFFA such as regional homogeneity, inter-site dependence, and distributional choices are reviewed. A brief discussion is then presented on identifying homogenous regions based on annual maximum flood series. The review of RFFA methods as outlined above is then presented. A summary of the findings from this review is given at the end of the chapter.. BASIC ISSUES..1 REGIONAL FLOOD FREQUENCY ANALYSIS The availability of streamflow data is an important aspect in any flood frequency analysis. The estimation of probability of occurrence of floods in the credible limit range (ARIs 100 years) and beyond credible limit (large to rare floods) is an extrapolation based on limited recorded flood data. Thus, the larger the recorded data set, the more accurate the estimates will be. From a statistical view point, estimation from a small sample may give unreasonable or physically unrealistic parameter estimates, especially for distributions with a large number of parameters (three or more). Large variations associated with small sample sizes cause the estimates to be uncertain and biased. In practice, however, data may be limited or in some cases may not be available for a site. In such situations, RFFA is most useful. 15

44 CHAPTER RFFA is a technique of transferring information from gauged suites to ungauged sites. RFFA serves two purposes. For sites where data are not available, the analysis is based on regional data (Cunnane, 1989). For sites with limited data, the joint use of data recorded at a site, called at-site data, and regional data from a number of stations in a region provides sufficient information to enable a probability distribution to be used with greater reliability. This type of analysis represents a substitution of space for time where data from different locations in a region are used to compensate for short records at a single site (National Research Council, 1988; Stedinger et al., 1993)... REGIONAL HOMOGENEITY RFFA is based on the concept of regional homogeneity which assumes that annual maximum flood populations at several sites in a region are similar in statistical characteristics and are not dependant on catchment size (Cunnane, 1989). Although this assumption may not be strictly valid, it is convenient and effective in most applications. One of the simplest RFFA procedures that have been used for a long time is the index flood method (IFM). The key assumption in the IFM is that the distribution of floods at different sites within a region is the same except for a site-specific scale or index flood factor. Homogeneity in regards to the index flood relies on the concept that the standardised regional flood peaks have a common probability distribution with identical parameter values. The identification of homogenous regions is an elementary step in RFFA (Bates et al., 1998). The application typically involves the allocation of an ungauged catchment to an appropriate homogenous group and the prediction of flood quantiles using developed models based on catchment characteristics (Bates et al., 1998). That is, the RFFA based on homogenous regions can transfer the information from similar gauged catchments to ungauged catchments to allow for flood prediction. There have been many techniques developed which attempt to establish homogenous regions. For example the probabilistic rational method (PRM) uses geographical contiguity as an indication of homogeneity that is the catchments which are nearby to each other should have similar runoff coefficients (I. E. Aust., 1987). 16

45 CHAPTER Looking at homogeneity from a theoretical point of view, two catchments annual maximum flood series may be treated as homogenous with respect to flood behaviour if they both satisfy two criteria: the inputs (such as rainfall) to the hydrological systems are identical, and the climatic and physical characteristics changing the input to flood peak are the same. No two catchments can satisfy these criteria perfectly based on the fact that each catchment has unique physical characteristics and that each catchment has different climatic inputs. In the search for practical homogeneity, one has to make decisions on the degree of similarity or dissimilarity that is acceptable to identify a cut-off point where a region is acceptably homogenous or heterogeneous, in consideration of the practical applications of the RFFA techniques. In defining homogenous regions for use in RFFA, a balance has to be made between including more sites for increased information and maintaining an acceptable level of homogeneity. In most situations when more sites are added to a region, certainly more information is gained about the flood regime; however sites that are hydrologically dissimilar can increase the heterogeneity in the region...3 INTER SITE DEPENDENCE Some RFFA methods make use of inter site dependence (see also section.4.1) while others do not. Inter site dependence as reported by Cunnane (1988) states that streamflow data points across a region will show similar behaviour within any given timeframe. This means that; 1) In some years the annual maximum flows at all sites are due to a single widespread meteorological event. ) In relatively dry years, peak flows are generally low over the entire region, in which case all annual maxima will be low. To be able to counteract these trends in RFFA, previous studies have indicated that a concurrent record of sufficient length should be adopted (Stedinger, 1983). Inter-site dependence can be viewed as disadvantageous, as it reduces the value of additional information for regional analysis, i.e. inter-site dependence limits the increase of information from an increase in the number of stations in a region. On the other hand, it is beneficial to the derivation of flood quantiles for ungauged sites, as it allows transfer of 17

46 CHAPTER information from gauged to ungauged sites. The effects of inter-site dependence on large flood estimation are discussed in more detail in Chapter DISTRIBUTIONAL CHOICES Selection of an appropriate probability distribution to be used in flood frequency analysis is of prime importance in at-site and RFFA. It has also been a topic of interest for a long time and one that is filled with controversies (Bobée et al., 1993). Selecting a probability distribution has received widespread attention by many researchers. The recent literature in this field is wide and varied and has been characterised by a proliferation of mathematical models, which lacks in theoretical justification but is applied in a simplistic manner to estimate flood flows. Benson (1968) and NERC (1975) devote considerable attention to this problem. Cunnane (1989) summarised the distributions commonly used in hydrology, mentioning 14 different distributions. Kidson and Richards (005) present an informative summary on the assumptions and alternatives for distributional choices. They cover aspects such as data choice, model choice and alternatives and the inclusion of historical and paleoflood data see (Stedinger and Cohn, 1986; Jin and Stedinger, 1989; Pilon and Adamowski, 1993; Salas et al., 1994; Cohn et al., 1997; Kuczera, 1999; Martins and Stedinger, 001; O Connell et al., 00; and Reis and Stedinger, 005). These studies generally show that the use of historical information can be of great value in the reduction of the uncertainty in flood quantile estimates. In some countries, a common distribution has been recommended to achieve uniformity between different design agencies. The U.S.A. Interagency Advisory Committee on Water Data (IACWD, 198) and the Institution of Engineers Australia (I. E. Aust., 1987) recommend the log Pearson type 3 (LP3) distribution for use in the United States and Australia, respectively. Other distributions that have received considerable attention include the extreme value types 1,, 3 (EV1, or 3), generalised extreme value (GEV) (NERC, 1975), wakeby (Houghton, 1978), generalised pareto (GPA) (Smith, 1987), twocomponent extreme value (Rossi et al., 1984) and the log-logistic distribution (Ahmad et al., 1988). The use of a standard distribution has been criticised by Wallis and Wood (1985) and Potter and Lettenmaier (1990). They argue that a reassessment of the use of the LP3 distribution for practical flood design is overdue. Vogel et al. (1993) studied the suitability 18

47 CHAPTER of a number of distributions (including the LP3) for Australia. They found that the GEV and wakeby distributions provide the best approximation to flood flow data in the regions of Australia that are dominated by rainfall during the winter months; for the remainder of the continent, the GPA and wakeby distributions provide better approximations. For the same data set, the LP3 performed satisfactorily, but not as well as either the GEV or GPA distribution. The distributions that have attracted the most interest as possible alternatives to the LP3 are the GEV and wakeby (Bates, 1994). Studies by Rahman et al. (1999b) and Haddad and Rahman (008) showed that GEV-LH moments method provide better results than the LP3 distribution in South east Australia in particular for New South Wales (NSW) and Victoria. Laio et al. (009) presented a procedure to identify suitable probability distributions for hydrological extremes. The objective of this study was to verify the most appropriate distribution using various goodness-of-fit tests. This study used real (data from the United Kingdom) and simulated data. It was found that no distribution gave the best fit, however the model selection tests were a step forward to identifying the most suitable probability distribution. More recent studies by Haddad and Rahman (011) (Journal paper can be found in Appendix A) compared seven probability distributions (EV1, log normal (LN), normal (NORM), GEV, Pearson type 3 (P3), LP3 and EV) for the state of Tasmania. Using the model selection based on the Aikake information criterion (AIC), Bayesian information criterion (BIC) and the modified Anderson Darling test (AD) as outlined by Laio et al. (009), they showed that the LN distribution with the Bayesian parameter fitting procedure provided more reliable results in terms of bias and standard error than the competing models for Tasmania..3 METHODS FOR IDENTIFICATION OF HOMOGENEOUS REGIONS The methods for obtaining homogenous regions are based on either geographical contiguity or flood characteristics alone or catchment characteristics alone. The theoretical aspects, limitations and associated problems with identification of homogenous regions based on flood data (annual maximum series) are discussed below. In this approach, the degree of homogeneity of a proposed group is judged on the basis of a dimensionless coefficient of the annual maximum flood series, such as the coefficient of variation (CV), coefficient of skewness (CS) or similar measures. Examples are given by 19

48 CHAPTER Dalrymple (1960), Wiltshire (1986a), Acreman and Sinclair (1986), Vogel and Kroll (1989), Chowdhury et al. (1991), Pilon and Adamowski (199), Lu and Stedinger (199), Hosking and Wallis (1993) and Fill and Stedinger (1995a, b). Dalrymple (1960) proposed a homogeneity test based on the sampling distribution of the standardised 10 year annual maximum flow, assuming an EV1 distribution. Wiltshire (1986a, b) presented a test based on the sampling distribution of CV to judge the degree of homogeneity in a region. He tested the efficiency of the proposed test on simulated data and concluded that it is clear that the test in its present form is unsuitable for use in assessing regional homogeneity. Acreman and Sinclair (1986) used a likelihood ratio test based on the assumption of an underlying GEV distribution. Hosking and Wallis (1991, 1993) proposed a heterogeneity measure based on the L moment ratios L coefficient of variation (LCV), L coefficient of skewness (LSK) and L kurtosis (LKT). The advantages of this test are that it is based on L moments and not distribution-specific like those mentioned above. This test has received considerable attention since its inception (e.g. Pearson, 1991; Thomas and Olsen, 199; Alila et al., 199; Guttman, 1993; Zrinji and Burn, 1996; Bates et al.,1998; Rahman et al.,1999b, Kjeldsen and Rosbjerg, 00; Madsen et al., 00; DiBaldassarre et al., 006; Castellarin et al., 007; Chebana and Ouarda, 008 and Gaume et al., 010), Cunnane (1988) mentioned that identification of a homogeneous region is necessarily based on statistical tests of hypothesis, the associated power of which, with currently available amounts of hydrological data, is low. Thus it is not possible to divide, with great assurance, a large number of catchments into homogeneous subgroups using flow records with limited lengths. Indeed from an Australian perspective homogeneity cannot always be satisfied (e.g. Haddad, 008; Haddad and Rahman, 01; Ishak et al., 011 and Rahman, 1997). With the existence of large predictive uncertainty, short record lengths and the heterogeneity that plagues Australian catchments, flood estimation methods that can deal with heterogeneity and predictive uncertainty in an efficient manner are needed. 0

49 CHAPTER.4 REGIONAL FLOOD FREQUENCY ANALYSIS METHODS DIFFERENT APPROACHES There are a number of RFFA methods based on streamflow data that have been reported. Some of the most commonly used methods are discussed below..4.1 INDEX FLOOD METHOD The index flood method (IFM) is a regional frequency approach for transferring flood or rainfall characteristics information from a group of gauged sites to an ungauged site of interest (Dalrymple, 1960; Madsen et al., 00 and Baldassarre et al., 006). The estimation of a flood quantile by the IFM can be expressed by: Q (.1) T Z T Where is the scaling factor and is called the index flood, and Z T is a dimensionless growth factor (or growth curve). In many cases the index flood is taken to be the mean of the annual flood maximum flood series, which is a site specific value; while the growth factor is assumed to be constant for the entire homogenous region under consideration. In the IFM, the dimensionless regional growth curve is used to estimate Z T. The flood quantile having an ARI of T year is then obtained from Equation.1. In the case of a gauged site, the at-site mean flood is used in Equation.1; for an ungauged site, is estimated using regional information. Equation in.1 is based on the following variables: Q T is the flood quantile at a site, with an ARI of T years; Z T is the regional growth factor, which defines the frequency distribution common to all the sites in a homogenous region; and is known as the index flood, which is typically represented (in gauged catchments) by the mean of the at site annual maximum flood series. Being used as a scale parameter, it is recognised as the term which dictates the difference in quantiles between individual sites within the homogenous region. 1

50 CHAPTER When the IFM is to be applied to the ungauged catchment case where there is no data available the difficulty in estimating becomes evident. Estimation such as this is typically performed via multiple regression between the mean annual flood now noted by (Q ) and catchment and climatic characteristics (catchment characteristics) within the region (e.g. Fill and Stedinger, 1998). The general form of this regression equation can be expressed as: Q b c d ab C D... (.) where B, C, D, are catchment characteristics and a, b, c, d, are parameters of the regression equation estimated by either ordinary or generalised least squares regression (OLSR and GLSR) (The GLSR method is discussed in more detail in section.5). The IFM or index frequency approach (being applicable to both flood and rainfall estimation) (e.g. Madsen et al., 00 and DiBaldassarre et al., 006) has been a popular approach for estimating flood quantiles since 1960 (Dalrymple, 1960). The assumption is made that the distribution of floods at different sites within a homogeneous region is the same except for a site-specific scale or index flood factor. Homogeneity with regards to the index flood relies on the concept that the standardised flood peaks from individual sites in the region follow a common probability distribution with identical parameter values. From all the methods to be discussed in this report, the IFM involves the strongest assumptions on homogeneity. Australian Rainfall and Runoff (ARR) (I. E Aust., 1987) did not favour the IFM as a design flood estimation technique for Australia. The IFM had been criticised on the grounds that the coefficient of variation of the flood series may vary approximately inversely with catchment area, thus resulting in flatter flood frequency curves for larger catchments. This had particularly been noticed in the case of humid catchments that differed greatly in size (Dawdy, 1961; Benson, 196; Riggs, 1973; Smith, 199). The IFM further developed in the late 1980 s is a vast improvement to the past methodologies, which uses regional average values of LCV and LSK with the at-site mean to fit a GEV or an alternative distribution (Hosking and Wallis, 1997). Hosking and Wallis

51 CHAPTER (1993) demonstrate that this approach is efficient when the region is relatively homogeneous and record lengths are relatively short. Alternatively, a regional GEV shape parameter can be adopted based upon a regional average (Stedinger and Lu, 1995; Hosking and Wallis, 1997 and Fill and Stedinger, 1998). This approach is more attractive than the typical index frequency method when record lengths and regional heterogeneity increase, but at-site data is sufficient to define the at-site LCV, but not long enough to resolve the shape parameter (LSK). The efficiency of using either the regional value or the at-site estimator clearly depends on the sample size. An obvious and natural solution is to combine the at-site and the regional estimators based on the precision of each estimator. This approach has been proposed before, for instance, Bulletin 17B (IAWCD, 198) recommends that a regional estimate of the shape parameter of the LP3 distribution be combined with the at-site estimate, to obtain a more precise estimator (see for example, Griffis and Stedinger, 004). Similarly, Fill and Stedinger (1998) have proposed such an extension to the original IFM. More recent studies in Australia, (Bates et al., 1998; Rahman et al., 1999a), assigned ungauged catchments to a particular homogenous group identified (through the use of L moments, (Hosking and Wallis, 1993)) on the basis of catchment and climatic characteristics as opposed to geographical proximity. However the deficiencies in this approach were evident in that it needed 1 catchment/climatic descriptors to be used. Therefore its practical use is somewhat limited by its complexity and the time needed to gather the relevant data. On an international scale Fill and Stedinger, 1998 demonstrated that the IFM can provide improved quantile estimation, when different sources of errors are reduced by including explicitly for the varying sampling errors and inter-site correlation from site to site in a region. The use of IFM however in Australia is undermined by the great heterogeneity among Australian catchments and any results obtained would be subject to substantial error. Therefore a method is needed where the assumption of homogeneity may be reduced by capturing the spatial variability from site to site within a region. This provides ground and motivation to explore the quantile regression technique (QRT) for design flood estimation in Australian conditions. 3

52 CHAPTER.4. STATION YEAR METHOD The standardised Q values of all the sites in the region are treated as if they form a single random sample of size n from a common parent population. The pooled standardised data are then fitted to a suitable distribution, and Z T values are calculated. Since this method ignores inter-site dependence, it may lead to greater uncertainty and bias, especially at large return periods (Cunnane, 1988 and Nanadakumar et al., 1997). The issue of inter-site dependence (see section..3) or spatial dependence is an issue that has been receiving a lot of attention in the field of flood and rainfall estimation. The main issues being researched are ways of (i) estimating spatial dependence based on the theory of max-stable spatial processes (e.g. Cooley et al., 006, 010; Vannitsem and Naveau, 007 and Vrac et al., 007 and Reich et al., 01) and (ii) incorporating spatial dependence to estimate the number of independent sites (N e ) in a region (e.g. Buishand, 1984; Hosking and Wallis, 1988; Dales and Reed, 1989; Nandakumar et al., 1997; Stewart et al., 1999; Nanadkumar et al., 000; Guse et al., 009 and Svensson and Jones, 010)..4.3 BAYESIAN ANALYSIS AND MONTE CARLO METHODS Bayesian inference is another alternative to classical estimation methods such as the method of moments and maximum likelihood. In Bayesian inference, the understanding of the likelihood parameters has different values as described by a probability density function (Reis and Stedinger, 005). In Bayesian inference, the information in the data can be represented by the entire likelihood function and also prior knowledge such as a numerical estimate of the degree of belief or a researcher s experience in a hypothesis before evidence has been observed. The method then calculates a numerical estimate of the degree of belief in the hypothesis after evidence has been observed. In flood frequency analysis parameter estimation is made through the posterior distribution which is calculated using Bayes theorem which is the probability that a frequency function P has parameters, given that we have observed the realisations d (defined in our data, any historical information, and limits to be placed on analysis and threshold exceedances). Bayes' theorem is given by Equation.3: P( d / ). P( ) P( / d) (.3) P( d) 4

53 CHAPTER where P( d) is the conditional probability of, given d (it is also called the posterior probability because it is derived from or depends upon the specified value of d) and is the result we are interested in. P() is the prior probability or marginal probability of (`prior' in the sense that it does not take into account any information about d). P(d ) is the conditional probability of d given and it is defined by choosing a distribution and depending on the availability of historical data. P(d) is the marginal probability of d, and acts as a normalising constant. Since complex models cannot be processed in closed form in a Bayesian analysis, namely because of the extreme difficulty in computing the normalisation factor P(d), simulation-based Monte Carlo techniques such as the MCMC approach which include Metropilis-Hasting algorithm are used in this analysis. More details about the Metropolis-Hastings algorithm can be found in Geman and Geman (1984), Casella and George (199), Metropolis et al., (1953) and Hastings (1970). The use of Bayes theorem for combining prior and sample flood information was introduced by Bernier (1967). Perrcchi and Rodriguez-Iturbe (1983) discussed some of the problems associated with Bayesian model choices in hydrology. They also discussed the use of prior information and tentative alternatives for improvements in Bayesian hydrological analysis. Ashkanasy (1985) advocated that the use of Bayesian methods would result in more reliable and credible flood frequency estimates. Bayesian methods in flood frequency analysis have since then been adopted by many researchers (e.g. Wood and Rodriguez-Iturbe 1975; Kuczera, 198a, 1983a, b, 1999; Fortin et al. 1997; Kuczera and Parent, 1998; Reis and Stedinger, 005; Reis et al., 005; Micevski and Kuczera, 009; Haddad et al., 010b, 01 and Haddad and Rahman, 01 the last 3 papers are based on the research in this thesis and can be seen in Appendix A)..4.4 PROBABILISTIC RATIONAL METHOD AS USED IN AUSTRALIA The rational method was introduced by Mulvaney (1851) and has been widely regarded as a deterministic representation of the flood generated from an individual storm. However, the rational method recommended in Australian Rainfall and Runoff (ARR) (I. E. Aust., 1987; Pilgrim and Cordery, 1993), is based on a probabilistic approach for use in estimating design floods. This probabilistic rational method (PRM) is represented by: Q C I, A (.4) T T tc T 5

54 CHAPTER where Q T is the peak flow rate (m 3 /s) for an ARI of T years; C T is runoff coefficient (dimensionless) for ARI of T years; I t c, T is average rainfall intensity (mm/h) for a design duration of time of concentration t c hours and ARI of T years; and A is the catchment area (km ). The method may be regarded as a regional model, with design rainfall intensity I t c, T and catchment area A as independent variables. The runoff coefficient C T is a factor which lumps the effects of climatic and physical characteristics, other than catchment area and rainfall intensity. It is noteworthy that in ARR 1987 the values of C T were estimated using conventional moment estimates from flow records of limited lengths e.g. some sites had only 10 years of records. Since conventional moment estimates are largely affected by sampling variability and extremes in the data, a higher degree of uncertainty in quantile estimation is likely to arise due to C T reported in the ARR The mapping and use of runoff coefficients are based on the assumption of geographical contiguity, an assumption that is unlikely to be satisfied. Pegram (00) and French (00) also discussed the strengths and weaknesses of the PRM. Rahman and Hollerbach (003) investigated the physical significance of runoff coefficients and assessed the extent of uncertainty of design flood estimates obtained by the PRM. By following the method of derivation in ARR, runoff coefficients were estimated for 104 gauged catchments in South east Australia. The mapping of these C 10 coefficients onto a suitable map of the area indicated that C 10 coefficients show little spatial coherence. The C coefficients are mapped according to the position of the gauging station and some interpolation is then required for areas where there is no data so that the contours can be developed. The error introduced into the contours is through the interpolation technique; this is due to the fact that some regions will be exposed to greater spatial changes in physical topography and other factors which directly affects the C 10 coefficients. In a very similar fashion Rahman and Holerbach (003) also stated that while nearby catchments shows similar meteorological characteristics, they may possess quite dissimilar physical characteristics, which clearly indicates that the method of simple linear interpolation over a geographical space on the map of C 10 in ARR (I. E Aust., 1987) has little validity. 6

55 CHAPTER More recently, Rahman et al. 009 and 011a, b (011b given in Appendix A) conducted a study comparing the PRM to the GLSR based QRT using 107 catchments in the state of NSW, Australia. The comparison was undertaken using a leave-one-out and split-sample validation approach examining specific features of each RFFA method. The conclusions that were drawn from this study were that the QRT-GLSR outperformed the PRM based on a range of evaluation statistics. Importantly it was found that the PRM and QRT-GLSR did not perform poorly for the smaller catchments used in the study. Overall, the QRT-GLSR was advantageous over the PRM in that no assumptions are needed regarding runoff coefficients as with the PRM. The QRT-GLSR also explicitly differentiates between sampling and model error thus allowing flexibility for further uncertainty analysis, whereas the PRM lacks scope for further development..5 QUANTILE AND PARAMETER REGRESSION TECHNIQUES.5.1 INTRODUCTION The United States Geological Survey (USGS) proposed a QRT where a large number of gauged catchments are selected from a region and flood quantiles are estimated from recorded streamflow data, which are then regressed against catchment variables that are most likely to govern the flood generation process. Studies by Benson (196) suggested that T-year flood peak discharges could be estimated directly using catchment characteristics data by multiple regression analysis. The QRT can be expressed as follows: b c d Q ab C D... (.5) T Where B, C, D, are catchment characteristics variables and with T year ARI (flood quantile), and a, b, c, are regression coefficients. Q T is the flood magnitude It has been noted the method can give design flood estimates that do not vary smoothly with ARI; however, hydrological judgment can be exercised in situations such as these when flood frequency curves can be adjusted to increase smoothly with T. There have been various techniques and many applications of regression models that have been adopted for hydrological regression. Most of these methods are derived from the methodology set out by the USGS as described above. 7

56 CHAPTER As an alternative to the QRT, the parameters of a probability distribution can be regressed against the explanatory variables (Tasker and Stedinger, 1989; Madsen et al., 00). In the case of the LP3 distribution, regression equations can be developed for the first three moments i.e. the mean, standard deviation and skewness of the logarithms of annual maximum flood series. For an ungauged catchment, these equations can then be used to predict the mean, standard deviation and skewness to fit an LP3 distribution. This method here is referred to as parameter regression technique (PRT). However, there has been little research on the applicability of the PRT as compared to the QRT in RFFA. Regionalising the parameters of a probability distribution (which is referred to as PRT in this study also offers three significant advantages over the QRT: 1. It ensures flood quantiles increase smoothly with increasing ARI, an outcome that may not always be achieved with the QRT. The flood quantiles obtained from the PRT may also be used to determine whether the flood quantiles derived from the QRT provides similar and consistent results.. It is straightforward to combine any at-site flood information with regional estimates using the approach described by Micevski and Kuczera (009) to produce more accurate quantile estimates; and 3. It permits quantiles to be estimated for any ARI within the limits of the developed RFFA method. Cunnane (1988) also reviewed methods that use regional information in the estimation of hydrologic statistics. One versatile approach employs regional information to derive a relationship between streamflow statistics and catchment characteristics using regional regression analysis such as the QRT or PRT. Such regional regression methods have been widely used to estimate hydrologic statistics at ungauged sites (Benson and Matalas, 1967; Matalas and Gilroy, 1968; Thomas and Benson, 1970; Moss and Karlinger, 1974; Jennings et al., 1993), and to increase the precision of the statistic of interest at sites with short record lengths by adding regional information (Kuczera, 198a; Stedinger, 1983; Madsen and Rosbjerg, 1997; Fill and Stedinger, 1998; Martins and Stedinger, 000; Reis and Stedinger, 003). Regional regression models such as the QRT or PRT aim to explain spatial variability of the hydrologic statistic by relating it to catchment variables, such as catchment area, mainstream slope, mean annual rainfall and percentage of forest cover. 8

57 CHAPTER The OLSR estimator has traditionally been used by hydrologists to estimate the regression coefficients in regional hydrological models. But in order for the OLSR model to be statistically efficient and robust, the annual maximum flood series in the region must be uncorrelated, all the sites in the region should have equal record length and all estimates of T year events have equal variance. Since the annual maximum flow data in a region do not generally satisfy these criteria, the assumption that the model residual errors in OLSR are homoscedastic is violated and the OLSR approach can provide very distorted estimates of the model s predictive precision (model error) and the precision with which the regression model parameters are being estimated (Stedinger and Tasker, 1985). To overcome the above problems in OLSR, Stedinger and Tasker (1985) proposed the GLSR procedure which can result in remarkable improvements in the precision with which the parameters of regional hydrologic regression models can be estimated, in particular when the record length varies widely from site to site. In the GLSR model, the assumptions of equal variance of the T year events and zero cross-correlation for concurrent flows are relaxed. The GLSR procedure as described by Stedinger and Tasker (1985) and Tasker and Stedinger (1989) require an estimate of the covariance matrix of residual errors ˆ ( Y ) for the hydrologic statistic of interest..5. GENERALISED LEAST SQUARES AND WEIGHTED LEAST SQUARES REGRESSION As discussed above, the coefficients of regional regression models have generally been estimated using the OLSR procedure. However, regionalisation using hydrological data violates the assumption that the residual errors associated with the individual observations are homoscedastic and independently distributed (Stedinger and Tasker, 1985). In the case of hydrological data, variations in streamflow record length and cross correlation among concurrent flows result in estimates of the T year events which vary in precision. Matalas and Benson (1961), Matalas and Gilroy (1968), Hardison (1971), Moss and Karlinger (1974) Tasker and Moss (1979) have examined the statistical properties of the OLSR procedures. 9

58 CHAPTER As shown in the studies cited above, OLSR estimates of the standard error of prediction and the estimated parameters are generally biased under many situations. Weighted and GLSR techniques were developed to deal with situations like those encountered in hydrology where a regression model residuals are heteroscedastic and perhaps cross correlated (Draper and Smith, 1981; Johnston, 197). Tasker (1980) used a weighted least squares regression (WLSR) procedure to account for unequal record lengths. Marin (1983) and Kuczera (198a, b, 1983a) developed an empirical Bayesian methodology, which can deal with these issues as well. An obstacle to the use of WLSR and GLSR procedures with hydrological data is the need to provide an estimate of the covariance matrix of residual errors; this covariance matrix is a function of the precision with which the true model can predict values of the streamflow statistics of concern as well as the sampling error in the available estimates of that statistic. The discussions and examples in the works by Tasker (1980) and Kuczera (1983b) illustrate the difficulties associated with the estimation of this matrix. Stedinger and Tasker (1985), in a Monte Carlo simulation with synthetically generated flow sequences, presented a comparison of the performance of the OLSR procedure with that of the GLSR one. In situations where the available streamflow records at gauged sites are of different and widely varying length and concurrent flows at different sites are crosscorrelated, the GLSR procedure provided more accurate parameter estimates, better estimates of the accuracy with the regression models coefficients being estimated, and almost unbiased estimates of the variance of the underlying regression model residuals. A simpler WLSR procedure neglects the cross correlations among concurrent flows. The WLSR algorithm has been shown to do as well as the GLSR procedure when the cross correlation among concurrent flows are relatively modest..5.3 PREVIOUS APPLICATION OF GENERALISED LEAST SQUARES AND BAYESIAN GENERALISED LEAST SQUARES REGRESSION The GLSR procedure introduced by Stedinger and Tasker (1985 and 1986) has been extensively used nationally and internationally to estimate the coefficients of regional regression models of hydrologic statistics (WMO, 1994; Robson and Reed, 1999). Tasker et al., 1986, 1996, Tasker and Stedinger, 1987; Rosbjerg and Madsen, 1994; Pandey and Nguyen, 1999; Kjeldsen and Rosbjerg 00; Feaster and Tasker, 00; Law and Tasker, 30

59 CHAPTER 003; Griffis and Stedinger, 007; Rosbjerg, 007; Kjeldsen and Jones, 009 and Haddad et al., 011a) have all applied the GLSR for regionalisation of flood quantiles. Madsen and Rosbjerg (1997) employed the GLSR procedure to obtain regional estimates of the parameters (i.e. index and LCV) of a GPA distribution employed as a prior distribution in an empirical Bayesian procedure for flood frequency analysis in New Zealand. Tasker and Driver (1988) developed regression equations using GLSR to predict mean loads for many chemical constituents at unguaged sites. GLSR has also been used as the basis of hydrologic network design (Moss and Tasker, 1991). Griffis and Stedinger (007) looked at the GLSR method in more detail. Previous studies by the US Geological Survey using the LP3 distribution have neglected the impact of uncertainty on the weighted skew on quantile estimation. The needed relationship was developed in this paper and its use was also illustrated in a regional flood study with 16 sites from South Carolina. The results were both accurate and hydrologically reasonable. This paper also looks at new statistical diagnostic metrics such as a condition number to check for multicollinearity, a new pseudo R appropriate for use with GLSR, and two error variance ratios. Micevski and Kuczera (009) presented a general Bayesian approach for inferring the GLSR regional regression model and for pooling with any available site information to obtain more accurate flood quantiles for a particular site in NSW, Australia. Tasker (1989), Vogel and Kroll (1990), Ludwig and Tasker (1993), Kroll and Stedinger (1999) and Hewa et al. (003) have used GLSR for regionalisation of low-flow statistics. Madsen et al. (1995), Madsen et al. (00 and 009) employed the GLSR procedure in the regional analysis of extreme rainfall in Denmark, while Overeem et al. (009) used a GLSR procedure to establish the correlation structure and infer uncertainty between parameters of the GEV distribution for extreme rainfalls in the Netherlands. Haddad et al. (011a) presented a GLSR procedure that regionalises the parameters of the GEV distribution for design rainfall estimation in Australia. Further examples are given below that address regional models of the log-space skewness coefficient, standard deviation and mean. The current methodology for flood frequency analysis in Australia and the United States consists of fitting a LP3 distribution to the gauged data by estimating the mean, standard deviation, and skew of the logarithms of the annual maximum flows. The problem is that the at-site skewness (shape parameter) 31

60 CHAPTER estimator is highly variable with typical record lengths often found in Australian data (average record length of 33 years, Rahman et al., 011). In order to improve the precision of the estimator and to reduce uncertainty, Bulletin 17B recommends combining the at-site estimator with a regional estimate of the skew coefficient (IACWD, 198; McCuen, 1979; Tung and Mays, 1981a, b; and McCuen and Hromadka, 1988). Tasker and Stedinger (1986) applied WLSR to derive a generalized skewness estimator for the Illinois River basin. They were unable to use GLSR because they did not know how to describe the correlations among skewness estimators. Martins and Stedinger (00a) have developed simple equations for the cross-correlation among skewness (and shape parameter of GEV and GPA distributions) estimators as a function of the cross-correlation of the flood flows themselves. Martins and Stedinger (00a) employed those equations to implement a GLSR model for regional skew estimation. Reis et al. (003 and 005) introduced a Bayesian approach to parameter estimation for the GLSR regional regression model developed by Stedinger and Tasker (1985) for hydrological analysis. The results presented in Reis et al, (005) show that for cases in which the model error variance is small compared to sampling error of the at site estimates, which is often the case for regionalisation of a shape parameter, the Bayesian estimator provides a more reasonable estimate of the model error variance than the method of moments (MOM) and maximum likelihood (ML) estimators. This paper also presented regression statistics for WLSR and GLSR models including pseudo analysis of variance, a pseudo R, error variance ratio (EVR) and variance inflation ratio (VIR), leverage and influence. The regression procedure was illustrated with two examples of regionalisation. Results obtained from OLSR, WLSR and GLSR procedures using the Bayesian and MOM model error variance estimators were compared. Gruber et al. (007) and Gruber and Stedinger (008) further develop the Bayesian GLSR (BGLSR) framework first presented by Reis et al. (005). This operational regression methodology is used in the estimation of regional shape parameters, as well as flood quantiles. The focus of this study was to also implement the Bayesian GLSR framework in conjunction with diagnostic statistics presented by Tasker and Stedinger (1989), Reis et al. (005), Reis (005) and Griffis and Stendinger, (007). The new diagnostics statistics for use with Bayesian GLSR provided comprehensive examination of the developed regression models. More recently, there have also been some further developments in the BGLSR area for log-space skew estimation for the non desert regions of California (Parrett et al., 010). Lamontagne et al. (011) and 3

61 CHAPTER Veilleux et al. (011) also used BGLSR for the estimation of log-space skews for annual maximum day rainfall flood volumes in the Central Valley and surrounding areas of California..6 FIXED REGIONS AND THE REGION OF INFLUENCE IN REGIONAL FLOOD FREQUENCY ANANALYS.6.1 FORMATION OF REGIONS In regional flood frequency analysis, regions have often been defined based on state/political boundaries. In ARR 1987, regional flood estimation methods were developed for various Australian states based on fixed regions. The problem with this type of fixed regions is that at state/regional boundaries, two different methods can provide quite different flood estimates. To avoid this problem, regions have also been identified in catchment characteristics data space using cluster analysis (Acreman and Sinclair, 1986), Andrews curves (Nathan and McMahon, 1990) and various other multivariate statistical techniques. One limitation with this type of region is that a correct method of assigning an ungauged catchment to a homogeneous region needs to be formulated, which is often problematic. If the ungauged catchment is assigned to the wrong region/group, the resulting flood estimation is associated with a high degree of error..6. REGION OF INFLUENCE VS FLEXIBLE REGION Since hydrological characteristics do not change abruptly across state boundaries, it is desirable to avoid fixed boundaries. Regionalisation without fixed regions was performed by Acreman and Wiltshire (1987) and Acreman (1987), and based on their work; the region of influence (ROI) approach was introduced by Burn (1990a, 1990b) where each site of interest (i.e. catchment where flood quantiles are to be estimated) has its own region. This way the defined regions may overlap and gauged sites can be part of more than one ROI for different sites of interest. The great advantage of the ROI approach is that it is not bounded by geographic regions often based on political boundaries such as state lines, and it thus avoids discontinuities at the boundaries of regions. The ROI for the site of interest is formed out of stations in close proximity, with proximity measured using a weighted Euclidean distance in an M-dimensional attribute space. The distance metric is defined by: 33

62 CHAPTER D M 1 m m i, j Wm X i X j m1 (.6) with D i,j as the weighted Euclidean distance between site i and j, M is the number of attributes included in the distance measure, and the X terms denote standardised values for attribute m at site i and site j, and W m is a weight applied to attribute m reflecting the relative importance of the attribute. Standardisation of attributes removes units and avoids introduction of bias due to scaling differences of the attributes. In a range of studies (Burn, 1990a; Zrinji and Burn, 1996; Tasker et al., 1996; Merz and Blöschl, 005; Eng et al., 005 and Eng et al., 007a) the attributes were standardised by the standard deviation over the entire dataset of attribute m. Attributes can arise from two sources, either based on physical features, such as catchment area, stream length, channel slope, stream density, or soil type, or statistical measures of climate and flow data, such as the coefficient of variation. Since the inception of the ROI procedure, it has been found the ROI can result in improved flood quantile estimates in terms of root mean square error and that ROI offers the flexibility of variable regions (Zringi and Burn, 1996). They went on further to refine the initial ROI approach into a hierarchal ROI approach. The hierarchical ROI approach was found to perform better for the estimation of higher order moments (i.e. skewness), this is the case where more sites are needed to form a region. It was found in this study that the hierarchical ROI approach improved flood estimates in the extreme range. Tasker et al. (1996) compared five different methods for developing regional regression models to estimate flood quantiles at ungauged sites in Arkansas, United States. The methods looked at traditional flood estimation regression approaches, multivariate techniques of cluster and discriminant analysis and a ROI approach based on geographical and catchment attribute space where the n gauging sites with the smallest distance made up the ROI for site i. The study concluded that the ROI approach (based on catchment attributes space) outperformed the other methods based on the lowest root mean square error. Eng et al. (005) used different ROI approaches for estimating the 50 years ARI flood quantile at ungauged sites in a case study for the Gulf Atlantic Rolling Plains of the southeastern United States. OLSR was used to regress flood statistics against catchment 34

63 CHAPTER characteristics for each ungauged site based on data from ROI containing the n closest gauging sites in both geographical (GROI) and catchment attributes space (CROI). Model performance was based on the prediction errors from independent testing. From this testing, it was shown for the two ROI approaches using the n closest gauging sites (based on geographical distance) was better than using a distance measure in catchment attributes space. They also found that GROI produced lower errors than CROI. Merz and Blöschl (005) examined the predictive performance of several flood regionalisation methods. They performed the assessment using a jackknife comparison of at-site estimated regionalised flood quantiles for 575 Austrian catchments. The ROI methods that only used catchment attributes performed relatively poorer to the methods that used geographical proximity. The ROI used in this study was then combined with multiple regression. Merz and Blöschl (005) were able to demonstrate that when spatial dependency was incorporated, the ROI showed less random errors. Eng et al. (007a) proposed a hybrid ROI (HROI) which combined the GROI and CROI in a GLSR framework. They applied this method to 1,091 catchments in the southeastern part of the United States to estimate the 50 years ARI flood quantile. Their study was able to show that the HROI yielded smaller root mean square estimation errors while also producing fewer extreme errors often found in either GROI or CROI. From this study it was concluded that for the 50 years ARI flood quantile, the similarity with respect to catchment attributes was important, however it was incomplete and that the consideration of the geographical proximity of the sites provided a useful surrogate for characteristics that were not included in the analysis. Eng et al. (007b) went on to also present an enhanced GLSR and ROI framework that is based on a leverage-guided ROI. This procedure used two newly defined ROI leverage and influence metrics. They applied their method to 996 catchments in the southeastern part of the United States. This new leverageguided ROI regression provided improvements in terms of lower root mean square errors while also eliminating all the influential observations. Gaál et al. (008) also presented a number of different regional approaches to regional frequency analysis utilising L-moments and the GEV distribution with the main focus on the ROI approach for modelling heavy rainfall amounts in Slovakia. This study used various pooling schemes using different alternatives of sites similarity (pooling groups 35

64 CHAPTER defined according to climatological characteristics and geographical proximity of sites, respectively) and pooled weighting factors. The performance of the ROI methods presented with at-site and other conventional regional methods was assessed through Monte Carlo simulation studies for rainfall annual maximum series for the 1 and 5 day durations. The results showed that all the frequency models based on the ROI produced growth curves that were superior to at-site and conventional regional estimates for most of the sites studied. The National Committee on Water Engineering intends to test the applicability of the Bayesian GLSR method for Australian catchments which may form the basis of the revision of the regional flood frequency methods in ARR (Project 5 Regional Flood Methods). While both the ROI and GLSR have been applied before in a QRT framework (see Eng et al., 007a,b), there has been no comprehensive comparison between ROI and fixed regions in a BGLSR framework. Moreover, there has been no solid comparison between the estimation of quantiles and the parameters of probability distributions in a ROI framework. This thesis as stated above uses the Bayesian approach to the analysis of a GLSR model for hydrologic statistics (Reis et al., 005). This relatively new approach is expanded on to allow computation of the posterior distributions of the parameters and quantiles of the LP3 distribution and the model error variance using a quasi-analytic approach. The Bayesian approach (Reis et al., 005) provides both a measure of the precision of the model error variance that the traditional GLSR lacks and a more reasonable description of the possible values of the model error variance in cases where the model error variance is smaller compared to the sampling errors. The ROI method used in this thesis improves on the current ROI approaches (e.g. Tasker et al., 1996) where the minimisation of the regression models predictive error variance rather than selecting or assuming a fixed number of sites to minimise a distance metric is sought. More details regarding the application of this method are provided in Chapter 3..7 MODEL VALIDATION IN HYDROLOGICAL REGRESSION ANALYSIS In multiple linear regression analysis, it is to be resolved which set of the predictor variables is the best suited or the most optimal for inclusion in the regression equation 36

65 CHAPTER without over fitting the model and which of the many candidate models is the most parsimonious one for making the most reliable prediction for the ungauged catchment case; i.e. addition of unnecessary predictor variables often leads to weaker models (e.g. producing greater uncertainty). Validation is generally used to assess a model s performance in hydrologic regression analysis. In the validation approach, a fixed percentage of the data (e.g. 10%, 0%) is left out while building the model, and then the developed model is tested on the left out data, which is not used in the model building (i.e. validation data set). The validation procedure has some appealing and important properties, e.g. it assists to select an appropriate model according to its prediction ability, while at the same time evaluating the prediction ability of the model for ungauged catchments..7.1 HISTORY OF MODEL VALIDATION During the last twenty years, the application of different validation methods has been widely used in different fields of sciences such as Chemometrics (Faber and Kowalski, 1997 and Song Xu et al., 005) and Econometrics (Racine, 000), examples include selection of a model in both univariate and multivariate calibrations using real and simulated data sets. Song Xu and Zeng Liang (001) and Song Xu et al. (005) provided a detailed study of leave-one-out (LOO) vs. Monte Carlo cross validation (MCCV) in multivariate calibration and quantitative structure-activity relationship research. The history of validation methods was summarised by Stone (1974) and Michaelsen (1987). Mosteller and Tuckey (1977) presented a good introduction of validation methods also. Efron (1983) and Bunke and Droge (1984) described the statistical behaviour of different validation methods. In classical statistical literature, validation is most often referred to as LOO. In LOO, one data point is left out while building a regression model (or other form of model) and then the model is tested on the previously left out data point. The procedure is repeated until all the data points are independently tested. Efron (1986) showed that LOO is not very efficient in estimating prediction error. Marter and Martern (001) pointed out that LOO often results in over fitting and provides underestimation of the true prediction error of the model. An asymptotically consistent method selects the best prediction model with probability one as the sample size tends to infinity (n). With this definition, LOO has a smaller chance of selecting the right model; that is, the probability becomes much smaller than one (see Shao, 1993). In hydrologic regression, often a large number of predictor 37

66 CHAPTER variables (e.g. catchment area, mean annual rainfall, design rainfall intensity, fraction forest, soil indices, elevation and slope) are available; here LOO is likely to include unnecessary predictor variables in the model (Shao, 1993). In such situations, the selected model tends to perform well in calibration but quite poorly during prediction. MCCV, a form of model validation, was first introduced by Picard and Cook (1984). Shao (1993) proved that this method is asymptotically consistent and has a greater chance than LOO of selecting the best model with more accurate prediction ability. The MCCV leaves out a notable part of the sample at a time during model building and validation and repeats the procedure many times. When compared with the ordinary methods for selecting the best predictor variables (i.e. stepwise regression and employing statistics such as Mallows C p or P-value hypothesis) MCCV may be more desirable as it evaluates the different models according to their predictive ability using many different combinations of validation data sets. Interestingly, MCCV has not been tested in hydrologic regression analysis where one often deals with a very limited and scarce observed data set..7. PREVIOUS APPLICATIONS OF LEAVE-ONE-OUT VALIDATION IN HYDROLOGY The LOO has often been used in hydrology mainly due to limited sample size of hydrological data; therefore to make the best use of the available data, a LOO has often been adopted. There is an abundance of literature in regards to LOO in hydrological applications; we present a few below on the various applications. LOO has found popularity in the application of estimating rainfall statistics, low-flow indices quantiles and flood quantile estimation. For example, Brath et al. (003) investigated the statistical properties of the rainfall extremes in northern central Italy, the reliability of the estimates to ungauged sites was assessed by using a LOO validation. Di Baldassarre et al. (006) used Monte Carlo experiments and LOO validation in the estimation of uncertainty for design rainfalls for ungauged sites in northern central Italy as well. Sun et al. (011) used a LOO validation for model evaluation in predicting monthly rainfall in the Daqing Mountains in northern China. The regionalisation of low flows has gained popularity recently and is of great importance in hydrological studies, it is also a critical issue for the PUB initiative (i.e. Prediction in Ungauged Basins of the International Association of Hydrological Sciences IAHS; e.g., 38

67 CHAPTER Sivapalan et al., 003). Castiglioni et al. (009) presented the estimation of low flow indices in ungauged catchments in Italy by applying deterministic and geostatistical techniques for interpolating low flow indices in physiographical space. A LOO validation procedure was adopted to quantify the accuracy of each technique when it is applied to ungauged catchments. Through the LOO validation the conclusion was drawn that the geostatistical techniques outperformed the deterministic ones. In Austria, Laaha and Blöschl (007) presented a national low flow estimation procedure for the whole country for both gauged and ungauged catchments. In each step of the estimation procedure, many alternative methods were tested by LOO validation. This led to the identification of the best performing method for low flow estimation in Austria. Canonical correlation analysis (CCA) was used in the estimation of low-flows in Greece as shown by Tsakiris et al. (011). Tsakiris et al. (011) also used LOO validation to conclude whether CCA could reliably assist in catchment classification into sub regions and also whether partitioning the region into two sub regions offered improvements of low flow quantile estimates through multiple linear regression. Many studies in the literature can also be found regarding the use of LOO for validation of RFFA models (for example see Sankarasubramanian and Lall, 003; Merz and Blöschl, 005; Juraj and Ouarda, 007; Chowdury and Sharma, 009; Kjeldsen, 010; Iacobellis et al., 011). Merz and Bloschl (005) examined the predictive performance of several flood regionalisation methods for 575 Austrian catchments using a LOO comparison. Regional flood-rainfall duration-frequency modelling at small ungauged sites was undertaken in south-western Ontario, Canada by Juraj and Ouarda (007). Model performance was evaluated by a LOO procedure using evaluation statistics such as average bias and relative root-mean square error. Chowdhury and Sharma (009) applied a similar validation technique which essentially resembled the LOO to predict and forecast arid river flows in Australia. In the UnitedKingdom (UK), Kjeldsen (010) used LOO in modelling the impacts of urbanisation on flood frequency relationships. The LOO showed that the developed adjustment factors were generally better at predicting the effects of urbanisation on the flood frequency curve than the adjustment factors currently used in the United Kingdom. It can be seen through the literature review that little attention has been paid to-date to the application of MCCV in RFFA and hydrological applications and the examination of the 39

68 CHAPTER possible benefits that could be gained from the application of this procedure. Hence, this thesis looks at three main issues as a part of its broad objective: (1) Demonstrating the application of MCCV method in hydrological regression analysis using both the OLSR and GLSR; () Comparison of the MCCV with the most commonly applied LOO validation for selecting the most parsimonious regression model to be applied for ungauged catchments; and (3) Demonstration of the best use of the limited datasets (often encountered in hydrology) which can hinder the detailed validation of hydrological regression models..8 REGIONAL FLOOD FREQUENCY FOR LARGE TO RARE FLOODS.8.1 BRIEF REVIEW OF LARGE FLOOD ESTIMATION AND PREVIOIUS APPLICATIONS Estimation of large to rare and even extreme return periods is of absolute importance for hydrological design and risk assessment for large infrastructure. The term large floods refer to floods with 50 to 100 years ARIs (Nathan and Weinmann, 001). Floods in the range from 100 years ARI to the credible limit of extrapolation (ARI in the order of 000 years) are referred to as rare floods, while floods from the credible limit of extrapolation to the PMF are termed extreme floods. Due to knowledge and data limitation and the uncertainty involved in extrapolating beyond available data, the errors in final estimates can be quite high. The average size of recorded flood data referring to Australian small to medium sized catchments is about 33 years (Rahman et al., 009). To make better use of this information and to be able to transfer this information to ungauged catchments again regional estimation methods are used as described in section.4. Some studies both in the past and present, on an international scale have looked at the advantages and disadvantages of different regional models for large, rare and extreme floods, (Ferrari et al., 1993; Kundzewicz et al., 1993; Katz et al., 00; Castellarin, 007; Castellarin et al., 007; Vogel et al., 007; Van Gelder et al., 007; Moisello, 007; Majone et al, 007; El Aldouni, 008; Laio et al., 009; Castellarin, 009; Calenda et al., 009 and Gaume et al., 010). In Australia, the issue of large to extreme flood estimation in the past has been addressed by some researchers (e.g. Pilgrim, 1986; Rowbottom et al., 1986; Pilgrim and Rowbottom, 1987; Stedinger et al., 1993; Nathan and Weinmann, 001 and Haddad et al., 010). Book VI of Australian Rainfall and Runoff (ARR) was upgraded in 1999 with guidance for estimation of large to probable maximum floods (PMF). The procedures outlined in 40

69 CHAPTER ARR1999 include flood frequency analysis and various rainfall-based methods. For flood frequency estimates in the range of rare floods, use of regional information plus paleohydrological information was suggested and for rainfall-based methods, an annual exceedance probability (AEP) neutral approach was recommended (Nathan and Weinmann, 001). The statistics of extremes have played an important role in engineering practice for water resources design and management. There have been recent developments in statistical theory of extreme values that can be applied to improve the rigour of these flood estimates and to make the estimates more physically meaningful. The development of more rigorous statistical methodology for regional analysis of large to rare floods as well as the extensions in Bayesian methods can help to improve and quantify uncertainty in the estimation procedure. Although the fundamental probabilistic theory of extreme values has been well developed for a long time(leadbetter et al., 1983; Coles, 001; Cooley et al., 006; Cooley et al., 010) the statistical modelling of large to rare floods remains a subject of active research. Probability weighted moments (PWM) or L-moments are popular in application to large and more extreme events hydrology than the ML approach (Katz et al., 00). L- moments possess computation simplicity and have very good performance in small samples (Hosking, 1990 and Hosking et al. 1985). Regional analysis is another way of making use of more available information that originated with estimation of large to rare flood hydrology in mind (Dalrymple, 1960 and Hosking et al., 1985; Jothityangkoon and Sivapalan, 003; Castellarin et al., 005; Castellarin et al., 007; Douglas and Vogel, 006; Vogel et al., 007; and Gaume et al., 010). The basic idea is that if a region is relatively homogenous then the estimation of large to rare flood quantiles at a given site may be improved by using the larger observations at other sites as well (i.e., a trade off between space and time). Castellarin et al (005) introduced an estimator of the exceedance probability associated with a regional envelope curve (REC) for extreme flood estimation, which accounts for the impact of intersite correlation of annual floods. Douglas and Vogel (006) provided an probabilistic behaviour interpretation of floods of record in the United States (U.S.) for use with REC. Castellarin (007) and Castellarin et al. (007) apply the probabilistic and multivariate probabilistic REC) to real data in Italy for extreme flood estimation. They documented that the multivariate extension outperforms the ordinary REC and provide flood quantile 41

70 CHAPTER estimation at ungauged sites that are nearly as reliable as index flood quantiles. Vogel et al. (007) went onto enhance the method presented by Castellarin et al (005) and Castellarin et al. (007) by introducing a general expression for the exceedance probability of an envelope curve. A case study was implemented using historical flood series from 6 sites located across the U.S. The results overall indicated that the approach introduced by Vogel et al. (007) offers significant promise for the estimation of large to extreme floods with envelope curves for heterogeneous regions. More recently Gaume et al. (010) proposed approach based on standard regionalisation index methods for extreme floods in Slovakia and the south of France. They created larger data samples by using historical, paleoflood or extreme floods occurring in ungauged catchments to reduce the uncertainties on high return period quantiles in a region. It is well known that regionalisation models based on the index flood method assume some sort of homogeneity in its application. In particular, it is assumed that the probability distribution of the standardised variable obtained by normalisation of the annual maximum flows with respect to the average of the population is the same in all the catchment sites inside the homogeneous region. As a matter of fact the sample values of these moments (coefficient of variation (CV) and the coefficient of skewness, if the analysis is limited to second and third order moments) can vary in a very wide range. Hence since there is a high variability associated with these parameters of a probability distribution the size of error in the derived quantile estimates could be very large. Recently, a new probabilistic model (PM) has been introduced (Majone and Tomirotti, 004; Majone et al., 007 and Haddad et al., 011b) specifically for this sort of analysis. Majone and Tomirotti (004) originally calibrated the PM for Italian rivers, and extended the method using 7300 historical series of annual maximum flows observed at gauging stations belonging to different geographical areas around the world. Majone et al. (007) applied the PM to flood data from 8,500 gauging stations across the world and found that the method can provide quite reasonable design flood estimates for ARIs in the range of 4,000-9,000 years. In a study conducted by Haddad et al. (011b) on a data set containing 7 gauging sites from Australia (Victoria and NSW), it was found the PM model when coupled with GLSR performs very well when applied to ungauged catchments and can estimate the year ARIs floods with reasonable accuracy. 4

71 CHAPTER The PM (Majone and Tomirotti, 004; Majone et al., 007; Haddad et al., 011b) is based on the assumption that the standardised maximum values (Q max ) of the annual maximum flood series from a large number of individual sites in a region can be pooled (considering the across-sites variations in the mean and CV values of annual maximum floods). The concept is similar to the Cooperative Research Centre for Catchment Hydrology Focussed Rainfall Growth Estimation (CRC-FORGE) method (Nandakumar et al., 1997) where extreme design rainfall estimates are based on pooled rainfall data from a large region up to several hundred gauges (concept of expanding region). The particular advantages of the Probabilistic Model is that it does not assume a constant CV across the sites as with the index frequency approach; this feature, in particular, allows the PM to pool data more efficiently over a very large region. The PM termed large flood regionalisation model (LFRM) for large to rare floods as described by Majone et al. (007) and Haddad et al. (011b) is chosen for further study as another objective of this thesis. This method as discussed above is an empirical approach that makes use of pooled data from various sites while taking into account differences in means and the varying CV from site to site. This unique form of standardisation allows the pooling of more data from many stations. As compared to standard methods, the application of LFRM can overcome many of the difficulties, limitations and assumptions of large to rare flood frequency analysis. The main focus of the LFRM in this thesis is expanded to couple it with a spatial dependence model (such as the CRC-FORGE constant spatial dependence model (e.g. Buishand, 1984; Hosking and Wallis, 1988; Dales and Reed, 1989; Nandakumar et al., 1997; Stewart et al., 1999; Nanadkumar et al., 000; Castellarin et al. 007; Vannitsem and Naveau, 007; Guse et al., 009) that reflects the reduction in the net information available for regional analysis using spatially dependant data (see also sections..3 and.4.). The LFRM is also to be extended to the ungauged catchment case by coupling it with the BGLSR method and the ROI to estimate the mean and CV of annual floods at sites where there is no or little data. The advantages of the BGLSR and ROI methods have been already discussed in sections.5 and.6. 43

72 CHAPTER.9 IMPACT OF CLIMATE CHANGE ON FLOOD FREQUENCY ANALYSIS In the literature, it has been noted that the frequency and magnitude of extreme flood events are likely to rise in the near future due to climate change (IPCC, 007; BOM, 01). This may have implications to typical flood frequency analysis that assumes that the stationarity assumption is valid (Khaliq et al., 006). Researchers in non-stationary flood frequency analysis in different parts of the world have questioned the validity of the traditional flood risk assumptions of stationarity (e.g. Franks and Kuczera, 00; Cunderlik and Burn, 003; Prudhomme et al., 003; Micevski et al., 006; Leclerc and Ouarda, 007; Pui et al., 011 and Pall et al., 011). There have been number of studies on the identification of trends in flood data. For example, Olsen et al. (1999) have reported positive trends in flood risk over time for gauged sites within the Mississippi, Missouri, and Illinois River basins. Douglas et al. (000) discovered no evidence of trends in flood flows but they did find evidence of upward trends in low flows at larger scale in the Midwest and at a smaller scale in Ohio, the north central and the upper Midwest regions in USA. Negative trends in total streamflow were most common for the analysed Pennsylvanian streamflow time series from 1971 to 001 due to climate variability (Zhu and Day, 005). Novotny & Stefan (007) investigated the streamflow records from 36 gauging stations in five major river basins of Minnesota, USA, for trend and correlations using the Mann-Kendall (MK) test and moving averages method. They found that trends diffsignificantly from one river basin to another, and became more prominent for shorter time windows. Pasquini and Depetris (007) presented an overview of flood discharge trends of South American rivers draining the southern Atlantic seaboard. Juckem et al. (008) found a decrease in annual flood peaks for stream gauging stations in the Driftless Area of Wisconsin. Ishak et al. (013) found that about 15% of Australian stream gauging stations showed a trend, mainly downward in eastern and south-eastern and south-westen parsts of Australia and upward in northern Australia. It should be noted here that this study excluded these 15% stations from the thesis as this focuses on stationary regional flood frequency analysis. However, the outcome of study by Ishak et al (013), part of the PhD thesis of Elias Ishak (another UWS PhD student), will be used to develop an adjustment factor to correct for the nonstationarity in regional flood frequency analysis. 44

73 CHAPTER.10 SUMMARY The estimation of flood behaviour (both in terms of credible limit and beyond credible limit of extrapolation) at ungauged catchments is a common problem in hydrology. Regional flood frequency analysis (RFFA) is commonly used to transfer flood characteristics information from gauged catchments to ungauged ones. In this chapter, the literature review has covered a range of currently applied RFFA techniques with particular emphasis to the quantile and parameter regression techniques (QRT and PRT). The index flood method (IFM) has been discussed; which assumes the probability distribution of floods at sites of homogenous regions is identical except for a site specific scaling factor. Recent studies have shown positive results based on L-moments based IFM in South-east Australia. However due to the large heterogeneity in Australian catchments a method that does not strictly require homogeneous regions is more suitable for Australia. The probabilistic rational method (PRM) is currently recommended in South east Australia for design flood estimation in small to medium sized ungauged catchments. Though considered a regional method and easy to apply it has been criticised by researchers because of the assumption of geographical contiguity in the mapping and application of the runoff coefficients. The QRT and PRT are multiple regression techniques which relate flood quantiles or the parameters of a probability distribution (i.e. location, scale and shape which are related to the mean, standard deviation and skew of the flood data) to catchment characteristics assuming linearity. The advantage of both the QRT and PRT is that no assumptions are made about runoff coefficients or geographical contiguity, or strict homogenous regions as with the PRM and IFM, respectively. The preferred methodology of the QRT and PRT is to use the generalised least squares regression (GLSR) and in particular the Bayesian GLSR (BGLSR) approach as further improvement in generalisations can be made with this method such as accounting for sampling variability and cross correlated data and more importantly distinguishing between model error and sampling error in the regional model. Furthermore, the Bayesian approach 45

74 CHAPTER provides both a measure of the precision of the model error variance that the traditional GLSR lacks and a more reasonable description of the possible values of the model error variance in cases where the model error variance is smaller compared to the sampling errors. The concept of fixed regions and region of influence (ROI) approach in RFFA has also been discussed. Both the advantages and disadvantages have been outlined. The past studies presented have all showed the improvements of the ROI over a fixed region approach. Keeping this in mind along with the high heterogeneity in Australian catchments it makes sense to combine and compare the methods of the QRT and PRT under a BGLSR framework with the ROI approach. Model validation is a very important part of RFFA especially in the area of hydrological regression. The concept of model validation using split-sample, leave-one-out validation (LOO) and Monte Carlo cross validation (MCCV) has been discussed. Past studies of the application of LOO in hydrology have been presented, while studies relating to the use of MCCV in other fields of science have also been discussed. Given the lack of application of MCCV in hydrological regression, this thesis will compare both LOO and MCCV for RFFA model validation in the state of New South Wales in Australia. Finally, estimation of large to rare and even extreme return periods is of absolute importance for hydrological design and risk assessment for large infrastructure. The statistical modelling of large to rare floods remains a subject of active research. A brief history of large flood estimation has been given in this chapter along with recent studies and applications in this field. This thesis will present a new large flood regionalisation model (LFRM) that pools data more efficiently over a very large region. The LFRM will be combined with a newly developed spatial dependence model that reflects the reduction in the net information available for regional analysis using spatially dependant data. 46

75 CHAPTER 3 CHAPTER 3: ADOPTED STATISTICAL TECHNIQUES FOR REGIONAL FLOOD FREQUENCY ANALYSIS AND MODEL VALIDATION 3.1 GENERAL This chapter provides an overall description of the statistical techniques adopted in this study for (i) regional flood frequency analysis (RFFA) in the range of 100 years average recurrence intervals (ARIs) and (ii) Validation of regional hydrological regression models using leave-one-out (LOO) and Monte Carlo cross validation (MCCV) techniques. At the outset, a flow chart (Figure ) is provided which summaries the statistical procedures and methodologies adopted in this thesis. At the beginning, log Pearson type 3 (LP3) distribution is described which is fitted to the observed annual maximum flood series data using a Bayesian parameter estimation procedure. A discussion is then be presented on the quantile and parameter regression techniques (QRT and PRT), while the basic theory has been introduced in Chapter, further emphasis is given here on the generalised least squares regression (GLSR), in particular, the Bayesian GLSR (BGLSR). The region of influence (ROI) approach is then discussed in the light of its application with the BGLSR. Here the application of the ROI is based on the minimisation of the predictive variance, which is applied with both the QRT and PRT regression techniques. The methodology outlined here is intended to highlight the assumptions involved and to give an overview of how to deal with the various uncertainty associated with the data to obtain reliable flood estimates. The next part of this chapter discusses the mathematical formulations used in the model validation. The theory behind LOO and MCCV is presented (as outlined in Song Xu et al., 005), with an emphasis on the hydrologic regression analysis using ordinary least squares regression (OLSR) and GLSR-based QRT.. 47

76 CHAPTER 3 Major steps in the research Data collation Streamflow data -Filling missing data, trend analysis, rating curve error analysis, outlier testing - Catchment and climatic characteristics data At site flood frequency analysis (LP3 distribution) Regional flood frequency analysis (ARIs 100 years) Comparison of Bayesian GLSR using QRT and PRT in a fixed region framework Comparison and validation of methods using data from NSW, VIC, QLD and TAS Comparison of Bayesian GLSR using QRT and PRT in a region of influence framework Application of LOO and MCCV for the validation of regional regression models Case study for NSW using OLSR and GLSR Large flood regionalisation model (LFRM) development using data from all Australia Collation of streamflow data, Homogeneity testing, Finding appropriate distribution LFRM and spatial dependence model development Ungauged catchment application At site flood frequency analysis (GEV distribution) Conclusions and Recommendations Figure Flow chart showing statistical techniques/ methods adopted in this thesis 48

77 CHAPTER 3 3. AT-SITE FLOOD FREQUENCY ANALYSIS 3..1 BASICS OF AT-SITE FLOOD FREQUENCY ANALYSIS At site flood frequency analysis is an elementary step in any RFFA study. The primary objective of flood frequency analysis is to relate the magnitude of extreme events to its frequency of occurrence through the use of probability distributions (Chow et al., 1988). Data observed over an extended period of time in a river system are analysed using frequency analysis techniques. The data for flood frequency analysis are assumed to be independent and identically distributed. The flood data are considered to be stochastic and space and time independent. Furthermore, it is assumed that the flood data have not been affected by natural or manmade changes in the hydrological regime and climate change (stationarity assumption). In flood frequency analysis, a unique relationship between a flood magnitude and the corresponding ARI T is sought. The task as stated is to extract information from a flow record to estimate the relationship between Q and T. Three different models may be considered for this purpose (Cunnane, 1989). These models are (1) annual maximum series, () partial duration series or peaks over threshold series, and (3) time series model. For this study, annual maximum series flood data is adopted. Australian Rainfall and Runoff (ARR) (I. E. Aust, 1987) recommends the LP3 distribution fitted with the method of moments (MOM) for use in at-site flood frequency analysis. However, research has shown that the reassessment of the LP3 distribution/mom estimation is overdue (Wallis and Wood, 1985; Vogel et al., 1993). The recommendations currently being prepared by the National Committee on Water Engineering include a variation to the current MOM to Bayesian fitting procedures to estimate the parameters of the probability distributions used in at-site flood frequency analysis (Kuczera and Franks, 005). Hence, this method is adopted in this study to estimate the at-site flood quantiles. The LP3 Bayesian procedure has shown satisfactory results in the study area as demonstrated by Haddad and Rahman (008) and Rahman et al. (011). 49

78 CHAPTER FLIKE SOFTWARE FOR AT-SITE FFA The at-site flood quantiles are estimated by FLIKE, which is a computer program developed by Professor George Kuczera of the University of Newcastle. The FLIKE program facilitates Bayesian analysis and the method of L-moments for parameter estimation. The following section briefly describes the LP3 probability distribution. Kuczera (1999b) presents how FLIKE obtains initial parameter values when searching for the most probable values LOG PEARSON TYPE 3 (LP3) DISTRIBUTION The LP3 probability model has the following probability distribution function (pdf): f(log x α,, ) e Γ( ) (log x ) α 1 exp (log x ) e for 0, x ; for e 0, x ; α 0 (3.1) with ( being the gamma function. The LP3 model has been widely accepted in practice because it consistently fits flood data as well if not better than other probability models. When the skew of log e x is zero, the model simplifies to the log normal. The model, however, is not well-behaved from an inference perspective. Direct inference of the shape parameter the scale parameter and the location parameter causes numerical problems. For example, when the skew of log e x is close to zero, the shape parameter tends to infinity. Experience indicates that it is preferable to fit the first three moments (, and ) of log e x rather than and (Kuczera, 1999b). This parameterisation is based on the mean (), standard deviation (), and skewness () which is often used to calculate the T-year event quantile: Q T K ( ) (3.) T where K ( ) is the frequency factor which is the T-year quantile of the Pearson type 3 (P3) T distribution with mean zero and standard deviation of 1, and skewness. The frequency 50

79 factor CHAPTER 3 KT can be approximated with sufficient accuracy by the Wilson-Hilferty transformation (Kirby, 197 and Rao and Hamed, 000) for < : 3 K T z 1 1 (3.3) 6 6 where z is the T-year quantile of the standard normal distribution. Problems however may arise when the skew of log e x is negative, the upper bound on flows can cause problems. FLIKE avoids this problem by starting the search for the most probable parameters using log normal MOM parameters fitted to the flood data. This strategy is quite robust because when the skew of log e x is zero, the flow bounds are pushed all the way to infinity. As a result, the search starts in a region of parameter space well removed from the constraints imposed by the flow bounds. Furthermore, a serious problem arises when the absolute value of the skew of log e x exceeds ; that is, when 1 and when < 1, the LP3 has a gamma-shaped density. However, when 1, the density changes to a J-shaped function. Indeed when = 1, the pdf degenerates to that of an exponential distribution with scale parameter and location parameter. For 1, the J-shaped density seems to be over parameterised with three parameters. The posterior density surface reveals extremely elongated contours which are suggestive of an over parameterised model. In such circumstances, it is pointless to use the LP3 distribution. Under this circumstances, it is suggested either the generalised extreme value (GEV) or generalised pareto (GPA) distributions be used as a substitute (Kuczera 1999b). In this study, no prior information was used in fitting the LP3 distribution. The parameters of the LP3 distribution (i.e. mean, standard deviation and skewness) were also extracted from the FLIKE software for use with the RFFA. 3.3 THE CLASSICAL GLS REGRESSION PROBLEM This section focuses on the basic generalised least squares regression (GLSR) model and discusses the classical assumptions for this procedure. Subsequent sections recast the analysis of the GLSR model in a Bayesian framework following Reis (005) and Reis et al. (005). Streamflow data, be it annual maximum or partial duration series data sets, can be used to derive an empirical relationship between catchment/climatic characteristics 51

80 CHAPTER 3 variables and the hydrologic statistic of interest. For instance, catchment area and design rainfall intensity may be used to estimate hydrologic characteristics at a site, such as the mean annual flow, the 10-year or 100-year peak flow, or the shape parameter of a theoretical probability distribution like the log-space skewness coefficient () used to fit a LP3 distribution, or the shape parameter () of a GEV or GPA distribution. The GLSR model assumes that the quantity of interest y i at a given site i can be described by a linear function of catchment/climatic characteristics (or a transformation there of) with an additive error. In matrix notation, the model is represented by: y Xβ (3.4) where X is a (n k) matrix of catchment characteristics augmented by a column of ones, is a (n 1) vector of regression parameters that must be estimated and is an (n 1) vector of random errors for each of the n sites used in the regression assumed to be normally distributed with zero mean and the covariance matrix of the form: E( T ) Ω (3.5) wherein is the model error variance and is a positive definite symmetric matrix (Johnston, 197; Rencher, 000; Koop, 005). Different choices of the matrix allow one to make different assumptions regarding the nature of the model errors. If Ω is equal to the identity matrix I, the problem is homoscedastic, and the GLSR model reduces to OLSR. Uncorrelated errors with different variances at different sites can be described using a matrix with different variances of the diagonal and zero off the diagonal. In this case, the GLSR model in Equation (3.5) reduces to weighted least squares regression (WLSR) model. In the more general case, is defined in such a way that it reflects both heteroscedasticity and correlation among residuals. According to the Gauss-Markov-Aitken theorem, where is known, the minimum variance unbiased estimator for does not depend on and is given by (Rao and Toutenburg, 1999 and Koop, 005): T 1 1 T 1 ( X Ω X) X Ω y (3.6) ˆGLS 5

81 CHAPTER 3 The equation above is defined as the GLSR estimator denoted by ˆ GLS. Note that the subscript GLS is sometimes omitted for brevity. The unbiased estimate of is given by: ˆ T 1 ( ) ( ˆ y Xβ GLS Ω y Xβ ) ˆ GLS GLS (3.7) n k 1 with sampling covariance matrix: var( ˆ 1 1 ) ( X T GLS Ω X) (3.8) The GLSR estimator is also the best linear unbiased (BLUE) estimator in all classes of linear estimators. Since the matrix is unknown, we need to use an estimator, which is usually denoted by ˆ GLSR, THE STEDINGER AND TASKER MODEL Stedinger and Tasker (1985, 1986) developed a GLSR model for regional hydrologic regression. The important difference from the OLSR and the classical GLSR model of the form given by Equation (3.4) lies in the development and partition of the covariance matrix of the errors. The GLSR model of Stedinger and Tasker (1985) assumes that the total error results from two sources: model errors i that are assumed to be independently distributed with mean zero, 0 i E and common variance: Cov, i 0 i j i j (3.9) j and sampling errors that arise due to the fact the actual values of y i are unknown and only estimates of the quantities of interest are available. Therefore, Equation (3.4) becomes (following Reis et al., 005): ˆ (3.10) y Xβ η Xβ where is the sampling error in the sample estimators. Thus, the regression-model errors i are a combination of: (i) time-sampling-error in the sample estimators ŷ i of y i and (ii) 53

82 CHAPTER 3 underlying model error i (lack of fit). The total error has mean zero and covariance matrix: T Λ I yˆ E (3.11) where Σ( ŷ ) is the covariance matrix of the sampling errors in the sample estimators (such as the flood quantiles or the parameters of the LP3 distribution see Equation (3.)). Timesampling errors in estimators of the y i s are usually correlated among sites because flows at nearby sites have similar hydrological mechanisms (e.g. meteorology). Reasonably accurate estimation of the sampling covariance matrix in the GLSR is very important and is of great concern and is vital to the solution of the GLSR equations. More details about the construction of Σ( ŷ ) for flood quantiles and statistics are given in section 3.5 and can be can be read in Stedinger and Tasker (1985 and 1986). In this regional framework can be viewed as a heterogeneity measure. Madsen et al. (1997 and 00) showed that the regional average GLSR estimator is a general extension of the record-length-weighted average commonly applied in the index flood method; however the record-length-weighted average estimator neglects inter-site correlation and regional heterogeneity (Stedinger et al., 1993 and Stedinger and Lu, 1995). The GLSR estimator of β is given by: ˆ 1 GLS δ δ T 1 1 T ( X Λ( ) X) X Λ( ) yˆ (3.1) The sampling covariance matrix thus becomes: ˆ T 1 1 ˆ ( X ( ) X) var( ˆ GLS ) GLS (3.13) The model error variance δ is due to an imperfect model and is a measure of the precision of the true regression model. Unfortunately the model error is not known and needs to be estimated. Stedinger and Tasker (1986) proposed a MOM estimator where δ can be solved iteratively solving Equation (3.1) along with generalized residual mean square error (MSE) equation given by Equation (3.14): 54

83 CHAPTER 3 ( yˆ Xβˆ GLS T ) [ ˆ I ( yˆ)] 1 ( yˆ Xβˆ GLS ) n ( k 1) (3.14) In some situations, the sampling covariance matrix explains all the variability observed in the data, which means that the left-hand side of Equation (3.14) will be less than n (k + 1) even if ˆ is zero. In these circumstances, the MOM estimator of the model error variance is generally taken to be zero (Stedinger and Tasker, 1985; 1986). Alternative methods for estimating the model error variance by maximum likelihood estimation (ML) can be seen in Kuczera (1983a) and Rencher (000). Based on Monte Carlo simulations, Stedinger and Tasker (1986) showed that the MOM model error variance ˆ procedure provides faster and more robust results since no assumptions are made about the distribution of the residuals, and less biased when the true model error variance is moderate to large (usually the case for flood quantiles and mean flood estimation). Stedinger and Tasker (1986) also from their simulation study using various cross-correlations among concurrent flows (0, 0.3 and 0.6) showed that for small ˆ MLs were much more accurate. Actually, the ML estimator for always had a smaller MSE than the MOM estimator. If the regional regression analysis exhibits a small model error variance ˆ, i.e. this is the case when sampling errors dominate the regional analysis (e.g. with the regionalisation of the shape parameters of probability distributions i.e. skewness estimators), the ML procedure should be preferred to the MOM estimator. Bayesian analysis, which is based on the likelihood function, is also a good candidate for these situations, and would address the bias concern because on average over the prior, Bayesian estimators are unbiased (Stedinger, 1983). 3.4 BAYESIAN METHODOLOGY ˆ Reis et al. (005) developed a Bayesian approach to estimate the regional model parameters and showed that the Bayesian approach can provide a realistic description of the possible values of the model error variance, especially in the case where sampling error tends to dominate the model errors in the regional analysis (Madsen et al., 00; Reis et al., 005 and Haddad et al., 01). This thesis extends the work of Reis et al. (005) and applies the BGLSR to estimate the parameters and flood quantiles of the LP3 distribution 55

84 CHAPTER 3 see Equation (3.)). The BGLSR is chosen as the desired framework as the current GLSR model analysis methodology based on Tasker and Stedinger (1989) and Griffis and Stedinger (007) do not provide an estimate of the uncertainty in the estimated model error variance of the flood quantiles and the first two moments of the LP3 distribution CLASSICAL BAYESIAN INFERENCE In a Bayesian framework, the parameters of the model are considered to be random variables, whose pdf should be estimated. The Bayesian approach combines any data with prior information (if available) about the parameters being estimated (see also section.4.3). This information usually is established from other data sets, previous studies or specific knowledge about the behavior of the system being analysed. Parameter estimation is made through the posterior distribution which is developed using Bayes rule: (Zellner, 1991): p( I ) ( ) p( I) (3.15) p( I ) ( ) d Here, p( I) is the posterior distribution of the parameter vector, p ( I ) is the likelihood function for the data, and ( ) is the prior distribution of. The denominator is a normalising constant that ensures that the area under the posterior pdf equals one. Reis et al. (005) developed a Bayesian approach to estimate the regional model coefficient of the log-space skewness and showed that the Bayesian approach can provide a realistic description of the possible values of the model error variance. It is advantageous to provide a full posterior distribution of the parameters which is done by the Bayesian approach as compared to classical methods which usually give a point estimate of the parameters. 3.5 BAYESIAN GLS REGRESSION APPROACH ADOPTED IN THIS STUDY FOR THE QUANTILE AND PARAMETER REGRESSION TECHNIQUES To regionalise the flood quantiles (Q T ), the sampling covariance matrix () of the LP3 distribution is required. Tasker and Stedinger (1989) and Griffis and Stedinger (007) (p. 84, Eq. 4) provide the approximate estimator of the components of matrix of the LP3 distribution which is given by: 56

85 CHAPTER 3 ( Q ) i 1 K 0.5K ( for i j T i, j i i i i ) n i mij i j 1 0.5K 0.5K 0.5K K ( 0.75 ) for i j ( QT ) i, j i i j j i j ij i j ij (3.16) n n i j where K is standard LP3 frequency factor, m ij is the concurrent record length between sites i and j, ρ ij is the lag zero cross correlation of flood peaks between sites i and j, and σ i and σ j are the population standard deviation at sites i and j respectively. The skew and standard deviation in the matrix (3.16) are subject to estimation uncertainty as well. In this study to avoid correlation between the residuals and the fitted quantiles, following methods are adopted: (i) the inter site correlation between the concurrent annual maximum flood series (ρ ij ) is estimated as a function of the distance between sites i and j; (ii) the standard deviations (of the logarithms of annual maximum flood series) σ i and σ j are estimated using a separate OLSR and GLSR using the explanatory variables used in the study (given in Chapter 4); and (iii) the regional skew (of the logarithms of annual maximum flood series) is used in place of the population skew as suggested by Tasker and Stedinger (1989). This analysis above used the regional estimates of the standard deviation and skew obtained from the BGLSR. The detailed information on the covariance matrices associated with the standard deviation and skew can be found in Reis et al. (005) and Griffis and Stedinger (007). Here we provide an overview of the covariance matrices. It is necessary to carry out GLSR on the sample standard deviation and skew, because both these parameters have an associated estimation error, the approximation in Equation 3.16 should be updated to reflect all the uncertainty associated with the sampling error in the quantile estimates. The needed estimator of the sample covariance matrix for the standard deviation and skew is given below in Equations 3.17 through to 3.0: ( s) ( s), i 0.75 for i j i, j 0.5 (1 i ) i j 0.5 ( ij 0.75 i j ) ijmij i j n n i n j i for i j (3.17) 57

86 CHAPTER 3 The off-diagonal elements of the sampling covariance matrix for the skew coefficient include the term Cov[g i, g j ] which is the covariance between the two at-site skew estimators g i and g j. This term is obtained from: Cov [ g, g ] Var[ g ]Var[ g ] (3.18) i j g g i j i j where the cross-correlation g i g is estimated using the approximation developed by Martins j and Stedinger (00a): ˆ Sign( ˆ ) cf ˆ (3.19) g g i j ij ij ij wherein cfij mij / ( mij ni )( mij n j ), mij is the common record period and n i, n j are the extra observation period for station i and j, respectively. Values of are tabulated by Martins and Stedinger (00a) for 1.0. In addition, Var[g i ] and Var[g j ] are evaluated using the following approximation derived by Griffis and Stedinger, (007): 6 Var[ gi] a( ni ) ni 9 1 b( ni ) 6 i 15 4 c( ni ) i 48 (3.0) wherein a n ), b n ) and c n ) are corrections for small samples: ( i ( i ( i a( ni ) n n 0.3 i i 0.59 i 0.6 i 1.18 i 3 i b( ni ) n n n 0.9 i c( ni ) n n n 1.77 i (3.1) The regional skew G i (Equation 3.) is used in Equation 3.0 in place of the population skew i to avoid correlation between the residuals and the at-site estimates of the skew. Wherein G i 6 1 ni 3 n ( xi x) i 1 (3.) 3 ( n 1)( n ) s i n i 58

87 CHAPTER 3 where x t is the logarithm of the annual maximum flows in the year t, and s is the sample standard deviation of x t. Because the true values of skews at each site are unknown, the regional mean of the skews is used in Equation 3.0. For the parameter regression technique (PRT), GLSR is also adopted (Tasker and Stedinger, 1989 and Griffis and Stedinger, 007) using a Bayesian framework (Reis et al., 005) to develop regression equations for the parameters of the LP3 distribution (i.e. mean, standard deviation, and skew coefficient of the logarithms of the annual maximum flood series (i.e.,, ). The regional values of standard deviation and skew were found based on Equations 3.18 to 3.. The sampling covariance matrix for the mean flood () was obtained following Stedinger and Tasker (1986) which is given below: ( q) i, j i n i for i j ijmij i j ( q) i, j for i j (3.3) n n i j 3.5. ADOPTED BAYESIAN REGRESSION APPROACH PRIOR FOR THE β COEFFICIENTS As discussed in section 3.4.1, in order to apply the Bayesian analysis to the regional regression problem in this study, one needs to define prior distributions for the coefficients and for the model error variance. With the Bayesian approach, it is assumed here that there is no prior information on any of the β coefficients; thus, a multivariate normal distribution with mean zero and a large variance (e.g. greater than 100) is used as a prior for the regression coefficients as suggested by Reis et al. (005). This prior is considered to be almost non-informative, which produces a pdf that is generally flat in the region of interest. A multivariate normal distribution prior is given by: 1/ P T ( ) exp 0.5( ) ( ) ( k 1) / p p (3.4) ( ) 59

88 CHAPTER 3 wherein β has dimension k + 1 and Equation 3.4 is modelled with a mean vector p and precision matrix P. Zellner (1971) also suggests that the prior can be represented by the reciprocal of the variance. Zellner (1971) and Congdon (001) also suggest that a parameter gamma distribution can be used to represent the prior information. The likelihood function for the data as suggested by Reis et al. (005) is considered to be a multivariate normal distribution, so that: n / 1 T 1 exp 0.5( y Xβ) ( y Xβ) L(, ) (3.5) 1/ where the covariance matrix is defined in Equation 3.11, n is the number of sites in the region, y is the vector with the sample values of the hydrologic statistic of interest (i.e. mean flood, flood quantile etc, and X is the matrix of explanatory variables (catchment area, design rainfall intensity etc) ANALYTICAL SOLUTION TO BAYESIAN APPROACH FOR THE POSTERIOR OF THE MODEL ERROR VARIANCE To compute the normalising constant in Equation 3.15 it is often useful to use Markov Chain Monte Carlo (MCMC) algorithms such as the Metropolis-Hastings or Gibbs sampler algorithms (e.g. Kuczera and Parent, 1998; Micevski and Kuczera, 009 and Reis et al., 003). These algorithms are usually adopted for computationally intense problems, which really depend on the dimension or complexity of the model being analysed. Given that the dimension of this problem is relatively straight forward, it can be solved more easily using the quasi-analytical approximation of the marginal posterior of the model error variance as discussed by Kitanidis (1986) and Reis et al. (003 and 005). Below a brief overview of equations and steps involved are outlined. In more simple cases, it is possible to integrate the joint posterior of and over the possible values of to obtain numerically the marginal posterior of normalising constant, and hence: except for the f ( I) f ( I, ) (, ) d (3.6) where 60

89 f ( I, ) is the likelihood function and (, ) is the joint prior for CHAPTER 3 and. The likelihood function is approximated by a multivariate normal distribution with the covariance matrix (Λ) Equation given by (3.11). n / 1 T 1 exp 0.5( y Xβ) ( y Xβ) L( I, ) (3.7) 1/ When a non-informative prior for is used with a truly non-informative uniform prior for results in the joint prior: (, ) e (3.8)_ The posterior distribution for alone by evaluation can be found: 1/ T 1 f ( I) e exp 0.5( y Xβ) ( y Xβ) d (3.9a) The expression above can be rewritten as: f ( I) e (3.9b) 1/ T exp 0.5 ( y Xβˆ) Λ 1 T ( y Xβˆ) ( ˆ) X T Λ 1 X( ˆ) d Now, the first term inside the brackets is not a function of and hence can be taken outside the integral. The integration then becomes: exp n / T T 1 ( ) 0.5( ˆ) X Λ X( ˆ) d 1/ (3.30) T 1 X Λ X Therefore, the posterior distribution of is proportional to: T 1 1/ T 1 X X exp 0.5( y Xβˆ) ( y Xβˆ) f ( I) e (3.31) The equation as expressed by (3.31) can then be used to calculate numerically the posterior pdf of the model error variance ( ), and its mean and variance without the need of using more sophisticated methods based on Monte Carlo simulation. The pdf of the model error 61

90 CHAPTER 3 variance may also be used to calculate the posterior distribution of the coefficients using: f I f I f I d (3.3) ( ) (, ) ( ) where f (, I) is a multivariate normal distribution. This result turns out to be a simple extension to the GLSR procedure developed in Stedinger and Tasker (1985). If one employs the use of efficient numerical integration procedures, the integral in Equation 3.3, as well as the mean and variance of, are easily computed PRIORS FOR THE PARAMETERS AND THE QUANTILES OF THE LP3 DISTRIBUTION It is well known that no model is perfect, any model that approximates a phenomenon will have error associated with it, hence the model error variance ( ) should be strictly positive. A model error variance of zero in real life is highly unlikely. It is also suspected, based on previous studies, that the model error variance for the regional skew model will be modest, this is especially the case when sampling error dominates the model error (or when the true model error variance is small compared to the sampling errors) (Reis, 005 and Reis et al., 005). For the mean flood, standard deviation and flood quantiles, the model error variance tends to dominate the regional analysis. In this case a zero or negative value for the model error variance is highly unlikely and a strict informative prior may not be required. However it is known that the model error variance in this case may suffer bias if it is estimated by a MOM estimator (Equation 3.14) (Stedinger and Tasker, 1986). This may introduce further uncertainty into the regional model; hence a Bayesian estimator may be attractive also in this case. A Bayesian estimator of the model error variance (Equation 3.31) as discussed above may be used to safeguard against uncertain model error variances, as adopted in this study. Further details can be found in Reis et al. (005) and Micevski and Kuczera (009). In summary, the Bayesian estimator offers a better way of dealing with the model error variance and quantifying associated uncertainty about it. The inverse-gamma distribution has been used in the past as it is a conjugate prior for normal regression problems. However, for GLSR model described by Stedinger and Tasker (1986) its use may not be attractive. The inverse-gamma is a heavy right-hand tailed distribution; as such it can assign reasonably large probabilities for big variances when 6

91 CHAPTER 3 compared to other distributions such as those with exponential tails for example. Given that issues may arise with the inverse-gamma distribution, in order to avoid these problems, an exponential distribution is used for the prior. The exponential distribution, because of its thin right-hand tail, is considered to be more consistent with what it is believed to be the likely values of the model error variances for regional regression models. It also has a nonzero pdf at zero, which would allow the data, represented by the likelihood function, to provide information about the error variance near zero. The exponential pdf is: (, ) e (3.33) Reis et al. (005) provides a detailed discussion of the derivation of the choice of a prior for the model error variance for regionalising the skew. For the regionalisation of skew, we employed a value for the prior mean of the model error variance equal to 6 following Reis et al. (005), hence: 6 (, ) 6e, 0 (3.34) To derive the prior distribution for the standard deviation, mean flood and flood quantiles of the LP3 distribution, we used an informative one-parameter exponential distribution where the reciprocal of the fractional form of residual error variance estimate taken from the OLSR is used as the prior mean of the model error variance. For example if the residual error variance ( ) from the OLSR is 0.1, we take the inverse of this value i.e. 1/0.1 = OLS Hence, the prior distribution of is an exponential distribution with mean equal to 1/8.33, therefore = (, ) 8.33e, 0 (3.35) In should be made clear that the parameter can have varying influences on the estimated coefficients of the regional regression model and on the estimated model error variances, as such we choose values that are most likely to be very close to the real values (i.e. taking the OLSR results of ). 63

92 CHAPTER SELECTING PREDICTOR VARIABLES This section describes the approach adopted for selecting the predictor variables that should be included in the prediction equations (regression models). The approach for selecting predictor variables used in this study provides improvements over current methods used to justify model selection in the BGLSR framework. Provided below is a discussion on the BGLSR statistics that guided the model selection. We use a procedure similar to forward stepwise regression utilising all the sites for each state (separate regression for each state) and initially adopting just a constant term in the regression equation. The model error variance and its standard error are noted. We then add predictor variables starting with area followed by different combinations of other variables. In all, 16 different combinations of predictor variables were used for the mean, standard deviation and skew models, while 5 combinations were trialled for the flood quantile models. Further information regarding the preparation and extraction of the catchment characteristics can be found in Chapter 4 of this thesis AVERAGE VARIANCE OF PREDICTION In RFFA, the objective is to make prediction at both gauged and ungauged sites; hence a statistic appropriate for evaluation of model selection is the variance of prediction, which in many cases depends on the explanatory variables at both gauged and ungauged sites. Hence, Tasker and Stedinger (1989) suggested the use of the average variance prediction (AVP). By using a GLSR model, one can predict a hydrological statistic on average over a new region. Thus, this becomes the average variance of prediction (AVP) new for a new site which is made up of the average sampling error and the average model error (Tasker and Stedinger, 1986). For BGLSR analysis according to Gruber et al. (007): AVP new n 1 T E[ ] x ivar[ β yˆ ] xi (3.36) n i1 Also, if the prediction is for a site that was used in the estimation of the regional regression model, the measure of prediction (AVP) old requires an additional term: AVP old 1 T 1 1 T 1 E x ( X Λ X) X Λ e (3.37) n T [ ] xivar[ β yˆ] xi n i1 i i 64

93 CHAPTER 3 where e i is a unit column vector with 1 at the ith row and 0 otherwise BAYESIAN AND AKAIKE INFORMATION CRITERIA In this study both the Akaike and Bayesian information criteria are used as statistics for model selection. The Akaike information criterion (AIC) developed by Akaike (1974) is given by Equation It is calculated based on the definition given by Greene (003), where SSTO is the Total-Sum-of-Squared deviations about the mean corrected for the sampling error, n is the sample size for regression and k is the number of predictor variables in the fitted regression model and R is the pseudo coefficient of determination used in BGLSR and is explained in section The first term on the right hand side of Equation 3.38 measures essentially the true lack of fit while the second term measures model complexity which is related to the number of predictor variables. AIC is given by: GLS AIC SSTO (1 R n GLS ( k 1) )exp n (3.38) In practice, after the computation of the AIC for all of the competing models, one selects the model with the minimum AIC value, AIC min.. The Bayesian information criterion (BIC) (Schwarz, 1978) is very similar to AIC, but is developed in a Bayesian framework and is calculated based on the definition given by Greene (003): BIC ( k 1) n SSTO ( 1 R GLS ) n (3.39) n The BIC penalises more heavily models with higher values of k than does AIC. Since SSTO and R GLS depends on the sample size, the competing models can be compared using AIC and BIC only if fitted using the same sample, as done in this study. As with the AIC, one selects the model with the minimum BIC value, i.e. BIC min BAYESIAN PLAUSIBILTY VALUE The significance of the regression coefficient values () obtained was evaluated using the Bayesian plausibility value (BPV) as developed by Reis et al. (005) and Gruber et al. (007), further reading of the mathematical derivations can also be read in the noted 65

94 CHAPTER 3 references. The BPV allows one to perform the equivalent of a classical hypothesis P-value test within a Bayesian framework. The advantage of the BPV is that it uses the posterior distribution of each parameter, which also reflects the prior. The BPV in this study was carried out at the 5% significance level COEFFICIENT OF DETERMINATION The traditional coefficient of determination (R ) measures the degree to which a model explains the variability in the dependent variable. It uses the partitioning of the sum of squared deviations and associated degrees of freedom to describe the variance of the signal versus the model error. Traditionally for OLSR, the Total-Sum-of-Squared deviations about the mean (SST) is divided into two separate terms, the Sum-of-Squared Errors explained by the regression model (SSR) and the residual Sum-of-Squared Errors (SSE), where SST = SSR + SSE. Reis et al. (005) proposed a pseudo co-efficient of determination ( R ) appropriate for use with the GLSR. For traditional R, both the SSE and SST include sampling and model error variances, and therefore this statistic can grossly misrepresent the true power of the GLSR model to explain the actual variation in the y i. Hence, for the GLSR a more appropriate pseudo co-efficient of determination is defined by: GLS R GLS n[ ˆ (0) ˆ ( k)] ˆ ( k) 1 n ˆ (0) ˆ (0) (3.40) where ˆ ( k) and ˆ (0 ) are the model error variances when k and no explanatory variables are used, respectively. Here, R GLS measures the improvement of a GLSR model with k explanatory variables against the estimated error variance for a model without any explanatory variable. If ˆ ( k) = 0, R GLS = 1 as it should, even though the model is not perfect because var[ i i ] is still not zero because var[ i ] > OTHER MODEL SELECTION CRITERIA A predictor variable having an estimated coefficient (other than the constant) that was less than two posterior standard deviations away from zero was rejected (this shows the relative 66

95 CHAPTER 3 importance of the predictor) (Hackelbusch et al., 009). In all the cases the simplest model was preferred. 3.7 FORMATION OF REGIONS The fixed region BGLSR analysis as above identifies the catchment characteristics that best account for heterogeneity by minimising the model error variance. However, it is assumed that there remains a possible spatial structure in the model error residuals. With this in mind the model error variance therefore within possible sub regions of the fixed region should be less than the fixed region model error variance. This is investigated further in this study (see Chapter 5). It is in this framework that the ROI approach was applied to the parameters (i.e. mean, standard deviation and skew) and flood quantiles of the LP3 distribution to further reduce the heterogeneity unaccounted for by the fixed region BGLSR model. The ROI approach in this study uses the distance between sites as the distance metric (i.e. geographic proximity). We apply the ROI within the state boundaries (see Figure 3) in the following way. For the ROI within the state boundaries, for the first iteration, the 15 nearest stations to the site of interest are selected and a regional BGLSR is performed and the predictive variance (Equations 3.14 and 3.31) is noted. The initial number of stations for the first iteration was chosen due to the fact that the smaller ROIs were causing the BGLSR not to run, i.e. there was a lot of instability in the running of the model. The second iteration proceeds with the next five closest stations being added to the ROI and repeating the regression. This procedure terminates when all the sites in the region have been included in the ROI. The ROI for the site of interest is then selected as the one which yields the lowest predictive variance. The ROI approach presented here is fundamentally different to that of Tasker et al. (1996) in that it seeks to minimise: (i) (ii) the regression model s predictive error variance rather than selecting or assuming a fixed number of sites that minimise a distance metric in catchment characteristic space; the ROI criterion of Tasker et al. (1996) cannot guarantee minimum predictive variance; and 67

96 CHAPTER 3 (iii) moreover, the selection of sites that are minimally different in catchment characteristic space may result in greater uncertainty in the estimated regression coefficients. It should be noted that the predictive error variance has two terms associated with it: (i) the model error variance; and (ii) the predictive variance arising from uncertainty in the estimated regression coefficients. The first term is the posterior expected value of the model error variance estimated using the approach of Reis et al. (005), see section and Equation 3.31 this is always nonzero and guards against situations where the most likely value of the model error variance is zero. The second term effectively guards against the ROI favouring fewer sites to minimise the model error variance; indeed, as the number of sites is reduced, the model error variance is likely to be offset by an increase in uncertainty in the estimated regression coefficients (i.e.β ). Figure 3 illustrates the ROI approach as adopted in this study. 68

97 CHAPTER 3 Site of interest within state boundaries Site of interest within state boundaries Figure 3 Example of ROI techniques applied in this study 3.8 REGRESSION DIAGNOSTICS The assessment of the regional regression model is made by using a number of statistical diagnostics such as a pseudo coefficient of determination (as discussed already in section 3.6.4) and the standard error of prediction. An analysis of variance for the BGLSR model is also undertaken to examine which portion of the total error (sampling or model) dominates the regional analysis for both the fixed region and ROI methods. This study also uses Cook s distance, the standardised residuals and Z-score analysis in a GLSR framework which is used to identify outlier sites; absence of outlier in regression diagnostics indicates the overall adequacy of the regional model. These statistics are described below. 69

98 CHAPTER STANDARD ERROR OF PREDICTION If the standardised residuals have a nearly normal distribution (to be determined in the residual analysis, see below), the standard error of prediction in percent (SEP) (Tasker et al., 1986) for the true flood quantile or parameter estimator is described by: 0.5 SEP (%) 100 [exp( AVPnew ) 1] (3.41) 3.8. RESIDUAL ANALYSIS Important to this study is the assessment of the adequacy of the regional regression model in its application to ungauged catchments. The measure of the raw residual (r i ), which is the difference between the sample (at-site estimate) and regional estimates of the LP3 parameter or flood quantile can be assessed initially for major deviations. However, interpreting the raw residual may be misleading as the raw residual has three sources of uncertainty: model error, sampling error and uncertainty due to regression coefficients being unknown. In this study, the standardised residual r si is used, which is the raw residual divided by its standard deviation defined as the square root of the sum of the predictive variance of the LP3 parameter or flood quantile and its sampling variance given by the appropriate diagonal element of the sampling covariance matrix. This yields the definition: r [ x ( X ri Λ si T 1 1 T 0. 5 i i X) x i ] where λ i is the diagonal of Λ (3.4) To assess the adequacy of the estimated LP3 parameters and flood quantiles from the QRT and PRT, standardised residuals, referred to as Z-scores are used. For site i and a given ARI, the Z-score is: Z ARI, i log ( Q ) log ( Qˆ ) e ARI, i e ARI, i (3.43) ˆ ARI, i ARI, i Here the numerator is the difference between the at-site flood quantile and regional flood quantile (estimated from the developed prediction equation) and the denominator is the 70

99 square root of the sum of the variances of the at-site ( ARI,i quantiles in natural logarithm space. CHAPTER 3 ) and regional ( ˆ ARI. i ) flood It is reasonable to assume that the errors in the two estimators are independent because Q ARI i, is an unbiased estimator of the true quantile estimators based upon the at-site data, whereas the error in ˆ is mostly due to the failure of the best regional model to estimate Q ARI, i accurately the true at-site flood quantile. The use of log space makes the difference approximately normally distributed and hence enables the use of standard statistical tests COOK S DISTANCE Tasker and Stedinger (1989) developed measures such as Cook s distance (D i ) from an OLSR to GLSR case. Tasker and Stedinger (1989) and Reis et al. (005) suggested that influence is large when D i is greater than 4/n, where n is the number of sites in the region. Further reading on the mathematical derivations associated with Cook s distance can be found in the noted references. 3.9 EVALUATION STATISITCS A LOO cross validation procedure is applied to assess the performance of the different RFFA methods. The site that is left out in building the model is in effect being treated as an ungauged site. Since all the sites in the database are being treated as ungauged for ROI this automatically satisfies the LOO validation approach. The following performance statistics are calculated from the fixed and ROI analysis: absolute (abs) median relative error (RE r ) in % over n sites, the relative root mean square error (RMSE r ) in % and the average ratio (r r ) of the predicted flood quantile to observed flood quantile as described below. RE r Median n i1 Q abs predi Q Q obsi obsi (3.44) RMSE r 1 n n i 1 Q predi Q Q obsi obsi (3.45) n 1 Qpred r i r (3.46) n i 1 Qobsi 71

100 CHAPTER 3 where Qobsi is the observed flood quantile at site i obtained from at-site flood frequency analysis estimated using FLIKE (Kuczera, 1999a), Qpredi is the predicted flood quantile at site i from the regional prediction equation from QRT and PRT and n is the number of sites in the region. The RE r (%) and RMSE r (%) provide an indication of the overall accuracy of the regional model. The model with minimum RE r is always preferred. For RMSE r the smallest value between the two competing models with the same number of parameters is generally preferred. It should be noted here that both the Q pred and Q obs values have uncertainties associated with them, and in particular, the Q obs values are subject to errors due to the annual maximum flood record length, rating curve extrapolation errors, selection of probability distribution and associated parameter estimation procedures. The above error statistics thus give some guidance about the relative accuracy of the method and should not be taken as the true uncertainty associated with the method. The average value of the Q pred /Q obs (r r ) gives an indication of the degree of bias (i.e. systematic over- or under estimation), where a value of 1 indicates good average agreement between the Q pred and Q obs as both of these values are essentially random variables. An r r value in the range of 0.5 to may be regarded as desirable (D), a value smaller than 0.5 may be regarded as gross underestimation (U), and a value greater than may be regarded as gross overestimation (O). It should be mentioned here that these are only arbitrary limits that are set at a relatively large width band recognising the significant uncertainty in the estimates from a RFFA method in Australia and hence would only provide a reasonable guide about the relative accuracy of the methods as far as the practical application of the methods are concerned REGIONAL UNCERTAINTY WITH FLOOD QUANTILE ESTIMATION For the ARIs considered in this study being the (ARIs 100 years) in this section we only consider the uncertainty in the regional flood quantile estimation based on the BGLSR-PRT and ROI where the ROI without state borders is used. In the annual maximum series models, the mean flood ( ˆ ), standard deviation of floods (ˆ ) and skewness (ˆ ) are considered as regional variables (i.e. the regional at-site estimates of the LP3 parameters). The regional T-year event estimate for the PRT is given by the following Equation. 7

101 CHAPTER 3 Q T Re K ) (3.47) Re R e T ( Re where the subscript R e refers to the site where the regional estimation is made. The uncertainty associated with the regional T-year event estimate can be found by combining the BGLSR method with multivariate normal distribution (MVN). The advantage of using the BGLSR is that it provides an estimate of the annual maximum series hydrologic statistics and their associated posterior variances. The posterior variance reflects the uncertainty related to the residual regional heterogeneity (model error variance) as well as sampling variability corrected for inter-site correlation while also reflecting the prior used. Thus the model error variance term is found by Equation 3.31 for the regional estimate of the ˆ, ˆ and ˆ parameters. These regional values along with the MVN can be used to quantity the uncertainty in the flood quantile estimates by deriving the 90% confidence limits THE MULTIVARIATE NORMAL DISTRIBUTION The MVN distribution model extends the univariate normal distribution model to fit vector observations. An np-dimensional vector of random variables may be defined as follows Y Y1, Y,... Ynp Yi, i 1,..., np (3.48) is said to have a multivariate normal distribution if its density function f(y) is of the form f ( Y ) f ( Y, Y,... Y / 1/ T 1 1/ exp 1/ Y y Y y 1 np ) (3.49) where y = (y 1,, y np ) is a vector of means (i.e. in this case the regional at-site estimate of the hydrological statistic of interest) and is the variance-covariance matrix of the MVN distribution. This can also be given by the notation in Equation The variance for use with the MVN distribution is taken from the BGLSR analysis (i.e. the posterior variances for each parameter estimate, see Figure 4). Y ( y, ) (3.50) N np 73

102 CHAPTER 3 For the univariate case, when np = 1, (i.e. ˆ parameter of the LP3 distribution) the onedimensional vector Y =Y 1 has the normal distribution with mean y and variance ˆ. For the bivariate case, when np =, (i.e. ˆ and ˆ parameters of the LP3 distribution), Y = (Y 1, Y ) has the bivariate normal distribution with two-dimensional vector of means, y = (y 1, y ) and covariance matrix with the correlation (ρ) between the two random variables is given by: ˆ ˆ mean mean, stdev mean ˆ ˆ ˆ mean, stdev mean stdev stdev ˆ stdev (3.51) For the trivariate case, when np = 3, (i.e. ˆ,ˆ and ˆ parameters of the LP3 distribution), Y = (Y 1, Y, Y 3 ) has the trivariate normal distribution with three-dimensional vector of means, y = (y 1, y, y 3 ) and covariance matrix with the correlation (ρ) between the three random variables is given by: ˆ mean, stdev mean, skew mean ˆ ˆ mean mean ˆ ˆ stdev skew mean, stdev mean ˆ stdev stdev, skew ˆ ˆ stdev ˆ ˆ stdev skew mean, skew ˆ ˆ mean stdev, skew stdev ˆ skew ˆ ˆ skew skew (3.5) By using Equations (3.50 and 3.5), 10,000 values are generated for each of the mean, standard deviation and skew of the LP3 distribution (see Equation 3.47). The T-year flood quantile is then fitted (see Equation 3.47) such that there will be 10,000 Q T Re. TheQ T Re values are then ranked in ascending order of magnitude and the 5 th, 50 th and 95 th percentile values are extracted. Figure 4 provides a good summary of the important steps involved in deriving the confidence limits. 74

103 CHAPTER 3 Mean N~ (y 1, ˆ mean ) Standard Deviation N~ (y, ˆ stdev ) Skew N~ (y 3, ˆ skew ) Correlation between parameters mean,stdev, mean, skew, stdev, skew Simulate 10,000 sets of mean, standard deviation and skew from the multivariate normal distribution Obtain 10,000 values of Equation QT Re from Order the 10,000 Q T Re values in ascending order and extract the 5 th and 95 th percentile values Figure 4 Use of multivariate normal distribution to develop confidence limits by Monte Carlo simulation 75

104 CHAPTER VALIDATION OF REGIONAL HYDROLOGICAL REGRESSION MODELS METHODOLOGY THE HYDROLOGICAL REGRESSION PROBLEM Suppose we have a dataset of n sites in a region with k potential catchment characteristics variables (independent variables) x i1, x i,, x ik and a response variable y i (i = 1,,, n) which can be a flood statistic (i.e mean flood)/quantile. The relationship between the response and independent variables is often assumed to be linear. There are also a few assumptions made on the data for hydrological regression; for instance, the dataset are representative of the regression relationship to be developed and the random errors are homoscedastic, (more about this can be seen in section 3.3.1). The OLSR and GLSR based regional regression model can be written in matrix notation as: y Xβ (3.53) Where y = (y 1, y,,y n ) T is the response vector of flood quantiles or flood statistic of interest (the superscript T denotes the transpose), X = (x i,j ) (i = 1,,, n; j = 1,,, k) is a [n k] matrix, is a k-dimensional vector of unknown regression coefficients to be estimated, is a n 1 random error vector assumed to have mean zero and covariance matrix defined by: E( T ) Ω (3.54) wherein is the model error variance and is a positive definite symmetric matrix (Johnston, 197; Rencher, 000; Koop, 005). Different choices of the matrix allow one to make different assumptions regarding the nature of the model errors. If Ω is equal to the identity matrix I n, the problem is homoscedastic, and the GLSR model reduces to OLSR. In more general cases when is defined to reflect heteroscedastisity and correlation among residuals GLSR is a more reasonable estimator. Stedinger and Tasker (1985, 1986) developed a GLSR model for regional hydrologic analysis. The important difference from OLSR is the development and partition of the covariance matrix of the errors. The GLSR model assumes that the total error results from 76

105 CHAPTER 3 two sources: model errors i that are assumed to be independently distributed with mean zero 0 i E and a common variance: Cov, i j i j (3.55) 0 i j and sampling errors that arise due to the fact the actual values of y i are unknown and only estimates of the quantities of interest are available. Therefore, Equation 3.53 becomes (following Reis et al., 005): ˆ (3.56) y Xβ η Xβ where is the sampling error in the sample estimators. Thus, the regression-model errors i are a combination of: (i) time-sampling-error in sample estimators i ŷ i of y i and (ii) underlying model error i (lack of fit). The total error has mean zero and covariance matrix: T Λ I yˆ E (3.57) where Σ( ŷ ) is the covariance matrix of the sampling errors in the sample estimators. Timesampling errors in estimators of the y i s are usually correlated among sites because flows at nearby sites have similar hydrological mechanisms (e.g. meteorology). Reasonably accurate estimation of the sampling covariance matrix in the GLSR is very important and is of great concern and is vital to the solution of the GLSR equations. More details about the construction of Σ( ŷ ) for flood quantiles and statistics can be read in Stedinger and Tasker (1985 and 1986), Reis et al. (005), Griffis and Stedinger (007) and section of this thesis. In both the regression approaches (OLSR and GLSR), the true values (i.e. the regression coefficients) are unknown. To be able to determine the best possible model, it is necessary to decide which of the different s should be included in the model. In typical ordinary stepwise regression this is equivalent to selecting the best set of independent variables for a regression model. Considering the case that uses GLSR (see Equation 3.56), where a more parsimonious model may be true such that: 77

106 CHAPTER 3 y X (3.58) where is a subset of {1,,, k}, X indicates the matrix whose columns are the ones in X that are indexed by the integers in, Λ R, C indicates the sampling covariance matrix whose rows and columns are the ones in Λ that are indexed by and indicates the vector whose components are the ones in that are also indexed by the integers in. Hence there are in total k -1 possible different models of the form represented by Equation For the model of the form of Equation 3.56, if is selected, the model is fitted based on Equation 3.58: T 1 1 T 1 X Λ ( ) X X Λ ( ) yˆ β ˆ GLSR R, C R, C (3.59) where βˆ GLSR is an estimate of β, when Σ( ŷ ) = 0 Equation 3.59 reduces to the OLSR solution. Further information on this can be seen in Stedinger and Tasker (1985 and 1986). Equation 3.59 is solved by employing an iterative procedure using a MOM estimator (see Stedinger and Tasker, 1985 and section of this thesis). After determining the model for use in hydrological regression the overall performance of the model is then evaluated according to its prediction ability, e.g. how well a model can predict flood quantiles for ungauged catchments? In most regression applications, the mean squared error of prediction (MSEP) of a model represents its prediction ability. In practice the lower the MSEP, the better is the prediction ability of the model MODEL SELECTION BY MONTE CARLO CROSS VALIDATION In statistical inference the term cross validation is usually used and has a wide meaning, however to avoid ambiguity this study uses the general term validation associated with either LOO or MCCV. In general, validation attempts to select a model based on the prediction ability of the model (Breiman et al., 1984; Zhang, 1993 and Burman, 1989). For general validation, when is selected, the n datasets (denoted by S) are split into two parts. The first part (calibration set), denoted by S c (with corresponding submatrix X Sc and subvector y Sc ), contains n c datasets for fitting the model. 78

107 CHAPTER 3 The second part (validation set), denoted by S v (with corresponding submatrix X Sv and subvector y Sv ), contains n v = n - n c datasets for validating the model. There are in total n v n C different forms of split samples. For each of the split samples, the model is fitted by the n c dataset of the first part of S c (Equation 3.59) to obtain β ˆ S c, GLSR. The datasets in the validation set (which are essentially gauged catchments) are treated as if they are ungauged. The fitted model then can predict the response vector ŷ Sv: yˆ X βˆ (3.60) T Sv Sv Sc, GLSR The average squared prediction error (ASPE) over all the dataset in the validation set is: ASPE( S v 1 ; ) ( y ˆ S v y S ) (3.61) v n v Therefore, letting S be the set whose elements are all from the validation sets corresponding to (n combination n v ) different forms of sample split. The cross validation criterion with n v datasets left out for validation is defined as: V n v S S ASPE( Sv; ) v ( ) (3.6) n nv where V ( ) is calculated for every. Equation 3.6 serves as an approximation of n v MSEP() in the situation of finite samples. Although LOO validation can select a model with bias b = 0 as n, it can however include unnecessary additional independent variables in the model. In this case the true model dimension k is not considered to be the most parsimonious and can lead to uncertainty in estimation due to over fitting. For general validation it has been proven, under the conditions n c and n v /n1 (Shao, 1993), that the probability for validation (with n v datasets left out for validation) to choose the model with the best prediction ability tends to one. In this framework the V ( ) (Equation 3.6) is n v asymptotically consistent; however, this is not the case for the computation of Vnv with large n v. In such situations, MCCV is an easy and effective procedure. 79

108 CHAPTER 3 For a selected, randomly split the dataset into two parts S c (i) (of size n c ) and S v (i) (of size n v ). Repeat the procedure N times. The repeated MCCV criterion is defined as: MCCV ( ) 1 N nv Nnv i1 ( yˆ y (3.63) s ( i) s ( i) ) v v ESTIMATING MSEP In hydrological regression, the estimate of MSEP is generally based on a finite dataset and datasets that are very small. Here we mainly consider using LOO or MCCV methods to estimate MSEP. As was noted in Efron (1986), the estimate of MSEP using observed data would tend to underestimate the true MSEP for new future observations, since the data have been used twice, both to fit the model and to check its accuracy. The results obtained however are an optimistic estimate at most of the models true prediction error. For Equation 3.58 the LOO validation criterion is: * 1( 1 1 V ) min{ V ( )} (3.64) * where V ) is obtained by Equation 3.6 (n c = 1) and denotes the optimal model index in 1 ( 1 Equation In all cases it should be mentioned that MSEP depends on the size of the calibration data set. MCCV can also be utilised to make the prediction. However, since MCCV uses only n c datasets for calibration it is considered unnecessary to use * MCCVn ( ) to estimate MSEP for the model with n datasets if n v is a large number. v n v 1 Let ) denote the optimal model index in Equation The expected difference between ( n * v * MCCVn ( ) and the mean squared error of prediction for the selected model is : v n v E n * * v MCCVn ( n ) MSEP( n ) v v v ncn (3.65) If a large portion of the dataset is left out for validation, the mean squared error should not * be very small. In such cases, MCCVn ( ) might be a poor estimate of MSEP ( ) (Burman, v n v * nv 80

109 CHAPTER ). In order to obtain slight improvements in accuracy of estimation, a correction term * is needed for MCCVn ( ) (Burman, 1989) given by: v n v CMCCV 1 N * * n ( ) MCCV ˆ n ( ) ( ˆ * * ) ( * * ) v n v n y X β v v, y X β GLSR S, GLSR ( i) n nv nv N nv nv i 1 n c 1 1 (3.66) where βˆ * in the second term is estimated based on n catchments and βˆ * in the nv, GLSR ( i) nv S c, GLSR third term is estimated based on n c catchments in S c (i) (i=1,,, N). MCCV ( ) indicates the average prediction ability of the model with n c catchments and, as stated above, it overestimates the MSEP of the model with n catchments. The second term in Equation 3.66 is the average residual sum of the squares of the model with n catchments. The third term in Equation 3.66 is the average residual sum of squares and prediction error of the model based on n c catchments. The latter two terms combine the effects of the model both with n c and with n catchments. n v * nv APPLICATION USING SIMULATED DATA Two Monte Carlo experiments are reported in this thesis which compared both OLSR and GLSR using LOO and MCCV. In the Monte Carlo simulation the following model is considered: y i o x x x (3.67) 1 1i i 3 3i i Where y i is the dependent variable data and is taken as the 0 year ARI, the simulated values are assumed to be independent normally distributed random variables estimated at i = 1,,, 50 which represents 50 stations: (8 sites with 50 years of data, 8 sites with 40 years of data, 1 sites with 3 years, 1 sites with 5 years of data and 10 sites with 15 years of data, this corresponds to an average record length of 33 years which is the average record length for most Australian catchments). Based on a previous analysis for Australian data in the Monte Carlo simulation analysis for the GLSR model to estimate the total error i we took to be low to high random errors in the range of 0 to 1 i.e. N(0, ) where is 0.5 and For the GLSR estimator we also need an estimate of the diagonals of Σ( ŷ ) (covariance matrix of sampling errors), for normally distributed y i (see Equations 5 81

110 CHAPTER 3 and 6, pg, 14 from Stedinger and Tasker 1985) are adopted to generate the sampling variance for each site of Σ( ŷ ). Furthermore, to estimate the off-diagonal elements of Σ( ŷ ) we also require estimates of cross correlations ( ij ) between concurrent record lengths (i.e. ˆ ( yˆ i, yˆ ) ) in the region. In the Monte Carlo simulation we generated cross correlated data j for ˆ ij = 0.30 (modest constant cross correlation between sites) and ˆ ij = 0.70 (medium to high constant correlation between sites). For OLSR, i are from N(0, ) i.e. standard normal, where are taken as 0. and 1 representing low level (smaller spread) and high level (larger spread) random errors respectively. Here x is the ith value of the kth variable x k, and the values of ki x k (k = 1,, i 3; i = 1,,, 50) and x represents three catchment descriptors (i.e. independent variables) sampled randomly from uniform and normal distributions in the interval of U[5, 1000] for x 1, N(x 1, 0.1) for x and, U[, 0] for x 3. The logarithms (base 10) of these descriptor variables are used in the simulation. To make the simulation more meaningful we also explore the influences of correlated descriptors (i.e. collinearity) which is very common in hydrological regression. The collinearity is explored using LOO and MCCV with both OLSR and GLSR. In this study, we allow x to have a high degree of collinearity with x 1 (such that the correlation coefficient between x 1 and x is taken to be 0.90, see above). In the Monte Carlo simulation, all different combinations of x 1, x and x 3 are considered and the model with best prediction ability is selected. The resulting regional models for OLSR and GLSR are respectively: yˆ x (3.68) i i OLSR GLSR 1 i i i yˆ x (3.69) 1 i The size of the validation sets is taken to be n v = 15, 0, 5,, 45, and the number of simulations 500. In order to assess the obtained model, a further,000 datasets are generated using the above procedure for the purpose of prediction. These datasets are used to calculate the MSEP for the models selected by LOO, MCCV and the assumed true model. 8

111 CHAPTER OBSERVED REGIONAL FLOOD DATA FROM NSW, AUSTRALIA A total of 96 unregulated rural catchments are selected from New South Wales (see more details in Chapter 4). The geographical distributions of these catchments are shown in Figure 5. The catchment areas are considered to be small to medium sized (I. E. Aust., 1987) ranging from 3 to 1000 km (mean: 353 km and median: 67 km ). The annual maximum flood series record lengths range from 5 to 75 years (mean: 37 years, median: 34 years and standard deviation: 11.4 years). More information regarding the preparation of the streamflow data can be found in Haddad et al. (010a) and also Chapter 4 of this thesis. The annual maximum flood series are assumed to follow the LP3 distribution for two reasons: (i). The LP3 distribution is the currently recommended at-site flood frequency probability model by ARR (I. E. Aust., 1987) and ii). It has also shown consistently better results in past studies for Australian catchments (see e.g. Haddad et al., 010a; Haddad et al., 01a and Haddad and Rahman, 01). The LP3 distribution is fitted using a Bayesian parameter fitting procedure (Kuczera, 1999a) for quantiles of ARIs of 10 and 100 years. These two ARIs are chosen because they cover both the high and low sides of the flood distribution. To apply the GLSR to regionalise the flood quantiles the sampling covariance matrix Σ( ŷ ) of the LP3 distribution is required. Tasker and Stedinger 1989 and Griffis and Stedinger (007) (p. 84, Equation 4, also see Equation 3.16 in this thesis) provide the approximate estimator of the components of Σ( ŷ ) matrix of the LP3 distribution. It should be mentioned here that other distributions like GEV could have been adopted; however, it is unlikely to affect the outcomes of the analysis. Furthermore, the LP3 distribution has been found to outperform the GEV distribution generally for eastern Australia (Zaman et al., 01). The skew and standard deviation in the Σ( ŷ ) matrix are subject to estimation uncertainty. In this study to avoid correlation between the residuals and the fitted quantiles, the following procedures are adopted: (iv) the inter site correlation between the concurrent annual maximum flood series (ρ ij ) is estimated as a function of the distance between sites i and j; (v) the standard deviations (of the logarithms of annual maximum flood series) σ i and σ j are estimated using a separate OLSR and using the predictor variables used in the study (see below); and 83

112 CHAPTER 3 (vi) the regional skew (of the logarithms of annual maximum flood series) is used in place of the population skew as suggested by Tasker and Stedinger (1989). This analysis above uses the regional estimates of the standard deviation and skew obtained from GLSR. The detailed information on the covariance matrices associated with the standard deviation and skew can be found in Reis et al. (005) and Griffis and Stedinger (007) and Equations 3.17 and ). Twelve climatic and catchment characteristics variables were selected. (More information regarding the extraction and preparation of the catchment characteristics can be found in Chapter 4 of this thesis). The predictor variables were log-transformed (base 10) and centered around the mean for the regression analysis. 3.1 SUMMARY A number of statistical techniques and formulations to be used in this thesis have been presented in this chapter. On the onset of this chapter, fitting the LP3 distribution to the observed flood data using a Bayesian parameter fitting procedure has been presented. The GLSR procedure has then been discussed both in its classical application and in hydrologic regression context to derive regional regression equations relating flood quantiles to catchment and climatic characteristics using both a QRT and PRT framework. The Bayesian GLSR (BGLSR) regression procedure was discussed in more detail. The setting up of the residual error covariance matrices with the BGLSR approach has also been discussed. This chapter has also discussed the formation of regions in RFFA which has included both the fixed and region of influence approaches. The second part of this chapter discussed the mathematical formulations used in the model validation in the context of hydrologic regression analysis using OLSR and GLSR. The statistical framework for the numerical experimentation and practical application demonstrating the use of LOO and MCCV in hydrologic quantile regression analysis has also been discussed. The next chapter will discuss the study areas and the different aspects of streamflow and catchment characteristics data collation and preparation. 84

113 CHAPTER 4 CHAPTER 4: STUDY AREA AND PREPARATION OF STREAMFLOW AND CATCHMENT CHARACTERISITICS DATA 4.1 GENERAL The assembly and preparation of streamflow data is an important step in any regional flood frequency analysis (RFFA) study. This chapter describes various aspects of the streamflow data collation adopted for this work e.g. selection of the study area, selection of stream gauging sites, checking annual maximum streamflow data, filling gaps in the streamflow data series, checking rating curve extrapolation errors associated with the streamflow data series, checking for outliers in the data series and testing for any significant trends that could undermine the purpose of flood frequency analysis. Because this study is primarily concerned with developing regional prediction equations for design flood estimation using both a quantile and parameter regression technique; an elementary step in any regional study such as this involves obtaining both climatic and catchment characteristics data. Identifying the most relevant catchment characteristics is difficult as there is no objective method for doing this; also many catchment characteristics are highly correlated, thus the presence of many of these in the model can cause problems with statistical analysis such as introducing multicollinearity and secondly it does not provide any extra useful information. Rahman (1997) indicated that there is no objective method for selecting catchment characteristics, thus an initial selection of candidate characteristics should be based on an evaluation and success of catchment characteristics used in past studies. Rahman (1997) considered in detail all possible climatic/catchment characteristics (referred as catchment characteristics henceforth) from over 0 previous studies to develop a reasonable starting point. Nevertheless, no general inference about the significance of a particular catchment characteristic can be made on the fact that an investigator has found it to be significant, since in a regional study such as this dominant characteristics may vary from region to region. 85

114 CHAPTER 4 In the second part of this chapter, the catchment characteristics to be used in this thesis are selected with the aim of developing a working database of catchment characteristics. Initially the selection of candidate catchment characteristics is described in sufficient detail and aspects of data collation/collection are presented later PUBLICATIONS A Journal paper (ERA, rank B) has been published on the materials presented in this chapter. This journal paper is given in Appendix A. The following is the reference of the paper. Haddad, K., Rahman, A., Weinmann, P.E., Kuczera, G. and Ball, J.E. (010a). Streamflow data preparation for regional flood frequency analysis: Lessons from south-east Australia. Australian Journal of Water Resources, 14 (1), STUDY AREA For this study, the Australian continent is selected as the study area. For flood quantile estimation in the range of 100 years average recurrence intervals (ARI), the quantile and parameter regression techniques (QRT and PRT) in a Bayesian generalised least squares regression (BGLSR), fixed and region of influence, (ROI) frameworks are applied in the states of Queensland (QLD), New South Wales (NSW), Victoria (VIC) and Tasmania (TAS). The model validation case study makes use of the data from NSW, while for the large flood analysis using the large flood regionalisation model (LFRM), 66 stations are used from all over the Australian continent, excluding the arid and semi arid regions. The selected study area is shown in Figure 5. 86

115 CHAPTER 4 Figure 5 Plot of the selected study area (i.e. NSW, VIC, QLD and TAS) 4.3 SELECTION OF CANDIDATE CATCHMENTS The following factors and criteria were considered in making the initial selection of the study catchments. Catchment area: The proposed regionalisation study aims at developing prediction equations for flood estimation in small to medium sized ungauged catchments. Since the flood frequency behaviour of large catchments has been shown to significantly differ from smaller catchments, the proposed method should be based on small to medium sized catchments. ARR (I. E Aust, 1987) suggests an upper limit of 1000 km for small to medium sized catchments, which seems to be reasonable and is adopted here. For larger catchments, the flood frequency curves are generally flatter as compared to the smaller ones. Since the focus of RFFA technique is design flood estimation to small ungauged catchments, the use of very large catchments in the development of RFFA techniques is not justified as per ARR (I. E Aust, 1987). 87

116 CHAPTER 4 Record length: The streamflow record at a stream gauging location should be long enough to characterise the underlying probability distribution with reasonable accuracy. In most practical situations, streamflow records at many gauging stations in a given study area are not long enough and hence a balancing act is required between obtaining a sufficient number of stations (which captures greater spatial information) and a reasonably long record length (which enhances accuracy of at-site flood quantile estimates. Selection of a cut-off record length appears to be difficult as this can affect the total number of stations available in a study area. However for this study, the stations having a minimum of 10 years of annual instantaneous maximum flow records are selected initially as candidate stations. This is because that sample size smaller than 10 years may not be useful in RFFA in Australia as this often suffers from long periods of droughts and flood quantile estimates with smaller record lengths this may provide biased results. Here 10 years is the cut-off record length; however, the adopted threshold was 4 years for most of the Australian states as noted later in this chapter. Regulation: Ideally, the selected streams should be unregulated, since major regulation affects the rainfall-runoff relationship significantly (storage effects). Streams with minor regulation, such as small farm dams, may be included because this type of regulation is unlikely to have a significant effect on annual floods. Gauging stations subject to major regulation are not included. Urbanisation: Urbanisation can affect flood behaviour dramatically (e.g. decreased infiltration losses and increased flow velocity). Therefore, catchments with more than 10% of the area affected by urbanisation are not included in the study. Landuse change: Major landuse changes, such as the clearing of forests or changing agricultural practices modify the flood generation mechanisms and make streamflow records heterogeneous over the period of record length. Catchments which have undergone major land use changes over the period of streamflow records are not included in the data set. Quality of data: Most of the statistical analyses of flood flow data assume that the available data are essentially error free; at some stations this assumption may be grossly violated. 88

117 CHAPTER 4 Stations graded as poor quality or with specific comments by the gauging authority regarding quality of the data were assessed in greater detail; if they are deemed low quality they are excluded. For example, if there were lots of missing data, and the gauging station location was sifted a long way from the previous location, the station was excluded. 4.4 STREAMFLOW DATA PREPARATION FILLING MISSING RECORDS IN ANNUAL MAXIMUM FLOOD SERIES Missing observations in streamflow records at gauging locations are very common and one of the elementary steps in any hydrological data analysis is to make decisions about dealing with these missing data points. Missing records in the annual maximum flood series are infilled where the extra data points can be estimated with sufficient accuracy to contribute additional information rather than noise. For this study, one of the following methods (a or b) is applied, as documented in Rahman (1997) and Haddad et al. (010a). (a) Comparison of the monthly instantaneous maximum (IM) data with monthly maximum mean daily (MMD) data at the same station for years with data gaps. If a missing month of instantaneous maximum flow corresponds to a month of very low maximum mean daily flow, then that is taken to indicate that the annual maximum did not occur during that missing month. (b) Application of a linear regression between the annual maximum mean daily flow series and the annual instantaneous maximum series of the same station. Regression equations developed are used for filling gaps in the IM record, but not to extend the overall period of record of instantaneous flow data. For in-filling the gaps, Method (a) is preferred over Method (b), as it is more directly based on observed data for the missing month and involves fewer assumptions TREND ANALYSIS Hydrological data for any flood frequency analysis, be it at-site or regional, should be stationary, consistent and homogeneous. The annual maximum flow series should not show any time trend to satisfy the basic assumption of stationarity with traditional flood frequency analyses methods. Thus, in this study, a trend analysis is carried out where 89

118 CHAPTER 4 possible to identify stations showing significant trend and the stations which do not show any trend are included in the primary data set for each Australian state. Two tests are initially applied to detect time trend, the Mann Kendall test (Kendall, 1970) and the distribution free CUSUM test (McGilchrist and Wodyer, 1975); both tests are applied at the 5% significance level. The Mann-Kendall test is concerned with testing whether there is an increase or decrease in a time series, whereas the CUSUM test concentrates on whether the mean values in two parts of a record are significantly different. As a useful guide and in addition to the trend tests, a simple time series plot and a cumulative flow graph of the station are also used to detect shifts in the annual maximum flood data RATING CURVE ERROR AND IDENTIFICATION Most stream gauging authorities establish a network of streamflow gauging stations to obtain continuous streamflow data. However, in most cases, these do not measure the actual discharge directly. Rather it is the stage that is recorded, and subsequently transformed to discharge by means of an estimated rating curve, which is constructed in most cases by correlating measurements of discharge with the corresponding observations of stage. However, the range of observed flood levels generally exceeds the range of measured flows, thus requiring different degrees of extrapolation of well established rating curves. Thus, most of the discharges calculated by rating curve are subject to uncertainty. Different methods of rating curve extrapolation are associated with a range of assumptions, from simple extension of fitted regression lines to hydraulic analysis methods requiring additional data. The magnitude of rating curve extrapolation errors depends on the stream and flood plain conditions near the gauging station, the strengths of the assumptions made in extrapolation, and the degree of extrapolation beyond the range of measured flows (Kuczera, 1999a). Any rating curve extrapolation error is directly transferred into the largest observations in the annual maximum flood series, and use of these extrapolated data in flood frequency analysis can result in grossly inaccurate flood estimates, particularly for higher ARIs. There are several studies that have examined the uncertainty of a single discharge estimate due to rating curve variability using a regression-based approach, e.g., Venetis (1970), 90

119 CHAPTER 4 Dymond and Christian (198) and Reitan and Petersen-Øverleir (008). On the other hand, the impact of rating curve error and imprecision in the estimation of the flood quantile has received less attention in hydrological literature (Petersen-Øverleir and Reitan, 009). Potter and Walker (1981), Rosso (1985), Shuzheng and Yinbo (1987) and Kuczera (199, 1996) provided some insights into the problem by analysing a multiplicative error model. Kuczera (1996) and Reis and Stedinger (005) adopted a multiplicative error model in a Bayesian framework to deal with rating curve error. From these studies, the main conclusion to be drawn is that multiplicative measurement error introduces bias into estimated flood quantiles. In this study, the stations having annual maximum flood data associated with high degree of rating curve extrapolation are identified by introducing a rating ratio (RR). The annual maximum flood series data point for each year (estimated flow Q E ) is divided by the maximum measured flow (Q M ) for that station to define the rating ratio (See Equation 4.1). Moreover the rating ratio is based on the highest measured flow over the total period of record, and the annual maximum flows are based on the gauging authorities best estimate of the rating curve applicable at the time of that flow event. Rating Ratio RR Q Q E ( ) (4.1) M If the RR value is below or near 1, the corresponding annual maximum flow may be considered to be free of rating curve extrapolation error. However, a RR value well above 1 indicates a rating curve error that can cause notable errors in flood frequency analysis. As an example, for Station 0, there are 11 data points with RR values greater than 1 (7% of total data points) and the maximum value of RR is 5.5 (Figure 6). This large degree of rating curve extrapolation is likely to affect flood frequency estimates at this station, especially the higher ARI floods such as Q 50 and Q 100, unless appropriate measures are taken. The application of RR is discussed further in the latter part of this chapter. For any RFFA, a large number of stations with reasonably long record lengths are required and hence a trade-off needs to be made between an extensive data set that includes stations with very large RR values (and thus lower accuracy) and a smaller data set with RR values 91

120 CHAPTER 4 restricted to what could be considered to be a reasonable upper limit of rating curve errors. A working method to decide on a cut-off RR value is determined by looking at the average and the maximum RR values for each station in a region/state. Based on the results from VIC and NSW, the RR values found to represent a reasonable compromise between accuracy at individual sites and total size of the regional data set are an average of 4 and a maximum of 0. 6 Likely Rating Curve Error Data points subject to possible rating curve errors QE/QM Data Point Figure 6 Plot of rating ratios (RR) for station SENSIVITY ANALYSIS AND IMPACT OF RATING CURVE EXTRAPOLATION ON FLOOD QUANTILE ESTIMATES Typically error arising from rating curve extension is smooth and can therefore introduce systematic error of both over- or under-estimation of the true discharge. The rating curve extension error coefficient of variation is not well known, however Potter and Walker (1981) suggest it could be as high as 30% in poor situations, such as errors in the extrapolation zone (see Figure 7). In the interpolation zone however where the rating curve is well defined by discharge-stage measurements, typically the error coefficient of variation 9

121 CHAPTER 4 would be small, say 1% to 5% (Kuczera, 1996 and Reis and Stedinger, 005). As noted by Kuczera (1999a), there are two cases in which smooth rating curve extension can introduce systematic error. Firstly an indirect estimate can be made for large floods well beyond the measured flow; it is this estimate that is then subject to extreme uncertainty. In such cases estimates that are well below the true discharge can cause significant underestimation in flood frequency analysis and vice versa. Rating curves are also extended by the slopeconveyance method, which mainly relies on extrapolation of gauged estimates of the friction slope so that this slope converges to a constant value. This can cause considerable systematic error which is difficult to quantify as compared to the log-log extrapolation. As it is the most commonly employed approach for rating-curve extrapolation, log-log extrapolation is explored in this study. In log-log extrapolation, the systematic error can be seen as the likely divergence from the true rating as the discharge increases. Thus, as the rating curve is extended from the true rating curve an extension zone is introduced. This extension zone depends on the distance from the anchor point and not from the origin. In this case the systematic error is incremental, as it originates from the anchor point. In this study, to implement the concept of systematic rating curve error, the flow that is closest to RR = 1 is used as the anchor point in the FLIKE rating curve error model (Kuczera 1999b). The assumption is then made that there is little error (1 to 5%) up to the anchor point (Figure 7). All discharge estimates with RRs > 1 (this means the true flood discharge exceeds the anchor value) have systematic error and deviate away from the anchor point. The application of the RR using a cut-off point value is introduced in this study to remove stations which are likely to be associated with high rating curve related errors. Further discussion on this is presented in this chapter, where the impacts on flood quantile estimates of different rating curve errors and RR values are examined, to demonstrate the importance of accurate flood discharge estimates. 93

122 CHAPTER 4 log discharge Actua1 rating curve (reported by gauging authority) Incremental error Anchor Point, RR=1 Maximum measured flow True but unknown rating curve Interpolation zone Extrapolation zone log stage Figure 7 Rating curve extension error TESTS FOR OUTLIERS In a set of annual maximum flood series there is a possibility of outliers being present. An outlier is an observation that deviates significantly from the bulk of the data, which may be due to errors in data collection or recording, or due to natural causes. In this study, the Grubbs and Beck (197) method is adopted in detecting high and low outliers. This method was recommended in Bulletin 17B by the United States Water Resources Council after large scale testing of a wide variety of procedures. The method is based on determining high outlier and low outlier thresholds by applying a one-sided 10% significance level test that considers the sample size. The test was developed by Grubbs and Beck (197) for detecting single outliers from a normal distribution but (when applied to the logs of a flood data series) has been shown to be also applicable to the log Pearson type 3 (LP3) distribution. The method is simple to use and has been widely applied in North America (Ng et al., 007). Its application to dealing with low outliers is straightforward. However, it should be noted here that special precaution is needed to treat any detected high outlier, given that there is a 10% chance of the null hypothesis of no outliers having been wrongly rejected. If not caused by data error, the 'high outlier' data point contains very useful information regarding the frequency of large floods. 94

123 CHAPTER RESULTS OF STREAMFLOW DATA PREPARATION PROCESS The methods described in section 4.4 are applied to gauged flood data to the entire Australian continent. In this section we present the detailed results for VIC and NSW for simplicity sake only; further results are summarised and further reading can be found in Rahman et al. (009 and 011a). This section summarises the main findings DATA PREPARATION FOR VICTORIA Based on the selection criteria presented in section 4.3, a total of 415 stations are initially selected as candidates from VIC each having a minimum of 10 years of streamflow record. For in-filling the gaps in the annual maximum flood series, Method (a) is preferred over Method (b) (see section for a description of these methods). The following points summarise the results of the in-filling of the annual maximum flood series data: (i) 73 data points from 187 stations are in-filled by Method (a); (ii) 60 data points from 44 stations are in-filled by Method (b); (iii) Regression equations used in gap filling have high R values (range , mean = 0.93 and SD = 0.041); and (d) 10% of stations do not have any missing records. After in-filling the gaps, the stations are checked for possible trends. Initially, the Mann- Kendall test is applied to the annual maximum flood series of the candidate stations. The results revealed that some 0% of the candidate stations exhibit a decreasing trend, a somewhat surprising result. However, the record lengths of many of these stations are less than 0 years, and, moreover, south-east Australia has experienced a severe drought since the mid 1990 s. To explore this issue further, time series plots and mass curves are prepared for the stations showing trend to detect visually if significant changes in slope can be identified. Figure 8 (a) presents the results for Station 3010, which shows a noticeable decrease in annual maximum flood data from the late 1980 s thus supporting the results from the Mann-Kendall test. The CUSUM test produced similar results see Figure 8 (b) - namely a downward shift in the mean from 1995 onwards. These results suggest that flood data at many stations are not independently and identically distributed from year to year. Thus there needs to be caution applied when using short records in estimating long term flood risks. The fact that data starting in the 1990s exhibited a significant downward trend for many stations in VIC makes the inclusion of 95

124 CHAPTER 4 stations with short records in RFFA questionable. Most RFFA methods can compensate for sampling variability but not for bias introduced by a drought-induced systematic downward trend in a short record. To overcome this problem, the introduction of a longer cut-off record length appears to be appropriate. However, the selection of a cut-off record length involves a trade-off between spatial coverage and bias. It is judged that a cut-off record length of 5 years is adequate for the purpose of this study. Although this has removed more than half of the candidate stations from VIC, the remaining stations would be less affected by bias and thus would yield more representative RFFA assessments of long-term flood risk. The number of eligible stations after the introduction of a cut-off length of 5 years, dropped to 144, which is only 35% of the initially selected 415 stations. This shows that the useful data set for RFFA in a given region is likely to be substantially smaller than the primary data set. Annual Maximum Flow (ML/ d) Station 3010 Decrease in flow magnitude a Year Station b 6 Vk Year Figure 8 (a) Time series plot showing significant trends after 1995 and (b) CUSUM test plot showing significant trends after Here Vk is CUSUM test statistic defined in McGilchrist and Wodyer (1975) 96

125 CHAPTER 4 In the remaining data set of 144 stations, many had rating ratios (RR) considerably greater than 1. From the histogram of RR values shown in Figure 9 it can be seen that 90% of the RR values for all the recorded annual maxima lie between 1 and 0. A RR value significantly greater than 1 could magnify the errors in flood frequency quantile estimates but, on the other hand, rejecting all stations with a RR greater than one would reduce the number of stations below the minimum required for a meaningful RFFA. Thus, it is decided that a cut-off RR value of 0 would be reasonable, which has reduced the eligible number of stations from 144 to 131 for VIC. The impacts of RR values on flood quantile estimates are presented in section Victoria Frequency % of rating ratio s lie between 1 & More Rating Ratio - RR 5 Figure 9 Histogram of rating ratios of annual maximum flood data in Victoria (stations with record lengths > 5 years) The results of the outlier detection procedure are summarised here: (a) Some 43% of the stations are found to have low outliers. The maximum number of low outliers detected in a data series is 5 and never exceed 19% of the total number of data points in a series. (b) Most of the detected low outliers occurr for stations which are located in low rainfall areas, especially in the western part of VIC. (c) 31% of low outliers occurred in the years 198 and Severe drought occurred during these years with the maximum annual flows in many rivers being baseflow rather than a flood. Similar results were reported by Rahman 97

126 CHAPTER 4 (1997). (d) 55% of the stations do not show any outliers. Even the values in the drought years of 1967 and 198 are not low enough to be treated as low outliers. The locations of most of these stations are in the south-eastern part of Victoria. (e) Only 1 station shows a high outlier. The detected low outliers are treated as censored flows in flood frequency analysis using FLIKE (that is, the information that there is no flood in that year is taken into account). The final VIC database contains 131 stations whose record lengths range from 5 to 5 years (mean and median: 3 years and standard deviation: 5 years). Some 87% of the stations have record lengths in the range 5-35 years, 8% of the stations in the range years and 5% of the stations in the range years. The catchment areas range from 3 to 997 km (mean: 31 km and median: 89 km ). Some 15 catchments (11%) are in the range of 3 to 50 km, 11 catchments (8%) are in the range of 51 to 100 km, 78 catchments (60%) are in the range of 101 to 500 km ; and 7 catchments (1%) are in the range of 501 to 997 km. The histogram of streamflow record lengths of the 131 stations is shown in Figure 10. The distribution of catchment areas is shown in Figure 11. The geographical distribution of these stations is shown in Figure 16, which shows that there is no station in north-western VIC that has passed the selection criteria. This region indeed is characterised by very low runoff and ephemeral streams Frequency Record Length (years) Figure 10 Distributions of streamflow record lengths of the selected 131 stations from Victoria 98

127 CHAPTER Frequency Catchment Area (km ) Figure 11 Distributions of catchment areas of the 131 catchments from Victoria 4.5. DATA PREPARATION FOR NSW AND ACT Initially, a total of 635 stations are selected from NSW and the Australian Capital Territory (ACT). After in-filling the gaps and using the selection criteria discussed in section 4.3, only 94 stations are retained with a minimum of 10 years of annual maximum flood data. The Mann-Kendall test, time series plot inspection and CUSUM test resulted in some 11% of the stations (31 stations) being identified as having a decreasing trend, generally after A cut-off record length of 5 years is adopted similar to Victoria, which has reduced the number of eligible stations to 106, which is only 17% of the initially selected 635 stations. In the remaining data set of 106 stations from NSW, many had RR values considerably greater than 1 see Figure 1. As for the VIC data, a cut-off RR value of 0 is adopted, which has reduced the eligible number of stations from 106 to

128 CHAPTER NSW Over 95% of rating ratios between 1 and 0 Frequency Rating Ratio - RR Figure 1 Histogram of rating ratios for 106 stations from NSW Some 40% of the stations from NSW and ACT are found to have low outliers. The maximum number of low outliers detected in a data series is 9 and has never exceeded 1% of the total number of data points in a series. Most of these detected low outliers occur for stations located in low rainfall areas, especially in the western parts of NSW. Some 31% of low outliers occur in the years 1967, 198 and About 47% of the stations do not show any outliers. Only 5 stations have shown a high outlier. The record lengths of the 96 stations range from 5 to 74 years (mean: 34 years, median: 31 years and standard deviation: 10 years). Some 77% of the stations have record lengths in the range 5-35 years, and 18% of the stations in the range years; and 5% in the range years. The catchment areas range from 8 to 1010 km, with an average value of 353 km, median of 67 km and a standard deviation of 76 km. Some 9 catchments (9%) are in the range of 8 to 50 km, 9 catchments (9%) are in the range of 51 to 100 km, 5 catchments (54%) are in the range of 101 to 500 km and 7 catchments (8%) are in the range of 501 to 1010 km. The histogram of streamflow record lengths of the 96 stations is shown in Figure 13. The distribution of catchment areas is shown in Figure 14. The geographical distribution of the 96 stations is shown in Figure 16. There is no station in far western NSW that has passed the selection criteria. 100

129 CHAPTER Frequency >75 Record Length (years) Figure 13 Distributions of streamflow record lengths of the selected 96 stations from NSW Frequency >1000 Catchment Area (km ) Figure 14 Distributions of catchment areas of the 96 catchments from NSW 101

130 CHAPTER SENSITIVITY ANALYSIS - IMPACT OF RATING CURVE ERROR ON FLOOD QUANTILE ESTIMATES To assess the impact of rating curve error (expressed in terms of RR) on flood quantile estimates, the FLIKE software, which implements the principles outlined in Kuczera (1999a, b), is employed to fit the LP3 distribution using the Bayesian parameter fitting procedure. In this application of FLIKE, no prior information is used with both the no rating curve error and the rating curve error cases. The flow closest to RR = 1 is used as the anchor point in the rating curve error model inbuilt in FLIKE. The flows greater than RR = 1 are expected to be associated with measurement errors i.e. the higher the RR value for a data point the greater the degree of rating curve extrapolation error associated with it (see Figure 7). In the flood frequency analysis using FLIKE for the rating error case, less weight is assigned to the flow data points beyond the anchor point (which represents higher flows). Three cases are considered here for illustration purposes where flows in excess of the anchor point are corrupted by a multiplicative error assumed to be log-normally distributed with mean one and coefficient of variation (CV) equal to 10%, 0% and 30%. Also, four different values of maximum RR are considered (5, 10, 0 and 40). Four stations from the database for VIC and NSW are selected with maximum RR values in the range of 5-40: Station (RR = 5), Station 13 (RR = 10), Station 3409 (RR = 0) and Station 101 (RR = 40). Table 1 presents the flood quantile estimates using FLIKE for four scenarios where the coefficient of variation of multiplicative errors CV equal to 0%, 10%, 0% and 30%. For each of these four scenarios, stations with maximum RR values of 5, 10, 0 and 40 are analysed. Table 1 presents the expected quantile and the lower and 95% confidence limits for the 50- and 100-year flood. To assist interpretation, the results for the cases where rating curve error is assumed present (i.e., CV > 0) are expressed as ratios for the case CV > 0 to the case CV = 0. 10

131 CHAPTER 4 Table 1 Flood quantile estimates and associated errors using ARR FLIKE with and without consideration of rating curve error (MMF = maximum measured flow) Station Maximum RR ARI of MMF (yr) LL 95% Rating error CV = 0% Expected UL 95% Ratio for CV = 10% and CV=0% Expected, % CL width, % Ratio for CV = 0% and CV=0% Expected, % CL width, % Ratio for CV = 30% and CV=0% Expected, % CL width, % 50-year flood quantile year flood quantile

132 CHAPTER 4 The results show that the width of the 95% quantile confidence limits increases with increasing rating curve error CV reflecting the fact that errors in estimating the bigger flood flows reduce the information content of the higher flows. Indeed, in the worst case, the confidence limit width increases by 50%. Moreover, the bias in quantile estimates increases with increasing CV, in some cases reaching 50% to 60%. This confirms the soundness of the eliminating stations judged to have poor quality ratings. Of interest is the relationship of quantile bias and accuracy with maximum RR. It appears that as the maximum RR increases, the bias and uncertainty in the quantiles tends to grow for a given rating curve error CV. The trend is somewhat obscured by the fact that the ARI of the maximum measured flow (i.e. the anchor point) varies. As the ARI of the anchor point grows fewer flows are affected by rating curve errors; for example, if the ARI of the anchor point is years, then half of the data will lie below the anchor point, largely unaffected by rating curve error. Thus one can see that station 101 which has maximum RR of 40 but an anchor point ARI of 3.77 years has similar bias and accuracy to Station 3409 which has a lower maximum RR of 0 but an anchor point ARI of 1.03 years. Although this analysis is not conclusive, it does suggest that stations with high maximum RR values are likely to be problematic unless some form of compensation for rating curve error is made. 4.6 SUMMARY RESULTS OF STREAMFLOW DATA PREPARATION FOR THE OTHER STATES The methods applied in section 4.5 are applied to gauged flood data in the entire Australian continent. In this section we present the summary results for QLD, TAS, Northern Territory (NT), Western Australia (WA) and South Australia (SA). Further results can be found in Rahman et al. (009 and 011b). This section also presents a summary of the final catchments adopted for this study TASMANIA A total of 53 catchments have been selected from TAS. The record lengths of annual maximum flood series of these 53 stations range from 19 to 74 years (mean: 30 years, median: 8 years and standard deviation: years). The catchment areas of the selected 53 catchments range from 1.3 km to 1900 km (mean: 33 km and median: 158 km ). The geographical distribution of the selected 53 catchments is shown in Figure

133 CHAPTER QUEENSLAND A total of 17 catchments have been selected from QLD. The record lengths of annual maximum flood series of these 17 stations range from 5 to 97 years (mean: 41 years, median: 36 years and standard deviation: 15. years). The catchment areas of the selected 17 catchments range from 7 km to 963 km (mean: 35 km, median: 54 km ). The geographical distribution of the selected 17 catchments is shown in Figure SOUTH AUSTRALIA A total of 9 catchments have been selected from SA. The record lengths of annual maximum flood series of these 9 stations range from 18 to 67 years (mean: 36 years, median: 34 years and standard deviation: 11. years). The catchment areas of the selected 9 catchments range from 0.6 km to 708 km (mean: 170 km and median: 76.5 km ). The geographical distribution of the selected 9 catchments is shown in Figure NORTHERN TERRITORY A total of 55 catchments have been selected from NT. The record lengths of annual maximum flood series of these 55 stations range from 19 to 54 years (mean: 35 years, median: 33 years and standard deviation: years). The catchment areas of the selected 55 catchments range from 1.4 km to 4,35 km (mean: 68 km and median: 360 km ). The geographical distribution of the selected 55 catchments is shown in Figure WESTERN AUSTRALIA A total of 146 catchments have been selected from WA. The record lengths of annual maximum flood series of these 146 stations range from 0 to 57 years (mean: 31 years, median: 30 years and standard deviation: 8.0 years). The catchment areas of the selected 146 catchments range from 0.1 km to 7,405.7 km (mean: 33 km and median: 60 km ). The geographical distribution of the selected 146 catchments is shown in Figure SUMMARY OF STREAMFLOW DATA AUSTRALIA WIDE A total of 68 catchments have been selected from all over Australia. The record lengths of the annual maximum flood series of these 68 stations range from 18 to 97 years (mean: 35 years, median: 33 years and standard deviation: 11.5 years). The distribution of record lengths is shown in Figure 15 (a). 105

134 CHAPTER 4 The catchment areas of the selected 68 catchments range from 0.1 km to 7,405.7 km (mean: 350 km, median: 14 km ). The geographical distribution of the selected 68 catchments is shown in Figure 16. The distribution of catchment areas of these stations is shown in Figure 15 (b) a Frequency Record Length (years) b 17 Frequency Catchment Area (km-sq) Figure 15 (a) Distribution of annual maximum flood record lengths of 68 stations from all over Australia (b) Distribution of catchment areas of 68 stations from all over Australia 106

135 CHAPTER 4 Figure 16 Geographical distributions of the selected 68 stations from all over Australia The summary of all the Australian data prepared as a part of this study is provided in Table. Table Summary of selected stations Australia wide No. of Median streamflow record length Median catchment size State stations (years) (km ) NSW and ACT VIC SA TAS QLD WA NT Total

136 CHAPTER SELECTION AND ABSTRACTION OF CATCHMENT CHARACTERISITCS Catchment characteristics used in many previous RFFA studies were summarised by Rahman (1997). He grouped the catchment characteristics under the headings of climatic characteristics, morphometric characteristics, catchment cover and land use characteristics, geological and soil characteristics, catchment storage characteristics, and location characteristics. Many catchment characteristics are highly correlated, and the inclusion of strongly correlated variables in prediction equations does not add any new information; it also causes problems in statistical analysis (e.g. multicollinearity). The following guidelines can be useful in making a reasonable selection: The characteristics should have a plausible role in flood generation. They should be unambiguously defined. Characteristics should be easily obtainable. When a simpler characteristic and a complex one are correlated and have similar effects then the simpler characteristic should be chosen. If a derived/combined characteristic is used, it should have a simple physical interpretation. The characteristics in the selected set should not be highly correlated, because this results in unstable parameters in hydrologic regression analysis. The prediction performance of a characteristic in other regionalisation studies should be taken into account, as this can give some general idea regarding the importance of the characteristic. Based on the hydrological significance, correlations and ease of the data abstraction, eight catchment characteristics are included in this study as listed in Table 3, and described below. Catchment area: Catchment area is the main scaling factor in the flood process and directly affects the potential flood magnitude from a given storm event. The total volume of runoff (Q) is proportional to the area of the catchment area (A), and of the general form: Q = ca m (4.) 108

137 CHAPTER 4 where the exponent m varies from 0.5 to Table 3 Catchment characteristics variables used in the study Catchment characteristics 1. area: Catchment area (km ). I: Design rainfall intensity (mm/h) 3. rain: Mean annual rainfall (mm) 4. evap: Mean annual areal potential evapotranspiration (mm) 5. S1085: Slope of the central 75% of mainstream (m/km) 6. sden: Stream density (km/km ) 7. forest: Fraction of catchment area under forest. 8. qsa: Fraction quaternary sediment area (VIC only). Almost all of the reported RFFA studies have found catchment area to be very significant. One of the reasons why the area variable has been so useful in statistical hydrology is its association with other significant morphometric characteristics like slope, stream length and stream order. Area was characterised by Anderson (1957) as the devil s own variable, because almost every watershed characteristic is correlated with it. As in the case of area, the mean annual flood is directly proportional to other morphometric characteristics, which are again directly proportional to area. In this study, catchment area is obtained from 1:100,000 topographic maps which are readily available for large parts of Australia. Rainfall intensity: Storm rainfall intensity (I ARI,d ), for an appropriate burst duration (d) and average recurrence interval (ARI), has been found to be the most significant predictor climatic characteristic in previous RFFA studies. This is to be expected given the strong causal link between intensity and peak flow. Importantly, this data is simple to obtain from the published data (e.g. ARR1987 Volume ). The use of rainfall intensity requires the selection of an appropriate storm burst duration and ARI. It seems to be logical to use a design rainfall intensity with a duration equal to the time of concentration (t c ), as suggested in the probabilistic rational method (I.E. Aust., 1987, 001). This is because as catchment area gets bigger, t c gets longer, which results in smaller average design rainfall intensity. However, there are different methods to estimate 109

138 CHAPTER 4 t c e.g. Bransby Williams formula and Friend formula (I.E. Aust., 001). For consistency, and ease of application, the formula recommended in ARR 1987 for VIC and eastern NSW, given by Equation 4.3, is adopted in this study t c 0.76A (4.3) where t c is time of concentration in hours and A is catchment area in km. In addition to the design rainfall intensity for a given ARI and t c (I ARI, tc ), rainfall intensities with fixed durations and ARIs are also trialled e.g. rainfall intensities with and 50 years ARIs and 1 and 1 hours durations. The various design rainfall intensities data for the selected study catchments are obtained using the intensity frequency duration (IFD) Calculator on the BOM website or the design data in ARR Volume. Mean annual rainfall: Mean annual rainfall has been used frequently in many previous RFFA studies. It may not have a direct link with flood peak, but it acts as a surrogate for some other characteristics (e.g. vegetation and wetness index) and is readily available. Thus, mean annual rainfall is included as a predictor variable in this study. The data for the mean annual rainfall for each catchment is extracted from the BOM Data CD of Annual Rainfall. Mean annual evaporation: This relates to the main loss component in the rainfall-runoff process. It is readily available and thus is included in this study. The mean annual areal potential evapotranspiration data for each catchment is extracted from the BOM Data CD of Evaporation. Slope: Slope is significant for any gravitational flow. With other catchment characteristics held constant the steeper the slope the greater the velocity of flow. Both overland and channel slope are important. Overland slope influences the velocity of shallow surface flow; hence, it can be expected to be of more importance for smaller catchments where the time spent in overland flow is a significant percentage of the total time needed for water to 110

139 CHAPTER 4 reach the catchment outlet. For larger catchments, channel slope is relatively more important than overland slope. There are several measures of slope; the most common of these are: Equal area slope: This is the slope of a straight line drawn on a profile of a stream such that the line passes through the outlet and has the same area under and above the stream profile. Average slope: This is equal to the total relief of the main stream divided by its length. S1085: This excludes the extremes of slope that can be found at either end of the mainstream. It is the ratio of the difference in elevation of the stream bed at 85% and 10% of its length from the catchment outlet, and 75% of the main stream length. Areal slope: This involves measuring the slope at a large number of points within a catchment and then determining an average areal slope. Taylor and Schwarz (195) slope: This assumes that velocity in each reach of a subdivided mainstream is related via the Manning s equation to the square root of slope. This index is equivalent to the slope of a uniform channel having the same length as the longest water course and an equal time of travel. In previous studies Strahler (1950) has shown that the overland slope and channel slope are strongly correlated. Benson (1959) found that S1085 gave the best prediction of the mean annual flood. The S1085 is closely correlated with the Taylor and Schwarz slope (NERC, 1975). From the different measures of slope, S1085 is deemed adequate and the simplest to estimate from 1:100,000 topographic maps and thus has been adopted in this study. Stream density: This is directly related to drainage efficiency of a catchment, and has been included in this study where possible. The definition of stream density is total stream length, which is taken as the sum of the length of all the blue lines in catchment as shown 111

140 CHAPTER 4 on 1:100,000 topographic maps, divided by catchment area. The length of the blue lines can be measured by opisometer/electronic distance meter or can be obtained using GIS. Stream density is not easy to measure and also the measured value depends on the map scale used. It should be retained in the final prediction equation only if it delivers significantly improved design flood estimates. Also, if it is used in final flood prediction equations, the procedure should stress the map scale to be used in its measurement. Forest area: The effect of vegetation on catchment response has been studied by many researchers (e.g. Flavell and Belstead, 1986; Williamson and Vand Der Wel, 1991; Flavell, 198). Forest reduces runoff by precipitation interception and transpiration. For a surface without a canopy or leaf litter layer, the interception loss is lower and overland flow travels more rapidly with less opportunity time for infiltration. Hence, Flavell (198) found that losses from rainfall decrease with increased clearing and that the runoff coefficient of the rational method increases with increased clearing. Fraction forest cover has been included in this study. The fraction of catchment covered by forest is estimated on 1:100,000 topographic maps by using a planimeter to measure the areas designated as dense and medium forest, and dense and medium scrub. Quaternary sediment area (VIC only): Storage directly affects the shape of the flood hydrograph, however defining storage as a single parameter is difficult. Quaternary sediment area appears to be an influential surrogate for storage, because it s a good indicator of floodplain extent variability in a catchment. Values for quaternary sediment area are determined from 1:50,000 geological maps. 4.8 SUMMARY The first part of this chapter has examined various aspects of the streamflow data collation adopted for this thesis. A total of 68 catchments have been selected from the continent of Australia (excluding the arid region see Figures 5 and 16). The annual instantaneous maximum flood series of the stations have been collected, gaps filled, rating curve extrapolation errors identified, trends and shifts in data analysis identified and outlier points censored. A sensitivity analysis has also been undertaken to understand the impacts of rating curve error on flood quantile estimation. The second part of this chapter has examined the candidate catchment characteristics for this study, a brief explanation has 11

141 CHAPTER 4 been given about each variable and how these data have been obtained. All the variables listed in Table 3 are used in the analyses presented in the subsequent chapters of this thesis. 113

142 CHAPTER 5 CHAPTER 5: RESULTS RFFA BASED ON FIXED REGIONS AND REGION OF INFLUENCE APPROACHES UNDER THE QUANTILE AND PARAMETER REGRESSION FRAMEWORKS 5.1 GENERAL This chapter develops flood prediction equations (for 6 average recurrence intervals (ARIs), which are, 5, 10, 0, 50 and 100 years) using both a fixed region and region of influence (ROI) approach in a quantile regression technique (QRT) and parameter regression technique (PRT) framework. The ROI approach is adopted to reduce the degree of heterogeneity present in Australian annual maximum flood regions to enhance the accuracy in design flood estimates. The Bayesian generalised least squares regression (BGLSR) technique is adopted for the parameter estimation which explicitly accounts for the inter-station correlation present in the annual maximum flood series (AMFS) data and it distinguishes between the sampling and model errors in regression analysis. The developed prediction equations allow for design flood or flood statistic estimates to be made at an ungauged catchment given the relevant catchment characteristics data. To assess the performances of the developed prediction equations, a Leave-one-out (LOO) validation procedure is adopted. The basic theory and assumptions associated with the QRT and PRT in a ROI BGLSR framework have been discussed in Chapter PUBLICATIONS Four journal papers (ERA, ranks A*, A, B and B) have been published based on the results presented in this chapter. These journal papers are given in Appendix A and noted below: Haddad, K. and Rahman, A. (01). Regional flood frequency analysis in eastern Australia: Bayesian GLS regression-based methods within fixed region and ROI framework: Quantile Regression vs. Parameter Regression Technique. Journal of Hydrology, , Haddad, K., Rahman, A. and Stedinger, J. R. (01). Regional Flood Frequency Analysis using Bayesian Generalized Least Squares: A Comparison between Quantile and Parameter Regression Techniques. Hydrological Processes, 5,

143 CHAPTER 5 Haddad, K., Rahman, A. and Kuczera, G. (011). Comparison of Ordinary and Generalised Least Squares Regression Models in Regional Flood Frequency Analysis: A Case Study for New South Wales. Australian Journal of Water Resources, 15(), 1-1. Haddad, K., Zaman, M. and Rahman, A. (010b). Regionalisation of skew for flood frequency analysis: a case study for eastern NSW. Australian Journal of Water Resources, 14(1), RESULTS FOR TASMANIA 5..1 SELECTING PREDICTOR VARIABLES WITH QRT AND PRT A total of 53 catchments were used from Tasmania for the analyses presented here. The locations of these catchments are shown in Figure 16. The AMFS record lengths of these 53 stations range from 19 to 74 years (mean 30 years, median 8 years and standard deviation 10 years). The catchment areas of these 53 stations range from 1.3 to 1,900 km (mean 33 km, median 158 km and standard deviation 417 km ). In the fixed region approach, all the 53 catchments were considered to have formed one region, however, one catchment was left out for cross-validation and the procedure was repeated 53 times to implement the LOO validation. Hence, the model data set contained 5 catchments in each iteration step. In the ROI approach, an optimum region was formed for each of the 53 catchments by starting with 15 stations in the first proposed region and then consecutively adding 1 station at each iteration step. Table 4 shows the different combinations of predictor variables for the Q 10 QRT model and the models for the first three parameters of the log Pearson Type 3 (LP3) distribution. Figure 17 and 18 show example plots of the statistics used in selecting the best set of predictor variables for the Q 10 and skew models. According to the model error variance (MEV), combinations 6, 16, 18, 0, 17, 19 and 4 were potential sets of predictor variables for the Q 10 model. Combinations 16, 18, 0, 17, 19 and 4 contained 3 to 4 predictor variables, while combinations 6 and 4 contained predictor variables. Indeed, combination 6 with the predictor variables (area and design rainfall intensity 50 I 1 ) showed the lowest MEV and the highest pseudo coefficient of determination ( R GLS ). The average variance of 115

144 CHAPTER 5 prediction old (AVPO), average variance of prediction new (AVPN), Akaike information criteria (AIC) and Bayesian information criteria (BIC) values favour combination 6 as well. Combination 6 was compared to combination 10 (the latter also contains predictor variables, area and design rainfall intensity I tc,10 ). Combination 6 had a smaller MEV while also showing the regression coefficient for variable 50 I 1 to be 5.5 times the posterior standard deviation away from zero, as compared to 4 times for I tc,10. Hence, combination 6 was finally selected as the best set of predictor variables for the Q 10 model. For the skew model, combination 4 showed the lowest MEV (0.034) and the highest R GLS (5%) (Figure 18), as well as the lowest AIC and BIC. Combination 1 without any explanatory variables ranked 13 out of the 16 possible combinations (MEV of 0.045); it also showed higher AVPO and AVPN as compared to combination 4, hence combination 4 was finally selected. A similar procedure was adopted in selecting the best set of predictor values for other models with the QRT and PRT. The sets of predictor variables selected as above were used in the LOO validation with fixed regions and ROI approaches. The Bayesian plausibility values (BPV) for the regression coefficients associated with the QRT over all the ARIs were between % and 8% for the variable area and 0.000% for design rainfall intensity 50 I 1. This justifies the inclusion of predictor variables area and 50 I 1 in the prediction equations for QRT. The BPVs for the skew model were 3% and 11% for area and 50 I 1, respectively indicating these variables are not very good predictors for skew. The BPVs for the mean model were close to 1% for both the predictor variables. For the standard deviation model, the BPV for the predictor variable rain was 1%. Regression equations developed for the QRT and PRT for the fixed region are given by Equations 5.1 to 5.9: ln(q ) = (area) + 3,35( 50 I 1 ) (5.1) ln(q 5 ) = (area) +.80( 50 I 1 ) (5.) ln(q 10 ) = (area) +.57( 50 I 1 ) (5.3) ln(q 0 ) = (area) +.39( 50 I 1 ) (5.4) ln(q 50 ) = (area) +.3( 50 I 1 ) (5.5) ln(q 100 ) = (area) +.0( 50 I 1 ) (5.6) 116

145 CHAPTER 5 ln(q ) = (area) ( I 1 ) (5.7) stdev = (rain) (5.8) skew = (area) + 1.0( 50 I 1 ) (5.9) It is reassuring to observe that the regression coefficients in the QRT set of equations vary in a regular fashion with increasing ARI. Table 4 Different combinations of predictor variables considered for the QRT models and the parameters of the LP3 distribution (QRT and PRT fixed region Tasmania) Combination Combinations for mean, standard deviation & skew Combinations for flood quantile models models 1 Const Const Const, area Const, area 3 Const, area, ( I 1 ) Const, area, I 1 4 Const, area, ( 50 I 1 ) Const, area, I 1 5 Const, area, ( I 1 ) Const, area, 50 I 1 6 Const, area, ( 50 I 1 ) Const, area, 50 I 1 7 Const, area, rain Const, area, rain 8 Const, area, forest Const, area, forest 9 Const, area, evap Const, area, forest, evap 10 Const, area, S1085 Const, area, I tc,ari 11 Const, area, sden Const, area, evap 1 Const, sden, rain Const, area, S Const, forest, rain Const, area, sden 14 Const, S1085, forest Const, sden, rain 15 Const, evap Const, forest, rain 16 Const, rain, evap Const, area, 50 I 1, rain 17 Const, rain Const, area, 50 I 1, sden 18 - Const, area, 50 I 1, rain, evap 19 - Const, area, 50 I 1, I tc,ari, evap 0 - Const, area, 50 I 1, I tc,ari, rain, evap 1 - Const, area, 50 I 1, I tc,ari, sden - Const, area, 50 I 1, I tc,ari, S Const, area, I tc,ari, evap 4 - Const, area, I tc,ari, rain 117

146 CHAPTER Const, area, I 1, I tc,ari MEV Standard Error of MEV R-sqd GLS Combination of Catchment Characteristics 100% 90% 80% 70% 60% 50% 40% 30% 0% 10% 0% AVPO AVPN AIC BIC Combination of Catchment Characteristics Figure 17 Selection of predictor variables for the BGLSR model for Q 10 (QRT, fixed region Tasmania), MEV = model error variance, AVPO = average variance of prediction (old), AVPN = average variance of prediction (new), AIC = Akaike information criterion, BIC = Bayesian information criterion, note R uses right hand axis GLS 118

147 CHAPTER MEV Standard Error of MEV R-sqd GLS 100% 0.5 AVPO AVPN AIC BIC % % % % % % % 0% % Combination of Catchment Characteristics 0% Combination of Catchment Characteristics Figure 18 Selection of predictor variables for the BGLSR model for skew 5.. PSUEDO ANOVA WITH QRT AND PRT MODELS FOR THE FIXED AND ROI REGIONS The pseudo analysis of variance (ANOVA) tables for the Q 0 and Q 100 models and the parameters of the LP3 distribution are presented in Tables 5 9 for the fixed regions and ROI. This is an extension of the ANOVA in ordinary least squares regression (OLSR) which does not recognise and correct for the expected sampling variance (Reis et al., 005). For the LP3 parameters, the sampling error increases as the order of moment increases i.e. the error variance ratio (EVR) increases with the order of the moments. An EVR of greater than 0.0 may indicate that the sampling variance is not negligible when compared to the model error variance, which suggests the need for a GLSR analysis (Gruber et al., 007). The ROI shows a reduced MEV (i.e. a reduced heterogeneity) as compared to the fixed regions, as fewer sites have been used. The model error dominates the regional analysis for the mean flood and the standard deviation models for both the fixed regions and the ROI. However, the ROI shows a higher EVR than the fixed region case, e.g. for the mean flood model the EVR is 0.0 for the ROI and 0.06 for the fixed region (Table 7). For the standard deviation model the EVR is 0.66 for the ROI and 0.54 for the fixed region, which is a 1% 119

148 CHAPTER 5 increase in EVR (Table 8). This shows that the ROI indeed deals better with heterogeneity, even if only slightly. The EVR values for the skew model are 9 and 9.3 for the fixed regions and ROI respectively (Tables 9), which are much higher than the recommended limit of 0.0. Again the GLSR should be the preferred modeling choice over the OLSR. Given that the skew model has a high sampling error component, an OLSR model would give misleading results. The advantage of GLSR is that it can distinguish between the variance due to model error and sampling error as explained in Chapter. Importantly, the Bayesian procedure adds another dimension to the analysis, by computing expectations over the entire posterior distribution. It has provided a more reasonable estimate of the MEV where the method of moment s estimator would have been grossly underestimated the model error variance, as the sampling error has overwhelmed the analysis. As far as the ROI is concerned, there is little change in the EVR as compared to the fixed region, as the skew model tends to include more stations in the regional analysis. Pseudo ANOVA tables were also prepared for the flood quantile models. For example, Tables 5 and 6 show the results for the Q 0 and Q 100 models, respectively. Here the ROI shows a higher EVR than the fixed region. This suggests that the BGLSR should be used with ROI in developing the flood quantile models, especially as the ARI increases. Table 5 Pseudo ANOVA table for Q0 model for Tasmania (QRT, fixed region and ROI) Source Degrees of freedom Sum of squares Fixed region ROI Equations Fixed region ROI Model k=3 k=3 n ( 0 ) = Model error n ( ) = n-k-1=48 n-k-1=30 Sampling error N = 5 N = 34 tr[ ( yˆ )] = Total Sum of the above n-1 = 103 n-1 = 67 = EVR

149 CHAPTER 5 Table 6 Pseudo ANOVA table for Q100 model for Tasmania (QRT, fixed region and ROI) Source Degrees of freedom Sum of squares Fixed region ROI Fixed ROI region Model k=3 k= Model error n-k-1=48 n-k-1= Sampling error N = 5 N = Total Sum of the above n-1 = 103 n-1 = 103 = EVR Table 7 Pseudo ANOVA table for the mean flood model for Tasmania (PRT, fixed region and ROI) Source Degrees of freedom Sum of squares Fixed region ROI Fixed region ROI Model k=3 k=3 n ( 0 ) = Model error n-k-1=48 n-k-1=4 n ( ) = Sampling error N = 5 N = 8 tr[ ( yˆ )] = Total Sum of the above n-1 = 103 n-1 = 55 = EVR Table 8 Pseudo ANOVA table for the standard deviation model for Tasmania (PRT, fixed region and ROI) Source Degrees of freedom Sum of squares Fixed Fixed region ROI region ROI Model k= k= Model error n-k-1=49 n-k-1= Sampling error N = 5 N = Total n-1 = 103 n-1 = 103 Sum of the above = EVR

150 CHAPTER 5 Table 9 Pseudo ANOVA table for the skew model for Tasmania (PRT, fixed region and ROI) Source Degrees of freedom Sum of squares Fixed region ROI Fixed ROI region Model k=3 k= Model error n-k-1=48 n-k-1= Sampling error N = 5 N = Total n-1 = 103 n-1 = 99 Sum of the above = EVR ASSESMENT OF MODEL ASSUMPTIONS AND REGRESSION DIAGNOSTICS To assess the underlying model assumptions (i.e. the normality of residuals), the plots of the standardised residuals vs. predicted values were examined. The predicted values were obtained from LOO validation. Figures 19 to 0 show the plots for the flood quantile Q 0 for the fixed region and ROI using the QRT and PRT framework. The underlying model assumptions are satisfied to a large extent, as 95% of the standardised residuals values fall between the limits of ±. The ROI shows standardised residuals closer to the ± limits. The results in Figures 19 to 0 reveal that the developed equations satisfy the normality of residual assumption quite satisfactorily. Also no specific pattern (heteroscedasicity) can be identified, with the standardised values being almost equally distributed below and above zero. Similar results were obtained for the skew, standard deviation and other flood quantile models, which are not shown in this thesis due to space constraints. 1

151 CHAPTER 5 Standardised Residual 3 BGLSR-QRT (FIXED REGION) BGLSR-PRT (FIXED REGION) Fitted LN(Q 0 ) Figure 19 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region, Tasmania) Standardised Residual 3 BGLSR-QRT (ROI) BGLSR-PRT (ROI) Fitted LN(Q 0 ) Figure 0 Plots of standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI, Tasmania) The QQ-plots of the standardised residuals (Equation 3.4) vs. normal score (Equation 3.43) for the fixed region (based on LOO validation) and ROI were also examined. Figures 1 and present results for the Q 0 flood quantile model, which shows that all the points closely follow a straight line. This indicates that the assumption of normality and the homogeneity of variance of the standardised residuals have largely been satisfied. The standardised residuals are indeed normally and independently distributed N(0,1) with mean 0 and variance 1 as the slope of the best fit line in the QQ-plot, which can be interpreted as the standard deviation of the normal score (Z score) of the quantile, should approach 1 and the intercept, which is the mean of the normal score of the quantile should approach 0 as the number of sites increases. It can be observed from Figures 1 and that the fitted lines 13

152 CHAPTER 5 for the developed models pass through the origin (0, 0) and have a slope approximately equal to one. The ROI approach approximates the normality of the residuals slightly better (i.e. a better match with the fitted line) than the fixed region approach. Similar results were also found for the mean, standard deviation, skew and other flood quantile models, which are not shown in this thesis due to space constraints. Normal Score ARI 0 (FIXED REGION) BGLSR-QRT - BGLSR-PRT Standardised Residual Figure 1 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, Tasmania) Normal Score ARI 0 (ROI) BGLSR-QRT BGLSR-PRT -3 Standardised Residual Figure QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, Tasmania) To assess the adequacy of the BGLSR models, Cook s distance values were also calculated. No outlier/influential sites were found for the mean, standard deviation and flood quantile models. For the skew model (Figure 3), sites 8 and 50 were above the threshold value of (i.e. 4/53, where 53 is the total number of sites). Site 8 showed the 14

153 CHAPTER 5 largest standardised residual value. The flow data, site history and flood frequency plots of these two sites were examined. It was found that site 8 had a record length of 33 years (in the top 0%) and a very small annual maximum flow value in 1968, which was not surprising as this was a drought year. This small flow caused a high negative skew of for the site. Site 50 had record length of 46 years (5 th largest record length) and a skew value 1.15, and it did show the largest influence value (Figure 3). The regression analysis was repeated by removing these two sites. Indeed site 8 did influence the analysis with a notable decrease in the expected MEV ( ) from 0.05 to The AVPO and AVPN dropped notably from and to and 0.049, respectively. The GLS R also increased from 36% to 53%, which is deemed to be a remarkable increase. The effective record length based on AVPN of in this case is 1 years, which is nearly 4 times the average record length for Tasmania. Site 8 did therefore influence the results notably and was therefore removed from the database in subsequent analyses. The removal of site 50 resulted in little improvement in the skew model with a negligible increase in R GLS (55%) and a slightly smaller (0.03) and was therefore retained. 0.5 GLSR Cooks D Site No. Figure 3 Cook s distance (D i ) for locating outlier sites for skew model based on variable combination 4 The summary of various regression diagnostics (the relevant equations are described in section 3.8) is provided in Table 10. This shows that for the mean flood model, the MEV and average standard error of prediction (SEP) are much higher than those of the standard deviation and skew models. This indicates that the mean flood models exhibits a higher 15

154 CHAPTER 5 degree of heterogeneity than the standard deviation and skew models, this result is also supported by the ANOVA analysis. Indeed the issue here is that sampling error becomes larger as the order of the moment increases, therefore, in case of the skew, the spatial variation is a second order effect that is not really detectable. For the mean flood model, the ROI shows a MEV which is 11% smaller than for the fixed region. Also, the R GLS value for the mean flood model with the ROI is % higher than for the fixed region. The reasonable reduction in MEV alone indicates that the ROI should be preferred over the fixed region analysis for developing the mean flood model. For the standard deviation model, ROI also shows 8% smaller SEP and 5% higher R GLS values. This indicates that the ROI is preferable to the fixed region for the standard deviation model. What is also noteworthy (as seen from Table 10) is that the SEP% for the skew model is slightly larger for the ROI than the fixed region analysis. This may be due to the fact that, if the number of sites are reduced (smaller ROI), the predictive variance may be slightly inflated in the skew region. The R GLS values for the skew models are similar for the fixed region and ROI, with the latter providing only a % increase. One can see from Table 10 that the SEP values for all the flood quanitle models are % to 11% smaller for the ROI cases than the fixed region; the best result is obtained for ARI = years. Also, the R GLS values for ROI cases are 3% to 6% higher than the fixed region. These results show that the ROI generally outperforms the fixed region approach. Table 10 Regression diagnostics for fixed region and ROI for Tasmania Model Fixed region ROI Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q 100 MEV AVP SEP (%) R GLS (%) MEV AVP SEP (%) R GLS (%)

155 CHAPTER POSSIBLE SUBREGIONS IN TASMANIA Table 11 shows the number of sites and associated MEVs for the ROI and fixed region models. This shows that the ROI mean flood model has fewer sites on average (8 out of 5 sites i.e. 54%) than the standard deviation and skew models. The ROI skew model has the highest number of sites which includes nearly all the sites in Tasmania (50 out of 5 i.e. 96%). The MEVs for all the ROI models (except the skew model) are smaller than the fixed region models. This shows that the fixed region models experience a greater heterogeneity than the ROI. If the fixed regions are made too large, the model error will be inflated by heterogeneity that will go unaccounted for by the catchment characteristics. Figure 4 shows the resulting sub-regions in Tasmania (with minimum MEVs) for the ROI mean flood and skew models. For the mean flood and skew models, there are two distinct sub-regions. The regions can be classified as east and west Tasmania for which there are two distinct types of rainfall regimes and districts. The significance of this is that if spatial variations do exist in the hydrological statistic of interest, they are most likely to be captured by the ROI, as has been the case in this study for Tasmania. The results of this analysis concur with previous studies (McConachy et al., 003, Gamble et al., 1998, Xuereb et al., 001) which showed that large rainfalls over Tasmania are not meteorologically homogeneous. In the east of the state, the largest rainfall events occur in the warmer spring and summer months when low pressure systems in the Tasman Sea can direct an easterly onshore air flow over Tasmania. The heaviest rainfalls in the west of the state are due to the passage of fronts, sometimes associated with an intense extratropical cyclone with a westerly or southwesterly airstream (Xuereb et al., 001). Table 11 Model error variances associated with fixed region and ROI for Tasmania (n = number of sites in the region) Parameter/ quantiles ROI (n) ˆ Fixed region (n) ˆ Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q

156 CHAPTER 5 (a) (b) Figure 4 Spatial variations of the grouped minimum model error variances for Tasmania (a) mean flood model and (b) skew model 5..5 EVALUATION STATISTICS Table 1 presents the relative root mean square error (RMSE r ) (Equation 3.45) and relative error (RE r ) (Equation 3.44) values for the PRT and QRT models with both the fixed region and ROI. In terms of RMSE r, ROI clearly gives smaller values than the fixed regions for all the ARIs. The PRT-ROI shows smaller RMSE r values than the QRT-ROI for all the ARIs, however for ARIs of 5, 10 and 0 years, the increase is noticeable (i.e. 0 to 30 %). In terms of RE r, ROI gives up to 9% smaller values than the fixed regions. The PRT-ROI gives larger values of RE r (by 13%) for both the 50 and 100 years ARIs. For ARIs of to 0 years, the QRT-ROI gives smaller RE r values (by 1% to 13%) than the PRT-ROI. Finally the results of counting the Q pred /Q obs (r r ) ratios for the QRT and PRT for the ROI and fixed regions are provided in Tables 13 and 14. The QRT-ROI has 85% of the r r values in the desirable range, compared to 81% for the QRT-fixed region. The PRT-ROI has 78% of the r r values in the desirable range, compared to 74% for the PRT-fixed region. These results show that ROI performs better than the fixed regions with both the QRT and PRT. The PRT-ROI shows 16% underestimation as compared to 8% for the QRT-ROI. The cases with overestimation were very similar for both the methods. 18

157 CHAPTER 5 Table 1 Evaluation statistics (RMSE r and RE r ) from leave-one-out (LOO) validation for Tasmania Model RMSE r (%) RE r (%) PRT QRT PRT QRT Fixed ROI Fixed ROI Fixed ROI Fixed ROI region region region region Q Q Q Q Q Q Table 13 Summary of counts/percentages based on the rr values for QRT and PRT for Tasmania (fixed region). U = gross underestimation, D = desirable range and O = gross overestimation Model Count (QRT) Percent (QRT) Count (PRT) Percent (PRT) U D O U D O U D O U D O Q Q Q Q Q Q Sum / average Table 14 Summary of counts/percentages based on the rr values for QRT and PRT for Tasmania (ROI). U = gross underestimation, D = desirable range and O = gross overestimation ARI Count (QRT) Percent (QRT) Count (PRT) Percent (PRT) (years) U D O U D O U D O U D O Sum / average

158 CHAPTER SECTION SUMMARY This section of the thesis has compared the fixed region and ROI approaches for the state of Tasmania. A BGLSR approach was used to develop prediction equations for flood quantiles of ARIs of to 100 years (for QRT) and the first three parameters of the LP3 distribution (for PRT). It has been found that area and design rainfall intensity are significant predictors for both the QRT and PRT based prediction equations. When compared to the fixed region approach, the ROI with both QRT and PRT shows improvements by reducing the negative influence of regional heterogeneity, with a decrease in the model error variance, average standard error of prediction and an increase in the average pseudo R GLS. Both the standardised residual and QQ-plots of the ROI approach satisfy the underlying model assumptions slightly better than those of the fixed region. It has also been observed that both the QRT-ROI and PRT-ROI produce similar average root mean square error, median relative error and median Q pred /Q obs ratio values. Overall, the PRT-ROI and QRT-ROI have performed very similarly for Tasmania. The ROI approach outperforms the fixed region approach for Tasmania. 5.4 RESULTS FOR NEW SOUTH WALES, VICTORIA AND QUEENSLAND The analysis undertaken in this section makes use of observed AMFS data of catchments ranging in areas from 3 to 1010 km. The finally selected data set consists of n = 399 catchments (Figure 16) with AMFS record lengths ranging from 5 to 94 years (maximum record length for New South wales (NSW): 75 years, mean and standard deviation: 37 and 11 years, respectively; maximum record length for Victoria (VIC): 5 years, mean and standard deviation: 33 and 5 years, respectively and maximum record length for Queensland (QLD): 94 years, mean and standard deviation: 40 and 15 years, respectively). In the fixed region approach, all the catchments within a state boundary were considered to have formed one region; however, one catchment was left out for cross-validation and the procedure was repeated n times to implement the LOO validation scheme. In the ROI approach, an optimum region was formed for each of the n catchments by starting with 15 stations and then consecutively adding 5 stations at each iteration (see section 3.7 for more details). 130

159 CHAPTER SELECTING PREDICTOR VARIABLES WITH QRT AND PRT The stepwise procedure for selecting the best set of catchment characteristics predictors resulted in the following equations for the LP3 mean (), standard deviation (), skewness () and the flood quantiles (Q ARI ) for each of the states of NSW, VIC and QLD. The regression equations are presented in general form below, while the final results of the equations for NSW are provided in Table 15. The final results of VIC and QLD can be seen in Appendix B. = (area) + ( I 1 ) for NSW, VIC and QLD (5.10) = 0-1 (rain) - (S1085) for NSW (5.11) = (area) - (forest) for NSW (5.1) = 0-1 (rain) + (evap) for VIC (5.13) = (rain) - (evap) for VIC (5.14) = 0-1 (area) - ( I 1 ) for QLD (5.15) = ( 50 I 7 ) + (rain) for QLD (5.16) ln(q ARI ) = (area) + (I tc,ari ) for NSW, VIC and QLD (5.17) Tables 16 and 17 summarise the model error variance (MEV) as expressed by its posterior mean value, for the regional models of the three LP3 parameters and the flood quantiles Q, Q 10 and Q 100 for each of the selected combinations of catchment characteristics for NSW. Table 15 Summary of the final BGLSR results for NSW BGLSR model (NSW) Regression coefficient Posterior moment Mean Standard deviation Mean (µ) (constant) (area) ( I 1 ) Standard deviation () (constant) (rain) (S1085) Skewness () (constant) (area) (forest) Flood quantiles Q ARI= 0 (constant) (area)

160 (I tc,ari = ) Q ARI= (constant) (area) (I tc,ari =5 ) Q ARI= (constant) (area) (I tc,ari =10 ) Q ARI= (constant) (area) (I tc,ari =0 ) Q ARI= (constant) (area) (I tc,ari =50 ) Q ARI= (constant) (area) (I tc,ari =100 ) CHAPTER 5 Also provided in Tables 16 and 17 is the summary of the statistical measures used i.e. AVPO and AVPN, AIC, BIC, BPV and pseudo R ( R GLS ) to assess the best combination of catchment characteristics to predict the three parameters and flood quantiles of the LP3 distribution. Figure 5 shows the MEV, standard error of the MEV and R GLS values for the skew model. Combination 9 with a constant and two predictor variables area and forest showed the lowest MEV and the highest RGLS as well as the lowest AIC and BIC values. However, the lowest AVPO and AVPN values were found for combination 1 (a constant value, representing the intercept term in the regression model - see Figure 5). The BPV values were used to carry out a hypothesis test (at the 5% significance level) on the predictors of combination 9. The BPVs were found to be 6% and 7% for area and forest, respectively, while this showed the variables are not to be significant; however, these values are not considered to be notably high. Both the posterior coefficients 1 and values were smaller than two posterior standard deviations (for the respective case) away from zero supporting the results from the BPV test that these variables are not really significant. In this case, it may be possible to adopt a regional average skew value for the entire NSW state without using any prediction equation/predictor variable in the regression equation. This finding is consistent with Gruber and Stedinger (008) who found that a constant 13

161 CHAPTER 5 model for a regional skewness was the best model for a large region in the southeastern part of the United States. This is also supported by the fact that there was only a modest difference in the MEV values. Combination 9 and 1 however were both adopted and tested in this study with the PRT approach. A similar outcome was observed for the standard deviation model where the MEVs were very similar for combinations 1 and 1 (figure not shown due to space constraint). Combination 1 was adopted that had slope and rain as predictor variables. Indeed, AVPO, AVPN, BIC and AIC values were the lowest for this combination. Both the posterior coefficients 1 and were well established in the regression equations being more than two times the respective posterior standard deviation away from zero. The BPVs were % indicating the relatively higher significance of these two variables. For the mean flood, combination 6 (constant, area and I 1, ) had the smallest MEV. The posterior coefficients of 1 and in this combination were at least 5 and 11 times the respective posterior standard deviation away from zero, which shows that 1 and are well established in the prediction equation. Indeed, all the statistical criteria were found to be in favour of combination MEV Standard Error of MEV R-sqd GLS AVPO AVPN AIC BIC Combination of Catchment Characterisitcs Combination of Catchment Characteristics Figure 5 Selection of predictor variables for the BGLSR model for the skew (note that right-hand axis) R GLS uses the 133

162 CHAPTER 5 Figure 6 shows an example plot of the statistics used in selecting the best set of predictor variables for the fixed region flood quantile (QRT) models. According to the MEV values, combinations 19, 18, 0, 3, 16, 6, 4, 5 and 10 were potential sets of predictor variables for the Q 10 model. Combinations 18, 19, 0 and 3 contained 3 to 4 predictor variables while combinations 16, 6, 4, 5 and 10 contained predictor variables with similar MEVs and R GLS values. The AVPO, AVPN, AIC and BIC values all favoured combination 10, and hence this was finally selected as the best set of predictor variables for the Q 10 model which includes area and design rainfall intensity I tc,10. Both posterior coefficients 1 and were found to be 9 times the respective posterior standard deviation away from zero suggesting that these two variables are well established in the prediction equation. Indeed, based on similar findings, combination 10 was selected for all the flood quantile prediction equations (ARIs = 100 years). The BPVs for the regression coefficients associated with the variable area and design rainfall intensity I tc,ari for the QRT over all the ARIs were found to be significant with values smaller than MEV Standard Error of MEV R-sqd GLSR Combination of Catchment Characteristics 90% 80% 70% 60% 50% 40% 30% 0% 10% 0% 134

163 CHAPTER AVPO AVPN AIC BIC Combination of Catchment Characteristics Figure 6 Selection of predictor variables for the BGLSR model for Q10 model (note that uses the right-hand axis), (QRT, fixed region NSW), MEV = model error variance, AVPO = average variance of prediction (old), AVPN = average variance of prediction (new) AIC = Akaike information criteria, BIC = Bayesian information criteria 135

164 CHAPTER 5 Table 16 Summary of the catchment characteristics and statistical measures used in the stepwise regression for the parameters of the LP3 distribution for NSW Combination Catchment LP3 parameter characteristics a Mean Standard deviation Skewness AVPO AVPN AIC BIC BPV R AVPO AVPN AIC BIC BPV GLS R AVPO AVPN AIC BIC BPV GLS R GLS % % % 1 Const % % <0.1 0% Const, area <0.1, , 0 39% , 10 4% % 3 Const, area, I 1 0, 0, 0,13, <0.1, % % , 68 5% 4 Const, area, 50 I 1 0, 0, 0,10, <0.1, % % , 7 5% 5 Const, area, 50 I 1 0, 0, 0,13, <0.1, % % , 7 51% 6 Const, area, I 1 0, 0, 0,14, <0.1, % % , 86 50% 7 Const, area, S , 0, % ,9, 8 8% <0.1, 4, 9 49% 8 Const, area, sden , 0, % ,14, 58 4% <0.1, 4, 81 49% 9 Const, area, forest , 0, 60 39% ,5, 7 9% <0.1, 6, 7 65% 10 Const, area, evap , 0, % ,14, 6 6% <0.1,, 49 53% 11 Const, area, rain 0, 0, 0,40, <0.1, % % , 87 49% 1 Const, rain, S ,37, 16 % ,, 1 35% ,74, 87 10% 13 Const, sden, S ,0.8, 8 9% ,60, 5 8% ,74, 51 14% 14 Const, evap, sden ,0.1, 36 18% ,7, 61 3% ,50, 38 17% 15 Const, forest , 3 6% , 11 4% , 4 51% 16 Const, S1085, forest , 17, 7% , 7, 3 9% , 17, 60% a Const is a constant term. Refer to text in Chapter 4 for a full description of the catchment characteristics predictor variables. 136

165 CHAPTER 5 Table 17 Summary of the catchment characteristics and statistical measures used in the forward stepwise regression for the flood quantiles of the LP3 distribution (ARIs =, 10 and 100 years) for NSW Combinatio n Catchment characteristic LP3 flood quantiles s a ARI = ARI = 10 ARI = 100 AVP O AVP N 1 Const Const, area Const, area, I Const, area, 0.3 I Const, area, I Const, area, I Const, area, 0.7 S Const, area, 0.6 sden Const, area, 0.6 sden, forest Const, area, 0. I tc,ari Const, area, 0.6 forest AI C BI C BPV % 0 0% 0, 0, 0 39% 0, 0, 0 71% 0, 0, 0 75% 0, 0, 0 73% 0, 0, 0 75% 0, 0, 69 39% 0, 0, % 0, 0, 1, 9 46% 0, 0, 0 75% 0, 0, 4% R AVP GLS O AVP N AI C BI C BPV % R AVP GLS O AVP N 0 0% , 0, 0 56% , 0, 0 78% , 0, 0 78% , 0, 0 77% , 0, 0 79% , 0, 34 56% , 0, 0. 55% , 0, 1, 90 54% , 0, 0 79% , 0, 40 57% AI C BI C BPV % R GLS 0 0% 0, 0, 0 48% 0, 0, 0 67% 0, 0, 0 7% 0, 0, 0 67% 0, 0, 0 68% 0, 0, 63 48% 0, 0, % 0, 0, 1, 0 51% 0, 0, 0 65% 0, 0, 59 48% 137

166 CHAPTER 5 1 Const, area, evap 13 Const, area, rain 14 Const, rain, S Const, sden, S Const, area, 50 I 1, S Const, area, 50 I 1, rain 18 Const, area, 50 I 1, S1085, forest 19 Const, area, 50 I 1, I tc,ari, forest 0 Const, area, 50 I 1, I tc,ari, S1085, forest Const, area, 0.3 I tc,ari, rain Const, area, 0.3 I tc,ari, evap Const, area, 0.3 I tc,ari, forest , 0, 0. 50% 0, 0, 0. 73% 0, 0, 4 19% 0, 15, 8% 0, 0, 0, 40 83% 0, 0, 0, % 0, 0, 0, 48, 79 7% 0, 0,,15, 16,7 0 74% 0, 0, 15, 18, 70,7 8 73% 0, 0, 0, 76% 0, 0, 0, 86 74% 0, 0, 0, 98 73% , 0, 0 0, 0, 0 69% % , 6, 1 11% , 5,0. 1 9% , 0, 0, 35 79% , 0, 0, 79% , 0, 0, 55, 75 0, 0,,, 43,70 0, 0, 3, 44, 95,90 80% % % , 0, 0, 76 78% , 0, 0, 80 79% , 0, 0, 8 79% , 0, % 0, 0, % 0, 36, 0.7 8% 0, 7,0. 1 8% 0, 0, 0, 6 67% 0, 0, 0, 8 67% 0, 0, 0, 55, 79 75% 0, 0,,10, 80,90 75% 0, 0, 7, 90, 95,90 73% 0, 0, 0, 81 66% 0, 0, 0, 95 65% 0, 0, 0, 98 69% 138

167 CHAPTER 5 4 Const, area, I tc,ari, S Const, area, I 1, I tc,ari , 0, 0, 9 73% 0, 0, 46, 0 74% , 0, 0, 50 79% , 0, 59, 1 79% a Const is a constant term. Refer to text in Chapter 4 for a full description of the catchment characteristics predictor variables , 0, 0, 95 65% 0, 0, 49, 0 67% 139

168 CHAPTER REGION OF INFLUENCE VS. FIXED REGIONS FOR PARAMETER AND QUANTILE REGRESSION TECHNIQUES REGRESSION DIAGNOSTICS PSEUDO ANALYSIS OF VARIANCE The pseudo analysis of variance (ANOVA) tables for the Q 0 model and the parameters of the LP3 distribution (mean and skew are shown only due to space constraint) are presented in Tables 18 to 0 for the fixed and ROI regions for NSW, VIC and QLD. The pseudo ANOVA table describes how the total variation among the ŷ i values (predicted values) can be apportioned between that explained by the model error and sampling error. This is an extension of the ANOVA in the OLSR which does not recognise and correct for the expected sampling variance (Reis et al., 005). An error variance ratio (EVR) is used in Pseudo ANOVA, which is the ratio of sampling error variance to model error variance. An EVR of greater than 0.0 may indicate that the sampling variance is not negligible when compared to the model error variance, which suggests the need for a GLSR analysis (Gruber et al., 007). For the LP3 parameters, the sampling error (i.e. EVR) increases as the order of moment increases, this can be clearly seen for all the three states in Tables 18 and 19. For example, for NSW the EVR for the mean flood model for ROI is 0.3 (i.e. the sampling error is only 0.3 times of the model error) (see Table 18), the corresponding EVR value for the skew model (Table 19) is 18 (i.e. the sampling error is 18 times of the model error). The ROI shows a reduced model error variance for all the three states (i.e. a reduced heterogeneity), in particular for the mean flood model, as compared to the fixed regions. For example, for NSW state (Table 18) the model error variances for the fixed region and ROI are 7.7 and 16.5, respectively. It was found that the model error dominated the regional analysis for the mean flood and the standard deviation models (results not shown) for both the fixed regions and ROI for all the states. For the ROI, the mean flood model also shows a much higher model error variance than those of the standard deviation and skew models. These results based on the model error variance alone indicate that the mean flood has the greater level of heterogeneity associated with its regionalisation as compared to the standard deviation and skew. The ROI, however shows a higher EVR than the fixed regions e.g. for the mean flood model for NSW, the EVR is 0.30 for the ROI and 0.17 for the fixed region (see Table 18), Table 18 also provides the 140

169 CHAPTER 5 EVR results for VIC and QLD states, which show a similar outcome as of NSW. For the standard deviation model for NSW the EVR is 0.77 for the ROI and 0.35 for the fixed region, again similar results were found for VIC and QLD states as of NSW. The EVR values for the skew models of NSW, VIC and QLD are shown in Table 18. It can be observed from Table 18 that the EVR values range from 8 to 19 and 9.5 to 19 for the fixed regions and ROI, respectively (Table 19), which are much higher than the recommended limit of 0.0. In this relation, two important points may be noted below: (i) (ii) This result clearly indicates that the GLSR is the preferred modeling option over the OLSR for the skew model. An OLSR model for the skew would have clearly given misleading results as it does not distinguish between the model and sampling errors as found in similar previous studies (e.g. Reis et al., 005 and Haddad et al., 010b). Importantly, what is clear is that if a method of moment estimator was used to estimate the model error variance ( ) for the skew model, the model error variance would have been grossly underestimated as the sampling error heavily dominated the regional analysis. A more reasonable estimate of the model error variance has been achieved with the Bayesian procedure as it represents the values of by computing expectations over the entire posterior distribution. Similar results were found by Reis et al. (005), Gruber and Stedinger (008) and Haddad et al. (010b). As far as the ROI approach is concerned there is little change in the EVR values as compared to the fixed region approach for all the three states as the skew model tends to include more stations in the regional analysis. Table 18 Pseudo ANOVA table for the mean flood model (PRT, fixed region and ROI, NSW, VIC and QLD states) (Here n = number of sites in the region, k = number of predictors in the regression equation, EVR = error variance ratio, = model error variance when no 0 predictor variable is used in the regression model, = model error variance when predictor variable is used in the regression model and tr[ ( yˆ )] = sum of the diagonals of the sampling covariance matrix) 141

170 CHAPTER 5 Source Degrees of freedom Sum of squares Fixed NSW Fixed region ROI region ROI Model k=3 k=3 n ( 0 ) Model error n-k-1=9 n-k-1=3 n ( ) Sampling error n = 96 n = 36 tr[ ( yˆ )] Sum of the above Total n-1 = 191 n-1 = EVR VIC Model k=3 k= Model error n-k-1=17 n-k-1= Sampling error n = 131 n = Total n-1 = 61 n-1 = 85 Sum of the above EVR QLD Model k=3 k= Model error n-k-1=168 n-k-1=34 39 Sampling error n = 17 n = Total n-1 = 343 n-1 = 75 Sum of the above EVR Table 19 Pseudo ANOVA table for the skew model (PRT, fixed region and ROI, NSW, VIC and QLD states) (variables are explained in Table 18 caption) Source Degrees of freedom Sum of squares Fixed NSW Fixed region ROI region ROI Model k=3 k=3 n ( 0 ) Model error n-k-1=9 n-k-1=91 n ( ) Sampling error n = 96 n = 95 tr[ ( yˆ )] 4 3 Total Sum of the above 5 3 n-1 = 191 n-1 = 189 EVR VIC Model k=3 k= Model error n-k-1=17 n-k-1= Sampling error n = 131 n = Total Sum of the above n-1 = 61 n-1 = 33 EVR QLD Model k=3 k= Model error n-k-1=168 n-k-1= Sampling error n = 17 n = Total n-1 = 343 n-1 = 99 Sum of the above EVR

171 CHAPTER 5 The pseudo ANOVA tables were also prepared for all the flood quantile models (i.e. QRT models). The results for the Q 0 for all the three states are shown in Table 0. Here the ROI shows a higher EVR values than that of the fixed region. Also, the sampling error generally increases with increasing ARIs. The reduction in the model error variance as seen in Table 0 for all the three states is due to the fact that ROI has found an optimum number of sites based on the minimum model error variance which generally uses fewer sites than that of the fixed region approach. This indeed suggests that sub regions may exist in larger state. The flood quantile Q was found to experience the lowest EVR values for NSW and QLD for both the fixed region and ROI as compared to the Q 0 and Q 100 model results. This reflects the much greater spatial variability of the mean which is dominated by local catchment factors (as compared to the higher moments). This is reflected in the Q flood as it is very close to the mean flood magnitude. The Q 0 shows an EVR of 0.43, 0.30 and 0.97, respectively for NSW, VIC and QLD states (see Table 0) for ROI approach, which suggests that the BGLSR combined with ROI should be the preferred option when modelling the larger ARI quantiles, even though in this particular case the ROI has been impacted by the relatively large model error variances that have dominated the regional flood quantile modelling results. Table 0 Pseudo ANOVA table for Q 0 model (QRT, fixed region and ROI for NSW, VIC and QLD states) (variables are explained in Table 18 caption) Source Degrees of freedom Sum of squares Fixed NSW Fixed region ROI region ROI Model k=3 k=3 n ( 0 ) Model error n-k-1=9 n-k-1=48 n ( ) Sampling error n = 96 n = 5 tr[ ( yˆ )] Total Sum of the above 9 86 n-1 = 191 n-1 = 103 EVR VIC Model k=3 k= Model error n-k-1=17 n-k-1= Sampling error n = 131 n = Total Sum of the above n-1 = 61 n-1 = 103 EVR QLD 143

172 CHAPTER 5 Model k=3 k= Model error n-k-1=168 n-k-1= Sampling error n = 17 n = Total n-1 = 343 n-1 = 161 Sum of the above EVR REGRESSION DIAGNOSTICS MODEL ADEQUACY AND OUTLIER ANANLYSIS To assess the underlying model assumptions (i.e. the normality of residuals), the plots of the standardised residuals [Equation (3.4)] vs. fitted quantiles were examined for all the flood quantiles (estimated from QRT and PRT) and the parameters of the LP3 distribution for all the three states. The predicted values were obtained from the LOO validation procedure. Figure 7 shows the plot for the Q 0 model for the state of NSW. 3 BGLSR-QRT (FIXED REGION) BGLSR-PRT (FIXED REGION) Fitted ln(q 0 ) Standardised Residual Standardised Residual 3 BGLSR-QRT (ROI) BGLSR-PRT (ROI) Fitted ln(q 0 ) Figure 7 Plots of the standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, fixed region and ROI, NSW) 144

173 CHAPTER 5 If the underlying model assumption is satisfied to a large extent the standardised residual values should not exceed the ± limits; in practice, 95% of the standardised residuals should fall between ±. The result in Figure 7 reveals that the developed flood quantiles from the prediction equations satisfy the normality of residual assumption quite satisfactorily for both the fixed and ROI approaches. Also no specific pattern (heteroscedasicity) can be identified with the standardised values, which are being almost equally distributed below and above zero. What is noteworthy is that ROI clearly provides fewer genuine outliers for both the quantiles estimated by the QRT and PRT methods than the fixed region approach. This indeed demonstrates the superiority of the ROI over the fixed region approach. Similar results were observed for the states of VIC and QLD. The figures associated with VIC and QLD can be seen in Appendix B. The QQ-plots of the standardised residuals [Equation (3.4)] vs. normal score [Equation (3.43)] for the fixed region (based on LOO validation) and ROI were then examined. The results for the Q 0 model for NSW are shown in Figure 8, which reveals that all the points closely follow a straight line; this is especially noticeable for the ROI approach for both the QRT and PRT methods. This indicates that the assumption of normality and the homogeneity of variance of the standardised residuals are better approximated with the ROI approach. Overall, no genuine outliers can be detected for the flood quantiles estimated by the QRT and PRT on a regional scale. ARI 0 (FIXED REGION) 3 Normal Score BGLSR-QRT BGLSR-PRT -3 Standardised Residual 145

174 CHAPTER 5 ARI 0 (ROI) 3 1 Normal Score BGLSR-QRT BGLSR-PRT -3 Standardised Residual Figure 8 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, fixed region, ROI, NSW) If the standardised residuals are indeed normally and independently distributed N(0, 1) with mean 0 and variance 1 then the slope of the best fit line in the QQ-plot, which can be interpreted as the standard deviation of the normal score (Z score) of the quantile, should approach 1 and the intercept, which is the mean of the normal score of the quantile should approach 0 as the number of sites increases. Figure 8 indeed shows that the fitted lines for the developed models pass approximately through the origin (0, 0) and have a slope approximately equal to one. It can be seen that the results of the ROI approach satisfy the model assumptions relatively better than the fixed region approach. The superiority of the ROI approach again here is demonstrated. Similar results were observed for VIC and QLD states. The figures associated with VIC and QLD can be seen in Appendix B. The assumption of the normality of the residuals for all the three states (NSW, VIC and QLD) could not be rejected at the 10% level of significance using the Anderson-Darling and Kolmogorov-Smirnov tests for normality. Below is presented the residual analysis results of the ROI method for the PRT using a weighted regional average standard deviation and skew values, which are weighted by the error covariance matrix (i.e. no predictor variables in the regression equation considered in this case) for the state of NSW (as an example). The main aspect of this analysis is to determine if there is any reasonable loss in accuracy and efficiency especially in the flood quantile estimation of the mid to higher ARIs (i.e. 0 to 100 years) when using a weighted regional average standard deviation and skew (obtained as above) as compared to ones with 146

175 CHAPTER 5 predictor variables. It should be stressed here that this weighted regional average standard deviation and skew do vary from site to site as each site has a unique ROI. The standardised residuals vs. the fitted quantile plot of Q 0 is shown in Figure 9 that superimposes the estimate made by the QRT-ROI, PRT-ROI and the PRT-ROI that uses a weighted regional average standard deviation and skew estimate. Indeed, one can observe that the PRT-ROI estimate of Q 0 with the weighted regional average standard deviation and skew performs equally well as the competing models. Nearly all the standardised residuals fall within the limits, suggesting that the use of predictor variables in the estimation of standard deviation and skew does not really add much meaningful information to the analysis. The QQ-plot (Figure 30) of the competing models shows that the use of a weighted regional average standard deviation and skew does not result in any major gross errors in the final quantile estimates. The residual analysis also reveals that the major assumptions of the regression have been largely satisfied (i.e. normality of the residuals). The results based on the evaluation statistics are given in section Standardised Residual BGLSR-QRT (ROI) BGLSR-PRT (ROI) BGLSR-PRT (ROI, Regional weighted average Stdev and Skew) Fitted ln(q 0 ) Figure 9 Plots of the standardised residuals vs. predicted values for ARI of 0 years (QRT and PRT, ROI and PRT-ROI with weighted average standard deviation and skew, NSW) 147

176 CHAPTER 5 ARI 0 (ROI) 3 1 Normal Score Standardised Residual BGLSR-QRT BGLSR-PRT BGLSR-PRT (ROI, regional weighted ave, stdev and skew) Figure 30 QQ-plot of the standardised residuals vs. Z score for ARI of 0 years (QRT and PRT, ROI, and PRT ROI with weighted average standard deviation and skew, NSW) DIAGNOSTIC STATISTICS The summary of the various regression diagnostics (as described in section 3.8 and Equation (3.41)) is provided in Table 1 for NSW, VIC and QLD states. This shows that for the mean flood model (for all the three states), MEV and SEP are much higher than those of the standard deviation and skew models. This indicates that the mean flood model exhibits a higher degree of heterogeneity than the standard deviation and skew models. This result supports the pseudo ANOVA results. Indeed the issue here is that sampling error becomes larger as the order of the moment increases, therefore in case of the skew the spatial variation is a second order effect (as compared to sampling variability) that it not really detectable, this is apparent in both the fixed region and ROI cases. 148

177 CHAPTER 5 Table 1 Regression diagnostics for the fixed region and ROI for NSW, VIC and QLD Model Fixed region ROI Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q 100 Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q 100 Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q 100 MEV AVP SEP (%) NSW R GLS (%) MEV AVP SEP (%) R GLS (%) VIC QLD For the mean flood model (all the three states), the ROI shows a MEV which is smaller than the fixed region analysis. The lower MEV in turn also provides the lower AVP values as can be seen in Table 1. Also, the RGLS values for the mean flood model (all the three states) with the ROI case are 8%, 1% and 1% higher than the fixed region for NSW, VIC and QLD, 149

178 CHAPTER 5 respectively. These results indicate that the ROI should be preferred over the fixed region for developing the mean flood model. For the standard deviation model, ROI shows % smaller and 9% higher SEP and R GLS values, respectively for NSW. The best result is found for QLD, here ROI shows a 14% smaller and 1% higher SEP and RGLS values, respectively. This indicates that the ROI is preferable than the fixed region for the standard deviation model. The SEP and RGLS values for the skew model are the same for the fixed region and ROI for NSW and QLD, respectively (see Table 1). This can be explained by the fact that the number of sites for the skew model in the ROI approach was very close to that of the fixed region approach. Interestingly, one can see from Table 1 that the SEP values for all the flood quanitle models for NSW, VIC and QLD respectively are 5% to 11%, 6% to 7% and 5% to 13% smaller for the ROI case than the fixed region one. Also, the RGLS values for ROI case for NSW, VIC and QLD respectively are 4% to 7%, % to 1% and 1% to 5% higher than the fixed region case. These results show the relative advantage of the ROI approach coupled with BGLSR over a fixed region BGLSR where further improvements have been achieved overall. Table shows the number of sites in a region, the associated MEVs and their percentage (%) differences for the ROI against the fixed region models for NSW, VIC and QLD. This shows that the ROI mean flood model for all the three states has fewer sites on average (36 out of 96 i.e. 37% of the available sites for NSW, 3% for VIC and 4% for QLD) than the standard deviation and skew models. The ROI skew model for each state has the highest number of sites which includes nearly all the sites in the respective states. The MEVs for all the flood quantile ROI models are smaller than those of the fixed region models with differences in order of 50% to 60%. This shows that the fixed region models experience a greater heterogeneity than the ROI. If the fixed region models are made too big, the model error is likely to be inflated by heterogeneity unaccounted for by the catchment characteristics predictor variables. Two important points should be noted here that spatial proximity (physical distance) may become a surrogate for unknown processes in regional flood frequency analysis (RFFA) and that the catchment characteristics variables available at the regional scale may not always be sufficient indicators of regional flood behaviour. In fact, these regional models are too simplistic in their form, predictor variables and data 150

179 CHAPTER 5 representation; there are lots of lumping and approximations involved along with many simplistic assumptions. Hence, regional flood models can never be highly accurate within the current modelling and data regime. Table Model error variances associated with the fixed region and ROI for NSW, VIC and QLD (n = number of sites needed for the LP3 parameters and flood quantiles) State NSW VIC QLD Parameter / ARI ROI (n) / ˆ Fixed region (n) / ˆ (%) diff in ˆ Mean Stdev Skew Q Q 5 Q 10 Q 0 Q 50 Q % 1% 0% 5% 30% 30% 8% 9% 17% ROI (n) / ˆ Fixed region (n) / ˆ (%) diff in ˆ 8% 7% 18% 6% 31% 34% 46% 43% 51% ROI (n) / ˆ Fixed region (n) / ˆ (%) diff in ˆ 35% 60% 7% 46% 53% 61% 53% 41% 40% Figure 31 plots the spatial variation of the MEVs (grouped in classes according to numerical values as specified in the legend) for the mean flood model (Figure 31a) and how the MEV varies with the number of sites within the ROI, for a typical site (Figure 31b) for the state of NSW. The plot reveals the relative advantage of the ROI approach. It can be seen that there are distinct spatial variations illustrating the heterogeneity of the mean flood model that would be often ignored in a fixed region approach. Similar results were observed in both VIC and QLD states. The spatial variation in the model error for the skew model captures the entire study area mostly (figure not shown) for NSW, VIC and QLD. Similar results were found by 151

180 CHAPTER 5 Hackelbusch et al. (009). The significance of this finding is that if any spatial variations exist in the hydrologic statistic of interest, they are most likely to be captured by the ROI. (a) (b) kilometres kilometres 00 New South Wales New South Wales LEGEND MEV = MEV = MEV = MEV = MEV => 0.4 SITES Australian Capital Territory Australian Capital Territory Victoria Victoria Figure 31 Spatial variations of the grouped minimum model error variances for (a) mean flood model and (b) number of sites which produced the lowest predictive variance for the mean flood model EVALUATION STATISTICS An objective assessment of the developed models can be made by using the numerical evaluation statistics given in Equation (3.45) and Equation (3.44), in which RMSE r is the relative root mean squared error and RE r is the absolute median relative error. The RMSE r is associated with the predictive error variance, where as RE r is related mostly with prediction bias. Using the model predicted flood quantiles (estimated by QRT and PRT, with fixed and ROI regions) using the LOO validation, the evaluation statistics were calculated. These are given in Table 3. Numerical values of these statistics show the relative advantage of the ROI approach (for both the QRT and PRT) for all the three states (i.e. NSW, VIC and QLD). The flood quantile estimates obtained from the fixed regions (QRT and PRT) are more biased (i.e. higher RE r ) and are of a lesser accuracy (i.e. higher RMSE r ). This is observed for all the three states. 15

181 CHAPTER 5 Table 3 Evaluation statistics (RMSE r and RE r ) from LOO validation for NSW (Results NSW for PRT using the weighted regional average standard deviation and skew models, i.e. no predictor variables given in brackets), VIC and QLD NSW Model RMSE r (%) RE r (%) PRT QRT PRT QRT Fixed ROI Fixed ROI Fixed ROI Fixed ROI region region region region Q (63) (37) Q (59) (3) Q (60) (33) Q (63) (34) Q (77) (35) Q (85) (39) VIC Q Q 5 Q 10 Q 0 Q 50 Q 100 Q Q 5 Q 10 Q 0 Q 50 Q QLD For the QRT and PRT (fixed region) it can be observed from Table 3 that there is not much difference in accuracy (RMSE r ) for NSW, VIC and QLD states. Indeed, in relation to bias (RE r ) both QRT and PRT fixed region models were found to be very similar for the three states. For QRT and PRT (ROI region), a similar result was found where there was no notable difference in accuracy (RMSE r ) between the competing models. For the bias (RE r ), both the QRT ROI and PRT ROI models achieved very similar values as seen in Table 3. While Table 3 does show slightly better accuracy and bias for the QRT over PRT, a point needs to 153

182 CHAPTER 5 be bought out to clarify this result. There is some underlying bias involved with the validation of the QRT (fixed and ROI) in that the predicted quantiles are being compared to the quantiles used in the regression analysis as dependent variables. Thus the result mostly seems to be slightly in favour of the QRT (see Table 3). How to compensate for this bias in the validation process needs further effort, which has not been done in this thesis. On the other hand, the validation procedure for the PRT is more stringent in that the parameters of the distribution are used in the regression and quantiles are then independently estimated and compared to the at-site flood quantiles. The results from the evaluation statistics therefore indicate that the PRT is indeed a viable approach for RFFA as an alternative to the commonly applied QRT method in the ungauged catchment application. Below the results are presented based on the evaluation statistics (i.e. Equations (3.45 and 3.44)) to compare the flood quantiles from PRT-ROI using a weighted regional average standard deviation and skew to the PRT-ROI using a standard deviation and skew as a function of predictor variables for the state of NSW. The evaluation statistics (see Table 3 values in the bracket) from the validation reveal that there is no real loss of accuracy (as compared to at-site flood quantiles) if a weighted regional average standard deviation and skew model is adopted to estimate the flood quantiles up to the 0 years ARI. The results at the higher ARIs (50 and 100 years) show that using a weighted regional average standard deviation and skew may slightly affect the outcome of the analysis (i.e. lesser accuracy and greater bias). The larger ARI estimation may require further information which may be provided by having predictor variables (such as catchment area, design rainfall intensity, forest and mean annual rainfall) for the standard deviation model as found in this study. This issue deserves further investigation before estimating larger ARI flood quantiles based on a weighted average standard deviation and skew estimates that do not use any predictor variables. The evaluation statistics presented above related to a particular aspect of the model validation over all the six ARIs for all the three states. Now it is worth looking at the overall performances of the different models (QRT and PRT, with fixed and ROI regions) based on a ratio statistics and case score analysis. The ratio is defined as Q pred /Q obs (i.e. r r ) and gives an indication of the degree of bias (i.e. systematic over- or under estimation), where a value of 1 indicates good average agreement between the Q pred and Q obs. Here Q pred values were 154

183 CHAPTER 5 obtained from LOO validation (fixed and ROI) using the developed QRT or PRT model. The distributions of the Q pred /Q obs ratio values for the state of NSW are shown in Figure 3 for 5, 0 and 100 years ARIs. Here, for the 5 years ARI, PRT-ROI shows the best results as the median ratio is the closest to the line corresponding to Q pred /Q obs = 1 (1-line) and the overall spread of the ratio values is the smallest. For the 0 years ARI, QRT-ROI median ratio is closer to the 1-line as compared to the PRT-ROI case; however, the overall spread of the ratio values for both the QRT-ROI and PRT-ROI is very similar. For the 100 years ARI, QRT-ROI shows noticeable overestimation and PRT-ROI shows some underestimation as the median ratio value is located just below the 1-line. Figure 3 Boxplots of Q pred /Q obs ratios for NSW for QRT and PRT, with fixed and ROI regions Considering all the three states, a case score analysis of the Q pred /Q obs ratio values is presented below. The criteria for the case score analysis can be seen in Chapter 3, section 3.9. The models are assessed based on which one receives the most desirable estimation on average over all the cases (i.e. 6 ARIs and 399 catchments (in total 394 cases for each PRT and QRT), combining NSW, VIC and QLD). Based on the criteria set out in section 3.9, from the 394 cases, the QRT and PRT with fixed region produce 1881 and 189 cases respectively with a desirable estimation, which is equivalent to 78% and 76% of the cases respectively. The QRT and PRT fixed region show that 11% and 13% of cases respectively have a gross underestimation. The gross overestimation for QRT and PRT fixed region achieves 11% of the cases each. 155

184 CHAPTER 5 The QRT-ROI and PRT-ROI methods provide 83% and 80% of cases with a desirable estimation. The gross underestimation is associated with 9% of cases for both the QRT and PRT, respectively. The gross overestimation sites for QRT-ROI and PRT-ROI are 8% and 11% of the cases, respectively. It can be seen that in both the fixed and ROI regions there are cases where the results do not have a very high degree of accuracy. Such results are typical of RFFA methods (see Rahman, 005) and are somewhat as expected due to simplistic nature of RFFA models, which involve many simplified assumptions. For example, addition of a greater number of predictor variables and/or use of a complex model form may increase accuracy marginally, but they are not generally significant as far as practical application of the RFFA methods is concerned (e.g. see Rahman et al., 1999a). Also, the error in at-site flood frequency analysis estimates (which is the base case for comparison) needs to be kept in perspective. While we see improvements in the ROI approach for QRT and PRT, the fact is that there remain a few cases where estimations are not of high accuracy. This needs further investigation to identify the reason for such high degree of error, which however, has not been done in this thesis. On average, however, only modest differences can be found for the QRT-ROI and PRT-ROI estimates for the majority of the cases (see Table 3). In looking at the cases where most of the gross overestimation and gross underestimation happened, it was found that the PRT in some cases under estimated the at-site flood quantles for the larger ARIs (50 and 100 years). Interestingly, it was also found that the QRT overestimated in many cases the lower ARI ( and 5 years) at-site flood quantile. These results were found for a range of catchments sizes over all the states. What can be concluded overall from this evaluation is that the PRT does not provide less accurate estimates than the commonly applied QRT method. In fact, the PRT is a useful way to check the results from QRT to make sure estimates make sense, especially in the case where the QRT results may not increase smoothly with ARI. 5.6 SECTION SUMMARY The main objectives of sections 5.4 and 5.5 were to compare the BGLSR approaches using a fixed and ROI framework that seeks to minimise the Bayesian model error variance (predictive uncertainty). For this purpose, data from 45 small to medium sized catchments in eastern Australia (covering Tasmania, VIC, NSW and QLD states) were used. Prediction equations were developed for the flood quantiles of ARIs of to 100 years using the QRT 156

185 CHAPTER 5 and for the first three moments of the LP3 distribution (i.e. PRT). Using a method similar to forward stepwise regression and adopting a number of statistical selection criteria it was possible to identify the optimal regression models to use in the ROI approach. It was found that area and design rainfall intensity were significant predictors for the estimation of the flood quantiles in these states using QRT, while area, design rainfall intensity, mean annual evaporation, mean annual rainfall, main stream slope and forest were relatively significant in the estimation of the second and third parameters of the LP3 distribution. LOO validation indicated that the ROI based on the minimisation of the predictive uncertainty leads to more efficient and accurate flood quantiles estimates by both the QRT and PRT. The regression diagnostics revealed that the catchment variables alone may not pick up all the heterogeneity in the regional model. Both BGLSR QRT-ROI and BGLSR PRT-ROI showed improvements in regional heterogeneity with an increase in the average pseudo coefficient of determination and a decrease in the model error variance, average variance of prediction and the average standard error of prediction. Both the standardised residual and QQ-plots of the ROI approach satisfied the underlying regression model assumptions better than the fixed region. It was shown that both BGLSR QRT-ROI and BGLSR PRT-ROI produce smaller average RMSE r and RE r values when compared to the fixed region regression approach. Based on the evaluation statistics overall it was found that there are only modest differences between the BGLSR QRT-ROI and BGLSR PRT-ROI which suggests that the PRT is a viable alternative to QRT in RFFA. The RFFA methods developed in this study was based on the database available in eastern Australia. It is expected that availability of a more comprehensive database (in terms of both quality and quantity) will further improve the predictive performance of both the fixed and ROI based RFFA methods presented in this study, which however needs to be investigated in future when such a database is available. 5.7 UNCERTAINTY ESTIMATION FOR NEW SOUTH WALES, VICTORIA, QUEENSLAND AND TASMANIA IN A ROI-PRT FRAMEWORK Here, uncertainty in design flood estimation is examined in a BGLSR multivariate normal distribution framework, in that the posterior variance of each flood statistic (i.e. mean, standard deviation, and skew) was combined and the correlation structure between statistics 157

186 CHAPTER 5 was preserved to assess the uncertainty associated with the flood quantiles (see section 3.10, Equations 3.50 to 3.5 and Figure 4). It should be noted that this method only considers the uncertainty arising from the estimation of the flood statistics i.e. sampling errors and intersite correlation (as mentioned in section and Equation 3.31). Other uncertainties were not considered, such as measurement errors and uncertainty about the choice of distribution. This method was applied to all the six ARIs and selected sites in the study regions for NSW, VIC, QLD and TAS. As an example, the results are shown for four catchments, 1 from each of the four states with varying record lengths (i.e. for NSW = 9 years, VIC = 41 years, QLD = 6 years and TAS = 4 years). Figure 33 plots the 95% confidence bands from the Monte Carlo simulation with 10,000 simulation runs (and the FLIKE at-site confidence bands) along with the at-site and regional estimation. Figure 33 shows that the predicted (expected) quantiles (blue triangles) are generally well matched with the observed at-site FFA estimates (black circles); however, the result for TAS is not overly good. It is also reassuring to see that the quantiles increase with increasing ARI. Taking the case of site 0301 for NSW and ARI = 100 years, the confidence interval ranges from 303 m 3 /s to 1597 m 3 /s, which show a rather medium to large uncertainty. However, the result may not be considered poor as they match up reasonably well with the FLIKE at-site confidence limit values (409 m 3 /s to 513 m 3 /s). Overall, the uncertainty bands estimated for the regional approach were larger than the at-site ones, which is as expected. Reasons for this may be due to the fact the BGLSR model corrects for sampling variability and that generally there is more uncertainty associated with regional estimation. Finally, it can also be seen that the uncertainty increases considerably with increasing ARI. In any case the framework presented here provides a relatively reliable basis for uncertainty analysis which would be of great benefit for in real world applications. 158

187 CHAPTER 5 Figure 33 Design flood quantile estimation and confidence limits curves for ARIs of to 100 years 159

188 CHAPTER SUMMARY This chapter has developed and compared flood prediction equations for the states of New South Wales, Victoria, Queensland and Tasmania (for 6 ARIs, Q to Q 100 ). Both fixed regions and ROI approaches in a QRT and PRT framework were used, where the quantiles and parameters (i.e. mean, standard deviation and skew) of the LP3 distribution were regressed against catchment characteristics predictor variables. The BGLSR procedure was adopted for the estimation of the regression model coefficients. To assess the performances of the developed prediction equations a LOO validation procedure was adopted. Overall, it was found that the QRT and PRT-ROI perform very similarly and that the PRT is a viable alternative for design flood estimation in ungauged catchments. The developed prediction equations allow for design flood or flood statistics estimates along with its associated uncertainty (in the form of confidence limits at any ungauged catchment) given the relevant catchment characteristics data. 160

189 CHAPTER 6 CHAPTER 6: RESULTS - MODEL VALIDATION USING LOO AND MCCV 6.1 GENERAL This chapter presents the results of the comparison of the Leave-one-out (LOO) and Monte Carlo cross validation (MCCV) techniques in a hydrological regression framework. Both ordinary least squares (OLSR) and generalised least squares regression (GLSR) are applied to the experimental and real datasets. This chapter aims to outline the overall advantages and disadvantages of the proposed methods for model selection and validation. The basic theory and assumptions associated with the LOO and MCCV both in an OLSR and GLSR framework have been discussed in Chapter PUBLICATIONS A Journal paper (ERA, rank A*) has been accepted regarding this chapter. The Journal paper can be found in Appendix A. The following is the reference where the paper can be found. Haddad, K., Rahman, A., Zaman, M. and Shrestha, S. (013). Applicability of Monte Carlo Cross Validation Technique for Model Development and Validation Using Generalised Least Squares Regression. Journal of Hydrology, doi.org/ /j.jhydrol

190 CHAPTER 6 6. RESULTS 6..1 PREDICTORS USED The summary statistics associated with the predictor variables used in this analysis are provided in Table 4, while Table 5 presents the correlation between the log-transformed predictor variables where it can be seen that there is significant collinearity and multicollinearity between the design rainfall intensities (ranging 0.73 to 0.94), medium correlation between rain and evap (0.5) and evap and the design rainfall intensities (ranging 0.40 to 0.58) and modest correlation between sden and rain (0.7) and sden and evap (0.36). Table 4 Summary of predictor variables (here log 10 is used) Predictor variable Minimum Maximum Mean Standard deviation log(area) (km ) log( I 1 ) (mm/h) log( I 1 ) (mm/h) log( 50 I 1 ) (mm/h) log( 50 I 1 ) (mm/h) log(i tc, ARI ), ARI = 10-year (mm/h) log(i tc,ari ), ARI = 100-year (mm/h) log(evap) (mm) log(rain) (mm) log(sden) (km/km ) log (S1085) (m/km) log(forest) (fraction)

191 CHAPTER 6 Table 5 Correlation between the log 10 predictor variables used in the analysis area I 1 I 1 50 I 1 50 I 1 I tc, ARI=10 I tc, ARI=100 rain evap sden S1085 forest area 1.00 I I I I I tc, ARI= I tc, ARI= rain evap sden S forest

192 CHAPTER SIMULATED DATA A number of simulation runs were undertaken on different models with varying random errors. Here, we discuss the simulation based on the model given by Equations (3.68 and 3.69). The results for OLSR are summarised in Tables 6 and 7 while the results for GLSR are provided in Tables 8 and 9. The summary tables also provide the results for the analysis based on the true (i.e. true model) MSEP for both the OLSR and GLSR models. For the LOO (i.e. for n v = 1), the model tends to include a greater number of predictor variables than that required as evidenced by the inclusion of many more predictor variables than those of the higher n v. This particular feature is evident for both the OLSR and GLSR techniques. As an example, in Tables 6 and 7, for the OLSR LOO (where n v = 1) and for = 1, x 1 is selected only 4% (10/500) of the cases, while for = 0., x 1 is selected 51% (53/500) of the cases. The GLSR results also suffer from over fitting (see Tables 8 and 9); however, the chances for selecting the right model do increase with the GLSR. As an example, for = 0.95, x 1 is selected 53% (63/500) of the cases, while for = 0.5, x 1 is selected 64% (318/500) of the cases. Another important aspect of the LOO for both the OLSR and GLSR that it tends to underestimate the MSEP of the true model and calibration data set as compared to the higher n v. Figure 34 illustrates this where it can be seen that as n v increases the MSEP also increases. It is thus evident that LOO lends itself to over fit the selected regional regression model. For the MCCV case, when n v = 45, x 1 is included 475 and 49 instances for the OLSR when = 1 and = 0., respectively. This gives an MSEP = 3.50 and 1.49 for the CMCCV case (see Tables 6 and 7) as compared to 491 and 499 instances for the GLSR for = 0.95 and 0.5, respectively and MSEPs = 1.61 and 0.5 (see Tables 8 and 9). For n v = 1 for both the OLSR and GLSR, the MSEPs = 1.68, 0.61, 0.7 and (see Tables 6 to 9) which are relatively smaller when compared to the LOO of the calibration data set (i.e..0, 0.77, 0.41 and 0.11). This implies that the LOO, particularly with the OLSR, has a much higher chance of selecting a larger model (i.e. a model with a higher number of predictor variables). From Tables 6 to 9 it can be seen that the MSEP values based on the 164

193 CHAPTER 6 model selected by the LOO are always greater than the true MSEPs (e.g. 53% (i.e. ( )/1.87) in Table 6 for the OLSR when = 1). Tables 6 and 7 also reveal that collinearity (i.e. between variables x 1 and x ) is more prominent for the OLSR LOO case especially when the random errors are highly spread (when = 1). This can be also seen in Figure 34 for n v = 1, where the combined variables x 1 and x have relatively closer MSEP values to the variable x 1. For the GLSR, the collinearity is not a major issue for both = 0.95 and = 0.5 (see Tables 8 and 9) and the varying cross correlation between sites. For example, x 1 and x (which are made highly correlated, see Chapter 3, section ) appear in the model many more instances in the OLSR (e.g. 155, 187,... times in Table 6) than in the GLSR (e.g. 93, 105,... times in Table 8). Since the GLSR analysis recognises the sampling error as a separate component to the total error, it seems that the GLSR can distinguish very well between the predictor variables in contrast to the OLSR. Since both sampling error and model error are lumped together, the OLSR model pushes for more predictor variables to compensate for the higher model uncertainty. From these results, it can be seen that the GLSR, with a relatively high spread of error (e.g. = 0.95) and modest correlation between sites, provides reasonable results with the LOO validation as compared to the OLSR LOO case. Hence, it may be concluded that the LOO is better suited with the GLSR than with the OLSR in regional hydrologic regression. From Tables 6 to 9 the following points may be noted. The chance for the MCCV to select the true model (that includes only x 1 as predictor) increases with increasing n v. This can be observed with both the OLSR and GLSR models; however, the results for the GLSR are slightly better. Uncertainty is therefore reduced for the model selected by the MCCV (i.e. decrease in over fitting). What is also noticeable from Tables 6, 7, 8 and 9 is that as n v increases some of the predictor variable combinations are not selected at all (i.e. shown as zero in the table). This illustrates that in most cases the MCCV method would choose the best model. Looking at Figure 34 for n v = 5 and 35, it is evident that both the OLSR and GLSR MCCV select the predictor variable x 1 consistently better than any other variable. This is especially the case for the GLSR MCCV case as it has the smaller MSEPs. When the MSEP (i.e. predictive variance) is smaller and when there is medium to high correlation between sites, the GLSR MCCV should be the preferred option for validation 165

194 CHAPTER 6 (as evident in Figure 34). The GLSR with modest cross correlation and larger random errors also provides relatively better results in most cases. In addition, the collinearity seems to have no major influence in choosing the correct predictor variable for the MCCV case (see Figure 34, i.e. n v = 15, 5 and 35). Furthermore, the GLSR appears to be the superior regression approach when the model errors are modest and when there is reasonable sampling uncertainty from site to site. In all the cases, the MSEP values for the MCCV depend significantly on n v. From Tables 6 to 9, it is clear that the use of MCCV to estimate the MSEP of the selected model when n v > 5 may not be appropriate as the MSEP increases modestly (i.e. n v also increases for the calibration set). In nearly all the cases for the OLSR and GLSR, with the varying random errors and cross correlations, the MCCV seems to estimate MSEP based on the selected model with similar level of accuracy to that of the CMCCV (for Equations 3.68 and 3.69). In nearly all the cases, CMCCV stays around the acceptable limits of the true MSEP up to n v = 5. From Tables 6 to 9, it is observed that CMCCV may be a good candidate to be used to estimate the prediction ability of the selected model overall, as CMCCV tends to stay within acceptable limits around the MSEP of the selected model (for Equations 3.68 and 3.69). Thus, with n v = 15 to 5 (representing 30% to 50% of the catchments), the MCCV and the CMCCV estimate the MSEP with reasonable accuracy. Table 6 Results from simulated data, OLSR when = 1 Frequencies of variables being selected n v x 1 x 1, x 3 x, x 3 x 1, x x or x 3 Values of optimal MSEP Based on Eq.(3.68) Simulated data True model LOO MCCV CMCCV LOO MCCV TMSEP

195 CHAPTER 6 Figure 34 The mean squared error of prediction (MSEP) associated with LOO and MCCV for OLSR and GLSR simulations 167