A new look at statistical evaluations of cloud seeding experiments WMA Meeting 9-12 April 2013 San Antonio, Texas Roelof Bruintjes, Dan Breed, Mike Dixon, Sarah Tessendorf, Courtney Weeks, DuncanAxisa, Omar Al Yazeedi
INTRODUCTION Historically randomized statistical experiments to evaluate rainfall enhancement have focused on target/control areas with surface precipitation gauges being the primary evaluation data set During the past twenty years radar based evaluation methods have become more generally used by either tracking a reflectivity maxima or advecting a circle around the seeding target for convective clouds.
DATA AND METHODS The radar reflectivity and storm (threshold reflectivity maxima or area polygon) specific derived parameters are calculated for each storm that is tracked objectively
Radar estimate of rainfall within the TITAN framework The storm The TITAN experimental unit Objective radar estimate of rainfall 55 45 30 TITAN identifies and tracks individual storms based on a specified reflectivity threshold
DATA AND METHODS Various characteristics are measured at 5- minute (volume scan) intervals for as long as the evaluation is intended (mostly to 60 minutes but some studies more than 10 hours) after seeding Primary measures of the effects of seeding include radar derived storm rain flux and mass, storm area and volume, and height of max reflectivity etc.
Statistical Experiments Hygroscopic seeding Exploratory analyses South African experiments (1990-1995; 127 cases) Mexico experiments (1996-1998; 94 cases) United Arab Emirates experiments (2000-2004; ~130 cases) Queensland experiments (2008-2009; 37 cases.
South African Experiments South African vs. Mexico Results Rain mass (kton) 500 450 400 350 300 250 200 150 100 50 0-5 5 15 25 35 45 55 Time from decision Rain Mass (Ktons) 0 100 200 300 400-10 0 10 20 30 40 50 60 Minutes After Decision Results were significant (Mather et al, 1997) Stratified by regions (Nelspruit more maritime; and Bethlehem more continental), the results were more pronounced and stronger for the Bethlehem area (Silverman,2001). The results could have been biased between the maritime and continental cases and the true increases may not be well represented for all cases
Mexico experiments 800 Mexican Randomized Experiment 600 Q3 Rainmass 400 Q2 200 Q1 0-10 0 10 20 30 40 50 Time From Decision Mexico experiment (based on South African experiment): - Double-blind randomization - Seeding with hygroscopic flares - Evaluation based on time-resolved radar-rainfall estimates from objective TITAN software
Classification of Mexico data using aerosol burden (satellite aerosol optical depth) Mexican Randomized Experiment 800 Original results without stratification 600 Rainmass Q3 400 Q2 200 0 Typical non-aerosol day (<0.1 optical depth) Typical aerosol day (>.1 optical depth) Q1 Effect is most apparent on the days with significant -10 0 10 20 30 40 50 aerosol burden Time From Decision On cleaner, non-aerosol days, little effect Non-Aerosol cases Aerosol cases Seed & control quartile results
UAE Experiments Initial measurements indicated similar droplet concentrations to South African and Mexican clouds Convection very common over the Oman Mountains Experiment conducted in the same manner as South African experiment Statistical experiment indicated no effect from seeding Similar amount of cases than the South African experiment
Precipitation Formation UAE clouds already ingest large dust Three Aspects particles aggregated with salts and sulfates mimicking the hygroscopic 1. Aerosol size distributions and seeding effect. hygroscopicity The mid-level sub-tropical high 2. Thermodynamic structure of pressure inversion provides for recirculation of drops and thus growth Capping atmosphere 3. Effects on ice processes before the cloud ascends through the inversion layers inversion level. Long period of cumulus growth below -5C the inversion; Recycling of droplets in repeated updrafts and broadening of the spectrum; Natural drizzle formation even before the rainstorm breaks through the inversion; Efficient ice multiplication process and Large lots CCN? of cold rain in thunderstorms; Efficient precipitation process without seeding
Humidifying experiments with saltmineral aggregates 10% 51% 60% 70% 2 µm 76% 82%
Queensland experiments Tendencies seem similar as South African and Mexican results (based on 39 cases with exactly the same criteria as for South African and Mexican analyses) Initial overall preliminary P-values for some parameters Duration after decision p > 96.5% area time series p > 77.7% Precipitation flux time series p > 73.0%
Randomized results: Cell duration (major response) Survival analysis: The results seem to indicate that seeded clouds tended to live longer than unseeded clouds with higher hazard rate (chance at each time interval that cloud would dissipate) for unseeded clouds Mean overall Hazard Rate: Unseeded = 30%, Seeded = 11%; P-value: 99.05% Sample size and initial biases needs to be further considered Both the time series results and survival analyses seem to indicate similar results Initial results seem similar to earlier experiments in Mexico and South Africa Results should be interpreted with great caution at this stage
Evaluation Issues Spatial and temporal variability due to meteorological factors has a much greater influence than the enhancement factor (No random draw from the same distribution of potential values; (Beare et al., 2010). Identification of meteorological factors and use as covariates in the analyses (e.g. aerosol loading and thermodynamic profiles). Simple statistical tests insufficient in this environment and multivariate statistical process models that exhibit spatial and temporal dependence are more appropriate than a single test (e.g. aerosol loading).
REGIME CHARACTERISTICS AND ANALYSES PROCEDURES Background aerosol concentrations, size and chemistry play an important role in the efficiency of precipitation formation processes in clouds (Aerosol loading and types of aerosols). Meteorological and thermodynamic effects play an important role in the evolution cloud droplet spectra in clouds (e.g. inversion levels). Objective tracking of storms with radar is complex and can introduce artifacts into the data set. (e.g. TITAN tracking). Large variability in size of observed radar detected storms can bias the statistical analyses due to outliers in certain volume categories (Outliers and convective complexes).
Effects of aerosols on convective clouds Higher aerosol concentrations produces higher cloud droplet concentrations and in some cases narrower cloud droplet spectra and require deeper warm clouds to initiate coalescence (example Indonesia). However, if cloud base temperatures are >18C sufficient cloud depth exist to initiate coalescence in both continental and maritime clouds (Indonesia, Australia, South Africa). If large drops (>.2mm) exist at 0C and lofted to higher levels, ice is detected at temperatures between -5 and -8C while if they do not exist ice is detected at temperatures <- 11C (Australia, South Africa, India, etc.). Why do the existence of large drops initiate ice at between - 5 and 8C in convective clouds?
Contrasts in Indonesia 1997-1998 and 2005 Studies Measured cloud base cloud droplet size distributions in different environments over Indonesia. Biomass smoke at an airport in Sumatra during the peak of the forest fires in Southeast Asia during the 1997/98 biomass smoke event.
Sulawesi microphysical measurements and precipitation processes Polluted cloud
Distribution of Ice and Water in a Convective Cloud FREEZING LEVEL IN INDIA CLOUD BASE WARMER THAN 15C In the atmosphere temperature decreases 10 o C per km with height in a dry environment and about 6 o C per km in a cloud
Aerosol, thermodynamic and Cloud base temperature effects Karnataka BANGALORE Cloud base Cloud top
Aerosols, CCN and Cloud droplet concentrations (India) High concentrations of droplets due to pollution CCN and aerosol conc.
Broadening of cloud droplet spectra by re-circulation Large drop (>.2mm) concentrations between 2 and 6L -1 CCN effect: Difficult to form rain in only warm clouds
Effects on Ice Processes Large drops freezing Secondary Ice Formation Concentrations between 200 to 400L -1 Concentrations: ~5-10L -1 Similar to concentrations of observed large drops POTENTIAL INVIGORATION OF CLOUD GROWTH DUE TO LATENT HEAT OF FREEZING
Radar Responses Strong invigoration of radar echo intensities after cloud penetrate below 0 o C Morning, only warm clouds Afternoon, Mixed phase convective clouds
Warm rain process (Typical in Southeast Queensland clouds) Collision and coalescence of droplets falling at different terminal velocities leads to raindrop formation Raindrops are millimeters in size Cloud droplets are 100 times smaller in diameter
Cloud base heights and warm cloud depths during Queensland project Two cases with cold cloud bases (<+15C) South Africa 22 November 2008 High cloud base No large drops at 0C First ice below -12C
Precipitation Processes: Australia example (19 cases with warm Continental cloud droplet spectra at cloud base (~400 to 600cm -3 Coalescence initiates before cloud top reaches 0 o C Drizzle/rain drops present as cloud rises through 0 o C level bases) Temperature versus time 22 January 2009 Images of cloud droplets and drizzle/rain drops Cloud droplet size distributions at cloud base and 0 o C. Due to warm cloud bases (~20 o C) clouds initially develop warm rain process
Precipitation Processes: Mixed-phase/ice processes initiated by freezing of large drizzle/rain drops and subsequent initiation of natural seeding (ice splintering) process rapidly depleting cloud liquid water content Large drop freezing at ~-5 o C Initiation of ice splintering process Rapid conversion of LWC to ice Rapid depletion of LWC inhibiting lightning in these cases Temperature versus time 27 January 2009
Implications for cloud seeding programs Include cloud base temperature as a covariate in the analyses Include aerosol loading as a covariate (e.g aerosol loading) Include inversion levels between 0 and -10C as a covariate Include depth and dryness of the boundary layer as a covariate for precipitation at he ground
Evaluation Issues cont. Definition of Experimental Unit (especially for objective radar identification) Especially on how to treat mergers and splits a) b) c) d) e) f) g) h)
Evaluation Issues cont. Treatment procedures and consistency of treatment especially from aircraft.
AREA
VOLUME DISTRIBUTIONS
Evaluation Issues cont. Randomization procedure and associated selection biases; multiplicity of the responses (Gabriel, 2002a) Choice of statistical tests and models including single and double ratio statistics. Significance and power of detection including confidence regions and pooling (Gabriel, 2002b). Treatment group interactions both for groundbased winter orographic and airborne summertime convective cloud seeding.
New technologies and measurements Dual polarization radar data providing new insights Satellite and remote sensor aerosol and cloud measurements providing real-time characterization of the characteristics. New airborne and in-situ measurements to better characterize cloud processes
Plots of mean Z DR versus mean Z H for four single cell storms on different days. The values are for a height of ~ 1.5 km above ground. Each point represents a different radar scan time from the beginning to the end of the cell. The red indicates the growing phase and the blue the decaying phase. The beginning and end are defined as a cell mean Z H of 10 dbz Rain Drop size distributions Continental versus Maritime Wilson et al., 2012
Summary Variations in meteorological conditions can dominate the effects of seeding and are often times much larger than the effect of seeding (10-100 times). These variations can occur in space and in time and can significantly affect the results from any randomized seeding experiments depending on a single statistical test assuming that the samples are randomly drawn from the same distribution of potential values (treatment application for these measurements was at random).
Summary cont. More statistically efficient means of analysis are required if we hope to gain significant results in realistic time frames such as multivariate statistical models by including covariates that influence the precipitation processes in a region to control for natural variations in rainfall. In contrast to pure randomization analysis, this type of analysis estimates the conditional contribution to rainfall by meteorological and for example aerosol effects.