Special Focus Session SF 1c CROWD SOURCING SCIENCE

Transcription

1 Special Focus Session SF 1c CROWD SOURCING SCIENCE Rapporteur: Chris Hennon University of North Carolina at Asheville 1 University Heights CPO #2450 Asheville, NC USA chennon@unca.edu Phone: Abstract: Cyclone Center is a website that collects Dvorak-like global tropical cyclone (TC) classification data from citizen scientists. The goal of Cyclone Center is to produce a homogeneous dataset of TC intensity and metadata that can serve as a source for a global TC reanalysis. In addition, Cyclone Center will be able to contribute to the reconciliation of interagency TC intensity differences arising from the inconsistent application of the Dvorak technique and other operational challenges. After two years of data collection and analysis of nearly 300,000 classifications from thousands of users, we demonstrate that: 1) citizen scientists, most with little or no formal meteorological training, as a group provide classification skill comparable to and sometimes exceeding an objective technique, and 2) classifications will be able to provide a strong foundation for a global reanalysis project. One of the biggest challenges remaining is recruiting enough classifiers to finish the 32-year record in a reasonable period of time. SF 1c.0 Introduction Global best track data suffer from inter-agency inconsistencies because of differences in available data and technology as well as divergent analysis methods, particularly with the Dvorak technique. The problem was recently highlighted by papers that showed that recent western North Pacific category 4-5 typhoons were increasing in frequency (Webster et al. (2005), decreasing in frequency (Wu et al., 2006) and displaying no significant trend (Kossin et al., 2007). What is the truth? Because tropical cyclones (TCs) are rarely observed directly, even in the North Atlantic, reliance on the Dvorak technique to determine the maximum TC wind is high. However, the method is inherently subjective and its rules have been changed at various forecast agencies to conform to perceived regional intensity differences. We recognize that a portion of the best track disagreements arise from several factors beyond Dvorak (pattern) interpretation, including: different wind averaging periods, inconsistent mapping of current intensity (CI) numbers to wind speeds and other perceived regional bias adjustments. But recent work (Barcikowska et al., 2012; Nakazawa and Hoshino, 2009) demonstrates that significant inter-agency intensity differences in operational Dvorak estimates significantly contribute to disparate best-track data, even after accounting for these other factors. A reanalysis of agency best track data that attempts to quantify and reconcile all of the recognized inter-agency differences would be extremely difficult and most likely unsatisfactory. 1

2 Cyclone Center addresses this problem at the foundational level by producing a set of homogeneous classification numbers. A website was developed to serve geostationary TC infrared satellite imagery ( ) to citizen scientists for classification. User responses are used to determine a maximum sustained surface wind through a Dvorak-like procedure. The advantages to this approach lie in the consistency of the method, the homogeneity of the input data, the accuracy of the crowd and the additional uncertainty information that falls out of the crowd sourcing approach. SF 1c.1 Data and Methods We serve homogenous, TC-centric geostationary satellite imagery (HURSAT, Knapp and Kossin 2007) to citizen scientists who log on to cyclonecenter.org. HURSAT provides nearly 300,000 images of 3,321 global tropical cyclones that formed from The dataset was created by merging global geostationary satellite data and calibrating for homogenization; this allows us to classify global TC intensity across a 32-year period without needing to account for differences in the observation platforms. Other satellite data were excluded because they were not universally available for all TCs, including other geostationary channels like visible. SF 1c.1.1 Data Collection Investigators are asked a series of simple questions for each image. The questions are designed so that non-experts can answer them well while still providing a minimum amount of classification data to determine the maximum surface wind. Questions include identifying the stronger storm (i.e. 24-hour trend), identifying the dominant cloud pattern (eye, curved band, shear, embedded centre) and pattern specific queries such as the degree of band wrap. Each image in the HURSAT record will be classified by at least 10 unique users, which will allow us to determine a degree of certainty in the responses. To encourage participation, dynamic guides are available each step of the way to assist users who may not be sure of the best responses. Interesting storm images can be collected by citizen scientists, shared on Facebook or Twitter and openly discussed in the online forum. We anticipate that valuable insights will be gained by mining the online discussions of our citizen scientists. SF1c.1.2 Analysis Methodology Citizen scientist responses from the first two years of the project have been evaluated in two ways. A bias-corrected TC intensity is calculated by performing an inter-comparison of citizen scientist classifications with one another. We have also used a Monte Carlo approach that randomly select individual classifications, which leads to the production of uncertainties. Each technique 1) estimates the intensity based on an image snapshot, and 2) applies temporal constraints to the instantaneous estimates. Currently we focus on the pattern analysis from the citizen scientists. That is, we are only concerned with their responses that pertain to the type of cloud pattern and the perceived maturity of the TC. Figure 1 is a screen shot that shows the relevant question. The full implementation of the Dvorak-like procedure, which will use citizen science responses along with the HURSAT brightness temperature data, is currently in development. We expect the classifications to be improved with the addition of the physical data. 2

3 Figure 1. Screen shot of a user selecting a curved band cloud pattern Of course, the true TC maximum wind speed is never known, even in cases where a storm is sampled well with aircraft reconnaissance. To test the robustness of Cyclone Center, a subset of recon-sampled North Atlantic TCs was selected ( Best Track/Recon ) to serve as the validation set. Results are compared to ADT-HURSAT (Kossin et al. 2013), an objective Dvoraklike algorithm applied to the same HURSAT dataset used in this project. SF 1c.2 Results Figure 2 shows the wind distributions for 722 validation points covering the North Atlantic TC record. The figure suggests both the Cyclone Center consensus (CC Consensus) and ADT-HURSAT appear to do a good job at capturing the observed TC intensity distribution. The CC consensus does a particularly good job at classifying TCs in the strong tropical storm/marginal hurricane wind speeds (55-75 kt), an area where ADT-HURSAT has traditionally struggled. 3

4 Figure 2. Frequency distribution of maximum wind speed (kt) of Cyclone Center classifications (CC Consensus) and ADT- HURSAT compared to 722 North Atlantic best track points with reconnaissance data (Best-Track/Recon) Using the same best-track/recon validation set as ground truth, we calculated the root mean square error (RMSE) and bias for both ADT-HURSAT and the CC consensus (Figure 3). Both datasets exhibit low bias and near-normal error distributions. The CC consensus RMSE is approximately 4 kt higher than ADT-HURSAT. The larger error is not surprising at this point. We expect that the CC errors will be reduced, perhaps substantially, when the images are subjected to a full Dvorak-like procedure that incorporates cloud brightness temperature data. Figure 4 shows a case study of Typhoon Yvette (1992). The left side of the figure shows a number of agency best tracks, ADT-HURSAT, and CC Consensus time series. Note that there is a general agreement between ADT-HURSAT, CC, and one of the best track agencies. It should be noted that wind speed averaging time differences have not been normalized, so the differences in wind speeds is not as large as suggested in the figure. An example of what can be accomplished with the Monte Carlo approach is shown on the right side of Figure 4. We randomly selected one classification from the available classifications for a snapshot. Performing this for each snapshot of the storm creates a simulated intensity analysis. Temporal rules (following ADT) are then applied to the random values, producing a final intensity time series of the system. However, there are numerous possible time series of intensity based on differences between each citizen scientist. For instance, for a storm that lasts 7 days with 8 images per day and 10 citizen scientists per image, there are ~10 56 possible time series of intensity (which is an upper limit given the likelihood that there would be some agreement in the classifications). In our analysis, we created 100 time series of intensity through random selection of classifications at each time. This produces a distribution of intensities at each snapshot, rather than one value. The variation of intensity at each snapshot provides an estimate of intensity and some measure of uncertainty. 4

5 Figure 3. Intensity estimate error distributions for CC consensus and ADT-HURSAT, as measured against best track estimates influenced by low-level aircraft reconnaisance. Figure 4. Left) Time series of Yvette (1992) intensities. Right) Monte Carlo analysis with blue shading and box whisker plots indicating degrees of spread and certainty. As shown in Figure 4, the method shows that there is a large degree of agreement early on in the Yvette s lifecycle (days 1-5) and larger uncertainty during the mature stages. We believe that the large uncertainty arises from the diversity of eye sizes, shapes, and eye wall cloud top temperature patterns that may make it difficult to identify a close match on the eye pattern canonical images (not shown). 5

6 SF 1c.3 Discussion and Future Work After two years of classifications, Cyclone Center has completed about 20% of the images in the HURSAT database. Efforts are ongoing to attract and retain citizen scientists to accelerate the data collection process. We hope that the TC community will become an active partner in this process, as the outcome from the project will provide clarity for a number of TCs where there is multi-agency disagreement and a useful dataset for climate studies. We have demonstrated that the crowd sourcing approach for TC classification has the potential to serve as a starting point for a global reanalysis (Hennon et al. 2014). Currently, we are moving beyond the pattern analysis approach described above by using the HURSAT brightness temperature data to develop a full Dvorak-like retrieval for each Cyclone Center image. We expect some improvement in the accuracy of the classifications. Readers are encouraged to visit cyclonecenter.org to participate in the project and provide feedback to the science team. Acknowledgements The current Cyclone Center science team is Ken Knapp (U.S. National Climatic Data Center (NCDC)), Jim Kossin (NCDC), Carl Schreck (Cooperative Institute for Climate and Satellites (CICS)), and Scott Stevens (CICS). Other contributions have been made by Peter Thorne, Paula Hennon (CICS), Michael Kruk (ERT, Inc.), and Jared Rennie (CICS). Many at Zooniverse worked closely with the team to design and maintain the Cyclone Center website. In particular, Brian Carstensen, Michael Parrish, Arfon Smith, Chris Snyder, David Weiner, Chris Lintott, David Miller and Kelly Borden were critical to the site s success. Over 7,400 registered citizen scientists have provided the classification data presented here. Acronyms used in the report ADT Advanced Dvorak Technique ADT HURSAT Advanced Dvorak Technique-Hurricane Satellite CC Cyclone Center CI Current Intensity HURSAT Hurricane Satellite RMSE Root Mean Square Error TC Tropical Cyclone References Barcikowska, M., F. Feser, and H. von Storch, 2012: Usability of best track data in climate statistics in the western North Pacific. Mon. Wea. Rev., 140, Hennon, C.C., K.R. Knapp, C.J. Schreck, S.E. Stevens, J.P. Kossin, P.W. Thorne, P.A. Hennon, M.C. Kruk, J. Rennie, J-M Gadea, M. Striegl, and I. Carley, 2014: Cyclone Center: Can citizen scientists improve tropical cyclone intensity records? Bull. Amer. Meteor. Soc., available at Knapp, K.R., and J.P. Kossin, 2007: Newe global tropical cyclone data from ISCCP B1 geostationary satellite observations. J. Appl. Rem. Sensing, 1, doi: / Kossin, J.P., K.R. Knapp, D.J. Vimont, R.J. Murname, and B.A. Harper, 2007: A globally consistent reanalysis of hurricane variability and trends. Geophys. Res. Lett., 34, doi: /2006gl Kossin, J.P., T.L. Olander, and K.R. Knapp, 2013: Trend analysis with a new global record of tropical cyclone intensity. J. Climate, 26, Nakazawa, T., and S. Hoshino, 2009: Intercomparison of Dvorak parameters in the tropical cyclone datasets over the western North Pacific. Sola, 5,

7 Webster, P.J, G.J. Holland, J.A. Curry, and H.R. Chang, 2005: Changes in tropical cyclone number, duration and intensity in a warming environment. Science, 309, Wu, M.C., K.H. Yeung, and W.L. Change, 2006: Trends in western north Pacific tropical cyclone intensity. Eos, Trans. Amer. Geophys. Union, 87,