Methods of forecast verification
|
|
- Amy Thornton
- 6 years ago
- Views:
Transcription
1 Methods of forecast verification Kiyotoshi Takahashi Climate Prediction Division Japan Meteorological Agency 1
2 Outline 1. Purposes of verification 2. Verification methods For deterministic forecasts For probabilistic forecasts Standardised Verification System for Long-Range Forecasts (SVSLRF) 3. Results on the TCC web page 2
3 1. Purposes of verification Forecast verification is a process of assessing quality of forecasts. to monitor forecast quality how accurate are forecasts and are they improving? to guide forecasters and users help forecasters understand model characteristics help us provide higher value forecasts to users to guide future developments of system identify model faults and improve systems help us compare and evaluate different forecasts (model or guidance) 3
4 2. Verification methods For deterministic forecast ACOR, RMSE, Bias (ME),MSSS (ROC) For probabilistic forecast ROC, Reliability diagram, BSS (Brel, Bres) 4
5 Root Mean Square Error (RMSE) 1 RMSE N N F i O i i1 Range: 0 to infinity, Perfect score: 0. 2 F : forecast O : observation N :sample size RMSE measures absolute magnitude of the forecast error. It does not indicate the direction the error. RMSE small RMSE large 5
6 Mean Error (ME) ME 1 N N F i O i i1 F : forecast O : observation N :sample size Range: variable, Perfect score: 0. ME measures average magnitude of the forecast error. It indicates the direction the error. RMSE N N i ME e e xi ai ME RMSE can be divided into ME(systematic error) and random error ( ). e 2 6
7 Anomaly Correlation (AC) Range: -1 to 1. Perfect score: 1. AC measures correspondence or phase difference between forecast and observation, subtracting out the climatological mean at each point. N i i i N i i i N i i i i i C O C F C O C F AC ) ( ) ( ) )( ( climatology observation forecast : : : C O F 7 AC <<1 AC 1
8 AC positive ( 1) RMSE large AC negative RMSE large AC positive( 1) RMSE small AC negative RMSE small 8
9 Mean Squared Skill Score (MSSS) MSSS 1 MSE MSE c Perfect score: 1 (when MSE=0) Climatology forecast score: 0 where MSE is the mean squared error MSE 1 N N F i O i i1 2 F : forecast O : observation and MSE c is the MSE of climatology forecast. MSSS can be expanded (Murphy,1988) as 2 2 s f s f f o MSSS 2 r fo 2 so so so 2n 1 2n n 1 n 1 s : squared root of variance r : correlation f : forecast o : observation n 1 Fi F O i O n i1 rfo The first 3 terms are related to S f SO 1 phase error (through the correlation) 2amplitude errors (through the ratio of the forecast to observed variances) 3bias error 9
10 Reliability diagram The reliability diagram plots the observed frequency(y-axis) against the forecast probability(x-axis). The diagonal line indicates perfect reliability (observed frequency equal to forecast probability for each category). climatology perfect reliability Brier Scores forecast frequency Points below(above) the diagonal line indicate overforecasting (underforecasting). 10
11 BS 1 N Brier (skill) score Brier score measures mean squared error of the probability forecasts. N i1 ( p i o i 2 ) Range: 0 to 1. Perfect score: 0 Random:1/3 p o i i N : forecast : observed : sample size probability Brier skill score measures skill relative to a reference forecast (usually climatology). BSS 1 BS BS reference Climatology: occurrence(0or 1) o( 1 o) Range: minus infinity to 1. BSS=0 indicates no skill when compared to the reference forecast. Perfect score: 1. 11
12 Decomposition of the Brier score Murphy(1973) showed that the Brier score could be partitioned into three terms(for K probability classes and N samples). These terms are shown separately to attribute sources of error. o :climatological occurrence BS 1 K K 1 2 n ( p o ) n ( o o) 2 k k k k k N N k1 k1 o(1 o) reliability (brel) resolution (bres) uncertainty (bunc) the mean squared difference between the forecast probability and the observed frequency. Perfect score: 0 the mean squared difference between the observed frequency and climatological frequency. - indicates the degree to which the forecast can separete different situations. climatologial forecast score:0 Perfect score: o( 1 o) measures the variability of the observations. 12
13 Brier skill score Brier skill score = the relative skill of the probabilistic forecast to the climatology BSS BS BS BS BS clim 1 0 BSc lim clim Reliability skill score brel Brel 1 BS c lim Skill score BS c bunc lim score score Range: minus infinity to 1. Perfect score: 1 BSS=0 indicates no skill when compared to the climatology. BSS>0 : better than clim. Resolution skill score Bres bres BS clim Perfect score: 1 Perfect score: 1 forecast perfect forecast score score reference reference Score 100 The larger these skill scores are, the better. 13
14 Interpretation of Reliability diagram and BSS Event : Z500 Anomaly > 0 Northern Hemisphere Spring of 2008 (2008/2/28 ~2008/5/29) First week forecast (day 2-8) Third and fourth week (day 16-29) overforecasting underforecasting BSS>0 better than climatology BSS<0 inferior to climatology 14
15 Relative Operating Characteristic (ROC) ROC is created by plotting the hit rate(y-axis) against the false alarm rate(x-axis) using increasing probability thresholds to make the yes/no decision. The area under the ROC curve (=ROC area) is frequently used as a score. 15
16 Forecast Steps for making ROC diagram 1. For each forecast probability category, count the number of hits,misses, false alarms, and correct non-events 2. Compute the hit rate and false alarm rate in each category k hit rate k = hits k / (hits k + misses k ) false alarm rate k = false alarms k / (false alarms k + correct non-events k ) 3. Plot hit rate vs false alarm rate 4. ROC area is the integrated area under the ROC curve yes no yes hits false alarms no misses correct nonevents total Observed Observed yes Observed no Contingency Table 16
17 Interpretation of ROC curves Event: Z500 anomaly > 0 Spring st week forecast 3rd and 4th week forecast Perfect performance high resolution (high potential skill) low resolution (low potential skill) ROC is not sensitive to bias in forecasts. Forecasts with bias may have a good ROC curve if resolution is still good. In this sense, the ROC can be considered as a measure of potential usefulness. On the other hand, reliability diagram is sensitive to the bias. It is needed to see both the ROC and the reliability diagram. 17
18 SVSLRF (Standard Verification System for Long-Range Forecast) WMO standard tool to verify skill in seasonal models It was introduced by the Commission for Basic Systems (CBS) of the World Meteorological Organization (WMO) in December, Users can appropriately evaluate forecast skill with common measures. 18
19 Outline of SVSLRF Mandatory part Parameters Verification regions Deterministic forecasts Probabilistic forecasts Level 1 T2m anomaly Tropics(20S-20N) MSSS ROC curves Precipitation anomaly (Nino3.4 Index) Northern extratropics(20n-90n) Southern extratropics(20s-90s) (N/A) ROC areas Reliability diagrams Frequency histograms Level 2 T2m anomaly Precipitation anomaly (SST anomaly) Grid-point verification on a 2.5 by 2.5 grid MSSS and its three-term decomposition at each gridpoint ROC areas at each grid-point Level 3 T2m anomaly Precipitation anomaly (SST anomaly) Grid-point verification on a 2.5 by 2.5 grid 3 by 3 contingency tables at each grid-point ROC reliability tables at each grid-point 19
20 LC-SVSLRF Lead Centre for the Long-Range Forecast Verification System Australian Bureau of Meteorology (BOM) Meteorological Service of Canada (MSC) URL: The Manual is here Lead Center web site You can get scores of several centers on Lead Centre website. 20
21 3. Verification results on TCC web page 1-month forecast 3-month forecast Warm/Cold season forecast 21
22 Verification of operational 1-month forecasts Deterministic forecast Ensemble mean forecast error maps, RMSE and Anomaly Correlation Probabilistic forecast Time sequence of RMSE and AC 22
23 Verification of 1-month Ensemble mean forecast maps (Deterministic) Z500 over the Northern Hemisphere Observation Forecast Error Stream function (850, 200hPa) RMSE and Anomaly Corr
24 Verification of 1-month Ensemble mean forecast maps (Deterministic) Z500 over the Northern Hemisphere Observation Forecast Error ANL-Clim (JRA-25/JCDAS) FCT-Clim (model) 24
25 Verification of operational 1-month forecasts Probabilistic forecast Reliability diagrams and Brier skill scores ROC curves and area for each season 25
26 Verification of hindcast based on SVSLRF 26
27 Hindcast verification methods based on Standardised Verification System for Long-Range Forecasts (SVSLRF) Verification of deterministic forecasts Mean Square Skill Score (MSSS) Contingency tables Verification of probabilistic forecasts Reliability diagrams ROC curves and ROC areas 27
28 Examples of MSSS for precipitation Dec-Feb started on 10 Nov. Jun-Aug started on 10 May. Positive MSSS indicates that the forecast is better than climatological forecast. 28
29 Deterministic Summary Index Random Climatology Perfect RMSE >0 0 ME 0 AC MSSS 0 +1 MSE >0 >0 0 Probabilistic Index Random Climatology Perfect Reliability diagram o( 1 o) Fit to the diagonal line BS 1/3 bunc= 0 BSS(x100) 100> Brel(x100) 100> 100> +100 Bres(x100) 100> Roc area(x100) 100>
30 References Murphy, A.H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, Murphy, A.H., 1988: Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon. Wea. Rev., 16,
Verification at JMA on Ensemble Prediction
Verification at JMA on Ensemble Prediction - Part Ⅱ : Seasonal prediction - Yukiko Naruse, Hitoshi Sato Climate Prediction Division Japan Meteorological Agency 05/11/08 05/11/08 Training seminar on Forecasting
More information4.3.2 Configuration. 4.3 Ensemble Prediction System Introduction
4.3 Ensemble Prediction System 4.3.1 Introduction JMA launched its operational ensemble prediction systems (EPSs) for one-month forecasting, one-week forecasting, and seasonal forecasting in March of 1996,
More informationStandardized Verification System for Long-Range Forecasts. Simon Mason
Standardized Verification System for Long-Range Forecasts Simon Mason simon@iri.columbia.edu MedCOF 2015 Training Workshop Madrid, Spain, 26 30 October 2015 SVS for LRF: Goal Provide verification information
More informationEnsemble Verification Metrics
Ensemble Verification Metrics Debbie Hudson (Bureau of Meteorology, Australia) ECMWF Annual Seminar 207 Acknowledgements: Beth Ebert Overview. Introduction 2. Attributes of forecast quality 3. Metrics:
More informationVerification of Probability Forecasts
Verification of Probability Forecasts Beth Ebert Bureau of Meteorology Research Centre (BMRC) Melbourne, Australia 3rd International Verification Methods Workshop, 29 January 2 February 27 Topics Verification
More informationEvaluating Forecast Quality
Evaluating Forecast Quality Simon J. Mason International Research Institute for Climate Prediction Questions How do we decide whether a forecast was correct? How do we decide whether a set of forecasts
More informationVERFICATION OF OCEAN WAVE ENSEMBLE FORECAST AT NCEP 1. Degui Cao, H.S. Chen and Hendrik Tolman
VERFICATION OF OCEAN WAVE ENSEMBLE FORECAST AT NCEP Degui Cao, H.S. Chen and Hendrik Tolman NOAA /National Centers for Environmental Prediction Environmental Modeling Center Marine Modeling and Analysis
More informationExercise Part 1 - Producing Guidance and Verification -
Exercise Part 1 - Producing Guidance and Verification - 1 Objectives To understand how to produce guidance for seasonal temperature and precipitation. To think rationally and find possible predictors.
More informationProducts of the JMA Ensemble Prediction System for One-month Forecast
Products of the JMA Ensemble Prediction System for One-month Forecast Shuhei MAEDA, Akira ITO, and Hitoshi SATO Climate Prediction Division Japan Meteorological Agency smaeda@met.kishou.go.jp Contents
More informationAkira Ito & Staffs of seasonal forecast sector
Exercise : Producing site-specific guidance using domestic data Akira Ito & Staffs of seasonal forecast sector Climate Prediction Division Japan Meteorological Agency TCC Training Seminar on One-month
More informationJMA s Seasonal Prediction of South Asian Climate for Summer 2018
JMA s Seasonal Prediction of South Asian Climate for Summer 2018 Atsushi Minami Tokyo Climate Center (TCC) Japan Meteorological Agency (JMA) Contents Outline of JMA s Seasonal Ensemble Prediction System
More informationSeasonal Climate Watch September 2018 to January 2019
Seasonal Climate Watch September 2018 to January 2019 Date issued: Aug 31, 2018 1. Overview The El Niño-Southern Oscillation (ENSO) is still in a neutral phase and is still expected to rise towards an
More informationTokyo Climate Center Website (TCC website) and its products -For monitoring the world climate and ocean-
Tokyo, 14 November 2016, TCC Training Seminar Tokyo Climate Center Website (TCC website) and its products -For monitoring the world climate and ocean- Yasushi MOCHIZUKI Tokyo Climate Center Japan Meteorological
More informationSeasonal Climate Watch July to November 2018
Seasonal Climate Watch July to November 2018 Date issued: Jun 25, 2018 1. Overview The El Niño-Southern Oscillation (ENSO) is now in a neutral phase and is expected to rise towards an El Niño phase through
More informationJMA s Ensemble Prediction System for One-month and Seasonal Predictions
JMA s Ensemble Prediction System for One-month and Seasonal Predictions Akihiko Shimpo Japan Meteorological Agency Seasonal Prediction Modeling Team: H. Kamahori, R. Kumabe, I. Ishikawa, T. Tokuhiro, S.
More informationDrought forecasting methods Blaz Kurnik DESERT Action JRC
Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 1 Drought forecasting methods Blaz Kurnik DESERT Action JRC Motivations for drought forecasting Ljubljana on 24 September 2009
More informationSeasonal Climate Watch June to October 2018
Seasonal Climate Watch June to October 2018 Date issued: May 28, 2018 1. Overview The El Niño-Southern Oscillation (ENSO) has now moved into the neutral phase and is expected to rise towards an El Niño
More informationShuhei Maeda Climate Prediction Division Global Environment and Marine Department Japan Meteorological Agency
Eighth Session of the Forum on Regional Climate Monitoring, Assessment and Prediction for Asia (FOCRAII) 5-7 April 2012, Beijing, China Shuhei Maeda Climate Prediction Division Global Environment and Marine
More informationWinter Forecast for GPC Tokyo. Shotaro TANAKA Tokyo Climate Center (TCC) Japan Meteorological Agency (JMA)
Winter Forecast for 2013 2014 GPC Tokyo Shotaro TANAKA Tokyo Climate Center (TCC) Japan Meteorological Agency (JMA) NEACOF 5, October 29 November 1, 2013 1 Outline 1. Numerical prediction 2. Interannual
More informationProbabilistic verification
Probabilistic verification Chiara Marsigli with the help of the WG and Laurie Wilson in particular Goals of this session Increase understanding of scores used for probability forecast verification Characteristics,
More informationAtmospheric circulation analysis for seasonal forecasting
Training Seminar on Application of Seasonal Forecast GPV Data to Seasonal Forecast Products 18 21 January 2011 Tokyo, Japan Atmospheric circulation analysis for seasonal forecasting Shotaro Tanaka Climate
More informationHow to Use the Guidance Tool (Producing Guidance and Verification)
How to Use the Guidance Tool (Producing Guidance and Verification) Hiroshi Ohno Tokyo Climate Center (TCC)/ Climate Prediction Division of Japan Meteorological Agency (JMA) Workflow of the Excel Guidance
More informationDevelopment of Multi-model Ensemble technique and its application Daisuke Nohara
Development of Multi-model Ensemble technique and its application Daisuke Nohara APEC Climate Center (APCC), Busan, Korea 2007/2/21 JMA Contents 1. Introduction of APCC 2. Seasonal forecast based on multi-model
More informationPredicting uncertainty in forecasts of weather and climate (Also published as ECMWF Technical Memorandum No. 294)
Predicting uncertainty in forecasts of weather and climate (Also published as ECMWF Technical Memorandum No. 294) By T.N. Palmer Research Department November 999 Abstract The predictability of weather
More informationPerformance of Multi-Model Ensemble(MME) Seasonal Prediction at APEC Climate Center (APCC) During 2008
2008/SOM3/ISTWG/SYM/034 Agenda Item: 4 Performance of Multi-Model Ensemble(MME) Seasonal Prediction at APEC Climate Center (APCC) During 2008 Submitted by: APEC Climate Center (APCC) APEC Climate Symposium
More informationProgram of the morning lectures
Program of the morning lectures 09:30~11:00 Lecture Explanation of the numerical weather prediction (NWP) system Introduction of the NWP Products 11:15~12:45 Exercise Download of the binary model outputs
More informationModel error and seasonal forecasting
Model error and seasonal forecasting Antje Weisheimer European Centre for Medium-Range Weather Forecasts ECMWF, Reading, UK with thanks to Paco Doblas-Reyes and Tim Palmer Model error and model uncertainty
More informationExtracting probabilistic severe weather guidance from convection-allowing model forecasts. Ryan Sobash 4 December 2009 Convection/NWP Seminar Series
Extracting probabilistic severe weather guidance from convection-allowing model forecasts Ryan Sobash 4 December 2009 Convection/NWP Seminar Series Identification of severe convection in high-resolution
More informationWassila Mamadou Thiaw Climate Prediction Center
Sub-Seasonal to Seasonal Forecasting for Africa Wassila Mamadou Thiaw Climate Prediction Center NOAA Forecast Con/nuum e.g. Disaster management planning and response e.g. Crop Selec6on, Water management
More informationTowards Operational Probabilistic Precipitation Forecast
5 Working Group on Verification and Case Studies 56 Towards Operational Probabilistic Precipitation Forecast Marco Turco, Massimo Milelli ARPA Piemonte, Via Pio VII 9, I-10135 Torino, Italy 1 Aim of the
More informationVerification of Continuous Forecasts
Verification of Continuous Forecasts Presented by Barbara Brown Including contributions by Tressa Fowler, Barbara Casati, Laurence Wilson, and others Exploratory methods Scatter plots Discrimination plots
More informationWMO Lead Centre activities for global sub-seasonal MME prediction
Workshop on sub-seasonal predictability, 2-5 November 2015, ECMWF WMO Lead Centre activities for global sub-seasonal MME prediction Suhee Park Korea Meteorological Administration Contents Introduction
More informationModel verification / validation A distributions-oriented approach
Model verification / validation A distributions-oriented approach Dr. Christian Ohlwein Hans-Ertel-Centre for Weather Research Meteorological Institute, University of Bonn, Germany Ringvorlesung: Quantitative
More informationMonthly probabilistic drought forecasting using the ECMWF Ensemble system
Monthly probabilistic drought forecasting using the ECMWF Ensemble system Christophe Lavaysse(1) J. Vogt(1), F. Pappenberger(2) and P. Barbosa(1) (1) European Commission (JRC-IES), Ispra Italy (2) ECMWF,
More informationHow to use the guidance tool (Producing Guidance and Verification)
Ex.1 How to use the guidance tool (Producing Guidance and Verification) Masayuki Hirai Tokyo Climate Center (TCC)/ Climate Prediction Division of Japan Meteorological Agency (JMA) Schedule of exercise
More informationInterpretation of Outputs from Numerical Prediction System
Interpretation of Outputs from Numerical Prediction System Hiroshi OHNO Tokyo Climate Center (TCC)/ Climate Prediction Division of Japan Meteorological Agency (JMA) Procedure of Seasonal Forecast (1) 1.
More informationDeveloping Operational MME Forecasts for Subseasonal Timescales
Developing Operational MME Forecasts for Subseasonal Timescales Dan C. Collins NOAA Climate Prediction Center (CPC) Acknowledgements: Stephen Baxter and Augustin Vintzileos (CPC and UMD) 1 Outline I. Operational
More informationSeasonal Prediction, based on Canadian Seasonal to Interannual Prediction system (CanSIPS) for the Fifth South West Indian Ocean Climate Outlook Forum
Seasonal Prediction, based on Canadian Seasonal to Interannual Prediction system (CanSIPS) for the Fifth South West Indian Ocean Climate Outlook Forum Dr. Marko Markovic NWP Section Canadian Centre For
More informationThe Impact of Horizontal Resolution and Ensemble Size on Probabilistic Forecasts of Precipitation by the ECMWF EPS
The Impact of Horizontal Resolution and Ensemble Size on Probabilistic Forecasts of Precipitation by the ECMWF EPS S. L. Mullen Univ. of Arizona R. Buizza ECMWF University of Wisconsin Predictability Workshop,
More informationECMWF 10 th workshop on Meteorological Operational Systems
ECMWF 10 th workshop on Meteorological Operational Systems 18th November 2005 Crown copyright 2004 Page 1 Monthly range prediction products: Post-processing methods and verification Bernd Becker, Richard
More informationDROUGHT FORECASTING IN CROATIA USING STANDARDIZED PRECIPITATION INDEX (SPI)
Meteorological and Hydrological Service of Croatia www.meteo.hr DROUGHT FORECASTING IN CROATIA USING STANDARDIZED PRECIPITATION INDEX (SPI) K. Cindrićand L. Kalin cindric@cirus.dhz.hr, kalin@cirus.dhz.hr
More informationSeasonal Climate Watch April to August 2018
Seasonal Climate Watch April to August 2018 Date issued: Mar 23, 2018 1. Overview The El Niño-Southern Oscillation (ENSO) is expected to weaken from a moderate La Niña phase to a neutral phase through
More informationFive years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system
Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy 11
More informationA simple method for seamless verification applied to precipitation hindcasts from two global models
A simple method for seamless verification applied to precipitation hindcasts from two global models Matthew Wheeler 1, Hongyan Zhu 1, Adam Sobel 2, Debra Hudson 1 and Frederic Vitart 3 1 Bureau of Meteorology,
More informationProbabilistic seasonal forecast verification
Probabilistic seasonal forecast verification Caio Coelho Centro de Previsão de Tempo e Estudos Climáticos (CPTEC) Instituto Nacional de Pesquisas Espaciais (INPE) Plan of lecture Introduction: Examples
More informationExtended-range/Monthly Predictions. WGSIP, Trieste
Extended-range/Monthly Predictions WGSIP, Trieste 1 Sub-seasonal to Seasonal Prediction: Met Office, Exeter (1 to 3 December 2010) Purpose Review of the current capabilities in sub seasonal to seasonal
More informationS e a s o n a l F o r e c a s t i n g f o r t h e E u r o p e a n e n e r g y s e c t o r
S e a s o n a l F o r e c a s t i n g f o r t h e E u r o p e a n e n e r g y s e c t o r C3S European Climatic Energy Mixes (ECEM) Webinar 18 th Oct 2017 Philip Bett, Met Office Hadley Centre S e a s
More informationEMC Probabilistic Forecast Verification for Sub-season Scales
EMC Probabilistic Forecast Verification for Sub-season Scales Yuejian Zhu Environmental Modeling Center NCEP/NWS/NOAA Acknowledgement: Wei Li, Hong Guan and Eric Sinsky Present for the DTC Test Plan and
More informationApplication and verification of the ECMWF products Report 2007
Application and verification of the ECMWF products Report 2007 National Meteorological Administration Romania 1. Summary of major highlights The medium range forecast activity within the National Meteorological
More informationCategorical Verification
Forecast M H F Observation Categorical Verification Tina Kalb Contributions from Tara Jensen, Matt Pocernich, Eric Gilleland, Tressa Fowler, Barbara Brown and others Finley Tornado Data (1884) Forecast
More informationSevere storm forecast guidance based on explicit identification of convective phenomena in WRF-model forecasts
Severe storm forecast guidance based on explicit identification of convective phenomena in WRF-model forecasts Ryan Sobash 10 March 2010 M.S. Thesis Defense 1 Motivation When the SPC first started issuing
More informationSeasonal Hydrometeorological Ensemble Prediction System: Forecast of Irrigation Potentials in Denmark
Seasonal Hydrometeorological Ensemble Prediction System: Forecast of Irrigation Potentials in Denmark Diana Lucatero 1*, Henrik Madsen 2, Karsten H. Jensen 1, Jens C. Refsgaard 3, Jacob Kidmose 3 1 University
More informationDownscaling of ECMWF seasonal integrations by RegCM
Downscaling of ECMWF seasonal integrations by RegCM Čedo Branković and Mirta Patarčić Croatian Meteorological and Hydrological Service Grič 3, 10000 Zagreb, Croatia (Thanks to Paul Dando and Manuel Fuentes
More informationstatistical methods for tailoring seasonal climate forecasts Andrew W. Robertson, IRI
statistical methods for tailoring seasonal climate forecasts Andrew W. Robertson, IRI tailored seasonal forecasts why do we make probabilistic forecasts? to reduce our uncertainty about the (unknown) future
More informationBackground of Symposium/Workshop Yuhei Takaya Climate Prediction Division Japan Meteorological Agency
Background of Symposium/Workshop Yuhei Takaya ytakaya@met.kishou.go.jp Climate Prediction Division Japan Meteorological Agency 1 Long-Range Forecast, Tokyo Japan, 8-10 December Outline Background of Dynamical
More informationTCC Recent Development and Activity
11 May 2015, FOCRAII TCC Recent Development and Activity Yasushi MOCHIZUKI Tokyo Climate Center Japan Meteorological Agency TCC Recent Development and Activity / Tokyo Climate Center Tokyo Climate Center
More informationEnvironment and Climate Change Canada / GPC Montreal
Environment and Climate Change Canada / GPC Montreal Assessment, research and development Bill Merryfield Canadian Centre for Climate Modelling and Analysis (CCCma) with contributions from colleagues at
More informationRecent Developments in Climate Information Services at JMA. Koichi Kurihara Climate Prediction Division, Japan Meteorological Agency
Recent Developments in Climate Information Services at JMA Koichi Kurihara Climate Prediction Division, Japan Meteorological Agency 1 Topics 1. Diagnosis of the Northern Hemispheric circulation in December
More informationFeature-specific verification of ensemble forecasts
Feature-specific verification of ensemble forecasts www.cawcr.gov.au Beth Ebert CAWCR Weather & Environmental Prediction Group Uncertainty information in forecasting For high impact events, forecasters
More informationEnsemble forecast and verification of low level wind shear by the NCEP SREF system
Ensemble forecast and verification of low level wind shear by the NCEP SREF system Binbin Zhou*, Jeff McQueen, Jun Du, Geoff DiMego, Zoltan Toth, Yuejian Zhu NOAA/NWS/NCEP/Environment Model Center 1. Introduction
More informationWhat is one-month forecast guidance?
What is one-month forecast guidance? Kohshiro DEHARA (dehara@met.kishou.go.jp) Forecast Unit Climate Prediction Division Japan Meteorological Agency Outline 1. Introduction 2. Purposes of using guidance
More informationWMO LC-LRFMME Website User Manual
WMO LC-LRFMME Website User Manual World Meteorological Organization Lead Centre for Long-Range Forecast Multi-Model Ensemble Last update: August 2016 Contents 1. WMO LC-LRFMME Introduction... 1 1.1. Overview
More informationStandardized Anomaly Model Output Statistics Over Complex Terrain.
Standardized Anomaly Model Output Statistics Over Complex Terrain Reto.Stauffer@uibk.ac.at Outline statistical ensemble postprocessing introduction to SAMOS new snow amount forecasts in Tyrol sub-seasonal
More informationForecasting precipitation for hydroelectric power management: how to exploit GCM s seasonal ensemble forecasts
INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 27: 1691 1705 (2007) Published online in Wiley InterScience (www.interscience.wiley.com).1608 Forecasting precipitation for hydroelectric power management:
More informationVerification of the operational NWP models at DWD - with special focus at COSMO-EU
Verification of the operational NWP models at DWD - with special focus at COSMO-EU Ulrich Damrath Ulrich.Damrath@dwd.de Ein Mensch erkennt (und das ist wichtig): Nichts ist ganz falsch und nichts ganz
More informationMonthly forecast and the Summer 2003 heat wave over Europe: a case study
ATMOSPHERIC SCIENCE LETTERS Atmos. Sci. Let. 6: 112 117 (2005) Published online 21 April 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/asl.99 Monthly forecast and the Summer 2003
More informationSpatial forecast verification
Spatial forecast verification Manfred Dorninger University of Vienna Vienna, Austria manfred.dorninger@univie.ac.at Thanks to: B. Ebert, B. Casati, C. Keil 7th Verification Tutorial Course, Berlin, 3-6
More informationBasic Verification Concepts
Basic Verification Concepts Barbara Brown National Center for Atmospheric Research Boulder Colorado USA bgb@ucar.edu May 2017 Berlin, Germany Basic concepts - outline What is verification? Why verify?
More informationAssessment of Ensemble Forecasts
Assessment of Ensemble Forecasts S. L. Mullen Univ. of Arizona HEPEX Workshop, 7 March 2004 Talk Overview Ensemble Performance for Precipitation Global EPS and Mesoscale 12 km RSM Biases, Event Discrimination
More informationVerification of the Seasonal Forecast for the 2005/06 Winter
Verification of the Seasonal Forecast for the 2005/06 Winter Shingo Yamada Tokyo Climate Center Japan Meteorological Agency 2006/11/02 7 th Joint Meeting on EAWM Contents 1. Verification of the Seasonal
More informationNOTES AND CORRESPONDENCE. Improving Week-2 Forecasts with Multimodel Reforecast Ensembles
AUGUST 2006 N O T E S A N D C O R R E S P O N D E N C E 2279 NOTES AND CORRESPONDENCE Improving Week-2 Forecasts with Multimodel Reforecast Ensembles JEFFREY S. WHITAKER AND XUE WEI NOAA CIRES Climate
More informationTCC Training Seminar on 17 th Nov 2015 JMA s Ensemble Prediction Systems (EPSs) and their Products for Climate Forecast.
TCC Training Seminar on 17 th Nov 2015 JMA s Ensemble Prediction Systems (EPSs) and their Products for Climate Forecast Takashi Yamada Climate Prediction Division Japan Meteorological Agency 1 Contents
More information3.6 NCEP s Global Icing Ensemble Prediction and Evaluation
1 3.6 NCEP s Global Icing Ensemble Prediction and Evaluation Binbin Zhou 1,2, Yali Mao 1,2, Hui-ya Chuang 2 and Yuejian Zhu 2 1. I.M. System Group, Inc. 2. EMC/NCEP AMS 18th Conference on Aviation, Range,
More informationAssessing high resolution forecasts using fuzzy verification methods
Assessing high resolution forecasts using fuzzy verification methods Beth Ebert Bureau of Meteorology Research Centre, Melbourne, Australia Thanks to Nigel Roberts, Barbara Casati, Frederic Atger, Felix
More informationSeasonal Predictions for South Caucasus and Armenia
Seasonal Predictions for South Caucasus and Armenia Anahit Hovsepyan Zagreb, 11-12 June 2008 SEASONAL PREDICTIONS for the South Caucasus There is a notable increase of interest of population and governing
More informationAn extended re-forecast set for ECMWF system 4. in the context of EUROSIP
An extended re-forecast set for ECMWF system 4 in the context of EUROSIP Tim Stockdale Acknowledgements: Magdalena Balmaseda, Susanna Corti, Laura Ferranti, Kristian Mogensen, Franco Molteni, Frederic
More informationClimate Outlook for Pacific Islands for July December 2017
The APEC CLIMATE CENTER Climate Outlook for Pacific Islands for July December 2017 BUSAN, 26 June 2017 Synthesis of the latest model forecasts for July December 2017 (JASOND) at the APEC Climate Center
More informationVerification of nowcasts and short-range forecasts, including aviation weather
Verification of nowcasts and short-range forecasts, including aviation weather Barbara Brown NCAR, Boulder, Colorado, USA WMO WWRP 4th International Symposium on Nowcasting and Very-short-range Forecast
More informationThe benefits and developments in ensemble wind forecasting
The benefits and developments in ensemble wind forecasting Erik Andersson Slide 1 ECMWF European Centre for Medium-Range Weather Forecasts Slide 1 ECMWF s global forecasting system High resolution forecast
More informationVerification statistics and evaluations of ECMWF forecasts in
635 Verification statistics and evaluations of ECMWF forecasts in 29-21 D.S. Richardson, J. Bidlot, L. Ferranti, A. Ghelli, T. Hewson, M. Janousek, F. Prates and F. Vitart Operations Department October
More informationQUANTIFYING THE ECONOMIC VALUE OF WEATHER FORECASTS: REVIEW OF METHODS AND RESULTS
QUANTIFYING THE ECONOMIC VALUE OF WEATHER FORECASTS: REVIEW OF METHODS AND RESULTS Rick Katz Institute for Study of Society and Environment National Center for Atmospheric Research Boulder, CO USA Email:
More informationThe 2010/11 drought in the Horn of Africa: Monitoring and forecasts using ECMWF products
The 2010/11 drought in the Horn of Africa: Monitoring and forecasts using ECMWF products Emanuel Dutra Fredrik Wetterhall Florian Pappenberger Souhail Boussetta Gianpaolo Balsamo Linus Magnusson Slide
More informationDetective Miss Marple in Murder at the Gallop. 3rd verification workshop, ECMWF 2007, Göber
m Detective Miss Marple in Murder at the Gallop Slide m Now, this is how we verify the forecasters at DWD: we watch them intensily and when they get it wrong, they'll get a bang over their head. OK, I'm
More informationClimate Outlook for Pacific Islands for December 2017 May 2018
The APEC CLIMATE CENTER Climate Outlook for Pacific Islands for December 2017 May 2018 BUSAN, 24 November 2017 The synthesis of the latest model forecasts for December 2017 to May 2018 (DJFMAM) from the
More informationIntroduction of Seasonal Forecast Guidance. TCC Training Seminar on Seasonal Prediction Products November 2013
Introduction of Seasonal Forecast Guidance TCC Training Seminar on Seasonal Prediction Products 11-15 November 2013 1 Outline 1. Introduction 2. Regression method Single/Multi regression model Selection
More informationModel verification and tools. C. Zingerle ZAMG
Model verification and tools C. Zingerle ZAMG Why verify? The three most important reasons to verify forecasts are: to monitor forecast quality - how accurate are the forecasts and are they improving over
More informationBasic Verification Concepts
Basic Verification Concepts Barbara Brown National Center for Atmospheric Research Boulder Colorado USA bgb@ucar.edu Basic concepts - outline What is verification? Why verify? Identifying verification
More informationFocus on parameter variation results
Accounting for Model Uncertainty in the Navy s Global Ensemble Forecasting System C. Reynolds, M. Flatau, D. Hodyss, J. McLay, J. Moskaitis, J. Ridout, C. Sampson, J. Cummings Naval Research Lab, Monterey,
More informationApplication and verification of ECMWF products: 2010
Application and verification of ECMWF products: 2010 Hellenic National Meteorological Service (HNMS) F. Gofa, D. Tzeferi and T. Charantonis 1. Summary of major highlights In order to determine the quality
More informationGlobal climate predictions: forecast drift and bias adjustment issues
www.bsc.es Ispra, 23 May 2017 Global climate predictions: forecast drift and bias adjustment issues Francisco J. Doblas-Reyes BSC Earth Sciences Department and ICREA Many of the ideas in this presentation
More informationTokyo Climate Center s activities as RCC Tokyo
First WMO Workshop on Operational Climate Prediction Pune, India, 09-11 November 2015 Tokyo Climate Center s activities as RCC Tokyo Arata Endo Tokyo Climate Center Japan Meteorological Agency E-mail:
More informationREPORT ON APPLICATIONS OF EPS FOR SEVERE WEATHER FORECASTING
WORLD METEOROLOGICAL ORGANIZATION COMMISSION FOR BASIC SYSTEMS OPAG DPFS EXPERT TEAM ON ENSEMBLE PREDICTION SYSTEMS CBS-DPFS/EPS/Doc. 7(2) (31.I.2006) Item: 7 ENGLISH ONLY EXETER, UNITED KINGDOM 6-10 FEBRUARY
More informationVerification of WAQUA/DCSMv5 s operational water level probability forecasts
Ministry of Infrastructure and Water Management Verification of WAQUA/DCSMv5 s operational water level probability forecasts N. Wagenaar KNMI Internal report IR-18-1 UTRECHT UNIVSERSITY DEPARTMENT OF
More informationJP3.7 SHORT-RANGE ENSEMBLE PRECIPITATION FORECASTS FOR NWS ADVANCED HYDROLOGIC PREDICTION SERVICES (AHPS): PARAMETER ESTIMATION ISSUES
JP3.7 SHORT-RANGE ENSEMBLE PRECIPITATION FORECASTS FOR NWS ADVANCED HYDROLOGIC PREDICTION SERVICES (AHPS): PARAMETER ESTIMATION ISSUES John Schaake*, Mary Mullusky, Edwin Welles and Limin Wu Hydrology
More informationExploring ensemble forecast calibration issues using reforecast data sets
NOAA Earth System Research Laboratory Exploring ensemble forecast calibration issues using reforecast data sets Tom Hamill and Jeff Whitaker NOAA Earth System Research Lab, Boulder, CO tom.hamill@noaa.gov
More informationExperimental MOS Precipitation Type Guidance from the ECMWF Model
Experimental MOS Precipitation Type Guidance from the ECMWF Model Phillip E. Shafer David E. Rudack National Weather Service Meteorological Development Laboratory Silver Spring, MD Development Overview:
More informationMulti-Model Ensembles in NWS Climate Prediction Center Subseasonal to Seasonal Forecasts: Metrics for Impact Events
Multi-Model Ensembles in NWS Climate Prediction Center Subseasonal to Seasonal Forecasts: Metrics for Impact Events Dan Collins, Climate Prediction Center Sarah Strazzo, CSIRO partners Q.J. Wang and Andrew
More information1.10 A NEW SPATIAL SCALE DECOMPOSITION OF THE BRIER SCORE FOR THE VERIFICATION OF PROBABILISTIC LIGHTNING FORECASTS. Abstract
1.10 A NEW SPATIAL SCALE DECOMPOSITION OF THE BRIER SCORE FOR THE VERIFICATION OF PROBABILISTIC LIGHTNING FORECASTS B. Casati L. J. Wilson Meteorological Service of Canada, Meteorological Research Branch,
More informationUpgrade of JMA s Typhoon Ensemble Prediction System
Upgrade of JMA s Typhoon Ensemble Prediction System Masayuki Kyouda Numerical Prediction Division, Japan Meteorological Agency and Masakazu Higaki Office of Marine Prediction, Japan Meteorological Agency
More informationProbabilistic Forecast Verification. Yuejian Zhu EMC/NCEP/NOAA
Probabilistic Forecast Verification Yuejian Zhu EMC/NCEP/NOAA Review NAEFS Products (FY07) (December 4 th 2007) Bias corrected NCEP/GFS forecast 4 times daily, every 6 hours, out to 180 hours Bias corrected
More information