Data science and engineering for local weather forecasts. Nikhil R Podduturi Data {Scientist, Engineer} November, 2016

Size: px
Start display at page:

Download "Data science and engineering for local weather forecasts. Nikhil R Podduturi Data {Scientist, Engineer} November, 2016"

Transcription

1 1

2 Data science and engineering for local weather forecasts Nikhil R Podduturi Data {Scientist, Engineer} November, 2016

3 Agenda About MeteoGroup Introduction to weather data Problem description Data science and weather forecasting Engineering Verification Results Questions 3

4 How many of you check weather forecasts frequently? 4

5 5

6 6 Weather data

7 7 1.5 TB/day

8 Types of data Observations: WMO weather stations (e.g: surface, upper-air, ships, drifting buoys, aircrafts etc) MeteoGroup measurement network 8

9 Types of data Observations: WMO weather stations (e.g: surface, upper-air, ships, drifting buoys, aircrafts etc) MeteoGroup measurement network Satellite data 9

10 Types of data Observations: WMO weather stations (e.g: surface, upper-air, ships, drifting buoys, aircrafts etc) MeteoGroup measurement network Satellite data Radar data 10

11 Types of data Observations: WMO weather stations (e.g: surface, upper-air, ships, drifting buoys, aircrafts etc) MeteoGroup measurement network Satellite data Radar data User data 11

12 Types of data Observations: WMO weather stations (e.g: surface, upper-air, ships, drifting buoys, aircrafts etc) MeteoGroup measurement network Satellite data Radar data User data Numerical weather prediction model data 12

13 Numerical weather prediction models Complex and Multidimensional data 13

14 Numerical weather prediction models Complex and multidimensional data 5 NWP models from different providers 14

15 Numerical weather prediction models Complex and multidimensional data 5 NWP models from different providers Data size per day TB 15

16 Data science and weather forecasting 16

17 17

18 Outcome Took 24 hours for 24 hour forecasts Grid interval km Poor results 18

19 MeteoGroup Forecasting system 19

20 MeteoGroup forecasting system 3 years of NWP data Machine learning model Trained model Forecasts 3 years of observation data Daily NWP data 20

21 MeteoGroup forecasting system Written in pascal 21

22 MeteoGroup forecasting system Written in pascal Runs on in house high performance computing cluster 22

23 MeteoGroup forecasting system Written in pascal Runs on in house high performance computing cluster Limitations Hard to maintain Not very transparent Scalability 23

24 24 Problem description

25 Next generation forecasting system Cloud based solution 25

26 Next generation forecasting system Cloud based solution Transparent 26

27 Next generation forecasting system Cloud based solution Transparent Scalable 27

28 Next generation forecasting system Cloud based solution Transparent Scalable Improve forecasting accuracy 28

29 Baseline model NWP data Downscale to location Interpolate missing values Linear model 29

30 Baseline model NWP data Downscale to location Interpolate missing values Linear model Outcome: Very fast Poor accuracy Multicollinearity 30

31 Iteration 1 Address multicollinearity using feature selection Scale the features NWP data Downscale to location Interpolate missing values Scale features Feature selection Linear model 31

32 Iteration 1 Address multicollinearity using feature selection Scale the features NWP data Downscale to location Interpolate missing values Scale features Feature selection Linear model Outcome: Improved accuracy 32

33 Iteration 2 Model selection between linear and non-linear models Advanced feature selection NWP data Downscale to location Interpolate missing values Scale features Advance feature selection Model selection (linear and non-linear models) 33

34 Iteration 2 Model selection between linear and non-linear models Advanced feature selection NWP data Downscale to location Interpolate missing values Scale features Advance feature selection Model selection (linear and non-linear models) Outcome: On par with existing forecasting system Slow training 34

35 35 Engineering to scale the product

36 Baseline model engineering (Scikit-learn, NumPy, Keras with TensorFlow) 36

37 Model engineering (Scikit-learn, NumPy, Keras with TensorFlow) Good: Python ML ecosystem Familiarity among the team Test driven and Agile Development Fail fast 37

38 Model engineering (Scikit-learn, NumPy, Keras with TensorFlow) Good: Python ML ecosystem Familiarity among the team Test driven and Agile Development Fail fast Bad: Not scalable 38

39 47000 * 15 * 360 model runs Locations Weather attributes e.g: temperature, wind etc Hours 39

40 Scaling with Apache Airflow 40 Apache Airflow By AirBnB Apache product since early 2016 Directed Acyclic Graph (DAG) Components UI Scheduler Executor(s)

41 Apache Airflow DAG Hooks (connections) Operators (tasks) Schedule Dependencies 41

42 Airflow and Mesos deploy persist AWS S3 Airflow scheduler Mesos cluster 42

43 Airflow and Mesos Cont Integ deploy Persist AWS S3 Airflow scheduler Mesos cluster 43

44 44 Verification

45 Model improvement cycle Deploy DAG Verify model Improve DAG 45

46 Forecast verification Forecast Engine AWS S3 with models JSON-LD 46

47 Verification metrics Mean absolute error Root mean squared error Mean error Heidke skill score Equitable threat score Probability density functions Error percentiles 47

48 48 Mean absolute error for different models (Temperature)

49 49 Probability distribution function for multiple models (Temperature)

50 Percentile graphs for each model (Temperature)

51 51 For demo please stop by MG booth

52 Results Cloud based solution AWS S3, EC2, ElastiCache Transparent Scalable Improve forecasting accuracy 52

53 Results Cloud based solution AWS S3, EC2, ElastiCache Transparent Verification microservice Scalable Improve forecasting accuracy 53

54 Results Cloud based solution AWS S3, EC2, ElastiCache Transparent Verification microservice Scalable Mesos cluster Training time a month to 5 hours (approx) Improve forecasting accuracy 54

55 Results Cloud based solution AWS S3, EC2, ElastiCache Transparent Verification microservice Scalable Mesos cluster Training time a month to 5 hours (approx) Improve forecasting accuracy On par or better 55

56 Improvements Hyperlocal AWS lambda integration Iterate for more accuracy 56

57 57 Questions?

58 We are hiring!

59 59

Implementation of global surface index at the Met Office. Submitted by Marion Mittermaier. Summary and purpose of document

Implementation of global surface index at the Met Office. Submitted by Marion Mittermaier. Summary and purpose of document WORLD METEOROLOGICAL ORGANIZATION COMMISSION FOR BASIC SYSTEMS OPAG on DPFS MEETING OF THE CBS (DPFS) TASK TEAM ON SURFACE VERIFICATION GENEVA, SWITZERLAND 20-21 OCTOBER 2014 DPFS/TT-SV/Doc. 4.1a (X.IX.2014)

More information

Scatterometer Wind Assimilation at the Met Office

Scatterometer Wind Assimilation at the Met Office Scatterometer Wind Assimilation at the Met Office James Cotton International Ocean Vector Winds Science Team (IOVWST) meeting, Brest, June 2014 Outline Assimilation status Global updates: Metop-B and spatial

More information

Global NWP Index documentation

Global NWP Index documentation Global NWP Index documentation The global index is calculated in two ways, against observations, and against model analyses. Observations are sparse in some parts of the world, and using full gridded analyses

More information

Hurricane Prediction with Python

Hurricane Prediction with Python Hurricane Prediction with Python Minwoo Lee lemin@cs.colostate.edu Computer Science Department Colorado State University 1 Hurricane Prediction Hurricane Katrina We lost everything. Katrina didn t care

More information

A Community Gridded Atmospheric Forecast System for Calibrated Solar Irradiance

A Community Gridded Atmospheric Forecast System for Calibrated Solar Irradiance A Community Gridded Atmospheric Forecast System for Calibrated Solar Irradiance David John Gagne 1,2 Sue E. Haupt 1,3 Seth Linden 1 Gerry Wiener 1 1. NCAR RAL 2. University of Oklahoma 3. Penn State University

More information

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2016

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2016 JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2016 New Zealand / Meteorological Service of New Zealand

More information

FORECASTING: A REVIEW OF STATUS AND CHALLENGES. Eric Grimit and Kristin Larson 3TIER, Inc. Pacific Northwest Weather Workshop March 5-6, 2010

FORECASTING: A REVIEW OF STATUS AND CHALLENGES. Eric Grimit and Kristin Larson 3TIER, Inc. Pacific Northwest Weather Workshop March 5-6, 2010 SHORT-TERM TERM WIND POWER FORECASTING: A REVIEW OF STATUS AND CHALLENGES Eric Grimit and Kristin Larson 3TIER, Inc. Pacific Northwest Weather Workshop March 5-6, 2010 Integrating Renewable Energy» Variable

More information

The benefits and developments in ensemble wind forecasting

The benefits and developments in ensemble wind forecasting The benefits and developments in ensemble wind forecasting Erik Andersson Slide 1 ECMWF European Centre for Medium-Range Weather Forecasts Slide 1 ECMWF s global forecasting system High resolution forecast

More information

WMO Aeronautical Meteorology Scientific Conference 2017

WMO Aeronautical Meteorology Scientific Conference 2017 Session 1 Science underpinning meteorological observations, forecasts, advisories and warnings 1.6 Observation, nowcast and forecast of future needs 1.6.1 Advances in observing methods and use of observations

More information

Application and verification of ECMWF products in Croatia

Application and verification of ECMWF products in Croatia Application and verification of ECMWF products in Croatia August 2008 1. Summary of major highlights At Croatian Met Service, ECMWF products are the major source of data used in the operational weather

More information

NUMERICAL EXPERIMENTS USING CLOUD MOTION WINDS AT ECMWF GRAEME KELLY. ECMWF, Shinfield Park, Reading ABSTRACT

NUMERICAL EXPERIMENTS USING CLOUD MOTION WINDS AT ECMWF GRAEME KELLY. ECMWF, Shinfield Park, Reading ABSTRACT NUMERICAL EXPERIMENTS USING CLOUD MOTION WINDS AT ECMWF GRAEME KELLY ECMWF, Shinfield Park, Reading ABSTRACT Recent monitoring of cloud motion winds (SATOBs) at ECMWF has shown an improvement in quality.

More information

How Computers Help Humans Root Cause Issues at Netflix SETH KATZ QCON NEW YORK, 2018

How Computers Help Humans Root Cause Issues at Netflix SETH KATZ QCON NEW YORK, 2018 How Computers Help Humans Root Cause Issues at Netflix SETH KATZ QCON NEW YORK, 2018 Hello!! Seth Katz! 5 years at Netflix! Focused on improving Netflix operations! Share what we ve learned on applying

More information

ECMWF global reanalyses: Resources for the wind energy community

ECMWF global reanalyses: Resources for the wind energy community ECMWF global reanalyses: Resources for the wind energy community (and a few myth-busters) Paul Poli European Centre for Medium-range Weather Forecasts (ECMWF) Shinfield Park, RG2 9AX, Reading, UK paul.poli

More information

Weather Forecasting: Lecture 2

Weather Forecasting: Lecture 2 Weather Forecasting: Lecture 2 Dr. Jeremy A. Gibbs Department of Atmospheric Sciences University of Utah Spring 2017 1 / 40 Overview 1 Forecasting Techniques 2 Forecast Tools 2 / 40 Forecasting Techniques

More information

SAWS: Met-Ocean Data & Infrastructure in Support of Industry, Research & Public Good. South Africa-Norway Science Week, 2016

SAWS: Met-Ocean Data & Infrastructure in Support of Industry, Research & Public Good. South Africa-Norway Science Week, 2016 SAWS: Met-Ocean Data & Infrastructure in Support of Industry, Research & Public Good South Africa-Norway Science Week, 2016 Marc de Vos, November 2016 South Africa: Context http://learn.mindset.co.za/sites/default/files/resourcelib/e

More information

Continuous Machine Learning

Continuous Machine Learning Continuous Machine Learning Kostiantyn Bokhan, PhD Project Lead at Samsung R&D Ukraine Kharkiv, October 2016 Agenda ML dev. workflows ML dev. issues ML dev. solutions Continuous machine learning (CML)

More information

A SOLAR AND WIND INTEGRATED FORECAST TOOL (SWIFT) DESIGNED FOR THE MANAGEMENT OF RENEWABLE ENERGY VARIABILITY ON HAWAIIAN GRID SYSTEMS

A SOLAR AND WIND INTEGRATED FORECAST TOOL (SWIFT) DESIGNED FOR THE MANAGEMENT OF RENEWABLE ENERGY VARIABILITY ON HAWAIIAN GRID SYSTEMS ALBANY BARCELONA BANGALORE ICEM 2015 June 26, 2015 Boulder, CO A SOLAR AND WIND INTEGRATED FORECAST TOOL (SWIFT) DESIGNED FOR THE MANAGEMENT OF RENEWABLE ENERGY VARIABILITY ON HAWAIIAN GRID SYSTEMS JOHN

More information

Swedish Meteorological and Hydrological Institute

Swedish Meteorological and Hydrological Institute Swedish Meteorological and Hydrological Institute Norrköping, Sweden 1. Summary of highlights HIRLAM at SMHI is run on a CRAY T3E with 272 PEs at the National Supercomputer Centre (NSC) organised together

More information

Operational use of ensemble hydrometeorological forecasts at EDF (french producer of energy)

Operational use of ensemble hydrometeorological forecasts at EDF (french producer of energy) Operational use of ensemble hydrometeorological forecasts at EDF (french producer of energy) M. Le Lay, P. Bernard, J. Gailhard, R. Garçon, T. Mathevet & EDF forecasters matthieu.le-lay@edf.fr SBRH Conference

More information

Computational Challenges in Big Data Assimilation with Extreme-scale Simulations

Computational Challenges in Big Data Assimilation with Extreme-scale Simulations May 1, 2013, BDEC workshop, Charleston, SC Computational Challenges in Big Data Assimilation with Extreme-scale Simulations Takemasa Miyoshi RIKEN Advanced Institute for Computational Science Takemasa.Miyoshi@riken.jp

More information

1 Introduction. Station Type No. Synoptic/GTS 17 Principal 172 Ordinary 546 Precipitation

1 Introduction. Station Type No. Synoptic/GTS 17 Principal 172 Ordinary 546 Precipitation Use of Automatic Weather Stations in Ethiopia Dula Shanko National Meteorological Agency(NMA), Addis Ababa, Ethiopia Phone: +251116639662, Mob +251911208024 Fax +251116625292, Email: Du_shanko@yahoo.com

More information

Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution

Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution Clément Gautrais 1, Yann Dauxais 1, and Maël Guilleme 2 1 University of Rennes 1/Inria Rennes clement.gautrais@irisa.fr 2 Energiency/University

More information

Application and verification of ECMWF products 2012

Application and verification of ECMWF products 2012 Application and verification of ECMWF products 2012 Met Eireann, Glasnevin Hill, Dublin 9, Ireland. J.Hamilton 1. Summary of major highlights The verification of ECMWF products has continued as in previous

More information

How to use the guidance tool (Producing Guidance and Verification)

How to use the guidance tool (Producing Guidance and Verification) Ex.1 How to use the guidance tool (Producing Guidance and Verification) Masayuki Hirai Tokyo Climate Center (TCC)/ Climate Prediction Division of Japan Meteorological Agency (JMA) Schedule of exercise

More information

GENERAL DESCRIPTION OF THE WEATHER FORECAST PROCESS WITH EMPHASIS ON FORECAST UNCERTAINTY. Zoltan Toth

GENERAL DESCRIPTION OF THE WEATHER FORECAST PROCESS WITH EMPHASIS ON FORECAST UNCERTAINTY. Zoltan Toth GENERAL DESCRIPTION OF THE WEATHER FORECAST PROCESS WITH EMPHASIS ON FORECAST UNCERTAINTY Zoltan Toth Environmental Modeling Center NOAA/NWS/NCEP USA Acknowledgements: Steve Lord, David Helms, Geoff DiMego,

More information

CARLOS F. M. COIMBRA (PI) HUGO T. C. PEDRO (CO-PI)

CARLOS F. M. COIMBRA (PI) HUGO T. C. PEDRO (CO-PI) HIGH-FIDELITY SOLAR POWER FORECASTING SYSTEMS FOR THE 392 MW IVANPAH SOLAR PLANT (CSP) AND THE 250 MW CALIFORNIA VALLEY SOLAR RANCH (PV) PROJECT CEC EPC-14-008 CARLOS F. M. COIMBRA (PI) HUGO T. C. PEDRO

More information

YOPP archive: needs of the verification community

YOPP archive: needs of the verification community YOPP archive: needs of the verification community B. Casati, B. Brown, T. Haiden, C. Coelho Talk Outline: P1 model and observation data P2 observation uncertainty P2 matched model and observation: time

More information

1. INTRODUCTION 2. QPF

1. INTRODUCTION 2. QPF 440 24th Weather and Forecasting/20th Numerical Weather Prediction HUMAN IMPROVEMENT TO NUMERICAL WEATHER PREDICTION AT THE HYDROMETEOROLOGICAL PREDICTION CENTER David R. Novak, Chris Bailey, Keith Brill,

More information

Numerical Weather Prediction: Data assimilation. Steven Cavallo

Numerical Weather Prediction: Data assimilation. Steven Cavallo Numerical Weather Prediction: Data assimilation Steven Cavallo Data assimilation (DA) is the process estimating the true state of a system given observations of the system and a background estimate. Observations

More information

The WMO Global Basic Observing Network (GBON)

The WMO Global Basic Observing Network (GBON) The WMO Global Basic Observing Network (GBON) A WIGOS approach to securing observational data for critical global weather and climate applications Robert Varley and Lars Peter Riishojgaard, WMO Secretariat,

More information

Ocean Modeling. Matt McKnight Boxuan Gu

Ocean Modeling. Matt McKnight Boxuan Gu Ocean Modeling Matt McKnight Boxuan Gu Engineering the system The Earth Understanding that the Oceans are inextricably linked to the world s climate is easy. Describing this relationship is more difficult,

More information

Application and verification of ECMWF products 2016

Application and verification of ECMWF products 2016 Application and verification of ECMWF products 2016 Met Eireann, Glasnevin Hill, Dublin 9, Ireland. J.Hamilton 1. Summary of major highlights The verification of ECMWF products has continued as in previous

More information

Impact of GPS and TMI Precipitable Water Data on Mesoscale Numerical Weather Prediction Model Forecasts

Impact of GPS and TMI Precipitable Water Data on Mesoscale Numerical Weather Prediction Model Forecasts Journal of the Meteorological Society of Japan, Vol. 82, No. 1B, pp. 453--457, 2004 453 Impact of GPS and TMI Precipitable Water Data on Mesoscale Numerical Weather Prediction Model Forecasts Ko KOIZUMI

More information

The ECMWF coupled data assimilation system

The ECMWF coupled data assimilation system The ECMWF coupled data assimilation system Patrick Laloyaux Acknowledgments: Magdalena Balmaseda, Kristian Mogensen, Peter Janssen, Dick Dee August 21, 214 Patrick Laloyaux (ECMWF) CERA August 21, 214

More information

Discovery Through Situational Awareness

Discovery Through Situational Awareness Discovery Through Situational Awareness BRETT AMIDAN JIM FOLLUM NICK BETZSOLD TIM YIN (UNIVERSITY OF WYOMING) SHIKHAR PANDEY (WASHINGTON STATE UNIVERSITY) Pacific Northwest National Laboratory February

More information

Progress on GCOS-China CMA IOS Development Plan ( ) PEI, Chong Department of Integrated Observation of CMA 09/25/2017 Hangzhou, China

Progress on GCOS-China CMA IOS Development Plan ( ) PEI, Chong Department of Integrated Observation of CMA 09/25/2017 Hangzhou, China Progress on GCOS-China CMA IOS Development Plan (2016-2020) PEI, Chong Department of Integrated Observation of CMA 09/25/2017 Hangzhou, China 1. Progress on GCOS-China 1 Organized GCOS-China GCOS-China

More information

DATA BROWSING AND ANALYSIS TOOL FOR MTSAT/LRIT. Ryoji Kumabe

DATA BROWSING AND ANALYSIS TOOL FOR MTSAT/LRIT. Ryoji Kumabe DATA BROWSING AND ANALYSIS TOOL FOR MTSAT/LRIT Ryoji Kumabe Meteorological Satellite Center / Japan Meteorological Agency 3-235 Nakakiyoto Kiyose Tokyo, Japan ABSTRACT The Multi-functional Transport Satellite

More information

Role of the forecaster

Role of the forecaster Role of the forecaster Anything new? Robert Mureau Why me? Answer : yes and no Historic perspective technology gap and the angry (white) forecaster Role of NVBM The two roles: guidance forecaster and communicator

More information

Introduction to Meteorology and Weather Forecasting

Introduction to Meteorology and Weather Forecasting Introduction to Meteorology and Weather Forecasting ENVI1400 : Meteorology and Forecasting : lecture 1 2 040909 ENVI1400 : Meteorology and Forecasting : lecture 1 3 040914 ENVI1400 : Meteorology and Forecasting

More information

Xinhua Liu National Meteorological Center (NMC) of China Meteorological Administration (CMA)

Xinhua Liu National Meteorological Center (NMC) of China Meteorological Administration (CMA) The short-time forecasting and nowcasting technology of severe convective weather for aviation meteorological services in China Xinhua Liu National Meteorological Center (NMC) of China Meteorological Administration

More information

Integrated Electricity Demand and Price Forecasting

Integrated Electricity Demand and Price Forecasting Integrated Electricity Demand and Price Forecasting Create and Evaluate Forecasting Models The many interrelated factors which influence demand for electricity cannot be directly modeled by closed-form

More information

EWGLAM/SRNWP National presentation from DMI

EWGLAM/SRNWP National presentation from DMI EWGLAM/SRNWP 2013 National presentation from DMI Development of operational Harmonie at DMI Since Jan 2013 DMI updated HARMONIE-Denmark suite to CY37h1 with a 3h-RUC cycling and 57h forecast, 8 times a

More information

Observing System Impact Studies in ACCESS

Observing System Impact Studies in ACCESS Observing System Impact Studies in ACCESS www.cawcr.gov.au Chris Tingwell, Peter Steinle, John le Marshall, Elaine Miles, Yi Xiao, Rolf Seecamp, Jin Lee, Susan Rennie, Xingbao Wang, Justin Peter, Alan

More information

Application and verification of ECMWF products 2008

Application and verification of ECMWF products 2008 Application and verification of ECMWF products 2008 RHMS of Serbia 1. Summary of major highlights ECMWF products are operationally used in Hydrometeorological Service of Serbia from the beginning of 2003.

More information

Application and verification of ECMWF products 2014

Application and verification of ECMWF products 2014 Application and verification of ECMWF products 2014 Israel Meteorological Service (IMS), 1. Summary of major highlights ECMWF deterministic runs are used to issue most of the operational forecasts at IMS.

More information

Forecasting demand in the National Electricity Market. October 2017

Forecasting demand in the National Electricity Market. October 2017 Forecasting demand in the National Electricity Market October 2017 Agenda Trends in the National Electricity Market A review of AEMO s forecasting methods Long short-term memory (LSTM) neural networks

More information

Verification of ECMWF products at the Deutscher Wetterdienst (DWD)

Verification of ECMWF products at the Deutscher Wetterdienst (DWD) Verification of ECMWF products at the Deutscher Wetterdienst (DWD) DWD Martin Göber 1. Summary of major highlights The usage of a combined GME-MOS and ECMWF-MOS continues to lead to a further increase

More information

WWRP Implementation Plan Reporting AvRDP

WWRP Implementation Plan Reporting AvRDP WWRP Implementation Plan Reporting AvRDP Please send you report to Paolo Ruti (pruti@wmo.int) and Sarah Jones (sarah.jones@dwd.de). High Impact Weather and its socio economic effects in the context of

More information

Application and verification of ECMWF products in Croatia - July 2007

Application and verification of ECMWF products in Croatia - July 2007 Application and verification of ECMWF products in Croatia - July 2007 By Lovro Kalin, Zoran Vakula and Josip Juras (Hydrological and Meteorological Service) 1. Summary of major highlights At Croatian Met

More information

BRINGING MARINE DATA ASSETS TO THE FUTURE INTERNET

BRINGING MARINE DATA ASSETS TO THE FUTURE INTERNET ENVIROfying the Future Internet BRINGING MARINE DATA ASSETS TO THE FUTURE INTERNET Leveraging the Future Internet for the Marine Usage Area Dr.Conor Delaney Galway Bay Smart Bay - Ireland MARINE SCENARIOS

More information

Data Short description Parameters to be used for analysis SYNOP. Surface observations by ships, oil rigs and moored buoys

Data Short description Parameters to be used for analysis SYNOP. Surface observations by ships, oil rigs and moored buoys 3.2 Observational Data 3.2.1 Data used in the analysis Data Short description Parameters to be used for analysis SYNOP Surface observations at fixed stations over land P,, T, Rh SHIP BUOY TEMP PILOT Aircraft

More information

Current research issues. Philippe Bougeault, Météo France

Current research issues. Philippe Bougeault, Météo France Current research issues for short range NWP Philippe Bougeault, Météo France The expectations from government and society as seen from Météo France Improve the accuracy of short range forecasts for security

More information

Developments towards multi-model based forecast product generation

Developments towards multi-model based forecast product generation Developments towards multi-model based forecast product generation Ervin Zsótér Methodology and Forecasting Section Hungarian Meteorological Service Introduction to the currently operational forecast production

More information

Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system

Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy 11

More information

Clustering Techniques and their applications at ECMWF

Clustering Techniques and their applications at ECMWF Clustering Techniques and their applications at ECMWF Laura Ferranti European Centre for Medium-Range Weather Forecasts Training Course NWP-PR: Clustering techniques and their applications at ECMWF 1/32

More information

Convective-scale NWP for Singapore

Convective-scale NWP for Singapore Convective-scale NWP for Singapore Hans Huang and the weather modelling and prediction section MSS, Singapore Dale Barker and the SINGV team Met Office, Exeter, UK ECMWF Symposium on Dynamical Meteorology

More information

ABSTRACT 3 RADIAL VELOCITY ASSIMILATION IN BJRUC 3.1 ASSIMILATION STRATEGY OF RADIAL

ABSTRACT 3 RADIAL VELOCITY ASSIMILATION IN BJRUC 3.1 ASSIMILATION STRATEGY OF RADIAL REAL-TIME RADAR RADIAL VELOCITY ASSIMILATION EXPERIMENTS IN A PRE-OPERATIONAL FRAMEWORK IN NORTH CHINA Min Chen 1 Ming-xuan Chen 1 Shui-yong Fan 1 Hong-li Wang 2 Jenny Sun 2 1 Institute of Urban Meteorology,

More information

Weather Analysis and Forecasting

Weather Analysis and Forecasting Weather Analysis and Forecasting An Information Statement of the American Meteorological Society (Adopted by AMS Council on 25 March 2015) Bull. Amer. Meteor. Soc., 88 This Information Statement describes

More information

Multi-Plant Photovoltaic Energy Forecasting Challenge with Regression Tree Ensembles and Hourly Average Forecasts

Multi-Plant Photovoltaic Energy Forecasting Challenge with Regression Tree Ensembles and Hourly Average Forecasts Multi-Plant Photovoltaic Energy Forecasting Challenge with Regression Tree Ensembles and Hourly Average Forecasts Kathrin Bujna 1 and Martin Wistuba 2 1 Paderborn University 2 IBM Research Ireland Abstract.

More information

Implementation Guidance of Aeronautical Meteorological Observer Competency Standards

Implementation Guidance of Aeronautical Meteorological Observer Competency Standards Implementation Guidance of Aeronautical Meteorological Observer Competency Standards The following guidance is supplementary to the AMP competency Standards endorsed by Cg-16 in Geneva in May 2011. Please

More information

AVIATION APPLICATIONS OF A NEW GENERATION OF MESOSCALE NUMERICAL WEATHER PREDICTION SYSTEM OF THE HONG KONG OBSERVATORY

AVIATION APPLICATIONS OF A NEW GENERATION OF MESOSCALE NUMERICAL WEATHER PREDICTION SYSTEM OF THE HONG KONG OBSERVATORY P452 AVIATION APPLICATIONS OF A NEW GENERATION OF MESOSCALE NUMERICAL WEATHER PREDICTION SYSTEM OF THE HONG KONG OBSERVATORY Wai-Kin WONG *1, P.W. Chan 1 and Ivan C.K. Ng 2 1 Hong Kong Observatory, Hong

More information

Finnish Open Data Portal for Meteorological Data

Finnish Open Data Portal for Meteorological Data 18.11.2013 1 Finnish Open Data Portal for Meteorological Data 14th Workshop on meteorological operational systems Roope Tervo Finnish Meteorological Institute Example of Data Sets -- Observations Data

More information

Application and verification of ECMWF products 2015

Application and verification of ECMWF products 2015 Application and verification of ECMWF products 2015 Instituto Português do Mar e da Atmosfera, I.P. 1. Summary of major highlights At Instituto Português do Mar e da Atmosfera (IPMA) ECMWF products are

More information

The Use of Analog Ensembles to Improve Short-Term Solar Irradiance Forecasting

The Use of Analog Ensembles to Improve Short-Term Solar Irradiance Forecasting ALBANY BARCELONA BANGALORE AMS Annual Meeting Atlanta, GA February 6, 214 The Use of Analog Ensembles to Improve Short-Term Solar Irradiance Forecasting Steve Young and John W. Zack AWS Truepower, LLC

More information

The Impacts of GPS Radio Occultation Data on the Analysis and Prediction of Tropical Cyclones. Bill Kuo, Xingqin Fang, and Hui Liu UCAR COSMIC

The Impacts of GPS Radio Occultation Data on the Analysis and Prediction of Tropical Cyclones. Bill Kuo, Xingqin Fang, and Hui Liu UCAR COSMIC The Impacts of GPS Radio Occultation Data on the Analysis and Prediction of Tropical Cyclones Bill Kuo, Xingqin Fang, and Hui Liu UCAR COSMIC GPS Radio Occultation α GPS RO observations advantages for

More information

Application and verification of ECMWF products 2018

Application and verification of ECMWF products 2018 Application and verification of ECMWF products 2018 National Meteorological Administration, Romania 1. Summary of major highlights In the field of numerical model verification, the daily GRID_STAT method

More information

The Impact of Observational data on Numerical Weather Prediction. Hirokatsu Onoda Numerical Prediction Division, JMA

The Impact of Observational data on Numerical Weather Prediction. Hirokatsu Onoda Numerical Prediction Division, JMA The Impact of Observational data on Numerical Weather Prediction Hirokatsu Onoda Numerical Prediction Division, JMA Outline Data Analysis system of JMA in Global Spectral Model (GSM) and Meso-Scale Model

More information

Can buoys predict hurricanes? Objectives Students will be able to: track drifter buoys determine the course of the gulf stream current

Can buoys predict hurricanes? Objectives Students will be able to: track drifter buoys determine the course of the gulf stream current Drifters Can buoys predict hurricanes? Objectives Students will be able to: track drifter buoys determine the course of the gulf stream current Materials computers with Internet access Student Worksheet

More information

MeteoGroup RoadMaster. The world s leading winter road weather solution

MeteoGroup RoadMaster. The world s leading winter road weather solution MeteoGroup RoadMaster The world s leading winter road weather solution Discover why RoadMaster is the world s leading winter road weather solution. Managing winter road maintenance means that you carry

More information

Nowcasting thunderstorms for aeronautical end-users

Nowcasting thunderstorms for aeronautical end-users Nowcasting thunderstorms for aeronautical end-users Jean-Marc Moisselin Météo-France, Nowcasting Department co-authors: Céline Jauffret (Météo-France) Overview Introduction SAT RADAR NWP image crédit:

More information

Categorical Verification

Categorical Verification Forecast M H F Observation Categorical Verification Tina Kalb Contributions from Tara Jensen, Matt Pocernich, Eric Gilleland, Tressa Fowler, Barbara Brown and others Finley Tornado Data (1884) Forecast

More information

Application and verification of ECMWF products 2012

Application and verification of ECMWF products 2012 Application and verification of ECMWF products 2012 National Meteorological Administration 1. Summary of major highlights The objective verification of all deterministic models forecasts in use have been

More information

Training: Climate Change Scenarios for PEI. Training Session April Neil Comer Research Climatologist

Training: Climate Change Scenarios for PEI. Training Session April Neil Comer Research Climatologist Training: Climate Change Scenarios for PEI Training Session April 16 2012 Neil Comer Research Climatologist Considerations: Which Models? Which Scenarios?? How do I get information for my location? Uncertainty

More information

CGE TRAINING MATERIALS ON VULNERABILITY AND ADAPTATION ASSESSMENT. Climate change scenarios

CGE TRAINING MATERIALS ON VULNERABILITY AND ADAPTATION ASSESSMENT. Climate change scenarios CGE TRAINING MATERIALS ON VULNERABILITY AND ADAPTATION ASSESSMENT Climate change scenarios Outline Climate change overview Observed climate data Why we use scenarios? Approach to scenario development Climate

More information

NextGen Update. Cecilia Miner May, 2017

NextGen Update. Cecilia Miner May, 2017 NextGen Update Cecilia Miner May, 2017 Agenda What s changed? NextGen Background FAA NextGen Weather Architecture NextGen Weather Processor Aviation Weather Display Common Support Services - Weather NWS

More information

David John Gagne II, NCAR

David John Gagne II, NCAR The Performance Impacts of Machine Learning Design Choices for Gridded Solar Irradiance Forecasting Features work from Evaluating Statistical Learning Configurations for Gridded Solar Irradiance Forecasting,

More information

HEPS. #HEPEX Quebec 2016 UPGRADED METEOROLOGICAL FORCING FOR OPERATIONAL HYDROLOGICAL ENSEMBLE PREDICTIONS: CHALLENGES, RISKS AND CHANCES

HEPS. #HEPEX Quebec 2016 UPGRADED METEOROLOGICAL FORCING FOR OPERATIONAL HYDROLOGICAL ENSEMBLE PREDICTIONS: CHALLENGES, RISKS AND CHANCES Zappa M, Andres N, Bogner K, Liechti K #HEPEX Quebec 2016 UPGRADED METEOROLOGICAL FORCING FOR OPERATIONAL HYDROLOGICAL ENSEMBLE PREDICTIONS: CHALLENGES, RISKS AND CHANCES HEPS Contact: massimiliano.zappa@wsl.ch

More information

Application and verification of ECMWF products 2015

Application and verification of ECMWF products 2015 Application and verification of ECMWF products 2015 Hungarian Meteorological Service 1. Summary of major highlights The objective verification of ECMWF forecasts have been continued on all the time ranges

More information

South African Weather Service SWIOCOF-5 Pre-Forum. Seasonal Forecast, Water Resources and Expected Outcomes

South African Weather Service SWIOCOF-5 Pre-Forum. Seasonal Forecast, Water Resources and Expected Outcomes South African Weather Service SWIOCOF-5 Pre-Forum Seasonal Forecast, Water Resources and Expected Outcomes Marc de Vos, September 2016 South Africa: Context BCRE, 2016, adapted from Boebel et al., 2003

More information

GL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays

GL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays GL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays Require accurate wind (and hence power) forecasts for 4, 24 and 48 hours in the future for trading purposes. Receive 4

More information

Forecasting AOSC 200 Tim Canty. Class Web Site: Lecture 26 Nov 29, Weather Forecasting

Forecasting AOSC 200 Tim Canty. Class Web Site:   Lecture 26 Nov 29, Weather Forecasting Forecasting AOSC 200 Tim Canty Class Web Site: http://www.atmos.umd.edu/~tcanty/aosc200 Topics for today: Forecasting Lecture 26 Nov 29, 2018 1 Weather Forecasting People have been trying to predict the

More information

METinfo Verification of Operational Weather Prediction Models December 2017 to February 2018 Mariken Homleid and Frank Thomas Tveter

METinfo Verification of Operational Weather Prediction Models December 2017 to February 2018 Mariken Homleid and Frank Thomas Tveter METinfo No. /8 ISSN 89-79X Meteorology Verification of Operational Weather Prediction Models December 7 to February 8 Mariken Homleid and Frank Thomas Tveter Photo: Jan Erik Haugen Contents Introduction

More information

Weather Forecasting. March 26, 2009

Weather Forecasting. March 26, 2009 Weather Forecasting Chapter 13 March 26, 2009 Forecasting The process of inferring weather from a blend of data, understanding, climatology, and solutions of the governing equations Requires an analysis

More information

Application and verification of ECMWF products 2010

Application and verification of ECMWF products 2010 Application and verification of ECMWF products Hydrological and meteorological service of Croatia (DHMZ) Lovro Kalin. Summary of major highlights At DHMZ, ECMWF products are regarded as the major source

More information

Importance of Numerical Weather Prediction in Variable Renewable Energy Forecast

Importance of Numerical Weather Prediction in Variable Renewable Energy Forecast Importance of Numerical Weather Prediction in Variable Renewable Energy Forecast Dr. Abhijit Basu (Integrated Research & Action for Development) Arideep Halder (Thinkthrough Consulting Pvt. Ltd.) September

More information

METinfo Verification of Operational Weather Prediction Models June to August 2017 Mariken Homleid and Frank Thomas Tveter

METinfo Verification of Operational Weather Prediction Models June to August 2017 Mariken Homleid and Frank Thomas Tveter METinfo No. /7 ISSN 89-79X Meteorology Verification of Operational Weather Prediction Models June to August 7 Mariken Homleid and Frank Thomas Tveter Contents Introduction Models..............................................

More information

Improving the accuracy of solar irradiance forecasts based on Numerical Weather Prediction

Improving the accuracy of solar irradiance forecasts based on Numerical Weather Prediction Improving the accuracy of solar irradiance forecasts based on Numerical Weather Prediction Bibek Joshi, Alistair Bruce Sproul, Jessie Kai Copper, Merlinde Kay Why solar power forecasting? Electricity grid

More information

Snowfall Detection and Rate Retrieval from ATMS

Snowfall Detection and Rate Retrieval from ATMS Snowfall Detection and Rate Retrieval from ATMS Jun Dong 1, Huan Meng 2, Cezar Kongoli 1, Ralph Ferraro 2, Banghua Yan 2, Nai-Yu Wang 1, Bradley Zavodsky 3 1 University of Maryland/ESSIC/Cooperative Institute

More information

Basic Verification Concepts

Basic Verification Concepts Basic Verification Concepts Barbara Brown National Center for Atmospheric Research Boulder Colorado USA bgb@ucar.edu May 2017 Berlin, Germany Basic concepts - outline What is verification? Why verify?

More information

Experiences of using ECV datasets in ECMWF reanalyses including CCI applications. David Tan and colleagues ECMWF, Reading, UK

Experiences of using ECV datasets in ECMWF reanalyses including CCI applications. David Tan and colleagues ECMWF, Reading, UK Experiences of using ECV datasets in ECMWF reanalyses including CCI applications David Tan and colleagues ECMWF, Reading, UK Slide 1 Main points Experience shows benefit of integrated & iterative approach

More information

The current status, functions, challenges and needs of South Sudan Meteorological Department (SSMD)

The current status, functions, challenges and needs of South Sudan Meteorological Department (SSMD) The current status, functions, challenges and needs of South Sudan Meteorological Department (SSMD) Presented: by Mojwok Ogawi Modo, South Sudan Meteorological Department. Background The present SSMD was

More information

Better Weather Data Equals Better Results: The Proof is in EE and DR!

Better Weather Data Equals Better Results: The Proof is in EE and DR! Better Weather Data Equals Better Results: The Proof is in EE and DR! www.weatherbughome.com Today s Speakers: Amena Ali SVP and General Manager WeatherBug Home Michael Siemann, PhD Senior Research Scientist

More information

Wind Assessment & Forecasting

Wind Assessment & Forecasting Wind Assessment & Forecasting GCEP Energy Workshop Stanford University April 26, 2004 Mark Ahlstrom CEO, WindLogics Inc. mark@windlogics.com WindLogics Background Founders from supercomputing industry

More information

AN ENSEMBLE STRATEGY FOR ROAD WEATHER APPLICATIONS

AN ENSEMBLE STRATEGY FOR ROAD WEATHER APPLICATIONS 11.8 AN ENSEMBLE STRATEGY FOR ROAD WEATHER APPLICATIONS Paul Schultz 1 NOAA Research - Forecast Systems Laboratory Boulder, Colorado 1. INTRODUCTION In 1999 the Federal Highways Administration (FHWA) initiated

More information

Operations Portugal (second half of 2010)

Operations Portugal (second half of 2010) Operations Portugal (second half of 2010) In the second half of 2010, important changes happened in the operational procedures, the main event being the operationalization of AROME for Madeira archipelago

More information

ATMOSPHERIC MODELLING. GEOG/ENST 3331 Lecture 9 Ahrens: Chapter 13; A&B: Chapters 12 and 13

ATMOSPHERIC MODELLING. GEOG/ENST 3331 Lecture 9 Ahrens: Chapter 13; A&B: Chapters 12 and 13 ATMOSPHERIC MODELLING GEOG/ENST 3331 Lecture 9 Ahrens: Chapter 13; A&B: Chapters 12 and 13 Agenda for February 3 Assignment 3: Due on Friday Lecture Outline Numerical modelling Long-range forecasts Oscillations

More information

PREDICTION OF OIL SPILL TRAJECTORY WITH THE MMD-JMA OIL SPILL MODEL

PREDICTION OF OIL SPILL TRAJECTORY WITH THE MMD-JMA OIL SPILL MODEL PREDICTION OF OIL SPILL TRAJECTORY WITH THE MMD-JMA OIL SPILL MODEL Project Background Information MUHAMMAD HELMI ABDULLAH MALAYSIAN METEOROLOGICAL DEPARTMENT(MMD) MINISTRY OF SCIENCE, TECHNOLOGY AND INNOVATION

More information

Supporting Information

Supporting Information Supporting Information Convolutional Embedding of Attributed Molecular Graphs for Physical Property Prediction Connor W. Coley a, Regina Barzilay b, William H. Green a, Tommi S. Jaakkola b, Klavs F. Jensen

More information

Be relevant and effective thinking beyond accuracy and timeliness

Be relevant and effective thinking beyond accuracy and timeliness The Fourth Technical Conference on the Management of Meteorological and Hydrological Services in WMO RA II (Asia) Be relevant and effective thinking beyond accuracy and timeliness C. Y. LAM Hong Kong Observatory

More information