Recent development at JMA (short-range and medium-range NWP)

Similar documents
Recent Developments of JMA Operational NWP Systems and WGNE Intercomparison of Tropical Cyclone Track Forecast

The Improvement of JMA Operational Wave Models

Upgrade of JMA s Typhoon Ensemble Prediction System

Global and Regional OSEs at JMA

JMA s Ensemble Prediction System for One-month and Seasonal Predictions

Masahiro Kazumori, Takashi Kadowaki Numerical Prediction Division Japan Meteorological Agency

AN OBSERVING SYSTEM EXPERIMENT OF MTSAT RAPID SCAN AMV USING JMA MESO-SCALE OPERATIONAL NWP SYSTEM

The next-generation supercomputer and NWP system of the JMA

11 days (00, 12 UTC) 132 hours (06, 18 UTC) One unperturbed control forecast and 26 perturbed ensemble members. --

Assimilation of Himawari-8 Atmospheric Motion Vectors into the Numerical Weather Prediction Systems of Japan Meteorological Agency

4.3.2 Configuration. 4.3 Ensemble Prediction System Introduction

Recent Developments of JMA Operational NWP Systems and WGNE Intercomparison of Tropical Cyclone Track Forecast

Operational Use of Scatterometer Winds at JMA

IMPACT STUDIES OF AMVS AND SCATTEROMETER WINDS IN JMA GLOBAL OPERATIONAL NWP SYSTEM

Value of Satellite Observation Sensitive to Humidity and Precipitation in JMA s Operational Numerical Weather Prediction

The Impact of Observational data on Numerical Weather Prediction. Hirokatsu Onoda Numerical Prediction Division, JMA

Observing System Experiments using a singular vector method for 2004 DOTSTAR cases

Activities of Numerical Weather Prediction for Typhoon forecast at Japan Meteorological Agency

Ninth Workshop on Meteorological Operational Systems. Timeliness and Impact of Observations in the CMC Global NWP system

Status and Plans of using the scatterometer winds in JMA's Data Assimilation and Forecast System

Recent Data Assimilation Activities at Environment Canada

2. Outline of the MRI-EPS

Recent Developments in Climate Information Services at JMA. Koichi Kurihara Climate Prediction Division, Japan Meteorological Agency

Operational Use of Scatterometer Winds in the JMA Data Assimilation System

Observing system experiments of MTSAT-2 Rapid Scan Atmospheric Motion Vector for T-PARC 2008 using the JMA operational NWP system

Development of JMA storm surge model

Assimilation of Himawari-8 data into JMA s NWP systems

Verification of the Seasonal Forecast for the 2005/06 Winter

Upgraded usage of MODIS-derived polar winds in the JMA operational global 4D-Var assimilation system

STATUS OF JAPANESE METEOROLOGICAL SATELLITES AND RECENT ACTIVITIES OF MSC

Direct assimilation of all-sky microwave radiances at ECMWF

非静力学 4 次元変分法. Kosuke Ito

Outline. Updates of JMA/MRI global aerosol prediction Updates of the geostationary satellite Himawari 8 Data assimilation

THE USE OF SATELLITE OBSERVATIONS IN NWP

TC/PR/RB Lecture 3 - Simulation of Random Model Errors

TCC Training Seminar on 17 th Nov 2015 JMA s Ensemble Prediction Systems (EPSs) and their Products for Climate Forecast.

Data Short description Parameters to be used for analysis SYNOP. Surface observations by ships, oil rigs and moored buoys

Current JMA ensemble-based tools for tropical cyclone forecasters

Study for utilizing high wind speed data in the JMA s Global NWP system

Background of Symposium/Workshop Yuhei Takaya Climate Prediction Division Japan Meteorological Agency

Scatterometer Utilization in JMA s global numerical weather prediction (NWP) system

Use of satellite radiances in the global assimilation system at JMA

Products of the JMA Ensemble Prediction System for One-month Forecast

Winter Forecast for GPC Tokyo. Shotaro TANAKA Tokyo Climate Center (TCC) Japan Meteorological Agency (JMA)

JMA s Seasonal Prediction of South Asian Climate for Summer 2018

The ECMWF Extended range forecasts

HIGH SPATIAL AND TEMPORAL RESOLUTION ATMOSPHERIC MOTION VECTORS GENERATION, ERROR CHARACTERIZATION AND ASSIMILATION

Current Status of COMS AMV in NMSC/KMA

A review on recent progresses of THORPEX activities in JMA

Assimilation of GNSS Radio Occultation Data at JMA. Hiromi Owada, Yoichi Hirahara and Masami Moriya Japan Meteorological Agency

Convective-scale NWP for Singapore

Data Impact Studies in the CMC Global NWP system

Assimilation of GPM/DPR at JMA

Quarterly numerical weather prediction model performance summaries April to June 2010 and July to September 2010

Satellite Soil Moisture Content Data Assimilation in Operational Local NWP System at JMA

Impact Evaluation of New Radiance data, Reduced Thinning and Higher Analysis Resolution in the GEM Global Deterministic Prediction System

Current Limited Area Applications

TIGGE at ECMWF. David Richardson, Head, Meteorological Operations Section Slide 1. Slide 1

Reduction of the Radius of Probability Circle. in Typhoon Track Forecast

2016 and 2017 Reviews of Probability-circle Radii in Tropical Cyclone Track Forecasts

Development of 3D Variational Assimilation System for ATOVS Data in China

The Nowcasting Demonstration Project for London 2012

Use of ATOVS raw radiances in the operational assimilation system at Météo-France

Big Ensemble Data Assimilation

The JRA-55 Reanalysis: quality control and reprocessing of observational data

Verification at JMA on Ensemble Prediction

AVIATION APPLICATIONS OF A NEW GENERATION OF MESOSCALE NUMERICAL WEATHER PREDICTION SYSTEM OF THE HONG KONG OBSERVATORY

WRF Model Simulated Proxy Datasets Used for GOES-R Research Activities

OSSE to infer the impact of Arctic AMVs extracted from highly elliptical orbit imagery

Work at JMA on Ensemble Forecast Methods

Evaluation and assimilation of all-sky infrared radiances of Himawari-8

Program of the morning lectures

Toward improved initial conditions for NCAR s real-time convection-allowing ensemble. Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell

Recent experience at Météo-France on the assimilation of observations at high temporal frequency Cliquez pour modifier le style du titre

Satellite Assimilation Activities for the NRL Atmospheric Variational Data Assimilation (NAVDAS) and NAVDAS- AR (Accelerated Representer) Systems

Center Report from KMA

Model error and seasonal forecasting

Some activities relative to China THORPEX TIGGE WG

An Overview of Atmospheric Analyses and Reanalyses for Climate

What s new for ARPEGE/ALADIN since Utrecht 2009?

Shuhei Maeda Climate Prediction Division Global Environment and Marine Department Japan Meteorological Agency

Assimilation of IASI data at the Met Office. Fiona Hilton Nigel Atkinson ITSC-XVI, Angra dos Reis, Brazil 07/05/08

The benefits and developments in ensemble wind forecasting

The Impact of Horizontal Resolution and Ensemble Size on Probabilistic Forecasts of Precipitation by the ECMWF EPS

NOTES AND CORRESPONDENCE. On Ensemble Prediction Using Singular Vectors Started from Forecasts

Assimilation of precipitation-related observations into global NWP models

JMA Contribution to SWFDDP in RAV. (Submitted by Yuki Honda and Masayuki Kyouda, Japan Meteorological Agency) Summary and purpose of document

Microwave-TC intensity estimation. Ryo Oyama Meteorological Research Institute Japan Meteorological Agency

Sub-grid parametrization in the ECMWF model

Recent progress in convective scale Arome NWP system and on-going research activities

Motivation & Goal. We investigate a way to generate PDFs from a single deterministic run

ENSO Outlook by JMA. Hiroyuki Sugimoto. El Niño Monitoring and Prediction Group Climate Prediction Division Japan Meteorological Agency

Review of GFS Forecast Skills in 2012

Methods of forecast verification

JMA s ATMOSPHERIC MOTION VECTORS In response to Action 40.22

Rosemary Munro*, Graeme Kelly, Michael Rohn* and Roger Saunders

ALASKA REGION CLIMATE OUTLOOK BRIEFING. November 16, 2018 Rick Thoman Alaska Center for Climate Assessment and Policy

ALASKA REGION CLIMATE OUTLOOK BRIEFING. December 22, 2017 Rick Thoman National Weather Service Alaska Region

Atmospheric circulation analysis for seasonal forecasting

Fog detection product

Transcription:

Recent development at JMA (short-range and medium-range NWP) Chiashi Muroi Numerical Prediction Division Japan Meteorological Agency Major changes in operational run Deterministic (short-range and medium-range) TL959L60 TL319L40 (Nov 2007) Reduced gaussian grid (Aug 2008) Ensemble (medium-range) TL319L50M51(SV) TL159L40M51(BGM) Typhoon Ensemble Prediction System (Feb 2008) TL319L50M11 Data assimilation RTTOV-8 (OCT 2008)

20,000km 2,000km 200km 20km 2km 200m New-suite (Deterministic) Spatial Scale Macro Meso Mesoscale Model tornado Micro Thunder storm GSM planetary wave RSM Baroclinic extratropical wave low front system tropical cloud cluster cyclone TYM 0.1 hour 1 hour 10 hour 1 day 100 hour1 week New GSM Temporal Scale New-suite (Ensemble) Spatial Scale 20,000km 2,000km 200km 20km 2km 200m Macro Meso cumulonimbus cumulonimbus tornado Micro New One-Week EPS 51 members Once a day extratropical low front system Thunder storm cloud cluster planetary wave Baroclinic wave Typhoon EPS 11 members 4 times a day Temporal Scale 0.1 hour 1 hour 10 hour 1 day 100 hour1 week

JMA Numerical Analysis and Prediction System Global Spectral Model MesoScale Model T L 959L60(~20km) 5km Non-hydrostatic model Typhoon Ensemble One-Week Ensemble T L 319L60(~60km)11members T L 319L60(~60km)51members Current NWP models of NPD/JMA Global Spectral Model (GSM) MesoScale Model (MSM) One-week Ensemble Typhoon Ensemble Objectives Forecast domain Horizontal resolution Vertical levels / Top Forecast Hours (Initial time) Initial Condition Short- and Mediumrange forecast Global Disaster reduction, Short-range forecast Japan and its surroundings (3600km x 2880km) One-week forecast Global Typhoon forecast T L 959(0.1875 deg) 5km T L 319(0.5625 deg) 60 0.1 hpa 84 hours (00, 06, 18 UTC) 216 hours (12 UTC) Global Analysis (4D-Var) 50 21800 m 15 hours (00, 06, 12, 18 UTC) 33 hours (03, 09, 15, 21 UTC) Mesoscale Analysis (4D-Var) 216 hours (12 UTC) 51 members 60 0.1 hpa 132 hours (00, 06, 12, 18 UTC) 11 members Global Analysis with ensemble perturbations Perturbations are produced by SV-method

Data assimilation systems of NPD/JMA Global Analysis Mesoscale Analysis Analysis scheme Four-dimensional variational method Analysis time 00, 06, 12, 18 UTC 00, 03, 06, 09, 12, 15, 18, 21 UTC Data cut-off time Horizontal resolution (inner-model resolution) 2 hours 20 minutes [Early Analysis] 11 hours 35 minutes (00, 12 UTC) 5 hours 35 minuts (06, 18 UTC) [Cycle Analysis] T L 959 / 0.1875 deg (T159 / 0.75 deg) 50 minutes 10 km (20 km) Vertical levels 60 levels up to 0.1 hpa 40 levels up to 10 hpa Assimilation window Remarks -3 hours to +3 hours of analysis time Based on a slightly old version of GSM -6 hours to analysis time Based on the previous MesoScale Model (hydrostatic, spectral) RMSE of 500 hpa Geopotential height in Northern Hemisphere Revision (5-day of cumulus forecast) Revision of cumulus T213L30 parameterization T213L40 Use of QuikSCAT 3D-Var parameterization Direct use of ATOVS Use of MODIS TL319L40 4D-Var Revision of cloud Revision of radiation Use of SSM/I and TMI Variational bias correction TL959L60

Major changes in NWP (1/2) 21 NOV 2007 Major upgrade was made to the GSM increase in the resolution from TL319L40 to TL959L60 with a topmost level raised from 0.4hPa to 0.1hPa use of a new high-resolution analysis of sea surface temperature and sea ice concentration as ocean surface boundary conditions use of surface snow depth data from the domestic dense observational network in the global snow depth analysis introduction of a convective triggering scheme into the deep convection parameterization introduction of a new 2-dimensional aerosol climatology derived from satellite observations for the radiation calculation increase in the resolution of inner loop model of the four-dimensional variational (4D-Var) data assimilation system from T106L40 to T159L60 change of forecast time to 84 hours for operations at 00, 06 and 18UTC. 21 NOV 2007 Upgrade of medium-range ensemble model 21 NOV 2007 disuse of RSM and TYM 7 DEC 2007 MA MTSAT-1R AMV in mesoscale analysis 10 JAN 2008 Calculation procedure of the convective triggering scheme was revised. Excessive limitation on cumulus mass flux from redundant vertical CFL condition was corrected. Physical constant was changed (Unified constant in forecast model and 4DVAR) 28 FEB 2008 Typhoon Ensemble Prediction System started Major changes in NWP (2/2) 24 JUN 2008 modified Hourly Analysys 5 AUG 2008 Reduced Gaussian Grids was implemented in GSM and number of grid points in calculation of nonlinear terms was reduced. 5 AUG 2008 New Doppler radar data was assimilated in mesoscale and hourly analysis 27 AUG 2008 Upgrade of global analysis Direct assimilation of clear-sky radiances of water vapour channels from geostationary satellites (MTSAT-1R/IMAGER, GOES11/IMAGER, GOES12/IMAGER, METEOSAT7/MVIRI, METEOSAT9/SEVIRI) started. Background errors of variational bias correcttion for radiance data were revised. 15 OCT 2008 Restructure of preprocess of direct assimilation of radiance Update of RTTOV (from RTTOV-7 to RTTOV-8)

The New High Resolution System Previous System Deterministic 9d-forecast: T L 319 L40 (Δt=15min) 4D-Var Analysis: T106 L40 (Eulerian) One-week EPS T L 159 L40 M51 (Δt=20min) New Operation Deterministic 9d-forecast: T L 959 L60 (Δt=10min) 4D-Var Analysis: T159 L60 (Eulerian) One-week EPS T L 319 L60 M51 (Δt=20min) Typhoon EPS T L 319 L60 M11 (Δt=20min) L40 0.1hPa L60 0.4hPa Position of levels and pressure layer thickness of L40(red) and L60 (blue)

Topography around TOKYO Old GSM(60km-mesh) New GSM(20km-mesh) High resolution new-suite (GSM0711) (cont.) Use of surface snow depth data from domestic dense observation network in the global snow depth analysis. Introduction of new two-dimensional aerosol climatology derived from satellite observations for the radiation calculation. Introduction of convective triggering scheme to the deep convection parameterization. Retuning of the stratocumulus cloud scheme for higher vertical resolution.

Precipitation forecast 高解像度化にともない極端現象の予報精度が向上 Old GSM Obs New GSM 12UTC 17 July 2005 initial, FT=24 Verification of the High Resolution System Anomaly correlation of mean-sea-level pressure for northern hemisphere 1 Mean- Sea- Level Pressure AC 20N- 90N Anomaly corr. 0.9 0.8 0.7 0.6 0.5 0.4 T319 L40 T959 L60 0.3 0 1 2 3 4 5 6 7 8 9 forecast day Mean over 31 12UTC cases from 1 October 2007

Fame and Gula: 28 Jan-5 Feb 2008 History of JMA unified GSM CTM Climate Medium-range Medium-range Ensemble Deterministic One-month Ensemble Three-month Ensemble Version Resolution ENSO MRI-GCM-II GSPM GSM8803 4 x5 L15 R24L23 T63L16 MJ98 T42L30 CGCM2.0 T42L30 Dust CGCM2.2 T106L30 T42L30 CGCM2.3 OZONE T42L30 T42L68 Earth-System CGCM3.0 TL95L46 CGCM3.1 TL159L48 GSMUV GSM8911 T106L21 GSM9603 T213L30 GSM9912 T213L30 GSM0103 T213L40 GSM0305 T213L40 GSM0407 T213L40 GSM0502 TL319L40 GSM0603 TL319L40 GSM0703 TL319L40 GSM0711 TL959L60 GSM0808 TL959L60 1996 空海 T42L21 GSM9603 T63L30 1998 GSM0103 T106L40 GSM0305 T106L40 GSM0407 T106L40 2002 GSM0103 T63L40 空海2003 T42L21 2004 2006 GSM0603 TL159L40 GSM0703 TL159L40 GSM0711 TL319L60 GSM0603C TL159L40 GSM0703C TL159L40 V0803 TL159L60 GSM0502C TL95L40 GSM0703C TL95L40 2008 JMA/MRI-CGCM TL95L40

Utilization of operational computer system(2005~2011 CPU Ensemble Spread and Error of Ensemble Mean Root Mean Square Error of ensemble mean and ensemble spread of the 500 hpa geopotential height over the Northern Hemisphere (20N-90N) in December 2005 (winter season). TL159L40 BGM (Previous system) The ensemble spread of previous system tended to be large. Such trend is noticeable especially in the early stage of the forecast period. TL319L60 SV (New system) In the new system, the growth of the ensemble spread is similar to the error growth.

Performance of Ensemble Mean Forecast Root Mean Square Error of ensemble mean forecast for the 500 hpa geopotential height over the Northern Hemisphere (20N-90N) in August, 2004 and December, 2005. 2004/08 (Northern Hemisphere Z500) 2005/12 (Northern Hemisphere Z500) X TL319L60 SV (new system) TL159L40 BGM (previous system) The Z500 anomaly correlation of Ensemble Mean Forecast for the new system is similar to that of the previous system. Performance of Probabilistic Forecasting (A) Previous system (TL159) (B) New system (TL319) Brier Skill Score Forecast Time (hour) Forecast Time (hour) TL319L60 SV (new system) TL159L40 BGM (previous system) Brier Skill Score for the probabilistic forecast of positive and negative anomaly in 850hPa temperature with magnitude less than and larger than 1.5 climatological standard deviation over East Asia (20N-60N 100E-170E ) in August 2004 (summer season). (C) Analysis Probability of precipitation (12mm/1day) at 4 days forecast from 12UTC 13 December 2005. (A) previous model (Tl159L40), (B) new model (Tl319L60) (C) Radar-Raingauge analyzed precipitation (24 hour accumulated precipitation).

Performance in Pre-operational Parallel Run Anomaly correlation of 500 hpa geopotential height (Z500) for the ensemble mean forecast over the Northern Hemisphere (20N-90N )in pre-operational parallel run (9-20 November, 2007). TL319L60 SV TL159L40 BGM Root Mean Square Error of the ensemble mean and ensemble spread for the 500 hpa geopotential height over the Northern Hemisphere (20N-90N) in pre-operational parallel run. x RMSE ( previous system ) Spread ( previous system ) x RMSE ( new system ) Spread ( new system ) The new EPS has a 0.5 day advantage that the Z500 anomaly correlation for ensemble mean forecast fall below 0.6. The new system improved the relationship between RMSE of the ensemble mean and the ensemble spread. JMA began operation of the Typhoon EPS The Japan Meteorological Agency (JMA) has developed a new ensemble prediction system (EPS) known as the Typhoon EPS, aiming to further improve both deterministic and probabilistic forecasting of TC movements. We started operation of the Typhoon EPS from the beginning of the typhoon season in 2008 following preliminary operation since May 2007. The 20 km GSM, which became operational from 21 st Nov, supports both TC track and intensity forecasting. JMA started the Typhoon EPS from the beginning of the typhoon season in 2008. Uncertainty information associated with TC track forecasts will be able to be provided using the Typhoon EPS. T-PARC could help us to further address TC predictability and improve NWP systems.

Singular Vector (SV) calculations in the TEPS SV calculations are conducted targeting on both TCs (up to three TCs are targeted in one forecast event) and mid-latitude. Moist singular vector (Barkmeijer et al. 2001) calculations are performed for TCs, on the other hand, singular vectors for midlatitude are dry singular vectors. The norm to evaluate the growth rate of singular vectors is based on the moist total energy norm (Ehrendorfer et al. 1999). The evaluation interval time is 24 hours for both moist and dry singular vector calculations. Example of target areas: 2007.08.07 1800 UTC initial Green:mid-latitude target area (fixed to 20N-60N, 100E-180E) Blue:TC target area (20 degrees in longitude and 10 degrees in latitude with its center at a 24-hour forecast TC position) TC 1 TC 2 Based on the strong spread-skill relationship, we categorized the reliability of track forecasts at each forecast time of each forecast event and assigned a reliability index, A, B, or C, to the forecast, where A, B, and C represented categories of the highest, the middlelevel, and the lowest reliability, respectively, and the frequency of each category was set to 40%, 40%, and 20%. Initial time: 2007.07.29 00UTC Reliability: B Reliability: A Reliability: C FT (hours) Spread Reliability 0 3 A 6 22 A 12 64 A 18 126 A 24 216 A 30 333 A 36 479 B 42 656 B 48 870 B 54 1128 B 60 1436 B 66 1800 B 72 2219 B 78 2698 B 84 3231 B 90 3807 B 96 4430 B 102 5090 C 108 5795 C 114 6572 C 120 7333 C

Reduced gaussian grid (AUG 2008) Implementation of reduced gaussian grid Computational cost has been reduced about 20%. Framework of spectral model has been completely revised. Grid space is basis of spectral model. 2-dim decomposition Openmp is used Physical processes are parallelized totally. Standard grid Reduced grid ATOVS 輝度温度直接同化開始 1D L1C データの利用 History of preprocess of direct assimilation of radiance Year GAnal RTM Ver. RAnal, MAna 1982 Retrieve (NESDIS,MSC) 1999 2001 TOVS 1D-Var 2003 Atvsvar 2005 2006 2007 2008 QCRT QCRTMW QCRTMT QCRTX Retrieve (NESDIS, MSC) RTTOV-5 RTTOV-6 RTTOV-7 RTTOV-8 QCRTX

Comparison of O-B between RTTOV-8 and RTTOV-7 Red: RTTOV-8 (FASTEM-3) Green: RTTOV-7 (FASTEM-2) RTTOV-8 and RTTOV-7 are identic except ocean emissivity model in the MW radianc calculation. O-B means Observed minus calculated brightness temperature. The JMA 6-hour global forecasts a used as Background. FASTEM-3 shows better performance (standard deviation) in the calculation for AMSR-E. Comparison of O-B between RTTOV-8 and RTTOV-7 for QC passed without bias correction AMSR-E data ( Statistics: 01 August, 2007 1day data set) Development in the current system Introduction of a non-hydrostatic model-based 4D-Var for MSM (JNoVA) Update of Global Analysis inner model Unified package modules for both forecast and analisys(4d- VAR) model Revision of physical parameterizations (GSM, MSM) Coupling of GSM with an ocean mixed-layer model Introduction of new observation data (especially, satellite data)

a non-hydrostatic model-based 4D-Var for MSM - JNoVA Comparison of operational Meso 4D-Var & JNoVA Meso 4D-Var JNoVA Resolution Inner / Outer Num of vertical layer Model Top 20km / 10km 40layers / 10hPa 15km / 5km 40layers / 22km (40hPa) Vertical coordinate ηcoordinate z*coordinate Domain 3600x2880km 3600x2880km Assimilation window 6hours 3hours Iteration 40~60 times 25~30times Cutoff time 50mins 50mins Computational Time 17mins(40node) 22mins(60node) A tentative NWP model upgrade plan for the next computer system (2012-) Enhancement of vertical resolution (GSM, MSM and One-week and Typhoon ensemble) Enhancement of horizontal resolution (Oneweek and Typhoon ensemble) Increase of ensemble members (Typhoon ensemble) Local Forecast Model (2km, hourly, 9 hour forecast, 3D-Var data assimilation) a mesoscale ensemble forecast system (10km, 5 members: pre-operational test)

Development of 2km-mesh Local Forecast Model (LFM) Objectives : Dynamical QPF in very short-range forecast Forecast period : 9 hr (hourly) Analysis: 3DVAR Horizontal resolution : 2km Number of grid point - 550x650 (north), 800x550 (center), 650x550 (south), Vertical resolution : Variable resolution (40-678 m, 50 layers) 3600 km Tokyo 2900 km Fig1 Computational domain of MSM Observation LFM(2km) MSM(5km) GSM(20km) Fig4. Quantitative Precipitation Forecast 2008 Aug. 9 18JST 1hourly precipitation. Initial time is 15JST Aug. 9. Miscellious We, JMA, started NWP in 1959. Happy 50 th birthday next year!! Main office of JMA will move (inside TOKYO) in 2013.