Application of the Fractions Skill Score for Tracking the Effectiveness of Improvements Made to Weather Research and Forecasting Model Simulations

Report No. ARL-TR-8217
Authors: John W Raby; Huaqing Cai
Date/Pages: November 2017; 44 pages
Abstract: Spatial forecasts from Numerical Weather Prediction (NWP) models of meteorological variables supporting US Army battlefield operations are an integral part of the products available for the Staff Weather Officer's use in providing mission forecasts. This report presents some preliminary results obtained from the application of a nontraditional fuzzy verification method to evaluate the ability of NWP to simulate spatial variable fields filtered using thresholds. Fuzzy methods have been developed in recent years to overcome limitations encountered when applying traditional verification techniques to high-resolution NWP forecasts, which often result in misleading assessments of forecast accuracy. This study illustrates how the Fractions Skill Score (FSS) generated by the Model Evaluation Tools can be applied to assess the US Army Research Laboratory's Weather Running Estimate–Nowcast (WRE–N) model forecasts. The FSS is widely recognized as an important metric for verifying model performance as a function of threshold value and spatial scale and, when used to characterize the baseline performance, provides the basis for comparison as model improvements are implemented. Preliminary results suggest that the FSS applied to assess the WRE–N provides a robust metric to track changes in model performance and a better metric of the skill in predicting objects that affect input to My Weather Impact Decision Aid.
Distribution: Approved for public release
  Download Report ( 1.355 MBytes )
If you are visually impaired or need a physical copy of this report, please visit and contact DTIC.

Last Update / Reviewed: November 1, 2017