Start Submission Become a Reviewer

Reading: Calibrating probabilistic forecasts from an NWP ensemble

Download

A- A+
Alt. Display

Original Research Papers

Calibrating probabilistic forecasts from an NWP ensemble

Authors:

Thomas Nipen ,

University of British Columbia, Vancouver, CA
X close

Roland Stull

University of British Columbia, Vancouver, CA
X close

Abstract

A post-processing method for calibrating probabilistic forecasts of continuous weather variables is presented. The method takes an existing probability distribution and adjusts it such that it becomes calibrated in the long run. The original probability distributions can be ones such as are generated from a numerical weather prediction (NWP) ensemble combined with a description of how uncertainty is represented by this ensemble. The method uses a calibration function to relabel raw cumulative probabilities into calibrated cumulative probabilities based on where past observations verified on past raw probability forecasts. Applying the calibration method to existing probabilistic forecasts can be beneficial in cases where the underlying assumptions used to construct the probabilistic forecast are not in line with nature’s generating process of the ensemble and corresponding observation. The method was tested on a forecast data set with five different forecast variables and was verified against the corresponding analyses. The calibration method reduced the calibration deficiency of the forecasts down to the level expected for perfectly calibrated forecasts. When the raw forecasts exhibited calibration deficiencies, the calibration method improved the ignorance score significantly. It was also found that the ensemble-uncertainty model used to create the original probability distribution affected the ignorance score.

How to Cite: Nipen, T. and Stull, R., 2011. Calibrating probabilistic forecasts from an NWP ensemble. Tellus A: Dynamic Meteorology and Oceanography, 63(5), pp.858–875. DOI: http://doi.org/10.1111/j.1600-0870.2011.00535.x
12
Citations
  Published on 01 Jan 2011
 Accepted on 21 Jun 2011            Submitted on 12 Dec 2010

References

  1. AMS 2008 . Enhancing weather information with probability forecasts . Bull. Am. Meteor Soc . 89 , 1049 – 1053 .  

  2. Anderson , J. L . 1996 . A method for producing and evaluating proba-bilistic precipitation forecasts from ensemble model integrations . J. Clim . 9 , 1518 – 1530 .  

  3. Bremnes , J. B . 2007 . Improved calibration of precipitation forecasts using ensemble techniques . Part 2: statistical calibration methods , Technical report , Norwegian Meteorological Institute .  

  4. Briicker , J. and Smith , L. A . 2007 . Increasing the reliability of reliability diagrams . Wea. Forecasting 22 , 651 – 661 .  

  5. Eckel , F. A. and Walters , M. K . 1998 . Calibrated probabilistic quan-titative precipitation forecasts based on the MRF ensemble . Wea. Forecasting 13 , 1132 – 1147 .  

  6. Gneiting , T. , Balabdaoui , F. and Raftery , A. E . 2007 . Probabilistic fore-casts, calibration and sharpness. J. R. Stat. Soc., B 69, 243-268. Gneiting, T. and Raftery, A. E. 2007. Strictly proper scoring rules, prediction, and estimation . J. Am. Stat. Assoc . 102 , 359 – 378 .  

  7. Good , I. J . 1952 . Rational decisions. J. R. Stat. Soc., B 14, 107-114. Hamill, T. M. 2001. Interpretation of rank histograms for verifying ensemble forecasts. Mon . Wea. Re v . 129 , 550 – 560 .  

  8. Hamill , T. M. 2007. Comments on “calibrated surface temperature fore-casts from the Canadian ensemble prediction system using Bayesian model averaging” . Mon. Wea. Rev . 135 , 4226-4230 .  

  9. Hamill , T. M. and Colucci , S. J . 1997 . Verification of Eta-RSM short-range ensemble forecasts. Mon . Wea. Re v . 125 , 1312 – 1327 .  

  10. Hamill , T. M. and Colucci , S. J . 1998 . Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts. Mon . Wea. Re v . 126 , 711 – 724 .  

  11. Hamill , T. M. and Whitaker , J. S . 2006 . Probabilistic quantitative precip-itation forecasts based on reforecast analogs: theory and application. Mon . Wea. Re v . 134 , 3209 – 3229 .  

  12. Hamill , T. M. , Whitaker , J. S. and Mullen , S. L . 2006 . Reforecasts: an important dataset for improving weather predictions . Bull. Am. Meteor Soc . 87 , 33 – 46 .  

  13. Hoeting , J. A. , Madigan , M. , Raftery , A. E. and Volinsky , C. T . 1999 . Bayesian model averaging: a tutorial. Stat. Sci. 14, 382-401. Hopson, T. M. and Webster, P. J. 2010. A 1-10-day ensemble forecasting scheme for the major river basins of Bangladesh: forecasting severe floods of 2003-07 . J. Hydrometeor 11 , 618 – 641 .  

  14. Jewson , S. , Brix , A. and Ziehmann , C . 2005 . Weather Derivative Val-uation (eds. Jewson , S. , Brix , A. and Ziehmann , C. ). Cambridge , Cambridge University Press .  

  15. Johnson , C. and Swinbank , R . 2009 . Medium-range multimodel en-semble combination and calibration . Q. J. R. Meteor Soc . 135 , 777 – 794 .  

  16. Murphy , A. H . 1973 . A new vector partition of the probability score . J. AppL Meteor 12 , 595 – 600 .  

  17. Nielsen , H. A. , Nielsen , T. S. , Madsen , H. , Giebel , G. , Badger , J. and co-authors . 2006 . From wind ensembles to probabilistic information about future wind power production: results from an actual appli-cation, In: 9th International Conference on Probabilistic Methods Applied to Power Systems , Stockholm, Sweden.  

  18. Pinson , P. , McSharry , P. and Madsen , H . 2010 . Reliability diagrams for non-parametric density forecasts of continuous variables: accounting for serial correlation . Q. J. R. Meteor Soc . 136 , 77 – 90 .  

  19. Raftery , A. E. , Gneiting , T. , Balabdaoui , F. and Polalcowski , M . 2005 . Us-ing Bayesian model averaging to calibrate forecast ensembles. Mon . Wea. Re v . 133 , 1155 – 1174 .  

  20. Roulston , M. S. and Smith , L. A . 2002 . Evaluating probabilistic forecasts using information theory. Mon . Wea. Re v . 130 , 1653 – 1660 .  

  21. Sloughter , J. M. , Raftery , A. E. and Gneiting , T . 2007 . Probabilistic quantitative precipitation forecasting using Bayesian model averag-ing. Mon . Wea. Re v . 135 , 3209 – 3220 .  

  22. Talagrand , O. , Vautard , R. and Strauss , B. 1997. Evaluation of probabilis-tic prediction systems, In: Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF , pp. 1-25.  

  23. Wilson , L. J. , Beauregard , S. , Raftery , A. E. and Verret , R . 2007 . Cal-ibrated surface temperature forecasts from the Canadian ensemble prediction system using Bayesian model averaging. Mon . Wea. Re v . 135 , 1364 – 1385 .  

comments powered by Disqus