• \
    Start Submission Become a Reviewer

    Reading: 4-D-Var or ensemble Kalman filter?

    Download

     A- A+
    Alt. Display

    Original Research Papers

    4-D-Var or ensemble Kalman filter?

    Authors:

    Eugenia Kalnay ,

    University of Maryland, College Park, 3431 CSS building, MD 20742-2425, US
    X close

    Hong Li,

    University of Maryland, College Park, 3431 CSS building, MD 20742-2425, US
    X close

    Takemasa Miyoshi,

    Numerical Prediction Division, Japan Meteorological Agency, Tokyo, JP
    X close

    Shu-Chih Yang,

    University of Maryland, College Park, 3431 CSS building, MD 20742-2425, US
    X close

    Joaquim Ballabrera-Poy

    Institut de Ciències del Mar (CSIC), Barcelona, ES
    X close

    Abstract

    We consider the relative advantages of two advanced data assimilation systems, 4-D-Var and ensemble Kalman filter (EnKF), currently in use or under consideration for operational implementation.With the Lorenz model, we explore the impact of tuning assimilation parameters such as the assimilation window length and background error covariance in 4-D-Var, variance inflation in EnKF, and the effect of model errors and reduced observation coverage. For short assimilation windows EnKF gives more accurate analyses. Both systems reach similar levels of accuracy if long windows are used for 4-D-Var. For infrequent observations, when ensemble perturbations grow non-linearly and become non-Gaussian, 4-D-Var attains lower errors than EnKF. If the model is imperfect, the 4-D-Var with long windows requires weak constraint. Similar results are obtained with a quasi-geostrophic channel model. EnKF experiments made with the primitive equations SPEEDY model provide comparisons with 3-D-Var and guidance on model error and ‘observation localization’. Results obtained using operational models and both simulated and real observations indicate that currently EnKF is becoming competitive with 4-D-Var, and that the experience acquired with each of these methods can be used to improve the other. A table summarizes the pros and cons of the two methods.

    How to Cite: Kalnay, E., Li, H., Miyoshi, T., Yang, S.-C. and Ballabrera-Poy, J., 2007. 4-D-Var or ensemble Kalman filter?. Tellus A: Dynamic Meteorology and Oceanography, 59(5), pp.758–773. DOI: http://doi.org/10.1111/j.1600-0870.2007.00261.x
    25
    Views
    3
    Downloads
      Published on 01 Jan 2007
    Peer Reviewed
     CC BY 4.0
     Accepted on 20 May 2007            Submitted on 4 Jun 2006

    References

    1. Anderson , J. L., 2001 . An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Re v . 129 , 2884 – 2903 .  

    2. Andersson , E. , Fisher , M. , Holm , E. , Isaksen , L. , Radnoti , G. and co-authors. 2005. Will the 4D-Var approach be defeated by nonlinearity? ECMWF Tech Memo 479. Available at: http://www.ecmwf.int/publications .  

    3. Baek , S.-J. , Hunt , B. R. , Kalnay , E. , Ott , E. and Szunyogh , I., 2006 . Local ensemble Kalman filtering in the presence of model bias . Tellus 58 , 293 – 306 .  

    4. Bengtsson , L. and Hodges , K. I., 2005 . On the impact of humidity observations in numerical weather prediction . Tellus 57A , 701 – 708 .  

    5. Bishop , C. H. , Etherton , B. J. and Majumdar , S. J., 2001 . Adaptive sampling with ensemble transform Kalman filter. Part I: theoretical aspects. Mon. Wea. Re v . 129 , 420 – 436 .  

    6. Burgers , G. , van Leeuwen , P. J. and Evensen , G., 1998 . On the analysis scheme in the ensemble Kalman filter. Mon. Wea. Re v . 126 , 1719 – 1724 .  

    7. Cohn , S. , Da Silva , A. , Guo , J. , Sienkiewicz , M. and Lamich , D., 1998 . Assessing the effects of data selection with the DA0 physical-space statistical analysis system. Mon. Wea. Re v . 126 , 2913 – 2926 .  

    8. Corazza , M. , Kalnay , E. , Patil , D. J. , Ott , E. , Yorke , J. A. and co-authors. 2002. Use of the breeding technique in the estimation of the background error covariance matrix for a quasi-geostrophic model. Paper 6.4 in the ANIS Symposium on Observations, Data Assimilation and Probabilistic Prediction, Orlando, FA, January 14-17 2002. Available at: http://ams.confex.com/ams/pdfpapers/28755.pdf .  

    9. Corazza , M. , Kalnay , E. , Patil , D. J. , Yang , S.-C. , Morss , R. and co-authors. 2003. Use of the breeding technique to estimate the structure of the analysis “error of the day”. Nonlinear Processes in Geophysics 10 , 233 - 243 .  

    10. Corazza , M. , Kalnay , E. and Yang , S.-C., 2007 . An implementation of the local ensemble Kalman filter in a quasi-geostrophic model and comparison with 3D-Var . Nonlinear Proc. Phys . 14 , 89 – 101 .  

    11. Danforth , C. M. , Kalnay , E. and Miyoshi , T. 2007 . Estimating and correcting global weather model error. Mon. Wea. Re v . 134 , 281 – 299 .  

    12. Davolio , S. and A. Buzzi . 2004 . A nudging scheme for the assimilation of precipitation data into a mesoscale model . Weather and Forecasting 19 ( 5 ), 855 – 871 .  

    13. Dee , D. P. and da Silva , A. M., 1998 . Data assimilation in the presence of forecast bias . Quart. J. Raoy. Meteor Soc . 126 , 269 – 295 .  

    14. Dee , D. P. and Todling , R., 2000 . Data assimilation in the presence of forecast bias: the GEOS moisture analysis. Mon. Wea. Re v . 128 , 3268 – 3282 .  

    15. De Pondeca , M. S. E V. , Purser , R. J. , Parrish , D. F. and Derber , J. C., 2006 . Comparison of strategies for the specification of anisotropies in the covariances of a three-dimensional atmospheric data assimilation system, NOAA/NCEP Office Note 452, 13 pp. Available at: www.emc.ncep.noaa.gov/officenotes/newemotes/on452.pdf.  

    16. Derber , J. , Puser , R. , Wu , W.-S. , Treadon , R. , Pondeca , M. and co-authors. 2003. Flow-dependent Thin grid-point 3D-Var. Available at: //www.ecmwf.int/publications/library/ecpublications/_pdf/seminar/2003/sem2003_derber.pdf  

    17. Ehrendorfer , M. and Tribbia , J., 1997 . Optimal prediction of forecast error covariances through singular vectors. J. Atmos. Sc i . 54 , 286 – 313 .  

    18. Evensen , G., 1994 . Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics . J. Geophys. Res. 99(C5) , 10 143-10 162 .  

    19. Evensen , G. and van Leewen , P. J., 1996 . Assimilation of Geosat altimeter data for the Aghulas current using the ensemble Kalman Filter with a quasi-geostrophic model. Mon. Wea. Re v . 124 , 85 – 96 .  

    20. Evensen , G., 1997 . Advanced data assimilation for strongly nonlinear dynamics. Mon. Wea. Re v . 125 , 1342 – 1354 .  

    21. Evensen , G., 2003 . The ensemble Kalman filter: theoretical formulation and practical implementation . Ocen. Dyn . 53 , 343 – 367 .  

    22. Fertig , E. , Harlim , J. and Hunt , B., 2007a . A comparative study of 4D-Var and 4D ensemble Kalman filter: perfect model simulations with lorenz-96 . Tellus 59 , 96 – 101 .  

    23. Fertig , E. , Hunt , B. , Ott , E. and Szunyogh , I., 2007b . Assimilating nonlocal observations with a local ensemble Kalman Filter . Tellus , in press .  

    24. Fisher , M. and Hollingsworth , A., 2004 . Evaluation of reduced rank Kalman filters. Paper J1.10 of ANIS 16th Conference on Numerical Weather Prediction, 84th ANIS Annual Meeting, Seattle, Wash, January 12-15 2004. Available at: ams.confex.com/ams/84Annual/techprogram/paper_74522.htm  

    25. Fisher , M., L, eutbecher , M. and Kelly , G. , 2005. On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation . Q. J.R. Meteorol. Soc . 3235-324 6 .  

    26. Gaspari , G. and Cohn , S. E., 1999 . Construction of correlation functions in two and three dimensions . Quart. J. Roy. Meteor Soc . 125 , 723 – 757 .  

    27. Houtekamer , P. L. and Mitchell , H. L., 1998 . Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Re v . 126 , 796 – 811 .  

    28. Houtekamer , P. L. and Mitchell , H. L., 2001 . A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Re v . 129 , 123 – 137 .  

    29. Houtekamer , P. L. , Mitchell , H. L. , Pellerin , G. , Buehner , M. , Charron , M. and co-authors. 2005. Atmospheric data assimilation with an ensemble Kalman filter: results with real observations. Mon. Wea. Rev . 133 , 604 - 620 .  

    30. Houtekamer , P. L. and Mitchell , H. L., 2005 . Ensemble Kalman filtering . Q. J. Roy. Met. Soc . 131 , 3269 – 3290 .  

    31. Hunt , B. R., 2005 . An efficient implementation of the local ensemble Kalman filter. Available at: http://arxiv.org/abs/physics/0511236.  

    32. Hunt , B. R. , Kalnay , E. , Kostelich , E. J. , Ott , E. , Patil , D. J. and co-authors. 2004. Four-dimensional ensemble Kalman filtering. Tellus, 56A , 273 - 277 .  

    33. Hunt , B. R. , Kostelich , E. J. and Szunyogh , I., 2007 . Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter . Physica D , 230 , 112 – 126 .  

    34. Ide , K. , Courtier , P. , Ghil , M. and Lorenc , A., 1997 . Unified notation for data assimilation: operational, sequential and variational . J. Meteor Soc. Jpn . 75 , 181 – 189 .  

    35. Järvinen , H. , Andersson , E. and Bouttier , E, 1999 . Variational assimilation of time sequences of surface observations with serially correlated errors . Tellus 51A , 469 – 488 .  

    36. Kalnay , E., 2003 . Atmospheric Modeling, Data Assimilation and Predictability . Cambridge University Press , Cambridge , 341 pp .  

    37. Keppenne , C. and Rienecker , H., 2002 . Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Wea. Re v . 130 , 2951 – 2965 .  

    38. Keppenne , C. , Rienecker , M. , Kurkowski , N. and Adamec , D., 2005 . Ensemble Kalman Filter assimilation of temperature and altimeter data with bias correction and application to seasonal prediction . Nonlinear Proc. Geophys . 12 , 491 – 503 .  

    39. Kistler , R. , Kalnay , E. , Collins , W. , Saha , S. , White , G. and co-authors. 2001. The NCEP-NCAR 50-year reanalysis: monthly means CD-ROM and documentation. Bull. Amer Meteor Soc . 82 ( 2 ), 247 - 267 .  

    40. Lawson , W. G. and Hansen , J. A., 2004 . Implications of stochastic and deterministic filters as ensemble-based data assimilation methods in varying regimes of error growth. Mon. Wea. Re v . 132 , 1966 – 1981 .  

    41. Li , Hong , Kalnay , E. , Miyoshi , T. and Danforth , C., 2007 . Ensemble Kalman Filter in the presence of model errors. Available at: http://ams.confex.com/ams/pdfpapers/120175.pdf  

    42. Liu , J. , Fertig , E. , Li , H. , Kalnay , E. , Hunt , B. and co-authors. 2007. Comparison between local ensemble transform Kalman filter and PSAS in the NASA finite volume GCM: perfect model experiments. Available at: http://arxiv.org/ftp/physics/papers/0703/0703066.pdf .  

    43. Lorenc , A. C., 2003 . The potential of the ensemble Kalman filter for NWP - a comparison with 4D-Var . Quart. J. Roy. Meteor Soc . 129 , 3183 – 3203 .  

    44. Lorenz , E., 1963 . Deterministic non-periodic flow. J. Atmos. Sc i . 20 , 130 – 141 .  

    45. Miller , R. , Ghil , M. and Gauthiez , F., 1994 . Advanced data assimilation in strongly nonlinear dynamical systems , J. Atmos. Sc i . 51 , 1037 – 1056 .  

    46. Miyoshi , T., 2005 . Ensemble Kalman filter experiments with a Primitive-Equation global model. Doctoral dissertation, University of Maryland, College Park, 197 pp. Available at: https://drum. umd.edu/dspace/handle/1903/3046.  

    47. Miyoshi , T. and Yamane , S., 2007 . Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution . Mon. Wea. Rev ., in press .  

    48. Molteni , E, 2003 . Atmospheric simulations using a GCM with simplified physical parameterizations. I: model climatology and variability in multi-decadal experiments . Clim. Dyn . 20 , 175 – 191 .  

    49. Morss , R. E. , Emanuel , K. A. and Snyder , C., 2001 . Idealized adaptive observations strategies for improving numerical weather prediction. J. Atmos. Sc i . 58 , 210 – 234 .  

    50. Nerger , L. , Hiller , W. and Scroeter , J., 2005 . A comparison of error subspace Kalman filters . Tellus 57A , 715 – 735 .  

    51. Ott , E. , Hunt , B. R. , Szunyogh , I. , Zimin , A. V. , Kostelich , E. J. and co-authors. 2002. Exploiting local low dimensionality of the atmospheric dynamics for efficient Kalman filtering. ArXiv :archive/paper 020358, Available at: http://arxiv.org/abs/physics/020358 .  

    52. Ott , E. , Hunt , B. R. , Szunyogh , I. , Zimin , A. V. , Kostelich , E. J. and co-authors. 2004. A local ensemble Kalman filter for atmospheric data assimilation. Tellus 56A , 415 - 428 .  

    53. Parrish , D. E and Derber , J. C., 1992 . The National Meteorological Center’s Spectral Statistical-Interpolation Analysis System. Mon. Wea. Re v . 120 , 1747 – 1763 .  

    54. Pires , C. , Vautard , R. and Talagrand , O., 1996 . On extending the limits of variational assimilation in chaotic systems . Tellus , 48A , 96 – 121 .  

    55. Rabier , E , Jarvinen , H. , Klinker , E. , Mahfouf , J. -E and Simmons , A. , 2000. The ECMWF operational implementation of four-dimensional variational physics. Q. J. R. MeteoroL Soc ., 126 , 1143 - 1170 .  

    56. Rabier , E and Liu , Z. , 2003. Variational assimilation: theory and overview. Available at: http://www.ecmwf.int/publications/library/ecpublications/_pdf/seminar/2003/sem2003_rabier.pdf  

    57. Riishojgaard , L.-P, 1998 . A direct way of specifying flow-dependent background error correlations for meteorological analysis systems . Tellus 50A , 42 – 57 .  

    58. Rotunno , R. and Bao , J. W., 1996 . A case study of cyclogenesis using a model hierarchy. Mon. Wea. Re v . 124 , 1051 – 1066 .  

    59. Schröter , J. , Seiler , U. and Wenzel , M., 1993 . Variational assimilation of Geosat into an eddy-resolving model of the Gulf Stream extension area . J. Phys. Ocean . 23 , 925 – 953 .  

    60. Szunyogh , I. , Kostelich , E. J. , Gyarmati , G. , Patil , D. J. , Hunt , B. R. and co-authors. 2005. Assessing a local ensemble Kalman filter: Perfect model experiments with the NCEP global model. Tellus 57A , 528 - 545 .  

    61. Szunyogh , I. , Kostelich , E. , Gyarmati , G. , Kalnay , E. , Hunt , B. R. and co-authors. 2007. Assessing a local ensemble Kalman filter: assimilating real observations with the NCEP global model. Under revision in Tellus .  

    62. Tippett , M. K. , Anderson , J. L. , Bishop , C. H. , Hamill , T. M. and Whitaker , J. S., 2003 . Ensemble square root filters. Mon. Wea. Re v . 131 , 1485 – 1490 .  

    63. Tremolet , Yannick, 2005 . Accounting for an imperfect model in 4D-Var. ECMWF Tech Memo #477. Available at: www.ecmwf.it/publications .  

    64. Wang , X. and Bishop , C. H., 2003 . A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sc i . 60 , 1140 – 1158 .  

    65. Whitaker , J. S. and Hamill , T. M., 2002 . Ensemble data assimilation without perturbed observations. Mon. Wea. Re v . 130 , 1913 – 1924 .  

    66. Whitaker , J. S. , Hamill , T. M. , Wei , X. , Song , Y. and Toth , Z., 2007 . Ensemble Data Assimilation with the NCEP Global Forecast System. Mon. Wea. Rev ., under revision. Available at http://www.cdc.moaa.gov/people/jeffrey.s.whitaker/Manuscripts/pubs.html  

    67. Yang , S-C. , Corazza , M. , Carrassi , A. and Kalnay , E., 2007 . Comparison of ensemble-based and variational-based data assimilation schemes in a quasi-geostrophic model. AMS 10th Symposium on In-tegrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface. Available at: http://ams.confex.com/ams/pdfpapers/101581.pdf  

    68. Yang , S-C. , Baker , D. , Li , H. , Huff , M. , Nagpal , G. and co-authors . 2006. Data assimilation as synchronization of truth and model: experiments with the 3-variable Lorenz system. J. Atmos. Sci . 63 , 2340-235 4 .  

    69. Zupanski , M., 2005 . Maximum likelihood ensemble filter: theoretical aspects. Mon. Wea. Re v . 133 , 1710 – 1726 .  

    Jump to Discussions Related content
    comments powered by Disqus