Start Submission Become a Reviewer

Reading: Data assimilation with the weighted ensemble Kalman filter

Download

A- A+
Alt. Display

Original Research Papers

Data assimilation with the weighted ensemble Kalman filter

Authors:

Nicolas Papadakis,

Barcelona Media/Av. Diagonal 177, 08017 Barcelona, ES
X close

Etienne Mémin ,

INRIA, Campus de Beaulieu, 35042 Rennes cedex, FR
X close

Anne Cuzol,

Université de Bretagne Sud, Campus de Tohannic BP 573, 56017 Vannes cedex, FR
X close

Nicolas Gengembre

INRIA, Campus de Beaulieu, 35042 Rennes cedex, FR
X close

Abstract

In this paper, two data assimilation methods based on sequential Monte Carlo sampling are studied and compared: the ensemble Kalman filter and the particle filter. Each of these techniques has its own advantages and drawbacks. In this work, we try to get the best of each method by combining them. The proposed algorithm, called the weighted ensemble Kalman filter, consists to rely on the EnsembleKalman Filter updates of samples in order to define a proposal distribution for the particle filter that depends on the history of measurement. The corresponding particle filter reveals to be efficient with a small number of samples and does not rely anymore on the Gaussian approximations of the ensemble Kalman filter. The efficiency of the new algorithm is demonstrated both in terms of accuracy and computational load. This latter aspect is of the utmost importance in meteorology or in oceanography since in these domains, data assimilation processes involve a huge number of state variables driven by highly non-linear dynamical models. Numerical experiments have been performed on different dynamical scenarios. The performances of the proposed technique have been compared to the ensemble Kalman filter procedure, which has demonstrated to provide meaningful results in geophysical sciences.

How to Cite: Papadakis, N., Mémin, E., Cuzol, A. and Gengembre, N., 2010. Data assimilation with the weighted ensemble Kalman filter. Tellus A: Dynamic Meteorology and Oceanography, 62(5), pp.673–697. DOI: http://doi.org/10.1111/j.1600-0870.2010.00461.x
7
Views
57
Citations
  Published on 01 Jan 2010
 Accepted on 13 Apr 2010            Submitted on 23 Oct 2009

References

  1. Anderson , J . 2001 . An ensemble adjustment Kalman filter for data assimilation . Mon. Wea. Rev . 129 ( 12 ), 2884 – 2903 .  

  2. Anderson , J . 2007 . An adaptive covariance inflation error correction algorithm for ensemble filters . Tellus 59A , 210 – 224 .  

  3. Anderson , J. and Anderson , S . 1999 . A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts . Mon. Wea. Rev . 127 ( 12 ), 2741 – 2758 .  

  4. Anderson , J. L . 2003 . A local least squares framework for ensemble filtering . Mon. Wea. Rev . 131 ( 4 ), 634 – 642 .  

  5. Arnaud , E. and Mémin , E . 2007 . Partial linear gaussian model for tracking in image sequences using sequential monte carlo methods . Int. J. Comput. Vision 74 ( 1 ), 75 – 102 .  

  6. Arulampalam , M. , Maskell , S. , Gordon , N. and Clapp , T . 2002 . A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking . IEEE Trans. Signal Process . 50 ( 2 ).  

  7. Bennet , A . 1992 . Inverse Methods in Physical Oceanography . Cambridge University Press, New York , NY .  

  8. Bertino , L. , Evensen , G. and Wackernagel , H . 2003 . Sequential data assimilation techniques in oceanography . Int. Statist. Rev . 71 ( 2 ), 223 – 241 .  

  9. Bishop , C. , Etherton , B. and Majumdar , S . 2001 . Adaptive sampling with the ensemble transform Kalman filter. Part I: theoretical aspects . Mon. Wea. Rev . 129 ( 3 ), 420 – 436 .  

  10. Burgers , G. , van Leeuwen , P. and Evensen , G . 1998 . Analysis scheme in the ensemble Kalman filter. Mon . Wea. Re v . 126 , 1719 – 1724 .  

  11. Corpetti , T. , Heas , R , Memin , E. and Papadalcis , N . 2009 . Pressure image asimilation for atmospheric motion estimation . Taus 61A , 160 – 178 .  

  12. Courtier , P . 1997 . Dual formulation of four-dimensional variational assimilation . Quart. J. Roy. Meteorol. Soc . 123 , 2249 – 2461 .  

  13. Courtier , R , Andersson , E. , Heckley , W. , Pailleux , J. , Vasiljevic , D. and co-authors . 1998. The ECMWF implementation of three-dimensional variational assimilation (3D-VAR). Part 1: formulation . Quart. J. Roy. Meteorol. Soc . 124 , 1783– 1807 .  

  14. Crisan , D. and Doucet , A . 2002 . Survey of convergence results on particle filering methods for practitioners . IEEE Trans. Signal Process . 50 ( 3 ), 736 – 746 .  

  15. Cuzol , A. and Memin , E . 2009 . A stochastic filter technique for fluid flows velocity fields tracking . IEEE Trans. Pattern Anal. Mach. Intell . 31 ( 7 ), 1278 – 1293 .  

  16. Del Moral , P . 2004 . Feynman-Kac Formulae Genealogical and Interacting Particle Systems with Applications . Series: Probability and Applications . Springer , New York .  

  17. Doucet , A. , Godsill , S. and Andrieu , C . 2000 . On sequential Monte Carlo sampling methods for Bayesian filtering . Stat. Comput . 10 ( 3 ), 197 – 208 .  

  18. Elliot , E , Horntrop , D. and Majda , A . 1997 . A Fourier-wavelet Monte Carlo method for fractal random fields . J. Comput. Phys . 132 ( 2 ), 384 – 408 .  

  19. Evensen , G . 1994 . Sequential data assimilation with a non linear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res . 99 ( C5 )(10), 143 – 162 .  

  20. Evensen , G . 2003 . The ensemble Kalman filter, theoretical formulation and practical implementation . Ocean Dyn . 53 ( 4 ), 343 – 367 .  

  21. Evensen , G . 2006 . Data Assimilation: The Ensemble Kalman Filter . Springer-Verlag , New-york .  

  22. Evensen , G. and van Leeuwen , P . 2000 . An ensemble Kalman smoother for nonlinear dynamics . Mon. Wea. Rev . 128 ( 6 ), 1852 – 1867 .  

  23. Fertig , E. , Harlim , J. and Hunt , B . 2007 . A comparative study of 4D-VAR and a 4D ensemble Kalman filter. Perfect model simulations with lorenz-96 . Tellus 59A , 96 – 100 .  

  24. Gauthier , R , Courtier , P. and Moll , P . 1993 . Assimilation of simulated wind lidar data with a Kalman filter . Mon. Wea. Rev . 121 ( 6 ), 1803 – 1820 .  

  25. Gordon , N. , Doucet , A. and Freitas , J. D . 2001 . Sequential Monte Carlo Methods in Practice . Springer-Verlag, New York , NY .  

  26. Gordon , N. , Salmond , D. and Smith , A . 1993 . Novel approach to non-linear/non-gaussian bayesian state estimation . IEEE Process.-F 140 ( 2 ).  

  27. Hamill , T. and Snyder , C . 2000 . A hybrid ensemble Kalman filter-3d variational analysis scheme . Mon. Wea. Rev . 128 ( 8 ), 2905 – 2919 .  

  28. Harlim , J. and Hunt , B . 2007a . Four-dimensional local ensemble transform Kalman filter: numerical experiments with a global circulation model . Tellus 59A ( 5 ), 731 – 748 .  

  29. Harlim , J. and Hunt , B . 2007b . A non-gaussian ensemble filter for assimilating infrequent noisy observations . Tellus 59A , 225 – 237 .  

  30. Harlim , J. and Majda , A . 2009 . Catastrophic filter divergence in filtering nonlinear dissipative systems. Commun . Math. Sc i . 7 ( 3 ).  

  31. Houtekamer , P. and Mitchell , H . 2001 . A sequential ensemble Kalman filter for atmospheric data assimilation . Mon. Wea. Rev . 129 ( 1 ), 123 – 137 .  

  32. Houtekamer , P. L. and Mitchell , H . 1998 . Data assimilation using an ensemble Kalman filter technique . Mon. Wea. Rev . 126 ( 3 ), 796 – 811 .  

  33. Houtekamer , P. L. and Mitchell , H . 2006 . Ensemble Kalman filtering . Quart. J. Roy. Meteorol. Soc . 131 ( 613 ), 3269 – 3289 .  

  34. Houtekamer , P. L. , Mitchell , H. , Pellerin , G. , Buehner , M. , Charron , M. and co-authors . 2005 . Atmospheric data assimilation with an ensemble Kalman filter: results with real observations. Mon. Wea. Rev . 133 ( 3 ), 604 – 620 .  

  35. Hunt , B. , Kalnay , E. , Kostelich , E. , Ott , E. , Patil , D. and co-authors . 2004 . Four-dimensional ensemble Kalman filtering. Tellus 56A , 273 – 277 .  

  36. Hunt , B. , Kostelich , E. and Szunyogh , J. 2007. Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D 230 , 112 – 126 .  

  37. Julier , S. and Uhlmann , J . 1997 . A new extension of the Kalman filter to nonlinear systems . In:Proceedings of the International Symposium Aerospace/Defense Sensing, Simul. Controls , 182 – 193 .  

  38. Kalman , R . 1960 . A new approach to linear filtering and prediction problems . Trans. ASME - J. Basic Eng . 82 , 35 – 45 .  

  39. Kalman , R. and Bucy , R . 1961 . New results in linear filtering and prediction theory . Trans. ASME - J. Basic Eng . 83 , 95 – 107 .  

  40. Kivman , G. A . 2003 . Sequential parameter estimation for stochastic systems . Nonlin. Process. Geophys . 10 , 253 – 259 .  

  41. Kong , A. , Liu , J. and Wong , W . 1994 . Sequential imputations and Bayesian missing data problems . J. Am. Stat. Assoc . 89 ( 425 ), 278 – 288 .  

  42. Kuhl , D. , Szunyogh , I. , Kostelich , E. J. , Patil , D. J. , Gyarmati , G. and co-authors . 2007 . Assessing predictability with a local ensemble Kalman filter. J. Atmos. Sci . 64 , 1116 – 1140 .  

  43. Le Dimet , F.-X. and Talagrand , O . 1986 . Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects . Tellus 38A , 97 – 110 .  

  44. Le Gland , F. , Monbet , V. and Tran , V . 2009 . Large sample asymptotics for the ensemble Kalman filter . Technical Report 7014 ,  

  45. Le Gland , F. , Monbet , V. and Tran , V . 2010 . Large sample asymptotics for the ensemble Kalman filter. In: Handbook on Nonlinear Filtering , ( eds D. Crisan and B. Rozovskii ), Oxford University Press , in press .  

  46. Liu , J. , Fertig , E. , Li , H. , Kalnay , E. , Hunt , B. and co-authors . 2008 . Comparison between local ensemble transform Kalman filter and PSAS in the nasa finite volume gcm-perfect model experiments. Nonlin. Process. Geophys . 15 ( 4 ), 645 – 659 .  

  47. Lorene , A . 1981 . A global three-dimensional multivariate statistical interpolation scheme . Mon. Wea. Rev . 109 ( 4 ), 701 – 721 .  

  48. Miyoshi , T. and Yamane , S . 2007 . Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution . Mon. Wea. Rev . 135 ( 11 ), 3841 – 3861 .  

  49. Musso , C. , Oudjane , N. and Le Gland , F . 2001 . Improving regularized particle filters. In: Sequential Monte Carlo Methods in Practice , (eds A. Doucet , N. de Freitas and N. Gordon ), Statistics for Engineering and Information Science. Springer-Verlag , New York , 247 – 271 .  

  50. Ott , E. , Hunt , B. R. , Szunyogh , I. , Zimin , A. V. , Kostelich , E. J. and co-authors . 2004 . A local ensemble Kalman filter for atmospheric data assimilation. Tellus 56A , 415 – 428 .  

  51. Patil , D. , Hunt , B. and Carton , J . 2001 . Identifying low-dimensional nonlinear behavior in atmospheric data . Mon. Wea. Rev . 129 ( 8 ), 2116 – 2125 .  

  52. Pham , D. T . 2001 . Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon . Wea. Re v . 129 , 1194 – 1207 .  

  53. Rémy , S. , Pannekoucke , O. , Bergot , T. and Baehr , C . 2010 . Adaptation of a particle filtering method for data assimilation in a 1D numerical model used for fog forecasting . Quart. J. Roy. Meteorol. Soc . , submitted .  

  54. Shu , C.-W . 1998 . Advanced Numerical Approximation of Nonlinear Hyperbolic Equations , Volume 1697 of Lecture Notes in Mathematics. Springer , Berlin/Heidelberg , 325 – 432 .  

  55. Snyder , C. , Bengtsson , T. , Bickel , P. and Anderson , J . 2008 . Obstacles to high-dimensional particle filtering . Mon. Wea. Rev . 136 ( 12 ), 4629 – 4640 .  

  56. Szunyogh , I. , Kostelich , E. J. , Gyarmati , G. , Kalnay , E. , Hunt , B. and co-authors . 2008 . A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus 60A , 113 – 130 .  

  57. Tippett , M. , Anderson , J. , Craig , C. B. , Hamill , T. and Whitaker , J . 2003 . Ensemble square root filters . Mon. Wea. Rev . 131 ( 7 ), 1485 – 1490 .  

  58. van der Merwe , R. , de Freitas , N. , Doucet , A. and Wan , E . 2001 . The unscented particle filter . Adv. Neural Inform. Process. Syst . , 13 , 584 – 590 .  

  59. van Leeuwen , P. J . 2002 . Ensemble Kalman filters, sequential importance resampling and beyond. In ECMWF Workshop on the Role of the Upper Ocean in the Medium and Extended Range Forecasting, ECMWF, Reading, UK , 46 – 56 .  

  60. van Leeuwen , P. J . 2003 . A variance-minimizing filter for large-scale applications . Mon. Wea. Rev . 131 ( 9 ), 2071 – 2084 .  

  61. van Leeuwen , P. J . 2009 . Particle filtering in geophysical systems . Mon. Wea. Rev . , 137 ( 2 ), 4089 – 4114 .  

  62. Whitaker , J. and Hamill , T . 2002 . Ensemble data assimilation without perturbed observations . Mon. Wea. Rev . 130 ( 7 ), 1913 – 1924 .  

  63. Whitaker , J. , Hamill , T. , Song , X. W. Y. and Toth , Z . 2008 . Ensemble data assimilation with the NCEP global forecast system . Mon. Wea. Rev . 136 ( 2 ), 463 – 482 .  

  64. Xiong , X. , Navon , I. and Uzunoglu , B . 2006 . A note on the particle filter with posterior gaussian resampling . Tellus 58A , 456 – 460 .  

  65. Zhou , Y. , McLaughlin , D. and Entelchabi , D . 2006 . Assessing the performance of the ensemble Kalman filter for land surface data assimilation . Mon. Wea. Rev . 134 ( 8 ), 2128 – 2142 .  

  66. Zupanski , M . 2005 . Maximum likelihood ensemble filter: theoretical aspects . Mon. Wea. Rev . 133 ( 6 ), 1710 – 1726 .  

comments powered by Disqus