• \
    Start Submission Become a Reviewer

    Reading: Measuring information content from observations for data assimilation: relative entropy vers...

    Download

     A- A+
    Alt. Display

    Original Research Papers

    Measuring information content from observations for data assimilation: relative entropy versus shannon entropy difference

    Author:

    Qin Xu

    NOAA/National Severe Storms Laboratory, Oklahoma, US
    X close

    Abstract

    The relative entropy is compared with the previously used Shannon entropy difference as a measure of the amount of information extracted from observations by an optimal analysis in terms of the changes in the probability density function (pdf) produced by the analysis with respect to the background pdf. It is shown that the relative entropy measures both the signal and dispersion parts of the information content from observations, while the Shannon entropy difference measures only the dispersion part. When the pdfs are Gaussian or transformed to Gaussian, the signal part of the information content is given by a weighted inner-product of the analysis increment vector and the dispersion part is given by a non-negative definite function of the analysis and background covariance matrices. When the observation space is transformed based on the singular value decomposition of the scaled observation operator, the information content becomes separable between components associated with different singular values. Densely distributed observations can be then compressed with minimum information loss by truncating the components associate with the smallest singular values. The differences between the relative entropy and Shannon entropy difference in measuring information content and information loss are analysed in details and illustrated by examples.

    How to Cite: Xu, Q., 2007. Measuring information content from observations for data assimilation: relative entropy versus shannon entropy difference. Tellus A: Dynamic Meteorology and Oceanography, 59(2), pp.198–209. DOI: http://doi.org/10.1111/j.1600-0870.2006.00222.x
      Published on 01 Jan 2007
    Peer Reviewed
     CC BY 4.0
     Accepted on 21 Nov 2006            Submitted on 16 Jun 2006

    References

    1. Bennett , A. F. 1992 . Inverse Method in Physical Oceanography . Cambridge University Press , New York , 346 pp .  

    2. Bernardo , J. M. and Smith , A. F. M. 1994 . Bayesian Theory . John Wiley and Sons , New York , 586 pp .  

    3. Daley , R. 1991 . Atmospheric Data Analysis . Cambridge University Press , New York , 457 pp .  

    4. Doviak , J. D. and Zrnic , D. S. 1993 . Doppler Radar and Weather Observations . 2nd Edition. Academic Press , New York , 562 pp .  

    5. Eyre , J. R. 1990 . The information content of data from satellite sounding systems. A simulation study . Q. J. R. Meteor Soc . 116 , 401 – 434 .  

    6. Golub , G. H. and Van Loan , C. F. 1983 . Matrix Computations . Johns Hopkins University Press , Baltimore , 476 pp .  

    7. Gong , J. , Wang , L. and Xu , Q. 2003 . A three-step dealiasing method for Doppler velocity data quality control . J. Atmos. Ocean. Technol . 20 , 1738 – 1748 .  

    8. Haven , K. , Majda , A. J. and Abramov , R. 2005 . Quantifying predictability through information theory. small sample estimation in a non-Gaussian framework . J. Comp. Phys . 206 , 334 – 362 .  

    9. Hodur , R. M. 1997 . The Naval Research Laboratory’s coupled ocean/atmosphere mesoscale prediction system (COAMPS). Mon. Wea. Re v . 125 , 1414 – 1430 .  

    10. Houtekamer , P. L. and Mitchell , H. L. 2001 . A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Re v . 129 , 123 – 137 .  

    11. Huang , H.-L. and Purser , R. J. 1996 . Objective measures of the information density of satellite data . MeteoroL Atmos. Phys . 60 , 105 – 117 .  

    12. Jazwinski , A. H. 1970 . Stochastic Processes and Filtering Theory . Academic Press , San Diego , 376 pp .  

    13. Kleeman , R. 2002 . Measuring dynamical prediction utility using relative entropy. J. Atmos. Sc i . 59 , 2057 – 2072 .  

    14. Kleeman , R. and Majda , A. J. 2005 . Predictability in a model for geo-physical turbulence. J. Atmos. Sc i . 62 , 2864 – 2879 .  

    15. Liu , S. , Xu , Q. and Zhang , P. 2005a . Quality control of Doppler velocities contaminated by migrating birds. PartBayes identification andprobability tests . J. Atmos. Oceanic Technol . 22 , 1114 – 1121 .  

    16. Liu , S. , Xue , M. , Gao , J. and Parrish , D. E 2005b . Analysis and impact of super-obbed Doppler radial velocity in the NCEP grid-point statistical interpolation (GSI) analysis system. Extended abstract. 17th Conf Num. Wea. Pred. Washington DC, Amer . Meteor. Soc . 13A . 4 .  

    17. Majda , A. J. , Kleeman , R. and Cai , D. 2002 . A mathematical framework for quantifying predictability through relative entropy . Methods AppL Anal . 9 , 425 – 444 .  

    18. Majda , A. J. and Wang , X. 2006 . Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows . Cambridge University Press , New York , 565 pp .  

    19. O’Hagan , A. 1994 . Kendall’s Advanced Theory of Statistics. Volume 2B. Bayesian Inference. Oxford University Press . New York , 332 pp.  

    20. Peckham , G. 1974 . The information content of remote measurements of atmospheric temperature by satellite infra-red radiometry and optimum radiometer configurations. Q. J. R. Meteor Soc . 100 , 406 - 419 .  

    21. Papoulis , A. 1991 . Probability, Random Variables, and Stochastic Processes . 3rd Edition , McGraw-Hill , New York , 666 pp .  

    22. Purser , R. J. , Parrish , D. F. and Masutani , M. 2000 . Meteorological observational data compression; An alternative to conventional “super-Obbing”. Office Note 430, National Centers for Environmental Prediction, Camp Springs, MD, 12 pp. [available on line http://www.emc.ncep.noaa.gov/officenotes/Fu11TOC.html#2000]  

    23. Purser , R. J. , Wu , W.-S. , Parrish , D. F. and Roberts , N. M. 2003 . Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Re v . 131 , 1524 – 1535 .  

    24. Shannon , C. E. 1949 . Communication in the presence of noise . Proc. I.C.E . 37 , 10 – 21 .  

    25. Whitaker , J. S. and Hamill , T. M. 2002 . Ensemble data assimilation without perturbed observations. Mon. Wea. Re v . 130 , 1913 – 1924 .  

    26. Wu , W.-S. , Purser , R. J. and Parrish , D. F. 2002 . Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Re v . 130 , 2905 – 2916 .  

    27. Xu , Q. , Nai , K. , Wei , L. , Zhang , P. , Wang , L. and co-authors. 2005. Progress in Doppler radar data assimilation. 32nd Conference on Radar Meteorology, 24-29 October 2005, Albuquerque, New Mexico, Amer. Meteor. Soc., CD-ROM, JP1J7.  

    28. Xu , Q. , Nai , K. and Wei , L. 2007 . An innovation method for estimating radar radialvelocity observation error and background wind error covariances . Q. J. R. Meteor. Soc ., in press .  

    29. Xu , Q. , Liu , S. and Xue , M. 2006 . Background error covariance functions for vector wind analyses using Doppler radar radialvelocity observations. Q. J. R. MeteoroL Soc . in press .  

    30. Zhang , P. , Liu , S. and Xu , Q. 2005 . Quality control of Doppler velocities contaminated by migrating birds. Part I: Feature extraction and quality control parameters . J. Atmos. Oceanic Technol . 22 , 1105 – 1113 .  

    Jump to Discussions Related content
    comments powered by Disqus