Methods and Applications of Analysis

Volume 9 (2002)

Number 3

A Mathematical Framework for Quantifying Predictability Through Relative Entropy

Pages: 425 – 444



David Cai

Richard Kleeman

Andrew Majda


Kleeman has recently demonstrated that the relative entropy provides a significant measure of the information content of a prediction ensemble compared with the climate record in several simplified climate models. Here several additional aspects of utilizing the relative entropy for predictability theory are developed with full mathematical rigor in a systematic fashion which the authors believe will be very useful in practical problems with many degrees of freedom in atmosphere/ocean and biological science. The results developed here include a generalized signal-dispersion decomposition, rigorous explicit lower bound estimators for information content, and rigorous lower bound estimates on relative entropy for many variables, N, through N, one-dimensional relative entropies and N, two-dimensional mutual information functions. These last results provide a practical context for rapid evaluation of the predictive information content in a large number of variables.

Full Text (PDF format)