Information Flow and Entropy Production in the Kalman-Bucy Filter ∗

We investigate the information theoretic properties of KalmanBucy filters in continuous time, developing notions of information supply, storage and dissipation. By introducing a concept of energy, we develop a physical analogy in which the unobservable signal describes a statistical mechanical system interacting with a heat bath. The abstract ‘universe’ comprising the signal and the heat bath obeys a non-increase law of entropy; however, with the introduction of partial observations, this law can be violated. The Kalman-Bucy filter behaves like a Maxwellian demon in this analogy, returning signal energy to the heat bath without causing entropy increase. This is made possible by the steady supply of new information. The analogy thus provides a quantitative example of Landauer’s Principle, operating in reverse. The filter sets up a stationary non-equilibrium state, in which energy flows around the loop comprising the heat bath, the signal and the filter. We define a rate of interactive entropy production, isolating the statistical mechanics of this flow from the marginal mechanics of the signal and filter. This leads naturally to a dual filtering problem in which information flows are reversed.