Directed information for communication problems with common side information and delayed feedback/feedforward

In this work, we consider the problems of channel coding with feedback and source coding with feedforward. We give a new interpretation of directed information [1] which helps us understand why it arises in the capacity and rate-distortion function of these two problems. Comparing the two problems studied, we nd that in channel coding feedback boosts capacity due to a larger constraint set of optimization of the same objective function. On the other hand, feedforward reduces the rate-distortion function due to a smaller objective function optimized over an unchanged constraint set. Using our interpretation of directed information, we nd the capacity of channels with arbitrarily delayed feedback and causal/non-causal state information. We also solve the corresponding source coding problem.