Approximating Difference Evaluations with Local Information

Difference evaluations can effectively shape agent feedback in multiagent learning systems, and have provided excellent results in a variety of domains, including air traffic control and distributed sensor network control. In addition to empirical evidence, there is theoretical evidence demonstrating how difference evaluations help shape agent utilities/objectives in order to promote system-wide coordination. However, analytically calculating difference evaluation functions requires knowledge of the states of all agents in the system, as well as the mathematical form of the system evaluation function. In practice, neither of these elements are typically available. In this work, we demonstrate that each agent can locally approximate difference evaluations using only local state and action information, as well as a broadcast value (rather than the mathematical form) of the system evaluation function, allowing for difference evaluations to be implemented in multiagent systems where global state information is unavailable. This approximation technique is tested in a multiagent congestion problem, and the results demonstrate that approximate difference evaluations perform similarly to analytically computed difference evaluations, while using far less system information.