Creating and Reusing Metric Graphs for Evaluating Agent Performance in the Supply Chain Management Domain

The overwhelming demand for efficient agent performance in Supply Chain Management systems, as exemplified by numerous international competitions, raises the issue of defining and using generalized methods for performance evaluation. Up until now, most researchers test their findings in an ad-hoc manner, often having to re-invent existing evaluationspecific knowledge. In this position paper, we tackle the key issue of defining and using metrics within the context of evaluating agent performance in the SCM domain. We propose the Metrics Representation Graph, a structure that organizes performance metrics in hierarchical manner, and perform a preliminary assessment by instantiating an MRG for the TAC SCM competition, one of the most demanding SCM competitions currently established. We envision the automated generation of the MRG, as well as appropriate contribution from the TAC community towards the finalization of the MRG, so that it will be readily available for future performance evaluations.