Oracle MCG: A first peek into COCO Detection Challenges

The recently presented COCO detection challenge will most probably be the reference benchmark in object detection in the next years. COCO is two orders of magnitude larger than Pascal and has four times the number of categories; so in all likelihood researchers will be faced with a number of new challenges. At this point, without any finished round of the competition, it is difficult for researchers to put their techniques in context, or in other words, to know how good their results are. In order to give a little context, this note evaluates a hypothetical object detector consisting in an oracle picking the best object proposal from a state-of-the-art technique. This oracle achieves a AP=0.292 in segmented objects and AP=0.317 in bounding boxes, showing that indeed the database is challenging, given that this value is the best one can expect if working on object proposals without refinement.