Detecting Objects with Graph Priors and Graph Refinement

The goal of this paper is to detect objects by exploiting their interrelationships. Rather than relying on predeﬁned and labeled graph structures, we infer a graph prior from object co-occurrence statistics. The key idea of our paper is to model object relations as a function of initial class predictions and co-occurrence priors to generate a graph representation of an image for improved classiﬁcation and bounding box regression. We additionally learn the object-relation joint distribution via energy based modeling. Sampling from this distribution generates a reﬁned graph representation of the image which in turn produces improved detection performance. Experiments on the Visual Genome and MS-COCO datasets demonstrate our method is detector agnostic, end-to-end trainable, and especially beneﬁcial for rare object classes. What is more, we establish a con-sistent improvement over object detectors like DETR and Faster-RCNN, as well as state-of-the-art methods modeling object interrelationships.