

OneFormer requires two inputs during inference: image and task token. 
During training, OneFormer only uses panoptic annotations.
If you want to train the model in a distributed environment across multiple nodes, then one should update the
  get_num_masks function inside in the OneFormerLoss class of modeling_oneformer.py. When training on multiple nodes, this should be
  set to the average number of target masks across all nodes, as can be seen in the original implementation here.
One can use [OneFormerProcessor] to prepare input images and task inputs for the model and optional targets for the model. [OneformerProcessor] wraps [OneFormerImageProcessor] and [CLIPTokenizer] into a single instance to both prepare the images and encode the task inputs.
To get the final segmentation, depending on the task, you can call [~OneFormerProcessor.post_process_semantic_segmentation] or [~OneFormerImageProcessor.post_process_instance_segmentation] or [~OneFormerImageProcessor.post_process_panoptic_segmentation]. All three tasks can be solved using [OneFormerForUniversalSegmentation] output, panoptic segmentation accepts an optional label_ids_to_fuse argument to fuse instances of the target object/s (e.g. sky) together.