The required output names (like ["last_hidden_state"]) can be obtained by taking a look at the ONNX configuration of 
each model. For example, for DistilBERT we have:
thon

from transformers.models.distilbert import DistilBertConfig, DistilBertOnnxConfig
config = DistilBertConfig()
onnx_config = DistilBertOnnxConfig(config)
print(list(onnx_config.outputs.keys()))
["last_hidden_state"]

The process is identical for TensorFlow checkpoints on the Hub. For example, export a pure TensorFlow checkpoint like so: