Adding a New tokens to ViT
Hey,I want to add 2 custom embeddings (tokens) to a pretrained transformersThose tokens will represent some argument to the imageI have seen there is a similar method to BERT and text transformers (add a word to the vocabulary) but I did not find something for image transformersCurrently I do it with ugly code that overrides the embeddings of the built-in Vit and adds the token.If you have a smarter solutions / similar ideas I would be happy to hearThanks