Converting CLIPModel to VisionTextDualEncoderModel
Hi HF team and thanks for your amazing work, for my research I would like to use the CLIPModel “openai/clip-vit-base-patch32” through the VisionTextDualEncoderModel class. I’ve tried to use thefrom_pretrained()method but it doesn’t support theclip_text_modelclass of the text tower.Do you have any suggestion from where to start to create such a script?
2024-03-21T12:45:10Z
cc@valhallawho might know this, since he worked on adding both CLIP and VisionTextDualEncoder to the Transformers library