Increasing pretrained CLIP max possible text sequence length
Hey!I’m quite new to this platform and I’ve been using it to work on my Msc Thesis. I’m wondering if it is possible to use a pretrained CLIP model but change how long the max sequence length is. In particular, the pretrained CLIP model I’m using isCLIP-RSCIDand I want to increase the maximum sequence length that the model can receive as input - currently it’s 77. Ideally I want  to keep the current model weights and simply add random weights for the new columns in the updated weights matrixes. I’ve tried to look around the CLIP documentation and couldn’t find anything that allows to keep the pretrained weights whilst changing the configuration but as I said I’m a beginner. Any help would be greatly appreciated!
2023-04-10T22:49:53Z
Did you ever figure out a solution to this? Running into the same issue formax_position_embeddings