Has vanilla transformer implemented in transformers library?
Hi all,Recently I’m going to do a translation task using the vanilla Transformer model in the paper “Attention is all you need”, but when I search for model list in Transformers docs, there seems no origin transformer. Anyone know where I can find the model for translation task, or I have to implement it manually?I know that pytorch library has a Transformer class, but it seems that class hasn’t embedding module and isn’t specified for particular task (like BertForxxx etc.).
2022-05-06T09:35:50Z
You may prefer to use libraries dedicated for that instead. For instance OpenNMT, fairseq, MarianMT. The latter has also been implemented in transformers.
2022-05-31T13:25:33Z
@BramVanroyI wonder is it possible to use a vanilla Transformer encoder with no any pre-training?
2022-06-05T11:09:50Z
What do you mean? If you do not train a model, it’s weights are randomly initialized. That means that without training, the model will give you random output. You can make use of models that someone else pretrained, though.Models - Hugging Face