Using time series for SequenceClassification models

[valhalla]

you might find some pointers here

Using transformers (BERT, RoBERTa) without embedding layer

I’m looking to train a RoBERTa model on protein sequences, which is in many ways similar to normal nlp training, but in others quite different. In the language of proteins, I have 20 characters instead of the normal 26 characters used in english (it is 26 right? :D), so that is rather similar. The big difference is that you don’t really combine the characters in proteins to actual words, but rather just keep each character as a distinct token or class. Hence essentially my input to the transfo…