Automatic sentence segmentation and encoding
Hello everyone,Is there a way to segment sentences and encode them using the tokenizers library automatically? I am currently using spacy and/or nltk for the sentence segmentation part and then tokenizers to encode the sentences.Many thanks.