Why is BertNormalizer not exposed on the tokenizers library?
Hi allI am trying to use the BertNormalizer() to train a ByteLevelBPETokenizer(). But it seems this normaliser is not exposed looking at the source(see attached image). I am using tokenizers==0.12.1tokenizersbert599×592 51 KBWas this intentional or is it a feature that will be added going forward?