File size: 223 Bytes
e8c6cb9 |
1 2 3 4 5 6 7 8 |
This repository doesn't contain a model, but only a tokenizer that can be used with the
`tokenizers` library.
```python
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_pretrained("anthony/tokenizers-test")
```
|