webdataset_filtered_converted / tokenizer_config.json

Commit History

pad token got lost in hf conversion
0f59ff7

sam-mosaic commited on

Upload tokenizer
4617d6f

codestar12 commited on