vVoxCore-non-1.3B_2-4-5-6-7 / added_tokens.json
iDSLR's picture
Upload tokenizer
75f4b12
raw
history blame contribute delete
95 Bytes
{
"<|endoftext|>": 50256,
"<||bos||>": 50257,
"<||pad||>": 50258,
"<||unk||>": 50259
}