Fix BOS (should be added by the tokenizer). Fix EOS (in config.json -> must be <|eot_id|>). Remove spaces that were aded after special tags (so that round-trip tests can pass and some bugs avoided when inferring with the model). More complete generation_config.json
e7e496d
Git LFS Details
- SHA256: d41d59729e4e0e21d6bbb41b6d146643cbe46cd49d849fbef8eb9d318255f883
- Pointer size: 135 Bytes
- Size of remote file: 4.07 GB
Git Large File Storage (LFS) replaces large files with text pointers inside Git, while storing the file contents on a remote server. More info.