Fix eos_token in tokenizer_config.json

#4
by hypocoristique - opened

This fix has been added by Meta 5 days ago : https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/commit/a8977699a3d0820e80129fb3c93c20fbd9972c41

Fixes a bug when Llama 3 does not stop after generating an end of sentence token.

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment