How can I make Bloom stop generating when it should?
How can I make Bloom stop generating when it should?
I want bloom stop generating when end of the response, do not generate other things until max_new_token.
Excuse my poor English
There is a stop parameter. It will stop generating when any of the given tokens in the list are generated:
payload = {
"inputs": promt,
"parameters": {
"do_sample": True,
"top_p": X,
"max_new_tokens": 150,
"temperature": X,
"stop": ['.', 'The next day']
}
response = hfrequests.post("/static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fbigscience%2Fbloom%3C%2Fa%3E", headers=headers, json=payload)
There is a stop parameter. It will stop generating when any of the given tokens in the list are generated:
payload = {
"inputs": promt,
"parameters": {
"do_sample": True,
"top_p": X,
"max_new_tokens": 150,
"temperature": X,
"stop": ['.', 'The next day']
}
response = hfrequests.post("/static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fbigscience%2Fbloom%3C%2Fa%3E", headers=headers, json=payload)
I want to let bloom stop when finish the content, I give a () at begin and end of example, it will let bloom put ) when finish the content