Using Bloom with detailed parameters?
Hello,Newbie here, so my apologies if this is a stupid question or if i post in the wrong section.I’m trying to use the bloom model through inference api and it works well, but when i try to add some parameters (from the detailed parameters list in the text generation category), i get this error:{‘error’: ‘Parameters are not accepted for this specific model’}import requests

API_URL = "https://api-inference.huggingface.co/models/bigscience/bloom"
#API_URL = "https://api-inference.huggingface.co/models/gpt2"

headers = {"Authorization": "****"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

prompt = "Hi i am an AI and "

output = query({
    "inputs": prompt,
    "parameters": {"max_new_tokens": 50, "return_full_text": False}
})

print(output)
#print(output[0]["generated_text"])This code works well (and the parameters are taken into account) when tried on gpt2, but fails on Bloom.Am i doing anything wrong ?ThanksCedric
2022-07-13T19:45:46Z
It worked for me.import requests

TOKEN = "Bearer <INSERT YOUR TOKEN HERE>"

API_URL = "https://api-inference.huggingface.co/models/bigscience/bloom"
headers = {"Authorization": TOKEN}

def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()

while True:

    text_input = input("Insert your input: ")
            
    output = query({
            "inputs": text_input,
            "parameters": {"max_new_tokens": 10,
                           "return_full_text": False}
    })

    print(output)
2022-07-13T20:27:41Z
@roschmid, when I try this, I receive{'error': "Authorization header is invalid, use 'Bearer API_TOKEN'"}Maybe you meantheaders = {"Authorization": f"Bearer {API_TOKEN}"}?
2022-07-13T22:18:56Z
@RylanSchaefferYou’re probably typing wrong your API Token.It has to go with the “Bearer” part.E.g. TOKEN = “Bearer 4EgJlma91939” (this is a made up Token, btw).
2022-07-14T01:47:32Z
I don’t thinkTOKEN = “Bearer 4EgJlma91939”is a token. Rather, you’ve preappendedBearerto the actual token (in your example, the actual token is4EgJlma91939).
2022-07-14T05:26:49Z
Hi@roschmidThanks for your answer.I wanted to try your code and first relaunched my script to ensure the error was still occuring with my code before trying yours, but it didn’t: now my old code works too !I guess they must have fixed something internally.Anyway, thanks a lot for taking the time to answer me, i marked you answer as a solution, although, for anyone bumping here, the code from the initial post works too.RgdsCedric
2022-07-14T13:09:15Z
I got your point. Fixed it.
2022-08-21T10:37:19Z
Thanks for the posts. I’m trying to add some parameters to a cURL request. Somehow it seems the parameters I’m trying to add are getting mixed up into the input string. It could be some kind of syntax error but I can’t see where I’m doing it wrong.This works:curl https://api-inference.huggingface.co/models/bigscience/bloom \
-X POST \
-d '{"inputs": "Two plus two equals "}' \
-H "Authorization: Bearer TOKEN"yielding the output[{"generated_text":"Two plus two equals  four.\nTwo plus two equals  four.\nTwo plus two equals  four.\nTwo plus two equals"}]However, when adding parameters, it seems that this code results in the attempted parameters being mixes up into the input text:curl https://api-inference.huggingface.co/models/bigscience/bloom \
-X POST \
-d '{"inputs": "Two plus two equals", "parameters": {"max_new_tokens": 50, "return_full_text": False} }' \
-H "Authorization: Bearer TOKEN"Resulting in this output:[{"generated_text":"{\"inputs\": \"Two plus two equals\", \"parameters\": {\"max_new_tokens\": 50, \"return_full_text\": False} }\n#     }\n#     return json.dumps(json_data)\n\n# def get_tokens"}]Maybe I just need a delimiter somewhere or the like?If someone can help me fix this I would be really appreciative. Thanks.
2023-02-18T13:11:12Z
This way the request seems to work:model_id="bigscience/bloom" 
payload="Two plus two equals"
params='{"max_new_tokens": 50, "return_full_text":false}'
url="https://api-inference.huggingface.co/models/"$model_id
data='{"inputs":"'$payload'","parameters":'$params'}'
post=$(curl -X POST $url -H "Content-Type: application/json" --data-raw "$data")
echo $post