HuggingFace Inference Endpoints: Pipeline Args
Hi! I’m trying to use the Huggingface Inference Toolkit with a pipeline using the guide below:GitHubGitHub - aws/sagemaker-huggingface-inference-toolkitContribute to aws/sagemaker-huggingface-inference-toolkit development by creating an account on GitHub.My question is this : I need a token-classification pipeline. I’m able to deploy it successfully, but is there a way to mention the aggregation strategy ?Thanks!
2023-09-07T16:03:27Z
It using the same paramters astransformers.  You can provide aparametersparameter when running inference withaggregation_strategy
2023-09-07T16:16:46Z
Thanks for replying Phil!Is this what you mean? ( this didn’t work)input_pack = {“inputs”:“Hello welcome to The Olympic Games in London”,“parameters”:{“aggregation_strategy”:“first”}}Or passing in the env parameters likeHF_TASK
2023-09-12T22:09:15Z
I’ve been running into some of the same things and getting different responses locally vs building to a docker space for api inference, super frustrating.From what I’m picking up, the tasks have different dictionary entries depending on task type. Typically an input and options.  Would love to see the source on this stuff.Relevant infohere
2024-01-22T20:36:12Z
Did you ever solve this issue? I’m facing the same problem
2024-01-22T20:52:57Z
Nvm I was able to solve it. In case anyone is interested on the solution:{“inputs”: “input your data here”,“parameters”: {“aggregation_strategy”: “first”}}