PretrainedConfig example to use it in GPT2 text-generation pipeline
I want to modify the default parameters in thePretrainedConfigand use it in the pipeline,prediction = pipeline('text-generation', model=model_path, tokenizer=tokenizerObject)Can someone please help me in constructing the PretrainedConfig object with parameters so that I can use it in the pipeline object.
2021-02-06T12:39:12Z
Hi@bala1802, since theTextGenerationPipelineaccepts a pretrained model as an argument perhaps you can adapt the following for your use case:from transformers import AutoModelWithLMHead, AutoTokenizer, AutoConfig, TextGenerationPipeline

model_ckpt= "gpt2"
# override default parameters here, eg output the hidden states
config = AutoConfig.from_pretrained(model_ckpt, output_hidden_states=True)
model = AutoModelWithLMHead.from_pretrained(model_ckpt, config=config)
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)

pipe = TextGenerationPipeline(model=model, tokenizer=tokenizer)You might not even need theTextGenerationPipelineclass since I think that this is equivalent to instantiating the pipeline as follows:pipe = pipeline('text-generation', model=model, tokenizer=tokenizer)