Hidden States of OpenAI GPT2 inconsistent
Hi, I am trying to use the OpenAI GPT2 and I just realized that the hidden states change every time I run the model and I cannot figure out why. When I use BertModel this does not happen.Does anyone have an explanation for that?Thank you so much in advance!
2021-10-25T13:43:05Z
Do you run the model in evaluation mode?i.e.model.eval()=> this will turn off any dropout modules
2021-10-25T14:27:23Z
IIRC GPT-2 does sampling which is not token-level deterministic.