Help converting model weights from polycoder gpt-neox
I’m fairly new to Hugging Face. I’m working on a project that involves using thePolycoder model, which is a variant of GPT-2 trained on source code. The model was trained using thegpt-neox framework, based on megatron and deepspeed. I have some weights, and I’d like to convert them for use with a Hugging Face model. I’ve poked around a bit, and I can see that there are several different GPT-based models in the library, like GPT-2, GPT-neox, GPT-J etc. I’m not quite sure which model to use, or how I can figure out how to convert the weights. Any help would be appreciated.
2022-08-11T20:51:47Z
Actually, I also just noticed that there’s anote in the READMEasking about doing this exact thing… anyone able to help?