Recommend an instance for MPT-7B and MPT-30B inference
I’d like to know what instance would be best for each.I’ve already tried with Google Collab on the free and pro plans - and 25 GB isn’t enough to load the model.I’ve tried Kaggle as well but haven’t figured out how to load the model.I can load into ml.m5.4xlarge on Sagemaker but that’s for General Compute so I’m thinking accelerated comput is better.
2023-07-18T13:13:39Z
I can train falcon-7b on ml.g5.2xlarge and falcon-40b on ml.g5.12xlarge
2023-07-19T12:11:01Z
appreciate it, that’s helpful, ronan