The Dolly v2 model but sharded to reduce the CPU memory required to load it on the GPU and also reduce the load time. By sharding it 
you don't need to load the full model on the CPU RAM before sending it to the GPU, just load it shard by shard.
{}
transformers pytorch gpt_neox text-generation autotrain_compatible endpoints_compatible has_space text-generation-inference region:us
TAGS
#transformers #pytorch #gpt_neox #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 

The Dolly v2 model but sharded to reduce the CPU memory required to load it on the GPU and also reduce the load time. By sharding it 
you don't need to load the full model on the CPU RAM before sending it to the GPU, just load it shard by shard.
TAGS
#transformers #pytorch #gpt_neox #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 

TAGS
#transformers #pytorch #gpt_neox #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
