# This is the 4bit converted version for use with oobabooga's text-generation-webui.

All credits go to this amazing project: https://github.com/FreedomIntelligence/LLMZoo

This is the chat-instruct version

Converted with `python llama.py ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save chimera7b-4bit-128g.pt`

It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there. 

Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)

{}
transformers pytorch llama text-generation autotrain_compatible endpoints_compatible has_space text-generation-inference region:us
TAGS
#transformers #pytorch #llama #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 

# This is the 4bit converted version for use with oobabooga's text-generation-webui.

All credits go to this amazing project: URL

This is the chat-instruct version

Converted with 'python URL ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save URL'

It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there. 

Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)

# This is the 4bit converted version for use with oobabooga's text-generation-webui.

All credits go to this amazing project: URL

This is the chat-instruct version

Converted with 'python URL ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save URL'

It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there. 

Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)
TAGS
#transformers #pytorch #llama #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
 # This is the 4bit converted version for use with oobabooga's text-generation-webui.

All credits go to this amazing project: URL

This is the chat-instruct version

Converted with 'python URL ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save URL'

It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there. 

Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)
TAGS
#transformers #pytorch #llama #text-generation #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
# This is the 4bit converted version for use with oobabooga's text-generation-webui.

All credits go to this amazing project: URL

This is the chat-instruct version

Converted with 'python URL ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save URL'

It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there. 

Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)