
<p><strong><font size="5">Information</font></strong></p>
OpenAssistant-Llama-13B-4-bit working with GPTQ versions used in Oobabooga's Text Generation Webui and KoboldAI.
<br>This was made using <a href="https://huggingface.co/serpdotai/llama-oasst-lora-13B">Serpdotai's Open Assistant 13b LoRa trained for 4 epochs</a> using Open Assistant's dataset.</p>

    python llama.py /KoboldAI/repos/gptq/llama13b-oasst-4-epochs-lora c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors llama13b-oasst-4-epochs-lora-4bit-128g.safetensors

<p><strong><font size="5">Benchmarks</font></strong></p>

<p><strong><font size="4">--true-sequential --groupsize 128</font></strong></p>

<strong>Wikitext2</strong>: 5.380471229553223

<strong>Ptb-New</strong>: 31.921072006225586

<strong>C4-New</strong>: 7.140256881713867

<strong>Note</strong>: This version uses <i>--groupsize 128</i>, resulting in better evaluations. However, it consumes more VRAM.
{"datasets": ["OpenAssistant/oasst1"]}
transformers llama text-generation dataset:OpenAssistant/oasst1 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #llama #text-generation #dataset-OpenAssistant/oasst1 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


<p><strong><font size="5">Information</font></strong></p>
OpenAssistant-Llama-13B-4-bit working with GPTQ versions used in Oobabooga's Text Generation Webui and KoboldAI.
<br>This was made using <a href="URL Open Assistant 13b LoRa trained for 4 epochs</a> using Open Assistant's dataset.</p>

    python URL /KoboldAI/repos/gptq/llama13b-oasst-4-epochs-lora c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors llama13b-oasst-4-epochs-lora-4bit-128g.safetensors

<p><strong><font size="5">Benchmarks</font></strong></p>

<p><strong><font size="4">--true-sequential --groupsize 128</font></strong></p>

<strong>Wikitext2</strong>: 5.380471229553223

<strong>Ptb-New</strong>: 31.921072006225586

<strong>C4-New</strong>: 7.140256881713867

<strong>Note</strong>: This version uses <i>--groupsize 128</i>, resulting in better evaluations. However, it consumes more VRAM.
TAGS
#transformers #llama #text-generation #dataset-OpenAssistant/oasst1 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

TAGS
#transformers #llama #text-generation #dataset-OpenAssistant/oasst1 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
