elisachen
/

bloom-GPTQquantized-model

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

elisachen commited on Mar 4, 2024

Commit

dd883e1

·

verified ·

1 Parent(s): 4450848

Update config.json

Set use_exllama to false

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -94,7 +94,7 @@
     "tokenizer": null,
     "true_sequential": true,
     "use_cuda_fp16": false,
-    "use_exllama": true
   },
   "skip_bias_add": true,
   "skip_bias_add_qkv": false,

     "tokenizer": null,
     "true_sequential": true,
     "use_cuda_fp16": false,
+    "use_exllama": false
   },
   "skip_bias_add": true,
   "skip_bias_add_qkv": false,