有办法量化吗

by zxgov - opened Jun 16, 2023

zxgov

Jun 16, 2023

如题

Owner Jun 20, 2023

可以的。
我使用AutoGPTQ实现了4-bits量化，可参考：baichuan-vicuna-chinese-7b-gptq
AutoGPTQ: https://github.com/PanQiWei/AutoGPTQ

xyzhou

Jun 20, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment