Loading checkpoint shards very slow
Loading checkpoint shards is very slow. Same model and same machine, sometimes it takes less than 1 minute, but sometimes it takes more than 10 minutes. May I know what’s the reason and how to speed up the process?
2023-12-19T19:10:45Z
check out your RAM vs VRAM usage… is it loading to your ram or GPU vram. typical vram is faster.try quantization… it may be faster.