Compared with 32b
I've heard that the first generation of Command-R models were better at creative writing than the subsequent 08-2024 models, but it's hard to overlook the dramatically lower VRAM usage of those new models. I'm curious, in your experience, is the creative-writer 35b model still worth it over the 32b?
Thanks
I've heard that the first generation of Command-R models were better at creative writing than the subsequent 08-2024 models, but it's hard to overlook the dramatically lower VRAM usage of those new models. I'm curious, in your experience, is the creative-writer 35b model still worth it over the 32b?
Yeah, I still think the writing is "better" (very subjective though) and a bit more creative for the older 35b, but it does seem prone to making weird mistakes like occasionally outputting Chinese or Cyrillic (the original does this too and not just my fine-tunes!).
If you are going to use the 35b then it's always advisable to use some small value of min_p like 0.05 to avoid these weird outputs.
If you are short on VRAM then maybe try the "plus" variant of the 32b model I uploaded today - it should be quite a bit more creative, and AFAIK it doesn't seem to have been harmed by increasing its Entropy like some of my early experiments.