text
Repurposed my older AI workstation to a homelab server, it has received 2xV100 + 1xP40
Repurposed my older AI workstation to a homelab server, it has received 2xV100 + 1xP40
new_line
text
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
new_line
text
Also able to Lora finetune with similar performace as an RTX3090.
Also able to Lora finetune with similar performace as an RTX3090.
new_line
text
It moved to the garage to no complaints for the noise from the family. Will  move to a Rack soon :D
It moved to the garage to no complaints for the noise from the family. Will  move to a Rack soon :D
new_line
Repurposed my older AI workstation to a homelab server, it has received 2xV100 + 1xP40
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
Also able to Lora finetune with similar performace as an RTX3090.
It moved to the garage to no complaints for the noise from the family. Will  move to a Rack soon :D

🚀
John6666
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/6aIIy7eNLLjepZfp3Aym3.png
George M
ge-or-ge
user
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6d37e02dee9bcb9d9fa18/os24VYiNCoyth9yQSdv_A.jpeg
Csaba  Kecskemeti
csabakecskemeti
user