text
As a GPU poor, I found a nice open source project.
As a GPU poor, I found a nice open source project.
new_line
new_line
text
dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)
dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)
new_line
new_line
text
Provision VM instances in 3 different use cases. These are the essential for any ML projects.
Provision VM instances in 3 different use cases. These are the essential for any ML projects.
new_line
text
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
new_line
text
- Task: run experiments (training, fine-tuning, ...) via SSH 
- Task: run experiments (training, fine-tuning, ...) via SSH 
new_line
text
- Service: run your model in production via HTTPS
- Service: run your model in production via HTTPS
new_line
new_line
text
dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.
dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.
new_line
new_line
text
The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.
The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.
new_line
new_line
text
If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
new_line
text
: 
: 
link
https://dstack.ai/
https://dstack.ai/
new_line
new_line
text
If you are looking for an alternative, there is SkyPilot project as well
If you are looking for an alternative, there is SkyPilot project as well
new_line
text
: 
: 
link
https://github.com/skypilot-org/skypilot
https://github.com/skypilot-org/skypilot
As a GPU poor, I found a nice open source project.

dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)

Provision VM instances in 3 different use cases. These are the essential for any ML projects.
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
- Task: run experiments (training, fine-tuning, ...) via SSH 
- Service: run your model in production via HTTPS

dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.

The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.

If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
: https://dstack.ai/

If you are looking for an alternative, there is SkyPilot project as well
: https://github.com/skypilot-org/skypilot
❤️
peterschmidt85 chansung osanseviero lysandre merve victor sayakpaul linoyts abhishek abidlabs KnutJaegersberg auditor johannhartmann julien-c radames bitdeep samusenps mishig gblazex juyongjiang greencookies akhaliq Citaman sbrandeis
🤗
peterschmidt85 lysandre merve KnutJaegersberg julien-c pierrci samusenps juyongjiang akhaliq
👍
bitdeep juyongjiang akhaliq
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
sayakpaul
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1659971187637-60d3b57ad7b174177faabd6e.jpeg
chansung park
chansung
user
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user