text
Unlocking the Power of locally running Llama-3 8B Model Agents with Chat-UI! 🔥🚀✨
Unlocking the Power of locally running Llama-3 8B Model Agents with Chat-UI! 🔥🚀✨
new_line
new_line
text
I'm thrilled to share my hackathon-style side project: 
I'm thrilled to share my hackathon-style side project: 
new_line
text
1. Finetuning Llama-8B for function calling using PEFT QLoRA as the instruct Llama-3 model doesn't support this. The colab notebook for it is here: 
1. Finetuning Llama-8B for function calling using PEFT QLoRA as the instruct Llama-3 model doesn't support this. The colab notebook for it is here: 
link
https://lnkd.in/ggJMzqh2
https://lnkd.in/ggJMzqh2
text
. 🛠️
. 🛠️
new_line
text
2. Finetuned model along with the 4-bit quants here: 
2. Finetuned model along with the 4-bit quants here: 
link
https://lnkd.in/gNpFKY6V
https://lnkd.in/gNpFKY6V
text
 ✨
 ✨
new_line
text
3. Clone Hugging Face 
3. Clone Hugging Face 
link
https://lnkd.in/gKBKuUBQ
https://lnkd.in/gKBKuUBQ
text
 and make it compatible for function calling by building upon the PR 
 and make it compatible for function calling by building upon the PR 
link
https://lnkd.in/gnqFuAd4
https://lnkd.in/gnqFuAd4
text
 for my model and local inferencing usecase using Ollama. This was a steep learning curve wherein I stayed awake the whole night to get it working. 💪🏽
 for my model and local inferencing usecase using Ollama. This was a steep learning curve wherein I stayed awake the whole night to get it working. 💪🏽
new_line
text
4. Above, I used SerpAPI for web browsing and Mongo DB Atlas free tier for persistence of conversations and assistant configs. 🔎
4. Above, I used SerpAPI for web browsing and Mongo DB Atlas free tier for persistence of conversations and assistant configs. 🔎
new_line
text
5. More work is required to switch between using tools and responding directly wherein I see the model breaks. 🧐
5. More work is required to switch between using tools and responding directly wherein I see the model breaks. 🧐
new_line
new_line
text
How cool is this wherein we are approaching experience akin to ChatGPT while using local hosted agent model running on your laptop! 💻
How cool is this wherein we are approaching experience akin to ChatGPT while using local hosted agent model running on your laptop! 💻
new_line
Unlocking the Power of locally running Llama-3 8B Model Agents with Chat-UI! 🔥🚀✨

I'm thrilled to share my hackathon-style side project: 
1. Finetuning Llama-8B for function calling using PEFT QLoRA as the instruct Llama-3 model doesn't support this. The colab notebook for it is here: https://lnkd.in/ggJMzqh2. 🛠️
2. Finetuned model along with the 4-bit quants here: https://lnkd.in/gNpFKY6V ✨
3. Clone Hugging Face https://lnkd.in/gKBKuUBQ and make it compatible for function calling by building upon the PR https://lnkd.in/gnqFuAd4 for my model and local inferencing usecase using Ollama. This was a steep learning curve wherein I stayed awake the whole night to get it working. 💪🏽
4. Above, I used SerpAPI for web browsing and Mongo DB Atlas free tier for persistence of conversations and assistant configs. 🔎
5. More work is required to switch between using tools and responding directly wherein I see the model breaks. 🧐

How cool is this wherein we are approaching experience akin to ChatGPT while using local hosted agent model running on your laptop! 💻

🤯
zarazi Kukedlc Abhaykoul AlekseiPravdin Tonic victor RedSparkie jaocseverywhere johnbendi
🔥
DmitryRyumin Tonic Nayer-ali med4u alleniver kpi parsapico johnbendi
/avatars/b87f7255d0a3ff3b50389d916e747a5c.svg
Isara Settavittayanukit
zarazi
user