text
🚨 Now you can run Starcoder- 2 models locally on your Mac M1 Pro Apple Silicon with 16GB memory! 🧑🏽‍💻 ⚡️✨
🚨 Now you can run Starcoder- 2 models locally on your Mac M1 Pro Apple Silicon with 16GB memory! 🧑🏽‍💻 ⚡️✨
new_line
new_line
text
Below is the UX with Twinny extension using 
Below is the UX with Twinny extension using 
resource
https://huggingface.co/bigcode/starcoder2-3b
model
bigcode/starcoder2-3b
https://huggingface.co/bigcode/starcoder2-3b
text
 for FIM and 
 for FIM and 
resource
https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf
model
codellama/CodeLlama-7b-Instruct-hf
https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf
text
 for chat. Dev tools is showing the prompt being sent to ollama server.
 for chat. Dev tools is showing the prompt being sent to ollama server.
new_line
new_line
text
Starcoder-2 is now supported in 
Starcoder-2 is now supported in 
inline_code
`llama.cpp`
llama.cpp
text
link
https://github.com/ggerganov/llama.cpp/pull/5795
https://github.com/ggerganov/llama.cpp/pull/5795
text
!
!
new_line
code_fence
```
cd llama.cpp
python convert-hf-to-gguf.py ../starcoder2-3b/ --outfile models/starcoder2-3b.gguf --outtype "f16"
./quantize models/starcoder2-3b.gguf models/starcoder2-3b-Q4_K_M.gguf Q4_K_M
```
cd llama.cpp
python convert-hf-to-gguf.py ../starcoder2-3b/ --outfile models/starcoder2-3b.gguf --outtype "f16"
./quantize models/starcoder2-3b.gguf models/starcoder2-3b-Q4_K_M.gguf Q4_K_M
text
new_line
text
For more details, please go through the following tweet thread: 
For more details, please go through the following tweet thread: 
link
https://x.com/sourab_m/status/1764583139798823235?s=20
https://x.com/sourab_m/status/1764583139798823235?s=20
🚨 Now you can run Starcoder- 2 models locally on your Mac M1 Pro Apple Silicon with 16GB memory! 🧑🏽‍💻 ⚡️✨

Below is the UX with Twinny extension using https://huggingface.co/bigcode/starcoder2-3b for FIM and https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf for chat. Dev tools is showing the prompt being sent to ollama server.

Starcoder-2 is now supported in `llama.cpp`  https://github.com/ggerganov/llama.cpp/pull/5795!
```
cd llama.cpp
python convert-hf-to-gguf.py ../starcoder2-3b/ --outfile models/starcoder2-3b.gguf --outtype "f16"
./quantize models/starcoder2-3b.gguf models/starcoder2-3b-Q4_K_M.gguf Q4_K_M
``` 
For more details, please go through the following tweet thread: https://x.com/sourab_m/status/1764583139798823235?s=20
🤯
osanseviero lvwerra clem Kukedlc notune jshuadvd codito rjmacarthy
❤️
clem Kukedlc Yoben BrigitteTousi rjmacarthy
🤝
clem