README.md

metadata

language:
  - en
tags:
  - 7B
  - mistralAI
  - v0.1
  - mistral
  - gguf

Steps to run model

1. install requirements

  pip install gguf
  pip install sentencepiece
  pip install typing
  pip install torch

2. For linux: use utorrent to download the model weights from magnet link

Commands

      sudo snap install utorrent
      utorrent

This opens utorrent UI, click "Add Torrent from URL" and paste

magnet:?xt=urn:btih:208b101a0f51514ecf285885a8b0f6fb1a1e4d7d&dn=mistral-7B-v0.1&tr=udp%3A%2F%http://2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=https%3A%2F%http://2Ftracker1.520.jp%3A443%2Fannounce

3. use convert.py script from https://github.com/ggerganov/llama.cpp/blob/master/convert.py for

python3 convert.py --outtype f16 "/path/to/mistral-7B-v0.1"

4. install ollama (link: https://github.com/jmorganca/ollama)

curl https://ollama.ai/install.sh | sh

(Hint: if there are errors with the above command, try uninstalling curl from snap and install with apt)

sudo snap remove curl
sudo apt install curl

modelfile for ollama

echo "FROM /path/to/mistral-7B-v0.1/ggml-model-f16.gguf" > modelfile

run ollama

ollama create mistral -f modelfile

make the folder that contains modelfile and downloaded mistral model weights traversable with a+x

sudo chmod -R a+x /path/to/mistral-7B-v0.1
ollama create name -f /path/to/mistral-7B-v0.1/modelfile

ollama run modelfile
ollama run mistral