How to Finetune phi-3 on MacBook Pro

<p>

In this blog, i'll show you how you can train/finetune the latest phi-3 model from Microsoft on your macbook pro! You'll need an M1 or M2 mac to do this. We will be using AutoTrain Advanced!</p>
<p>To install AutoTrain Advanced, you can do:</p>
<pre><code class="language-bash">$ pip install autotrain-advanced
</code></pre>
<p>Note: autotrain doesnt install pytorch, torchvision etc. So, you need to install it yourself. You can create a conda environment and install these dependencies:</p>
<pre><code class="language-bash">$ conda create -n autotrain python=3.10
$ conda activate autotrain
$ conda install pytorch::pytorch torchvision torchaudio -c pytorch
$ pip install autotrain-advanced
</code></pre>
<p>Once done, you can use the AutoTrain CLI or the UI on your mac machine! We will take a look at both!</p>
<p>AutoTrain not only offers LLM finetuning but many other tasks such as text classification, image classification, dreambooth lora, etc. But in this blog post, we are looking into LLM finetuning.</p>
<p>You can see all parameters you can adjust for llm finetuning by doing</p>
<pre><code class="language-bash">$ autotrain llm --<span class="hljs-built_in">help</span>
</code></pre>
<p>The next step is grabbing the data. In this blog I'm going to show you how you can train on your macbook with SFT training and ORPO tuning (the big but small brother of DPO).</p>
<ul>
<li>For SFT training, we need a dataset with a single text column. We can use something like <a href="https://huggingface.co/datasets/timdettmers/openassistant-guanaco">timdettmers/openassistant-guanaco</a> or alpaca like datasets. Note: these datasets are already formatted as text with system prompt, user instruction and assistant message. If you have them in a format like the following:</li>
</ul>
<pre><code>[ { "content": "Definition: In this task, you need to count the number of vowels (letters 'a', 'e', 'i', 'o', 'u') / consonants (all letters other than vowels) in the given sentence.\nInput: Sentence: 'a baseball player is in his hitting stance as a few people watch'. Count the number of consonants in the given sentence.\nOutput:", "role": "user" }, { "content": "32", "role": "assistant" } ]
</code></pre>
<p>you can use AutoTrain's chat-template parameter. We will see it later in this post, but for ORPO training. So, we will cover SFT training using pre-formatted dataset and do ORPO training with chat template.</p>
<ul>
<li>For ORPO training, you can use a dataset like <a href="https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized">argilla/distilabel-capybara-dpo-7k-binarized</a>. This dataset has many columns but we are interested only in <code>chosen</code> &amp; <code>rejected</code> columns.</li>
</ul>
<p>With AutoTrain, only creating or finding the dataset will be the most time consuming part. Now, when we have the dataset, we can do a SFT training using:</p>
<pre><code class="language-bash">autotrain llm \
--train \
--model microsoft/Phi-3-mini-4k-instruct \
--data-path timdettmers/openassistant-guanaco \
--lr 2e-4 \
--batch-size 2 \
--epochs 1 \
--trainer sft \
--peft \
--project-name my-own-phi-3-on-mac \
--username abhishek \
--push-to-hub \
--token <span class="hljs-variable">$HF_TOKEN</span>
</code></pre>
<p>Where $HF_TOKEN is your hugging face write token in case you want to push the trained model to hugging face hub for easy deployment and sharing. You can find your tokens <a href="https://huggingface.co/settings/tokens">here</a>.</p>
<p>Note that, we are using lora and thats why we have the <code>--peft</code> parameter. Also, if the text column is not called <code>text</code> in your dataset, you can add another parameter <code>--text-column your_datasets_text_column</code>. In case you want to use your own CSV/JSON file instead of a hugging face hub dataset, you can call it train.csv / train.jsonl and place it in a local folder. Command to train will change slightly:</p>
<pre><code class="language-bash">autotrain llm \
--train \
--model microsoft/Phi-3-mini-4k-instruct \
--data-path /path/to/folder/containing/training/file \
--text-column text_column_in_your_dataset \
--lr 2e-4 \
--batch-size 2 \
--epochs 1 \
--trainer sft \
--peft \
--project-name my-own-phi-3-on-mac \
--username abhishek \
--push-to-hub \
--token <span class="hljs-variable">$HF_TOKEN</span>
</code></pre>
<p>Next, we come to orpo training. For orpo training we change <code>--trainer sft</code> to <code>--trainer orpo</code>.</p>
<pre><code class="language-bash">autotrain llm \
--train \
--model microsoft/Phi-3-mini-4k-instruct \
--data-path argilla/distilabel-capybara-dpo-7k-binarized \
--text-column chosen \
--rejected-text-column rejected \
--lr 2e-4 \
--batch-size 2 \
--epochs 1 \
--trainer orpo \
--chat-template chatml \
--peft \
--project-name my-own-phi-3-on-mac-orpo \
--username abhishek \
--push-to-hub \
--token <span class="hljs-variable">$HF_TOKEN</span>
</code></pre>
<p>There are 4 changes above. Its only the column mappings that have changed and the trainer and ofcourse, the dataset. One other major change is usage of <code>--chat-template</code> parameter which is set to <code>chatml</code>. For <code>--chat-template</code>, the choices are: <code>zephyr</code>, <code>chatml</code>, <code>tokenizer</code> or None. None is used if you already formatted the data properly yourself as we did in SFT training.</p>
<p>Now if CLI is too much for you, you can also use the UI! Which is even easier and allows you to upload files too.</p>
<p>To use the UI:</p>
<pre><code class="language-bash">$ <span class="hljs-built_in">export</span> HF_TOKEN=your_huggingface_write_token

$ autotrain app --host 127.0.0.1 --port 10000
</code></pre>
<p>Then go to <a href="http://127.0.0.1:10000" rel="nofollow">http://127.0.0.1:10000</a> in your browser and enjoy AutoTrain UI! 🚀 A screenshot with same params as for ORPO training above is shown below:</p>
<p><a href="https://cdn-uploads.huggingface.co/production/uploads/5fa19f4ba13e063b8b2b5e11/-YLNMBbo4mer1zytmuyDs.png" rel="nofollow"><img alt="image/png" src="https://cdn-uploads.huggingface.co/production/uploads/5fa19f4ba13e063b8b2b5e11/-YLNMBbo4mer1zytmuyDs.png"/></a></p>
<p>If you are not able to find phi3 in the model dropdown list, you can use this URL instead: <code>http://127.0.0.1:7860/?custom_models=microsoft/Phi-3-mini-4k-instruct</code>. Note: I've added phi-3 as custom model. You can do the same for any other compatible model from the Hub. ;)</p>
<p>Both SFT and ORPO trainings were successfully tested on a M2 Max MacBook Pro.</p>
<p>In case of any issues or feature requests, use <a href="https://github.com/huggingface/autotrain-advanced" rel="nofollow">github repo</a>.</p>
<p>P.S. An extensive documentation of AutoTrain can be found <a href="https://hf.co/docs/autotrain" rel="nofollow">here</a></p>
<p>Happy (auto)training! 🚀 </p>
