File size: 2,578 Bytes
402ba77 3d270b8 7e77200 3d270b8 8639bf9 dfe3709 8639bf9 3a43bae |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
datasets:
- vicgalle/alpaca-gpt4
base_model:
- mistralai/Mistral-7B-v0.3
---
## Model Fine-Tuning
This model was fine-tuned using supervised fine-tuning (SFT) with the following key configuration details:
### Pre-trained Model
- **Model**: `mistralai/Mistral-7B-v0.3`
- **Tokenizer**: Corresponding tokenizer for the Mistral-7B model.
### Dataset
- **Dataset**: `vicgalle/alpaca-gpt4`
- **Subset Size**: First 1,000 examples.
### Training Parameters
- **Epochs**: 1
- **Batch Size**: 8 (with gradient accumulation of 2)
- **Learning Rate**: 2e-4
- **Optimizer**: `paged_adamw_8bit`
- **Weight Decay**: 0.001
- **Max Grad Norm**: 0.3
- **Warm-up Ratio**: 0.3
- **Gradient Accumulation**: 2 steps
- **FP16/BF16**: Disabled
- **Max Steps**: -1 (training will stop when dataset is exhausted)
- **Scheduler**: Linear learning rate scheduler
- **Monitoring**: Weights & Biases (`wandb`)
### PEFT Configuration
The model uses **LoRA (Low-Rank Adaptation)** for efficient fine-tuning with the following configuration:
- **lora_alpha**: 8
- **lora_dropout**: 0.1
- **r**: 16
- **Bias**: "none"
- **Task Type**: `CAUSAL_LM`
- **Target Modules**: `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj"]`
### Quantization Configuration
The model is quantized to **4-bit** using `BitsAndBytes` to reduce memory usage for training and inference:
- **Load in 4-bit**: Yes
- **Quantization Type**: `nf4`
- **Compute Data Type**: `float16`
### Training Environment
- **Platform**: Google Colab (GPU enabled)
- **Monitoring**: Weights & Biases (W&B)
### Model Saving and Upload
- **Fine-tuned Model**: Saved to Hugging Face repository: `nicksnlp/shrimp`
- **Tokenizer**: Pushed to Hugging Face repository.
### Example Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load the fine-tuned model
model = PeftModel.from_pretrained("nicksnlp/shrimp")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.3")
# Example inference
prompt = "What is Newton's 3rd Law and its formula?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### Code references:
- [Aarne Talman, aarnetalman: supervised_finetuning.ipynb](https://github.com/Helsinki-NLP/LLM-course-2024/tree/main/week-4/supervised_finetuning.ipynb)
- [Brett Young, byyoung3: Fine-Tuning-Mistral-7B-on-Python-Code-With-A-Single-GPU](https://wandb.ai/byyoung3/ml-news/reports/Fine-Tuning-Mistral-7B-on-Python-Code-With-A-Single-GPU---Vmlldzo1NTg0NzY5) |