en es dataset:cerebras/SlimPajama-627B dataset:bigcode/starcoderdata dataset:sam-mosaic/orca-gpt4-chatml dataset:alvations/globalvoices-en-es license:apache-2.0 region:us
---
license: apache-2.0
datasets:
- cerebras/SlimPajama-627B
- bigcode/starcoderdata
- sam-mosaic/orca-gpt4-chatml
- alvations/globalvoices-en-es
language:
- en
- es
---
<div align="center">

  # TinyLlama-1.1B-translate-en-es

</div>

This is a finetuned version with a partial dataset from alvations/globalvoices-en-es to test performance on translation task. It has been trained to translate english to spanish and viceversa with only 20k rows from the dataset.

The translation is not very accurate but it shows a lot of potential.

In order to use it you have to follow the chatml standard like so:
---
english to spanish:
```
<|im_start|>user Translate this to spanish: ```A father and son, who have been living off grid for 20 years, encounter an outsider who threatens to destroy the utopia they've built.```
<|im_start|>assistant 
```
This will provide the following result:
```
Un padre y hijo, que han vivido sin comida desde hace 20 años, encuentran un invitado quien amenaza con destruir la utopía que ellos han creado.
```
---
spanish to english:
```
<|im_start|>user Traduce esto al ingles: ```España se queda sin Copilot para Windows 11: la regulación de la UE frena su despliegue en Europa.```
<|im_start|>assistant 
```
Which will be completed as:
```
Spain is left without Copilot for Windows 11: the control of the UE has halted its deployment in Europe.
```

---
The results are far from perfect but there are A LOT of room to improvement since it was finetuned with only 20k rows from the dataset (which has 355k rows) for 2 epoch. This training took only about 5 hours on a "M1 Pro" processor. 

The base model used is a fine-tuned model with orca dataset [acalatrava/TinyLlama-1.1B-orca-gpt4](https://huggingface.co/acalatrava/TinyLlama-1.1B-orca-gpt4)

### Training
 - **Method**: QLORA
 - **Time**: 5h on a M1 Pro 32GB
 - **Based on**: [https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g](https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g) removing quantization since it's not supported on MPS
