
# LLaMA 7B trained on the ru_turbo_alpaca, Russian instructions dataset

**Important**: You should probably use [Saiga](https://huggingface.co/IlyaGusev/saiga_7b_lora). It has regular updates and should be better in every task.

Based on [LLaMA 7B](https://huggingface.co/decapoda-research/llama-7b-hf).

* This is an adapter-only version.
* Merged version: [link](https://huggingface.co/IlyaGusev/llama_7b_ru_turbo_alpaca_lora_merged).
* llama.cpp version: [link](https://huggingface.co/IlyaGusev/llama_7b_ru_turbo_alpaca_lora_llamacpp).

April 2, 2023: A new model version was uploaded, trained with outputs capped at 512 tokens instead of 256. Also, alpaca-cleaned was mixed with ru_turbo_alpaca. To use the old version, specify this:
```
revision="dd4b2c63d6639fd9490cd1d9b0d6d8b58e708d93"
```

Colab: [link](https://colab.research.google.com/drive/1JLoHOjDJQIa8SDqsEXrGHj4Z4aTnaajN)

Training code: [link](https://github.com/IlyaGusev/rulm/tree/master/self_instruct)

```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

MODEL_NAME = "IlyaGusev/llama_7b_ru_turbo_alpaca_lora"

config = PeftConfig.from_pretrained(MODEL_NAME)
model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    load_in_8bit=True,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, MODEL_NAME)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
```

Examples:
```
Вопрос: Почему трава зеленая? 
Выход: Трава зеленой из-за того, что она содержит хлорофиллы, пигменты, которые помогают ей фотосинтезировать энергию из солнечного света. Хлорофилл способен перерабатывать углекислый газ и воду в органические вещества, такие как углеводы, аминокислоты и жиры, которые необходимы растениям для их роста и развития.
```

```
Задание: Сочини длинный рассказ, обязательно упоминая следующие объекты.
Вход: Таня, мяч
Выход: Таня любила футбол с детства. Она играла на улице со своими друзьями и мечтала стать профессиональным футболистом. Когда ей исполнилось 12 лет, она начала тренироваться в футбольной школе. Она была очень талантливой и упорной, и вскоре ей удалось попасть в молодежную команду.
Таня играла в футбол каждый день, тренируясь и участвуя в матчах. Она старалась улучшать свои навыки и стать лучшей футболисткой в своей команде. Ее талант не остался незамеченным, и ей предложили попробовать свои силы в более серьезной команде.
Она приняла предложение и начала играть за взрослую команду.
```

* Final loss: 1.132
* Time: 35 hours, 34 minutes
* Hardware: 1x RTX 3090
{"language": ["ru", "en"], "datasets": ["IlyaGusev/ru_turbo_alpaca", "yahma/alpaca-cleaned"], "pipeline_tag": "text2text-generation", "inference": false}
text2text-generation ru en dataset:IlyaGusev/ru_turbo_alpaca dataset:yahma/alpaca-cleaned has_space region:us
TAGS
#text2text-generation #ru #en #dataset-IlyaGusev/ru_turbo_alpaca #dataset-yahma/alpaca-cleaned #has_space #region-us 


# LLaMA 7B trained on the ru_turbo_alpaca, Russian instructions dataset

Important: You should probably use Saiga. It has regular updates and should be better in every task.

Based on LLaMA 7B.

* This is an adapter-only version.
* Merged version: link.
* URL version: link.

April 2, 2023: A new model version was uploaded, trained with outputs capped at 512 tokens instead of 256. Also, alpaca-cleaned was mixed with ru_turbo_alpaca. To use the old version, specify this:


Colab: link

Training code: link



Examples:




* Final loss: 1.132
* Time: 35 hours, 34 minutes
* Hardware: 1x RTX 3090
# LLaMA 7B trained on the ru_turbo_alpaca, Russian instructions dataset

Important: You should probably use Saiga. It has regular updates and should be better in every task.

Based on LLaMA 7B.

* This is an adapter-only version.
* Merged version: link.
* URL version: link.

April 2, 2023: A new model version was uploaded, trained with outputs capped at 512 tokens instead of 256. Also, alpaca-cleaned was mixed with ru_turbo_alpaca. To use the old version, specify this:


Colab: link

Training code: link



Examples:




* Final loss: 1.132
* Time: 35 hours, 34 minutes
* Hardware: 1x RTX 3090
TAGS
#text2text-generation #ru #en #dataset-IlyaGusev/ru_turbo_alpaca #dataset-yahma/alpaca-cleaned #has_space #region-us 
 # LLaMA 7B trained on the ru_turbo_alpaca, Russian instructions dataset

Important: You should probably use Saiga. It has regular updates and should be better in every task.

Based on LLaMA 7B.

* This is an adapter-only version.
* Merged version: link.
* URL version: link.

April 2, 2023: A new model version was uploaded, trained with outputs capped at 512 tokens instead of 256. Also, alpaca-cleaned was mixed with ru_turbo_alpaca. To use the old version, specify this:


Colab: link

Training code: link



Examples:




* Final loss: 1.132
* Time: 35 hours, 34 minutes
* Hardware: 1x RTX 3090
TAGS
#text2text-generation #ru #en #dataset-IlyaGusev/ru_turbo_alpaca #dataset-yahma/alpaca-cleaned #has_space #region-us 
# LLaMA 7B trained on the ru_turbo_alpaca, Russian instructions dataset

Important: You should probably use Saiga. It has regular updates and should be better in every task.

Based on LLaMA 7B.

* This is an adapter-only version.
* Merged version: link.
* URL version: link.

April 2, 2023: A new model version was uploaded, trained with outputs capped at 512 tokens instead of 256. Also, alpaca-cleaned was mixed with ru_turbo_alpaca. To use the old version, specify this:


Colab: link

Training code: link



Examples:




* Final loss: 1.132
* Time: 35 hours, 34 minutes
* Hardware: 1x RTX 3090