# Polpaca: The Alpaca Speaks Polish

The blogpost: https://medium.com/@mmosiolek/can-alpacas-learn-languages-df48a03b6d8


[LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/) is a state-of-the-art, foundational, open-source large language model designed to help engineers and researchers advance their work in NLP. 
For example, Stanford researchers have fine-tuned [LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/) to construct an alternative to the famous ChatGPT - a model called [Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html).
Unfortunately, [LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/) was trained on a dataset consisting mainly of English texts, with only 4.5% of the data relating to other languages.
In addition, the [Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html) instruction training dataset consists only of examples of English instructions.
So [Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html) simply doesn't work for the other languages.

This repo makes [Alpaca-Lora-7B](https://huggingface.co/tloen/alpaca-lora-7b) speak Polish. 

### Usage

```python
from transformers import LlamaTokenizer, LlamaForCausalLM
from peft import PeftModel
import bitsandbytes as bnb

base = "decapoda-research/llama-7b-hf"
finetuned = "mmosiolek/polpaca-lora-7b"

tokenizer = LlamaTokenizer.from_pretrained(base)
tokenizer.pad_token_id = 0
tokenizer.padding_side = "left"

model = LlamaForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, finetuned).to("cuda")
```

For output generation use the following code. Feel free to adjust the parameters according to your needs.


```python
from transformers import GenerationConfig

config = GenerationConfig(
  temperature=0.1,
  top_p=0.5,
  top_k=40,
  num_beams=4,
  max_new_tokens=128,
  repetition_penalty=1.2
)

def run(instruction, model, tokenizer):
    encodings = tokenizer(instruction, padding=True, return_tensors="pt").to('cuda')
    generated_ids = model.generate(
        **encodings,
        generation_config=GENERATION_CONFIG,
    )
    decoded = tokenizer.batch_decode(generated_ids)
    del encodings, generated_ids
    torch.cuda.empty_cache()
    return decoded[0].split("\n")[-1]
```

### Example input/output
instruction:
```
Wymyśl kilka zapytań w google na temat kodowania.
```

output
```
Jaki jest najpopularniejszy język programowania?
Jaki jest najbardziej powszechnie używany język programowania?
Jaki jest najnowszy język programowania?
Jaki jest najbardziej efektywny język programowania?
```
---
instruction
```
Zrób listę pożądanych umiejętności inżynierów oprogramowania, którą możesz dodać do LinkedIn.
```
output
```
Pożądane umiejętności dla inżyniera oprogramowania obejmują:
- Programowanie w językach takich jak Java, C++, C#, Python, JavaScript i HTML
- Wnioskowanie logiczne
- Rozwiązywanie problemów
- Inżynieria oprogramowania
- Uczenie maszynowe
- Przetwarzanie języka naturalnego (NLP)
- Systemy zarządzania bazami danych (DBMS)
- Architektura
```
---
instruction
```
W odniesieniu do danego scenariusza pogody podaj kilka wskazówek, jak dostosować do niego plany podróży.
```

input
```
nagłą zmianę temperatury
```

output
```
Kilka wskazówek dotyczących dostosowania planów podróży w przypadku nagłej zmiany temperatury obejmują:
1. Upewnij się, że masz odpowiednie ubrania i akcesoria, takie jak krem przeciwsłoneczny, kapelusze i okulary przeciwsłoneczne.
2. Upewnij się, że masz wystarczającą ilość płynów, takich jak woda i napoje gazowane.
```
{"language": ["pl"], "license": "apache-2.0", "tags": ["alpaca", "llama", "self-instruct", "casual language model", "llm", "gpt", "chat-gpt"], "datasets": ["mmosiolek/pl_alpaca_data_cleaned"], "pipeline_tag": "question-answering"}
alpaca llama self-instruct casual language model llm gpt chat-gpt question-answering pl dataset:mmosiolek/pl_alpaca_data_cleaned license:apache-2.0 region:us
TAGS
#alpaca #llama #self-instruct #casual language model #llm #gpt #chat-gpt #question-answering #pl #dataset-mmosiolek/pl_alpaca_data_cleaned #license-apache-2.0 #region-us 

# Polpaca: The Alpaca Speaks Polish

The blogpost: URL


LLaMA is a state-of-the-art, foundational, open-source large language model designed to help engineers and researchers advance their work in NLP. 
For example, Stanford researchers have fine-tuned LLaMA to construct an alternative to the famous ChatGPT - a model called Alpaca.
Unfortunately, LLaMA was trained on a dataset consisting mainly of English texts, with only 4.5% of the data relating to other languages.
In addition, the Alpaca instruction training dataset consists only of examples of English instructions.
So Alpaca simply doesn't work for the other languages.

This repo makes Alpaca-Lora-7B speak Polish. 

### Usage



For output generation use the following code. Feel free to adjust the parameters according to your needs.




### Example input/output
instruction:


output

---
instruction

output

---
instruction


input


output

# Polpaca: The Alpaca Speaks Polish

The blogpost: URL


LLaMA is a state-of-the-art, foundational, open-source large language model designed to help engineers and researchers advance their work in NLP. 
For example, Stanford researchers have fine-tuned LLaMA to construct an alternative to the famous ChatGPT - a model called Alpaca.
Unfortunately, LLaMA was trained on a dataset consisting mainly of English texts, with only 4.5% of the data relating to other languages.
In addition, the Alpaca instruction training dataset consists only of examples of English instructions.
So Alpaca simply doesn't work for the other languages.

This repo makes Alpaca-Lora-7B speak Polish. ### Usage



For output generation use the following code. Feel free to adjust the parameters according to your needs. ### Example input/output
instruction:


output

---
instruction

output

---
instruction


input


output
TAGS
#alpaca #llama #self-instruct #casual language model #llm #gpt #chat-gpt #question-answering #pl #dataset-mmosiolek/pl_alpaca_data_cleaned #license-apache-2.0 #region-us 
 # Polpaca: The Alpaca Speaks Polish

The blogpost: URL


LLaMA is a state-of-the-art, foundational, open-source large language model designed to help engineers and researchers advance their work in NLP. 
For example, Stanford researchers have fine-tuned LLaMA to construct an alternative to the famous ChatGPT - a model called Alpaca.
Unfortunately, LLaMA was trained on a dataset consisting mainly of English texts, with only 4.5% of the data relating to other languages.
In addition, the Alpaca instruction training dataset consists only of examples of English instructions.
So Alpaca simply doesn't work for the other languages.

This repo makes Alpaca-Lora-7B speak Polish. ### Usage



For output generation use the following code. Feel free to adjust the parameters according to your needs. ### Example input/output
instruction:


output

---
instruction

output

---
instruction


input


output
TAGS
#alpaca #llama #self-instruct #casual language model #llm #gpt #chat-gpt #question-answering #pl #dataset-mmosiolek/pl_alpaca_data_cleaned #license-apache-2.0 #region-us 
# Polpaca: The Alpaca Speaks Polish

The blogpost: URL


LLaMA is a state-of-the-art, foundational, open-source large language model designed to help engineers and researchers advance their work in NLP. 
For example, Stanford researchers have fine-tuned LLaMA to construct an alternative to the famous ChatGPT - a model called Alpaca.
Unfortunately, LLaMA was trained on a dataset consisting mainly of English texts, with only 4.5% of the data relating to other languages.
In addition, the Alpaca instruction training dataset consists only of examples of English instructions.
So Alpaca simply doesn't work for the other languages.

This repo makes Alpaca-Lora-7B speak Polish.### Usage



For output generation use the following code. Feel free to adjust the parameters according to your needs.### Example input/output
instruction:


output

---
instruction

output

---
instruction


input


output