text
GPU Poor POV: Low Hanging Fruits
GPU Poor POV: Low Hanging Fruits
new_line
new_line
new_line
text
Sometimes we had to work with different language than English (what a surprise!) and it can be problematic, because as you may know many algorithms are mainly developed in English. 
Sometimes we had to work with different language than English (what a surprise!) and it can be problematic, because as you may know many algorithms are mainly developed in English. 
new_line
text
I was involved in building RAG in Polish language. At first, we need an proper embeddings for Polish language to feed them into lightweight LLM. 
I was involved in building RAG in Polish language. At first, we need an proper embeddings for Polish language to feed them into lightweight LLM. 
new_line
text
Looking through possible solution I become aware that existing/possible models are not accurate enough, and worked much worse than its 'english equivalent'. 
Looking through possible solution I become aware that existing/possible models are not accurate enough, and worked much worse than its 'english equivalent'. 
new_line
text
First thing that comes to mind is:
First thing that comes to mind is:
new_line
code_fence
```
 Lets become a mad scientist, download all possible data and train model for months to get the proper one. 
```
 Lets become a mad scientist, download all possible data and train model for months to get the proper one. 
new_line
text
But there are few cons of this. 
But there are few cons of this. 
new_line
text
- Its computionally heavy
- Its computionally heavy
new_line
text
- You are not full time researcher 
- You are not full time researcher 
new_line
text
- you have potential clients who want to use your solution, and they really happy to use it (in optimistic mood). 
- you have potential clients who want to use your solution, and they really happy to use it (in optimistic mood). 
new_line
text
Here comes the low hanging fruits.
Here comes the low hanging fruits.
new_line
text
 We developed a easier, workable solution. Instead of training new SOTA, we can use translation module like this one:
 We developed a easier, workable solution. Instead of training new SOTA, we can use translation module like this one:
new_line
new_line
resource
https://huggingface.co/Helsinki-NLP/opus-mt-pl-en
model
Helsinki-NLP/opus-mt-pl-en
https://huggingface.co/Helsinki-NLP/opus-mt-pl-en
new_line
text
translate your knowledge base to english, and use proper embedding model accurately. 
translate your knowledge base to english, and use proper embedding model accurately. 
new_line
text
I converted existing model using ctranslate2, 
I converted existing model using ctranslate2, 
new_line
new_line
code_fence
```
ct2-transformers-converter --model Helsinki-NLP/opus-mt-pl-en --output_dir opus-mt-pl-en
```
ct2-transformers-converter --model Helsinki-NLP/opus-mt-pl-en --output_dir opus-mt-pl-en
new_line
text
so making an inference  is not heavy (we observe 5 times speedup in compare to original version).
so making an inference  is not heavy (we observe 5 times speedup in compare to original version).
new_line
new_line
text
And by indexing knowledge base, we can return answer to LLM  in any language. (Indexes of context found in english language are equal to indexes in native language knowledge base). 
And by indexing knowledge base, we can return answer to LLM  in any language. (Indexes of context found in english language are equal to indexes in native language knowledge base). 
new_line
new_line
text
Of course there are some tweaks required, we have to validate accuracy of the translation.
Of course there are some tweaks required, we have to validate accuracy of the translation.
new_line
new_line
text
It was nice episode, we have our work done, there are people who can use it, so additive value exists. 
It was nice episode, we have our work done, there are people who can use it, so additive value exists. 
new_line
text
Have a great day and I wish you more effective deploys! <3
Have a great day and I wish you more effective deploys! <3
new_line
GPU Poor POV: Low Hanging Fruits


Sometimes we had to work with different language than English (what a surprise!) and it can be problematic, because as you may know many algorithms are mainly developed in English. 
I was involved in building RAG in Polish language. At first, we need an proper embeddings for Polish language to feed them into lightweight LLM. 
Looking through possible solution I become aware that existing/possible models are not accurate enough, and worked much worse than its 'english equivalent'. 
First thing that comes to mind is:
```
 Lets become a mad scientist, download all possible data and train model for months to get the proper one. 
```
But there are few cons of this. 
- Its computionally heavy
- You are not full time researcher 
- you have potential clients who want to use your solution, and they really happy to use it (in optimistic mood). 
Here comes the low hanging fruits.
 We developed a easier, workable solution. Instead of training new SOTA, we can use translation module like this one:

https://huggingface.co/Helsinki-NLP/opus-mt-pl-en
translate your knowledge base to english, and use proper embedding model accurately. 
I converted existing model using ctranslate2, 

```
ct2-transformers-converter --model Helsinki-NLP/opus-mt-pl-en --output_dir opus-mt-pl-en
```
so making an inference  is not heavy (we observe 5 times speedup in compare to original version).

And by indexing knowledge base, we can return answer to LLM  in any language. (Indexes of context found in english language are equal to indexes in native language knowledge base). 

Of course there are some tweaks required, we have to validate accuracy of the translation.

It was nice episode, we have our work done, there are people who can use it, so additive value exists. 
Have a great day and I wish you more effective deploys! <3

👍
mlabonne Tonic victor Dlbk clem NeuralNovel Ji-Ha
❤️
afrideva MaziyarPanahi osanseviero clem saudi-agi
https://cdn-avatars.huggingface.co/v1/production/uploads/61caeda441f9432649f03ab6/0UdRCrzIqhedZblgfpMBk.png
s3nh
s3nh
user
https://cdn-avatars.huggingface.co/v1/production/uploads/5fd5e18a90b6dc4633f6d292/gZXHW5dd9R86AV9LMZ--y.png
Maziyar Panahi
MaziyarPanahi
user