text
GPU Poor POV: Building a RAG which solves specific task.
GPU Poor POV: Building a RAG which solves specific task.
new_line
new_line
text
Everyone loves benchmarks.
Everyone loves benchmarks.
new_line
text
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.
new_line
new_line
text
I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 
I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 
new_line
new_line
text
Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
new_line
text
Something like:
Something like:
new_line
new_line
resource
https://huggingface.co/BAAI/llm-embedder
model
BAAI/llm-embedder
https://huggingface.co/BAAI/llm-embedder
new_line
new_line
text
LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 
LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 
new_line
new_line
text
Have a great day and think about problem which your models have to solve <3
Have a great day and think about problem which your models have to solve <3
GPU Poor POV: Building a RAG which solves specific task.

Everyone loves benchmarks.
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.

I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 

Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
Something like:

https://huggingface.co/BAAI/llm-embedder

LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 

Have a great day and think about problem which your models have to solve <3
👍
100rab RASMUS victor merve osanseviero tomaarsen samusenps clem Citaman lunarflu Tanvir1337 sethuiyer arpitingle Ji-Ha
❤️
merve samusenps akashicmarga clem lunarflu TuringsSolutions Tanvir1337 sbrandeis
🤗
victor merve clem lunarflu TuringsSolutions Tanvir1337 s3nh
https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png
Omar Sanseviero
osanseviero
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6527e89a8808d80ccff88b7a/CuGNmF1Et8KMQ0mCd1NEJ.jpeg
Lain
not-lain
user