text
Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation (RAG)
new_line
text
Redeemer of the "hallucination problem"
Redeemer of the "hallucination problem"
new_line
new_line
text
It is fair enough to argue that "hallucinations" in LLMs are just mere reflections of what we humans occasionally do - well it gets worse as we get older, but these models are brain inspired, thus such behaviors are likely inherently unavoidable. After all, we are just dreamers trying make sense of this life.
It is fair enough to argue that "hallucinations" in LLMs are just mere reflections of what we humans occasionally do - well it gets worse as we get older, but these models are brain inspired, thus such behaviors are likely inherently unavoidable. After all, we are just dreamers trying make sense of this life.
new_line
new_line
text
The best we can do is minimize and control it - but humanly how? By first feeding on relevant facts and then developing a habit that allows us to easily access those facts when needed. This is what RAG is all about - it's just a control mechanism that keeps the LLM aligned with reality and fact.
The best we can do is minimize and control it - but humanly how? By first feeding on relevant facts and then developing a habit that allows us to easily access those facts when needed. This is what RAG is all about - it's just a control mechanism that keeps the LLM aligned with reality and fact.
new_line
new_line
text
But How Does RAG Work?
But How Does RAG Work?
new_line
new_line
text
Well, to some extent it is domain-specific but the overall workflow boils down to the following:
Well, to some extent it is domain-specific but the overall workflow boils down to the following:
new_line
new_line
text
1. It makes use of a retrieval mechanism that hunts for facts relevant to a query - this involves an end-to-end backpropagation that leverages a retriever (Query Encoder + Document Index or Source of Truth) with a pre-trained generative model.
1. It makes use of a retrieval mechanism that hunts for facts relevant to a query - this involves an end-to-end backpropagation that leverages a retriever (Query Encoder + Document Index or Source of Truth) with a pre-trained generative model.
new_line
new_line
text
2. The generative model then uses the facts retrieved, performs some verification to give a more accurate response.
2. The generative model then uses the facts retrieved, performs some verification to give a more accurate response.
new_line
new_line
text
To summarize, the RAG architecture houses a pre-existing knowledge source model (termed parametric memory), which then utilizes a Source-of-Truth model or vector indexed data (termed non-parametric memory) that is accessed by a pre-trained neural retriever, in order to produce more informed, contextually appropriate and factually correct responses.
To summarize, the RAG architecture houses a pre-existing knowledge source model (termed parametric memory), which then utilizes a Source-of-Truth model or vector indexed data (termed non-parametric memory) that is accessed by a pre-trained neural retriever, in order to produce more informed, contextually appropriate and factually correct responses.
new_line
new_line
text
Sort of a "Genius Engine" if you might say. If only we humans could harness such, AGI would be much much sooner lol.
Sort of a "Genius Engine" if you might say. If only we humans could harness such, AGI would be much much sooner lol.
new_line
new_line
text
In the meantime, I have been Jaward Sesay (Chinese name 苏杰 Sujie) - a young Sierra Leonean, aspiring AI Researcher. I like to read, share and try implementing AI research papers. Also like dunking on big tech while rooting for open-source. My mentor 
In the meantime, I have been Jaward Sesay (Chinese name 苏杰 Sujie) - a young Sierra Leonean, aspiring AI Researcher. I like to read, share and try implementing AI research papers. Also like dunking on big tech while rooting for open-source. My mentor 
mention
@karpathy
karpathy
text
, I dream of him following me back on X lol. Thanks.
, I dream of him following me back on X lol. Thanks.
new_line
Retrieval-Augmented Generation (RAG)
Redeemer of the "hallucination problem"

It is fair enough to argue that "hallucinations" in LLMs are just mere reflections of what we humans occasionally do - well it gets worse as we get older, but these models are brain inspired, thus such behaviors are likely inherently unavoidable. After all, we are just dreamers trying make sense of this life.

The best we can do is minimize and control it - but humanly how? By first feeding on relevant facts and then developing a habit that allows us to easily access those facts when needed. This is what RAG is all about - it's just a control mechanism that keeps the LLM aligned with reality and fact.

But How Does RAG Work?

Well, to some extent it is domain-specific but the overall workflow boils down to the following:

1. It makes use of a retrieval mechanism that hunts for facts relevant to a query - this involves an end-to-end backpropagation that leverages a retriever (Query Encoder + Document Index or Source of Truth) with a pre-trained generative model.

2. The generative model then uses the facts retrieved, performs some verification to give a more accurate response.

To summarize, the RAG architecture houses a pre-existing knowledge source model (termed parametric memory), which then utilizes a Source-of-Truth model or vector indexed data (termed non-parametric memory) that is accessed by a pre-trained neural retriever, in order to produce more informed, contextually appropriate and factually correct responses.

Sort of a "Genius Engine" if you might say. If only we humans could harness such, AGI would be much much sooner lol.

In the meantime, I have been Jaward Sesay (Chinese name 苏杰 Sujie) - a young Sierra Leonean, aspiring AI Researcher. I like to read, share and try implementing AI research papers. Also like dunking on big tech while rooting for open-source. My mentor @karpathy, I dream of him following me back on X lol. Thanks.

❤️
samusenps osanseviero paralym Jafta rohitdavas
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/WeO0AziKGQ9OLFjWmXfee.jpeg
Rohit Kumar
rohitdavas
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6438a9027de34e8ea7e4b257/vib8QSd1AWMr_bR9ig_xJ.jpeg
Jaward Sesay
Jaward
user