text
✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs
✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs
new_line
new_line
text
Paper: arxiv.org/abs/2401.09967
Paper: arxiv.org/abs/2401.09967
new_line
new_line
text
Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 
Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 
new_line
new_line
text
Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.
Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.
new_line
new_line
text
Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (
Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (
link
https://github.com/outlines-dev/outlines
https://github.com/outlines-dev/outlines
text
), but they assume access to logits.
), but they assume access to logits.
new_line
new_line
text
This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.
This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.
new_line
new_line
text
This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.
This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.
new_line
new_line
text
They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs

Paper: arxiv.org/abs/2401.09967

Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 

Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.

Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (https://github.com/outlines-dev/outlines), but they assume access to logits.

This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.

This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.

They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
❤️
s3nh clem osanseviero samusenps Dlbk AsadullaH777 merve yahma lunarflu sbarman25 Aseetpatel21