text
𝐍𝐞𝐰 𝐠𝐮𝐢𝐝𝐞 𝐢𝐧 𝐨𝐮𝐫 𝐎𝐩𝐞𝐧-𝐒𝐨𝐮𝐫𝐜𝐞 𝐀𝐈 𝐜𝐨𝐨𝐤𝐛𝐨𝐨𝐤: 𝙎𝙩𝙧𝙪𝙘𝙩𝙪𝙧𝙚𝙙 𝙜𝙚𝙣𝙚𝙧𝙖𝙩𝙞𝙤𝙣! ✨
𝐍𝐞𝐰 𝐠𝐮𝐢𝐝𝐞 𝐢𝐧 𝐨𝐮𝐫 𝐎𝐩𝐞𝐧-𝐒𝐨𝐮𝐫𝐜𝐞 𝐀𝐈 𝐜𝐨𝐨𝐤𝐛𝐨𝐨𝐤: 𝙎𝙩𝙧𝙪𝙘𝙩𝙪𝙧𝙚𝙙 𝙜𝙚𝙣𝙚𝙧𝙖𝙩𝙞𝙤𝙣! ✨
new_line
new_line
text
Many use LLM use cases involve generating outputs with a specific structure. 
Many use LLM use cases involve generating outputs with a specific structure. 
new_line
new_line
text
➡️ For instance when using an LLM as a judge to evaluate another model's outputs, you need it to give you not only a score, but also the rationale for this score, and maybe a confidence level.
➡️ For instance when using an LLM as a judge to evaluate another model's outputs, you need it to give you not only a score, but also the rationale for this score, and maybe a confidence level.
new_line
text
So you do not need only "score: 1", but more a dictionary like:
So you do not need only "score: 1", but more a dictionary like:
new_line
code_fence
```
{
     "rationale": "The answer does not match the true answer at all."
     "score": 1,
     "confidence_level": 0.85
}
```
{
     "rationale": "The answer does not match the true answer at all."
     "score": 1,
     "confidence_level": 0.85
}
new_line
new_line
text
🤔 How to force your LLM to generate such a structured output?
🤔 How to force your LLM to generate such a structured output?
new_line
new_line
text
🏗️ 𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 is a great technique to generate structured output: you can specify a grammar (=set of rules) that the output should follow, and 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 𝘁𝗵𝗲𝗻 𝗳𝗼𝗿𝗰𝗲𝘀 𝘁𝗵𝗲 𝗱𝗲𝗰𝗼𝗱𝗲𝗿 𝘁𝗼 𝗼𝗻𝗹𝘆 𝗽𝗶𝗰𝗸 𝘁𝗼𝗸𝗲𝗻𝘀 𝘁𝗵𝗮𝘁 𝗿𝗲𝘀𝗽𝗲𝗰𝘁 𝘆𝗼𝘂𝗿 𝗴𝗿𝗮𝗺𝗺𝗮𝗿.
🏗️ 𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 is a great technique to generate structured output: you can specify a grammar (=set of rules) that the output should follow, and 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 𝘁𝗵𝗲𝗻 𝗳𝗼𝗿𝗰𝗲𝘀 𝘁𝗵𝗲 𝗱𝗲𝗰𝗼𝗱𝗲𝗿 𝘁𝗼 𝗼𝗻𝗹𝘆 𝗽𝗶𝗰𝗸 𝘁𝗼𝗸𝗲𝗻𝘀 𝘁𝗵𝗮𝘁 𝗿𝗲𝘀𝗽𝗲𝗰𝘁 𝘆𝗼𝘂𝗿 𝗴𝗿𝗮𝗺𝗺𝗮𝗿.
new_line
new_line
text
I've created a guide to show you how to use it, both via our Inference API and locally using 𝘰𝘶𝘵𝘭𝘪𝘯𝘦𝘴!
I've created a guide to show you how to use it, both via our Inference API and locally using 𝘰𝘶𝘵𝘭𝘪𝘯𝘦𝘴!
new_line
new_line
text
👉 Read it here: 
👉 Read it here: 
link
https://huggingface.co/learn/cookbook/structured_generation
https://huggingface.co/learn/cookbook/structured_generation
new_line
new_line
text
Thank you 
Thank you 
mention
@stevhliu
stevhliu
text
 for your great help in improving it!
 for your great help in improving it!
𝐍𝐞𝐰 𝐠𝐮𝐢𝐝𝐞 𝐢𝐧 𝐨𝐮𝐫 𝐎𝐩𝐞𝐧-𝐒𝐨𝐮𝐫𝐜𝐞 𝐀𝐈 𝐜𝐨𝐨𝐤𝐛𝐨𝐨𝐤: 𝙎𝙩𝙧𝙪𝙘𝙩𝙪𝙧𝙚𝙙 𝙜𝙚𝙣𝙚𝙧𝙖𝙩𝙞𝙤𝙣! ✨

Many use LLM use cases involve generating outputs with a specific structure. 

➡️ For instance when using an LLM as a judge to evaluate another model's outputs, you need it to give you not only a score, but also the rationale for this score, and maybe a confidence level.
So you do not need only "score: 1", but more a dictionary like:
```
{
     "rationale": "The answer does not match the true answer at all."
     "score": 1,
     "confidence_level": 0.85
}
```

🤔 How to force your LLM to generate such a structured output?

🏗️ 𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 is a great technique to generate structured output: you can specify a grammar (=set of rules) that the output should follow, and 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 𝘁𝗵𝗲𝗻 𝗳𝗼𝗿𝗰𝗲𝘀 𝘁𝗵𝗲 𝗱𝗲𝗰𝗼𝗱𝗲𝗿 𝘁𝗼 𝗼𝗻𝗹𝘆 𝗽𝗶𝗰𝗸 𝘁𝗼𝗸𝗲𝗻𝘀 𝘁𝗵𝗮𝘁 𝗿𝗲𝘀𝗽𝗲𝗰𝘁 𝘆𝗼𝘂𝗿 𝗴𝗿𝗮𝗺𝗺𝗮𝗿.

I've created a guide to show you how to use it, both via our Inference API and locally using 𝘰𝘶𝘵𝘭𝘪𝘯𝘦𝘴!

👉 Read it here: https://huggingface.co/learn/cookbook/structured_generation

Thank you @stevhliu for your great help in improving it!
🚀
tomaarsen stevhliu
🧠
trollek