resource
https://huggingface.co/google/gemma-7b-it
model
google/gemma-7b-it
https://huggingface.co/google/gemma-7b-it
text
 is super good!
 is super good!
new_line
new_line
text
I wasn't convinced at first, but after vibe-checking it...I'm quite impressed.
I wasn't convinced at first, but after vibe-checking it...I'm quite impressed.
new_line
new_line
text
I've got a notebook here, which is kind of a framework for vibe-checking LLMs.
I've got a notebook here, which is kind of a framework for vibe-checking LLMs.
new_line
new_line
text
In this notebook, I take Gemma for a spin on a variety of prompts:
In this notebook, I take Gemma for a spin on a variety of prompts:
new_line
text
•  [nonsensical tokens](
•  [nonsensical tokens](
resource
https://huggingface.co/datasets/harpreetsahota/diverse-token-sampler]
dataset
harpreetsahota/diverse-token-sampler
https://huggingface.co/datasets/harpreetsahota/diverse-token-sampler]
new_line
text
• [conversation where I try to get some PII)(
• [conversation where I try to get some PII)(
resource
https://huggingface.co/datasets/harpreetsahota/red-team-prompts-questions
dataset
harpreetsahota/red-team-prompts-questions
https://huggingface.co/datasets/harpreetsahota/red-team-prompts-questions
text
)
)
new_line
text
• [summarization ability](
• [summarization ability](
resource
https://huggingface.co/datasets/lighteval/summarization
dataset
lighteval/summarization
https://huggingface.co/datasets/lighteval/summarization
text
)
)
new_line
text
• [instruction following](
• [instruction following](
resource
https://huggingface.co/datasets/harpreetsahota/Instruction-Following-Evaluation-for-Large-Language-Models]
dataset
harpreetsahota/Instruction-Following-Evaluation-for-Large-Language-Models
https://huggingface.co/datasets/harpreetsahota/Instruction-Following-Evaluation-for-Large-Language-Models]
new_line
text
• [chain of thought reasoning](
• [chain of thought reasoning](
resource
https://huggingface.co/datasets/ssbuild/alaca_chain-of-thought
dataset
ssbuild/alaca_chain-of-thought
https://huggingface.co/datasets/ssbuild/alaca_chain-of-thought
text
)
)
new_line
new_line
text
I then used LangChain evaluators (GPT-4 as judge), and track everything in LangSmith. I made public links to the traces where you can inspect the runs.
I then used LangChain evaluators (GPT-4 as judge), and track everything in LangSmith. I made public links to the traces where you can inspect the runs.
new_line
new_line
text
I hope you find this helpful, and I am certainly open to feedback, criticisms, or ways to improve.
I hope you find this helpful, and I am certainly open to feedback, criticisms, or ways to improve.
new_line
new_line
text
Cheers: 
Cheers: 
new_line
new_line
text
You can find the notebook here: 
You can find the notebook here: 
link
https://colab.research.google.com/drive/1RHzg0FD46kKbiGfTdZw9Fo-DqWzajuoi?usp=sharing
https://colab.research.google.com/drive/1RHzg0FD46kKbiGfTdZw9Fo-DqWzajuoi?usp=sharing
https://huggingface.co/google/gemma-7b-it is super good!

I wasn't convinced at first, but after vibe-checking it...I'm quite impressed.

I've got a notebook here, which is kind of a framework for vibe-checking LLMs.

In this notebook, I take Gemma for a spin on a variety of prompts:
•  [nonsensical tokens](https://huggingface.co/datasets/harpreetsahota/diverse-token-sampler]
• [conversation where I try to get some PII)(https://huggingface.co/datasets/harpreetsahota/red-team-prompts-questions)
• [summarization ability](https://huggingface.co/datasets/lighteval/summarization)
• [instruction following](https://huggingface.co/datasets/harpreetsahota/Instruction-Following-Evaluation-for-Large-Language-Models]
• [chain of thought reasoning](https://huggingface.co/datasets/ssbuild/alaca_chain-of-thought)

I then used LangChain evaluators (GPT-4 as judge), and track everything in LangSmith. I made public links to the traces where you can inspect the runs.

I hope you find this helpful, and I am certainly open to feedback, criticisms, or ways to improve.

Cheers: 

You can find the notebook here: https://colab.research.google.com/drive/1RHzg0FD46kKbiGfTdZw9Fo-DqWzajuoi?usp=sharing
❤️
clem mvaloatto lunarflu osanseviero diwank harpreetsahota danielhanchen eramax arjunpatel adamelliotfields sbarman25 pansapiens