Leandro von Werra's picture

Leandro von Werra

lvwerra

·

https://github.com/lvwerra

AI & ML interests

NLP and RL

Recent Activity

reacted to lewtun's post with 🔥 2 days ago

This paper (https://huggingface.co/papers/2412.18925) has a really interesting recipe for inducing o1-like behaviour in Llama models: * Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting. * Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases) * Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1 * Use the resulting data for SFT & RL * Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement. Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!

liked a Space 10 days ago

data-agents/jupyter-agent

updated a Space 13 days ago

data-agents/jupyter-agent

View all activity

Articles

LeMaterial: an open source initiative to accelerate materials discovery and research

CinePile 2.0 - making stronger datasets with adversarial refinement

FineVideo: behind the scenes

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

A failed experiment: Infini-Attention, and why we should keep trying?

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Welcome Llama 3 - Meta's new open LLM

StarCoder2 and The Stack v2

Constitutional AI with Open LLMs

Preference Tuning LLMs with Direct Preference Optimization Methods

Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face

The N Implementation Details of RLHF with PPO

Finetune Stable Diffusion Models with DDPO via TRL

Spread Your Wings: Falcon 180B is here

Code Llama: Llama 2 learns to code

Fine-tune Llama 2 with DPO

The Falcon has landed in the Hugging Face ecosystem

Creating a Coding Assistant with StarCoder

StarCoder: A State-of-the-Art LLM for Code

StackLLaMA: A hands-on guide to train LLaMA with RLHF

Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Evaluating Language Model Bias with 🤗 Evaluate

Announcing Evaluation on the Hub

Organizations

lvwerra's activity

liked a Space 10 days ago

Jupyter Agent

liked a Space 16 days ago

Scaling test-time compute

liked a dataset 27 days ago

microsoft/RedStone

Updated 27 days ago • 551 • 27

liked a dataset about 1 month ago

ylecun/mnist

Viewer • Updated Aug 8, 2024 • 70k • 35.3k • 143

liked a Space about 2 months ago

Scaling FineWeb to 1000+ languages: Step 1: finding signal in 100s of evaluation tasks

liked a model 2 months ago

HuggingFaceTB/SmolLM2-1.7B-Instruct

Text Generation • Updated 28 days ago • 98.1k • 449

liked a Space 2 months ago

CinePileLeaderboard

Video-LLM evaluations on CinePile's evaluation split.

liked a Space 3 months ago

TxT360: Trillion Extracted Text

liked a dataset 4 months ago

HuggingFaceFV/finevideo

Viewer • Updated 16 days ago • 39.5k • 6k • 284

liked 2 models 5 months ago

meta-llama/Llama-3.1-8B-Instruct

Text Generation • Updated Sep 25, 2024 • 4.76M • • 3.35k

google/gemma-2-2b

Text Generation • Updated Aug 7, 2024 • 234k • 467

liked a Space 6 months ago

BigCodeBench Leaderboard

liked a Space 7 months ago

FineWeb: decanting the web for the finest text data at scale

liked a dataset 8 months ago

tomg-group-umd/cinepile

Viewer • Updated Oct 23, 2024 • 608k • 234 • 76

liked 2 models 8 months ago

bigcode/starcoder2-15b-instruct-v0.1

Text Generation • Updated Nov 3, 2024 • 796 • 100

bigcode/starcoder2-15b

Text Generation • Updated Jun 5, 2024 • 29.9k • • 575

liked a dataset 8 months ago

HuggingFaceFW/fineweb

Viewer • Updated 3 days ago • 47.5B • 213k • 1.79k

liked a model 9 months ago

mistral-community/Mixtral-8x22B-v0.1

Text Generation • Updated Jul 1, 2024 • 2.6k • 673

liked a Space 10 months ago

StarCoder2 Search

liked a dataset 10 months ago

bigcode/the-stack-v2

Viewer • Updated Apr 23, 2024 • 5.45B • 14.9k • 303