Leandro von Werra's picture

Leandro von Werra

lvwerra

·

https://github.com/lvwerra

AI & ML interests

NLP and RL

Recent Activity

reacted to lewtun's post with 🔥 2 days ago

This paper (https://huggingface.co/papers/2412.18925) has a really interesting recipe for inducing o1-like behaviour in Llama models: * Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting. * Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases) * Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1 * Use the resulting data for SFT & RL * Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement. Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!

liked a Space 10 days ago

data-agents/jupyter-agent

updated a Space 13 days ago

data-agents/jupyter-agent

View all activity

Articles

LeMaterial: an open source initiative to accelerate materials discovery and research

CinePile 2.0 - making stronger datasets with adversarial refinement

FineVideo: behind the scenes

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

A failed experiment: Infini-Attention, and why we should keep trying?

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Welcome Llama 3 - Meta's new open LLM

StarCoder2 and The Stack v2

Constitutional AI with Open LLMs

Preference Tuning LLMs with Direct Preference Optimization Methods

Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face

The N Implementation Details of RLHF with PPO

Finetune Stable Diffusion Models with DDPO via TRL

Spread Your Wings: Falcon 180B is here

Code Llama: Llama 2 learns to code

Fine-tune Llama 2 with DPO

The Falcon has landed in the Hugging Face ecosystem

Creating a Coding Assistant with StarCoder

StarCoder: A State-of-the-Art LLM for Code

StackLLaMA: A hands-on guide to train LLaMA with RLHF

Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Evaluating Language Model Bias with 🤗 Evaluate

Announcing Evaluation on the Hub

Organizations

lvwerra's activity

commented a paper 2 months ago

SelfCodeAlign: Self-Alignment for Code Generation

Paper • 2410.24198 • Published Oct 31, 2024 • 23 •

New activity in bigcode/starcoder 3 months ago

Adding `safetensors` variant of this model

#112 opened 9 months ago by

New activity in bigcode/starcoderbase-7b 4 months ago

Adding `safetensors` variant of this model

#2 opened 9 months ago by

New activity in HuggingFaceTB/SmolLM-1.7B-Instruct 5 months ago

v0.2 release deleted all files

#13 opened 5 months ago by

New activity in bigcode/README 7 months ago

Fix format & Add BigCodeBench

#10 opened 7 months ago by

New activity in bigcode/in-the-stack 9 months ago

re-launch the space to solve the build error

#16 opened 10 months ago by

Update app.py

#19 opened 9 months ago by

Update app.py

#18 opened 9 months ago by

New activity in bigcode/starcoder 10 months ago

Update README.md

#110 opened 10 months ago by

New activity in bigcode/the-stack-v2 10 months ago

🚩 Report: Legal issue(s)

#11 opened 10 months ago by

New activity in bigcode/starcoder2-3b 10 months ago

add dropout

#6 opened 10 months ago by

New activity in bigcode/starcoder2-7b 10 months ago

add dropout

#7 opened 10 months ago by

New activity in bigcode/starcoder2-15b 10 months ago

add dropout

#13 opened 10 months ago by

commented a paper 10 months ago

StarCoder 2 and The Stack v2: The Next Generation

Paper • 2402.19173 • Published Feb 29, 2024 • 136 •

New activity in bigcode/starcoder2-3b 10 months ago

Add architectures field to config.json

#2 opened 10 months ago by

New activity in bigcode/README 10 months ago

add SC2 and stack v2

#6 opened 10 months ago by

New activity in bigcode/starcoder2-15b 10 months ago

NVIDIA framework and contribution updates.

#3 opened 10 months ago by

New activity in bigcode/the-stack-v2 10 months ago

Update README.md

#2 opened 10 months ago by

New activity in bigcode/starcoder2-15b 10 months ago

Update README.md

#2 opened 10 months ago by

New activity in bigcode/starcoder2-7b 10 months ago

Update README.md

#1 opened 10 months ago by