SmolLM - blazingly fast and remarkably powerful
•
294
Anton Lozhkov
anton-l
AI & ML interests
Generative Models, Distributed Training, Photo and Video Enhancement
Recent Activity
new activity
2 days ago
HuggingFaceTB/finemath:Update README.md
new activity
3 days ago
HuggingFaceTB/finemath:Upload re.zip
new activity
3 days ago
HuggingFaceTB/finemath:Create test
Articles
Organizations
Posts
1
Post
2030
Introducing 📐𝐅𝐢𝐧𝐞𝐌𝐚𝐭𝐡: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
🛠️ carefully extracting math data from Common Crawl;
🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! 🚀
We’re also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
🛠️ carefully extracting math data from Common Crawl;
🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! 🚀
We’re also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
spaces
3
models
70
anton-l/bert_snowflake_regression
Updated
anton-l/ddpm-butterflies-128
Updated
•
433
•
8
anton-l/ddpm-butterflies-128-test
Updated
anton-l/dream-sna2
Text-to-Image
•
Updated
•
2
anton-l/dream-sna
Text-to-Image
•
Updated
•
3
anton-l/ddpm-ema-flowers-64-2gpu
Updated
•
2
anton-l/ddpm-ema-flowers-64-testt
Updated
•
7
anton-l/wav2vec2-base-superb-sv
Audio Classification
•
Updated
•
1.29k
•
3
anton-l/ddpm-ema-flowers-64-test
Updated
•
2
anton-l/gpt-j-tiny-random
Text Generation
•
Updated
•
991
•
1
datasets
24
anton-l/superb_dummy
Updated
•
1.76k
anton-l/superb
Updated
•
50
•
1
anton-l/superb_demo
Updated
•
1.82k
•
1
anton-l/fw_edu_200k_3_clusters
Viewer
•
Updated
•
100k
•
84
anton-l/dclm_edu_200k_clusters
Viewer
•
Updated
•
100k
•
69
anton-l/stanford_prompts_1M_rag
Viewer
•
Updated
•
50k
•
6
•
2
anton-l/math_fw_sample
Viewer
•
Updated
•
11
anton-l/wiki-embed-mxbai-embed-large-v1
Viewer
•
Updated
•
19.4M
•
51
anton-l/wiki-chunked-mxbai-embed-large-v1
Viewer
•
Updated
•
2.64M
•
18
anton-l/wiki_embeddings
Viewer
•
Updated
•
58.7k
•
35