pre-training - a jshin49 Collection

jshin49 's Collections

multi-lingual llms

mixture-of-experts

alignment-learning

pre-training

updated Apr 19, 2024

Pre-training Small Base LMs with Fewer Tokens

Paper • 2404.08634 • Published Apr 12, 2024 • 35
Ziya2: Data-centric Learning is All LLMs Need

Paper • 2311.03301 • Published Nov 6, 2023 • 16
How to Train Data-Efficient LLMs

Paper • 2402.09668 • Published Feb 15, 2024 • 40
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

Paper • 2404.06395 • Published Apr 9, 2024 • 22