Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
41
Zeyu hUANG
zeroyhuang
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
about 9 hours ago
Demons in the Detail: On Implementing Load Balancing Loss for Training
Specialized Mixture-of-Expert Models
upvoted
a
paper
5 months ago
Layerwise Recurrent Router for Mixture-of-Experts
authored
a paper
5 months ago
Layerwise Recurrent Router for Mixture-of-Experts
Organizations
None yet