NeurIPS.cc/2024/Conference
How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers
Xin Lu Yanyan Zhao Bing Qin Liangyu Huo Qing Yang Dongliang Xu