2503.05029v2
CC BY-NC-SA 4.0
Université de Montréal、Mila – Quebec AI Institute、Capital One、University of Chicago、Concordia University
MoE 的持續預訓練：你的路由器有多穩健？
Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien、Charles-Étienne Joseph、Zain Sarwar、Ashwinee Panda、Anirban Das、Shi-Xiong Zhang、Steph...
MoELLM持續預訓練RoutingLoad BalancingCatastrophic ForgettingReplay
Liang-Hsun Huang 許願