讀紙 ReadPaper

由 Claude Code Max 翻譯的 AI 論文繁體中文版 · 101 篇

篩選標籤: Pre-training 5 篇 × 清除
2404.06395v3 CC BY 4.0

MiniCPM:以可擴展的訓練策略揭示小型語言模型的潛力

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

Shengding Hu、Yuge Tu、Xu Han、Chaoqun He、Ganqu Cui、Xiang Long、Zhi Zheng、Yewei Fang、Yuxiang Huang 等(清華大...

Small Language ModelWSD SchedulerScaling LawPre-trainingMoELong Context開源模型
Liang-Hsun Huang 許願
2305.16264v5 arXiv License Hugging Face、Harvard University、University of Turku、Cornell University

資料受限下的語言模型 Scaling

Scaling Data-Constrained Language Models

Niklas Muennighoff、Alexander M. Rush、Boaz Barak、Teven Le Scao、Aleksandra Piktus、Nouamane Tazi、Sampo ...

LLMScaling LawPre-trainingDataChinchillaRepeated DataNeurIPS
Liang-Hsun Huang 許願
2512.07783v1 arXiv License Carnegie Mellon University

論預訓練、中期訓練與強化學習在推理語言模型上的交互作用

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models

Charlie Zhang、Graham Neubig、Xiang Yue(Carnegie Mellon University, Language Technologies Institute)

LLMReinforcement LearningReasoningPre-trainingMid-trainingGRPOProcess Reward
kerg kerg 許願
2509.20186v4 CC BY 4.0

思考增強預訓練 (Thinking Augmented Pre-training)

Thinking Augmented Pre-training

Liang Wang, Nan Yang, Shaohan Huang, Li Dong, Furu Wei

LLMPre-trainingReasoningSynthetic DataData AugmentationMid-training
Teds Lin 許願
2510.25741v4 CC BY-SA 4.0 ByteDance Seed, UC Santa Cruz, Princeton University, Mila - Quebec AI Institute, University of Montreal, Peking University, CMU, University of Pennsylvania, Conscium, University of Manchester, M-A-P

透過循環語言模型擴展潛在推理 / Ouro

Scaling Latent Reasoning via Looped Language Models

Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin,...

LoopLMLatent ReasoningParameter EfficiencyPre-trainingAdaptive ComputationUniversal TransformerLLM開源模型
Teds Lin 許願