讀紙 ReadPaper

由 Claude Code Max 翻譯的 AI 論文繁體中文版 · 101 篇

篩選標籤: Scaling Law 4 篇 × 清除
2608.27370v2 CC BY-SA 4.0 Alibaba / Qwen Team

Puro-2B:窮實驗室在 5090 美元內、用 RTX 5090 訓出的 Qwen2-1.5B

Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

Kairong Luo、Jiarui Cui、Yaorui Yin、Shengqi Chen 等(清華大學、鵬城實驗室;通訊作者 Kaifeng Lyu、Wenguang Chen)

PretrainingLow-Cost TrainingRTX 5090FP8MuonOpen RecipeScaling Law開源模型
Liang-Hsun Huang 許願
2404.06395v3 CC BY 4.0

MiniCPM:以可擴展的訓練策略揭示小型語言模型的潛力

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

Shengding Hu、Yuge Tu、Xu Han、Chaoqun He、Ganqu Cui、Xiang Long、Zhi Zheng、Yewei Fang、Yuxiang Huang 等(清華大...

Small Language ModelWSD SchedulerScaling LawPre-trainingMoELong Context開源模型
Liang-Hsun Huang 許願
2305.16264v5 arXiv License Hugging Face、Harvard University、University of Turku、Cornell University

資料受限下的語言模型 Scaling

Scaling Data-Constrained Language Models

Niklas Muennighoff、Alexander M. Rush、Boaz Barak、Teven Le Scao、Aleksandra Piktus、Nouamane Tazi、Sampo ...

LLMScaling LawPre-trainingDataChinchillaRepeated DataNeurIPS
Liang-Hsun Huang 許願
gh-MoonshotAI-Moonlight MIT License Moonshot AI、UCLA

Muon 可以擴展到 LLM 訓練

Muon is Scalable for LLM Training

Jingyuan Liu、Jianlin Su、Xingcheng Yao、Zhejun Jiang、Guokun Lai、Yulun Du、Yidao Qin、Weixin Xu、Enzhe Lu、...

LLMMuonOptimizerMoEScaling Law開源模型
Liang-Hsun Huang 許願