2305.10429v4
arXiv License
DoReMi：最佳化資料混合比例加速語言模型預訓練
DoReMi Optimizing Data Mixtures Speeds Up Language Model Pretraining
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V. Le, T...
LLMData MixingPretrainingDomain ReweightingDistributionally Robust Optimization資料選擇語言模型
Liang-Hsun Huang 許願