Xiaonan Nie

According to our database, Xiaonan Nie authored at least 20 papers between 2021 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

FlashFlex: Accommodating Large Language Model Training over Heterogeneous Environment.

Ran Yan, Youhe Jiang, Wangcheng Tao, Xiaonan Nie, Bin Cui, Binhang Yuan

CoRR, 2024

DataSculpt: Crafting Data Landscapes for LLM Post-Training through Multi-objective Partitioning.

Keer Lu, Zheng Liang, Xiaonan Nie, Da Pan, Shusen Zhang, Keshi Zhao, Weipeng Chen, Zenan Zhou, Guosheng Dong, Wentao Zhang, Bin Cui

CoRR, 2024

BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline.

Guosheng Dong, Da Pan, Yiding Sun, Shusen Zhang, Zheng Liang, Xin Wu, Yanjun Shen, Fan Yang, Haoze Sun, Tianpeng Li, Mingan Lin, Jianhua Xu, Yufan Zhang, Xiaonan Nie, Lei Su, Bingning Wang, Wentao Zhang, Jiaxin Mao, Zenan Zhou, Weipeng Chen

CoRR, 2024

PQCache: Product Quantization-based KVCache for Long Context LLM Inference.

Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, Bin Cui

CoRR, 2024

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs.

Pinxue Zhao, Hailin Zhang, Fangcheng Fu, Xiaonan Nie, Qibin Liu, Fang Yang, Yuanbo Peng, Dian Jiao, Shuaipeng Li, Jinbao Xue, Yangyu Tao, Bin Cui

CoRR, 2024

Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge.

Bin Xiao, Chunan Shi, Xiaonan Nie, Fan Yang, Xiangwei Deng, Lei Su, Weipeng Chen, Bin Cui

CoRR, 2024

Enabling Parallelism Hot Switching for Efficient Training of Large Language Models.

Hao Ge, Fangcheng Fu, Haoyang Li, Xuanyu Wang, Sheng Lin, Yujie Wang, Xiaonan Nie, Hailin Zhang, Xupeng Miao, Bin Cui

Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, 2024

2023

Hetu: a highly efficient automatic parallel distributed deep learning system.

Xupeng Miao, Xiaonan Nie, Hailin Zhang, Tong Zhao, Bin Cui

Sci. China Inf. Sci., January, 2023

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent.

Xiaonan Nie, Yi Liu, Fangcheng Fu, Jinbao Xue, Dian Jiao, Xupeng Miao, Yangyu Tao, Bin Cui

Proc. VLDB Endow., 2023

FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement.

Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, Bin Cui

Proc. ACM Manag. Data, 2023

Improving Automatic Parallel Training via Balanced Memory Workload Optimization.

Yujie Wang, Youhe Jiang, Xupeng Miao, Fangcheng Fu, Xiaonan Nie, Bin Cui

CoRR, 2023

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning.

Youhe Jiang, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Bin Cui

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, 2023

2022

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism.

Xupeng Miao, Yujie Wang, Youhe Jiang, Chunan Shi, Xiaonan Nie, Hailin Zhang, Bin Cui

Proc. VLDB Endow., 2022

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning.

Youhe Jiang, Xupeng Miao, Xiaonan Nie, Bin Cui

CoRR, 2022

HetuMoE: An Efficient Trillion-scale Mixture-of-Expert Distributed Training System.

Xiaonan Nie, Pinxue Zhao, Xupeng Miao, Tong Zhao, Bin Cui

CoRR, 2022

HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training.

Xupeng Miao, Yining Shi, Hailin Zhang, Xin Zhang, Xiaonan Nie, Zhi Yang, Bin Cui

Proceedings of the SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12, 2022

TSPLIT: Fine-grained GPU Memory Management for Efficient DNN Training via Tensor Splitting.

Xiaonan Nie, Xupeng Miao, Zhi Yang, Bin Cui

Proceedings of the 38th IEEE International Conference on Data Engineering, 2022

2021

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework.

Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, Bin Cui

Proc. VLDB Endow., 2021

Dense-to-Sparse Gate for Mixture-of-Experts.

Xiaonan Nie, Shijie Cao, Xupeng Miao, Lingxiao Ma, Jilong Xue, Youshan Miao, Zichao Yang, Zhi Yang, Bin Cui

CoRR, 2021

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce.

Xupeng Miao, Xiaonan Nie, Yingxia Shao, Zhi Yang, Jiawei Jiang, Lingxiao Ma, Bin Cui

Proceedings of the SIGMOD '21: International Conference on Management of Data, 2021