Yanghua Peng

According to our database, Yanghua Peng authored at least 24 papers between 2017 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

HybridFlow: A Flexible and Efficient RLHF Framework.

Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, Chuan Wu

CoRR, 2024

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation.

Weiqi Feng, Yangrui Chen, Shaoyu Wang, Yanghua Peng, Haibin Lin, Minlan Yu

CoRR, 2024

ByteCheckpoint: A Unified Checkpointing System for LLM Development.

Borui Wan, Mingji Han, Yiyao Sheng, Zhichao Lai, Mofan Zhang, Junda Zhang, Yanghua Peng, Haibin Lin, Xin Liu, Chuan Wu

CoRR, 2024

LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization.

Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin, Chuan Wu

CoRR, 2024

POSTER: LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization.

Juntao Zhao, Borui Wan, Chuan Wu, Yanghua Peng, Haibin Lin

Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, 2024

MegaScale: Scaling Large Language Model Training to More Than 10, 000 GPUs.

Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, Xin Liu

Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation, 2024

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices.

Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin, Yibo Zhu, Chuan Wu

Proceedings of the IEEE International Parallel and Distributed Processing Symposium, 2024

CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs.

Hanpeng Hu, Junwei Su, Juntao Zhao, Yanghua Peng, Yibo Zhu, Haibin Lin, Chuan Wu

Proceedings of the Nineteenth European Conference on Computer Systems, 2024

2023

Deep Learning-Based Job Placement in Distributed Machine Learning Clusters With Heterogeneous Workloads.

Yixin Bao, Yanghua Peng, Chuan Wu

IEEE/ACM Trans. Netw., April, 2023

SP-GNN: Learning structure and position information from graphs.

Yangrui Chen, Jiaxuan You, Jun He, Yuan Lin, Yanghua Peng, Chuan Wu, Yibo Zhu

Neural Networks, April, 2023

BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing.

Tianfeng Liu, Yangrui Chen, Dan Li, Chuan Wu, Yibo Zhu, Jun He, Yanghua Peng, Hongzheng Chen, Hongzhi Chen, Chuanxiong Guo

Proceedings of the 20th USENIX Symposium on Networked Systems Design and Implementation, 2023

2022

dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training.

Hanpeng Hu, Chenyu Jiang, Yuchen Zhong, Yanghua Peng, Chuan Wu, Yibo Zhu, Haibin Lin, Chuanxiong Guo

CoRR, 2022

Multi-resource interleaving for deep learning training.

Yihao Zhao, Yuanqiang Liu, Yanghua Peng, Yibo Zhu, Xuanzhe Liu, Xin Jin

Proceedings of the SIGCOMM '22: ACM SIGCOMM 2022 Conference, Amsterdam, The Netherlands, August 22, 2022

SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training.

Yangrui Chen, Cong Xie, Meng Ma, Juncheng Gu, Yanghua Peng, Haibin Lin, Chuan Wu, Yibo Zhu

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

dPRO: A Generic Performance Diagnosis and Optimization Toolkit for Expediting Distributed DNN Training.

Hanpeng Hu, Chenyu Jiang, Yuchen Zhong, Yanghua Peng, Chuan Wu, Yibo Zhu, Haibin Lin, Chuanxiong Guo

Proceedings of the Fifth Conference on Machine Learning and Systems, 2022

2021

DL2: A Deep Learning-Driven Scheduler for Deep Learning Clusters.

Yanghua Peng, Yixin Bao, Yangrui Chen, Chuan Wu, Chen Meng, Wei Lin

IEEE Trans. Parallel Distributed Syst., 2021

2020

Preemptive All-reduce Scheduling for Expediting Distributed DNN Training.

Yixin Bao, Yanghua Peng, Yangrui Chen, Chuan Wu

Proceedings of the 39th IEEE Conference on Computer Communications, 2020

Elastic parameter server load distribution in deep learning clusters.

Yangrui Chen, Yanghua Peng, Yixin Bao, Chuan Wu, Yibo Zhu, Chuanxiong Guo

Proceedings of the SoCC '20: ACM Symposium on Cloud Computing, 2020

2019

A generic communication scheduler for distributed DNN training acceleration.

Yanghua Peng, Yibo Zhu, Yangrui Chen, Yixin Bao, Bairen Yi, Chang Lan, Chuan Wu, Chuanxiong Guo

Proceedings of the 27th ACM Symposium on Operating Systems Principles, 2019

Deep Learning-based Job Placement in Distributed Machine Learning Clusters.

Yixin Bao, Yanghua Peng, Chuan Wu

Proceedings of the 2019 IEEE Conference on Computer Communications, 2019

2018

Online Job Scheduling in Distributed Machine Learning Clusters.

Yixin Bao, Yanghua Peng, Chuan Wu, Zongpeng Li

Proceedings of the 2018 IEEE Conference on Computer Communications, 2018

Optimus: an efficient dynamic resource scheduler for deep learning clusters.

Yanghua Peng, Yixin Bao, Yangrui Chen, Chuan Wu, Chuanxiong Guo

Proceedings of the Thirteenth EuroSys Conference, 2018

2017

Dynamic Scaling of Virtualized, Distributed Service Chains: A Case Study of IMS.

Jingpu Duan, Chuan Wu, Franck Le, Alex X. Liu, Yanghua Peng

IEEE J. Sel. Areas Commun., 2017

deTector: a Topology-aware Monitoring System for Data Center Networks.

Yanghua Peng, Ji Yang, Chuan Wu, Chuanxiong Guo, Chengchen Hu, Zongpeng Li

Proceedings of the 2017 USENIX Annual Technical Conference, 2017