Haibin Lin

According to our database, Haibin Lin authored at least 35 papers between 2013 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Minder: Faulty Machine Detection for Large-scale Distributed Model Training.

Yangtao Deng, Xiang Shi, Zhuo Jiang, Xingjian Zhang, Lei Zhang, Zhang Zhang, Bo Li, Zuquan Song, Hang Zhu, Gaohong Liu, Fuliang Li, Shuguang Wang, Haibin Lin, Jianxi Ye, Minlan Yu

CoRR, 2024

SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training.

Jinda Jia, Cong Xie, Hanlin Lu, Daoce Wang, Hao Feng, Chengming Zhang, Baixi Sun, Haibin Lin, Zhi Zhang, Xin Liu, Dingwen Tao

CoRR, 2024

HybridFlow: A Flexible and Efficient RLHF Framework.

Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, Chuan Wu

CoRR, 2024

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation.

Weiqi Feng, Yangrui Chen, Shaoyu Wang, Yanghua Peng, Haibin Lin, Minlan Yu

CoRR, 2024

ByteCheckpoint: A Unified Checkpointing System for LLM Development.

Borui Wan, Mingji Han, Yiyao Sheng, Zhichao Lai, Mofan Zhang, Junda Zhang, Yanghua Peng, Haibin Lin, Xin Liu, Chuan Wu

CoRR, 2024

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion.

Li-Wen Chang, Wenlei Bao, Qi Hou, Chengquan Jiang, Ningxin Zheng, Yinmin Zhong, Xuanrun Zhang, Zuquan Song, Ziheng Jiang, Haibin Lin, Xin Jin, Xin Liu

CoRR, 2024

LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization.

Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin, Chuan Wu

CoRR, 2024

POSTER: LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization.

Juntao Zhao, Borui Wan, Chuan Wu, Yanghua Peng, Haibin Lin

Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, 2024

MegaScale: Scaling Large Language Model Training to More Than 10, 000 GPUs.

Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, Xin Liu

Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation, 2024

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices.

Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin, Yibo Zhu, Chuan Wu

Proceedings of the IEEE International Parallel and Distributed Processing Symposium, 2024

LEMON: Lossless model expansion.

Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, Hongxia Yang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs.

Hanpeng Hu, Junwei Su, Juntao Zhao, Yanghua Peng, Yibo Zhu, Haibin Lin, Chuan Wu

Proceedings of the Nineteenth European Conference on Computer Systems, 2024

2023

Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies.

Zhuang Wang, Haibin Lin, Yibo Zhu, T. S. Eugene Ng

Proceedings of the Eighteenth European Conference on Computer Systems, 2023

2022

Espresso: Revisiting Gradient Compression from the System Perspective.

Zhuang Wang, Haibin Lin, Yibo Zhu, T. S. Eugene Ng

CoRR, 2022

dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training.

Hanpeng Hu, Chenyu Jiang, Yuchen Zhong, Yanghua Peng, Chuan Wu, Yibo Zhu, Haibin Lin, Chuanxiong Guo

CoRR, 2022

SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training.

Yangrui Chen, Cong Xie, Meng Ma, Juncheng Gu, Yanghua Peng, Haibin Lin, Chuan Wu, Yibo Zhu

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

dPRO: A Generic Performance Diagnosis and Optimization Toolkit for Expediting Distributed DNN Training.

Hanpeng Hu, Chenyu Jiang, Yuchen Zhong, Yanghua Peng, Chuan Wu, Yibo Zhu, Haibin Lin, Chuanxiong Guo

Proceedings of the Fifth Conference on Machine Learning and Systems, 2022

ResNeSt: Split-Attention Networks.

Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R. Manmatha, Mu Li, Alexander J. Smola

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022

The World of 5G - Volume 5: Intelligent Medicine

Wenhua Huang, Haibin Lin

WorldScientific, ISBN: 9789811244216, 2022

2021

Compressed Communication for Distributed Training: Adaptive Methods and System.

Yuchen Zhong, Cong Xie, Shuai Zheng, Haibin Lin

CoRR, 2021

2020

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing.

Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, Yi Zhu

J. Mach. Learn. Res., 2020

Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes.

Shuai Zheng, Haibin Lin, Sheng Zha, Mu Li

CoRR, 2020

Is Network the Bottleneck of Distributed Training?

Zhen Zhang, Chaokun Chang, Haibin Lin, Yida Wang, Raman Arora, Xin Jin

Proceedings of the 2020 Workshop on Network Meets AI & ML, 2020

CSER: Communication-efficient SGD with Error Reset.

Cong Xie, Shuai Zheng, Oluwasanmi Koyejo, Indranil Gupta, Mu Li, Haibin Lin

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Temporal-Contextual Recommendation in Real-Time.

Yifei Ma, Balakrishnan (Murali) Narayanaswamy, Haibin Lin, Hao Ding

Proceedings of the KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2020

2019

Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates.

Cong Xie, Oluwasanmi Koyejo, Indranil Gupta, Haibin Lin

CoRR, 2019

Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs.

Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander J. Smola, Zheng Zhang

CoRR, 2019

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing.

Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng

CoRR, 2019

Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources.

Haibin Lin, Hang Zhang, Yifei Ma, Tong He, Zhi Zhang, Sheng Zha, Mu Li

CoRR, 2019

Just-in-Time Dynamic-Batching.

Sheng Zha, Ziheng Jiang, Haibin Lin, Zhi Zhang

CoRR, 2019

Dive into Deep Learning for Natural Language Processing.

Haibin Lin, Xingjian Shi, Leonard Lausen, Aston Zhang, He He, Sheng Zha, Alexander J. Smola

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

2017

Self-Driving Database Management Systems.

Andrew Pavlo, Gustavo Angulo, Joy Arulraj, Haibin Lin, Jiexi Lin, Lin Ma, Prashanth Menon, Todd C. Mowry, Matthew Perron, Ian Quah, Siddharth Santurkar, Anthony Tomasic, Skye Toor, Dana Van Aken, Ziqi Wang, Yingjun Wu, Ran Xian, Tieying Zhang

Proceedings of the 8th Biennial Conference on Innovative Data Systems Research, 2017

2013

Salience-based feature preserving resizing for 3D models.

Yongwei Miao, Haibin Lin

Proceedings of the SIGGRAPH Asia 2013, 2013

3D reconstruction of complex geometric solids from 2D line drawings.

Yongwei Miao, Haibin Lin

Proceedings of the SIGGRAPH Asia 2013, 2013

Visual Saliency Guided Global and Local Resizing for 3D Models.

Yongwei Miao, Haibin Lin

Proceedings of the 2013 International Conference on Computer-Aided Design and Computer Graphics, 2013