Zhenyu Zhang

According to our database, Zhenyu Zhang authored at least 36 papers between 2021 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients.

Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang

CoRR, 2024

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.

Zhenyu Zhang, Ajay Jaiswal, Lu Yin, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang

CoRR, 2024

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding.

Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang

CoRR, 2024

Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache.

Zhenyu Zhang, Shiwei Liu, Runjin Chen, Bhavya Kailkhura, Beidi Chen, Atlas Wang

Proceedings of the Seventh Annual Conference on Machine Learning and Systems, 2024

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection.

Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once.

Zhangheng Li, Shiwei Liu, Tianlong Chen, Ajay Kumar Jaiswal, Zhenyu Zhang, Dilin Wang, Raghuraman Krishnamoorthi, Shiyu Chang, Zhangyang Wang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference.

Harry Dong, Xinyu Yang, Zhenyu Zhang, Zhangyang Wang, Yuejie Chi, Beidi Chen

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity.

Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Kumar Jaiswal, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, Shiwei Liu

Proceedings of the Forty-first International Conference on Machine Learning, 2024

CaM: Cache Merging for Memory-efficient LLMs Inference.

Yuxin Zhang, Yuxuan Du, Gen Luo, Yunshan Zhong, Zhenyu Zhang, Shiwei Liu, Rongrong Ji

Proceedings of the Forty-first International Conference on Machine Learning, 2024

JoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention.

Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Shaolei Du

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy.

Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention.

Zhen Tan, Tianlong Chen, Zhenyu Zhang, Huan Liu

Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

2023

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity.

Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, Shiwei Liu

CoRR, 2023

H₂O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen

CoRR, 2023

QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits.

Tianlong Chen, Zhenyu Zhang, Hanrui Wang, Jiaqi Gu, Zirui Li, David Z. Pan, Frederic T. Chong, Song Han, Zhangyang Wang

Proceedings of the IEEE International Conference on Quantum Computing and Engineering, 2023

H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Are Large Kernels Better Teachers than Transformers for ConvNets?

Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu

Proceedings of the International Conference on Machine Learning, 2023

Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?

Ruisi Cai, Zhenyu Zhang, Zhangyang Wang

Proceedings of the International Conference on Machine Learning, 2023

Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!

Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, Ajay Kumar Jaiswal, Zhangyang Wang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers.

Tianlong Chen, Zhenyu Zhang, Ajay Kumar Jaiswal, Shiwei Liu, Zhangyang Wang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Accelerable Lottery Tickets with the Mixed-Precision Quantization.

Zhangheng Li, Yu Gong, Zhenyu Zhang, Xingyun Xue, Tianlong Chen, Yi Liang, Bo Yuan, Zhangyang Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

2022

Can You Win Everything with A Lottery Ticket?

Tianlong Chen, Zhenyu Zhang, Jun Wu, Randy Huang, Sijia Liu, Shiyu Chang, Zhangyang Wang

Trans. Mach. Learn. Res., 2022

QuanGCN: Noise-Adaptive Training for Robust Quantum Graph Convolutional Networks.

Kaixiong Zhou, Zhenyu Zhang, Shengyuan Chen, Tianlong Chen, Xiao Huang, Zhangyang Wang, Xia Hu

CoRR, 2022

Sparse Winning Tickets are Data-Efficient Image Recognizers.

Mukund Varma T., Xuxi Chen, Zhenyu Zhang, Tianlong Chen, Subhashini Venugopalan, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Randomized Channel Shuffling: Minimal-Overhead Backdoor Attack Detection without Clean Datasets.

Ruisi Cai, Zhenyu Zhang, Tianlong Chen, Xiaohan Chen, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness.

Tianlong Chen, Huan Zhang, Zhenyu Zhang, Shiyu Chang, Sijia Liu, Pin-Yu Chen, Zhangyang Wang

Proceedings of the International Conference on Machine Learning, 2022

Data-Efficient Double-Win Lottery Tickets from Robust Pre-training.

Tianlong Chen, Zhenyu Zhang, Sijia Liu, Yang Zhang, Shiyu Chang, Zhangyang Wang

Proceedings of the International Conference on Machine Learning, 2022

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training.

Tianlong Chen, Zhenyu Zhang, Pengjun Wang, Santosh Balachandra, Haoyu Ma, Zehao Wang, Zhangyang Wang

Proceedings of the Tenth International Conference on Learning Representations, 2022

Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free.

Tianlong Chen, Zhenyu Zhang, Yihua Zhang, Shiyu Chang, Sijia Liu, Zhangyang Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy.

Tianlong Chen, Zhenyu Zhang, Yu Cheng, Ahmed Awadallah, Zhangyang Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

2021

You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership.

Xuxi Chen, Tianlong Chen, Zhenyu Zhang, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Efficient Lottery Ticket Finding: Less Data is More.

Zhenyu Zhang, Xuxi Chen, Tianlong Chen, Zhangyang Wang

Proceedings of the 38th International Conference on Machine Learning, 2021

GANs Can Play Lottery Tickets Too.

Xuxi Chen, Zhenyu Zhang, Yongduo Sui, Tianlong Chen

Proceedings of the 9th International Conference on Learning Representations, 2021

Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning.

Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, Zhangyang Wang

Proceedings of the 9th International Conference on Learning Representations, 2021

Robust Overfitting may be mitigated by properly learned smoothening.

Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, Zhangyang Wang

Proceedings of the 9th International Conference on Learning Representations, 2021

"BNN - BN = ?": Training Binary Neural Networks Without Batch Normalization.

Tianlong Chen, Zhenyu Zhang, Xu Ouyang, Zechun Liu, Zhiqiang Shen, Zhangyang Wang

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2021