Kaifeng Lyu

According to our database, Kaifeng Lyu authored at least 23 papers between 2018 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks.

Binghui Li, Zhixuan Pan, Kaifeng Lyu, Jian Li

CoRR, 2024

AI-Assisted Generation of Difficult Math Questions.

Vedant Shah, Dingli Yu, Kaifeng Lyu, Simon Park, Nan Rosemary Ke, Michael Mozer, Yoshua Bengio, Sanjeev Arora, Anirudh Goyal

CoRR, 2024

Safety Alignment Should Be Made More Than Just a Few Tokens Deep.

Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter Henderson

CoRR, 2024

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates.

Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, Sanjeev Arora

CoRR, 2024

RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval.

Kaiyue Wen, Xingyu Dang, Kaifeng Lyu

CoRR, 2024

Efficient Stagewise Pretraining via Progressive Subnetworks.

Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar

CoRR, 2024

DistillSpec: Improving Speculative Decoding via Knowledge Distillation.

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal

Proceedings of the Twelfth International Conference on Learning Representations, 2024

The Marginal Value of Momentum for Small Learning Rate SGD.

Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking.

Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee, Wei Hu

Proceedings of the Twelfth International Conference on Learning Representations, 2024

A Quadratic Synchronization Rule for Distributed Deep Learning.

Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing.

Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee

Proceedings of the International Conference on Machine Learning, 2023

Why (and When) does Local SGD Generalize Better than SGD?

Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

On the SDEs and Scaling Rules for Adaptive Gradient Algorithms.

Sadhika Malladi, Kaifeng Lyu, Abhishek Panigrahi, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction.

Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound.

Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

2021

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias.

Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning.

Zhiyuan Li, Yuping Luo, Kaifeng Lyu

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate.

Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Gradient Descent Maximizes the Margin of Homogeneous Neural Networks.

Kaifeng Lyu, Jian Li

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Fine-grained Complexity Meets IP = PSPACE.

Lijie Chen, Shafi Goldwasser, Kaifeng Lyu, Guy N. Rothblum, Aviad Rubinstein

Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, 2019

Theoretical Analysis of Auto Rate-Tuning by Batch Normalization.

Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Single-Source Bottleneck Path Algorithm Faster than Sorting for Sparse Graphs.

Ran Duan, Kaifeng Lyu, Hongxun Wu, Yuanhang Xie

CoRR, 2018

Single-Source Bottleneck Path Algorithm Faster than Sorting for Sparse Graphs.

Ran Duan, Kaifeng Lyu, Yuanhang Xie

Proceedings of the 45th International Colloquium on Automata, Languages, and Programming, 2018