Depen Morwani

According to our database, Depen Morwani authored at least 12 papers between 2020 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2025

Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants.

Depen Morwani, Nikhil Vyas, Hanlin Zhang, Sham M. Kakade

CoRR, February, 2025

2024

How Does Critical Batch Size Scale in Pre-training?

Hanlin Zhang, Depen Morwani, Nikhil Vyas, Jingfeng Wu, Difan Zou, Udaya Ghai, Dean P. Foster, Sham M. Kakade

CoRR, 2024

SOAP: Improving and Stabilizing Shampoo using Adam.

Nikhil Vyas, Depen Morwani, Rosie Zhao, Itai Shapira, David Brandfonbrener, Lucas Janson, Sham M. Kakade

CoRR, 2024

Deconstructing What Makes a Good Optimizer for Language Models.

Rosie Zhao, Depen Morwani, David Brandfonbrener, Nikhil Vyas, Sham M. Kakade

CoRR, 2024

A New Perspective on Shampoo's Preconditioner.

Depen Morwani, Itai Shapira, Nikhil Vyas, Eran Malach, Sham M. Kakade, Lucas Janson

CoRR, 2024

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning.

Nikhil Vyas, Depen Morwani, Rosie Zhao, Gal Kaplun, Sham M. Kakade, Boaz Barak

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Feature emergence via margin maximization: case studies in algebraic tasks.

Depen Morwani, Benjamin L. Edelman, Costin-Andrei Oncescu, Rosie Zhao, Sham M. Kakade

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Simplicity Bias in 1-Hidden Layer Neural Networks.

Depen Morwani, Jatin Batra, Prateek Jain, Praneeth Netrapalli

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales.

Nikhil Vyas, Alexander B. Atanasov, Blake Bordelon, Depen Morwani, Sabarish Sainathan, Cengiz Pehlevan

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2022

Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets.

Depen Morwani, Harish G. Ramaswamy

Proceedings of the International Conference on Algorithmic Learning Theory, 29 March, 2022

2020

Using noise resilience for ranking generalization of deep neural networks.

Depen Morwani, Rahul Vashisht, Harish G. Ramaswamy

CoRR, 2020

Inductive Bias of Gradient Descent for Exponentially Weight Normalized Smooth Homogeneous Neural Nets.

Depen Morwani, Harish G. Ramaswamy

CoRR, 2020