Sashank J. Reddi

According to our database, Sashank J. Reddi authored at least 75 papers between 2010 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

On the Role of Depth and Looping for In-Context Learning with Task Diversity.

Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar

CoRR, 2024

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs.

Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar

CoRR, 2024

On the Inductive Bias of Stacking Towards Improving Reasoning.

Nikunj Saunshi, Stefani Karp, Shankar Krishnan, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar

CoRR, 2024

Efficient Document Ranking with Learnable Late Interactions.

Ziwei Ji, Himanshu Jain, Andreas Veit, Sashank J. Reddi, Sadeep Jayasumana, Ankit Singh Rawat, Aditya Krishna Menon, Felix Yu, Sanjiv Kumar

CoRR, 2024

Landscape-Aware Growing: The Power of a Little LAG.

Stefani Karp, Nikunj Saunshi, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar

CoRR, 2024

Efficient Stagewise Pretraining via Progressive Subnetworks.

Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar

CoRR, 2024

Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?

Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Simplicity Bias via Global Convergence of Sharpness Minimization.

Khashayar Gatmiry, Zhiyuan Li, Sashank J. Reddi, Stefanie Jegelka

Proceedings of the Forty-first International Conference on Machine Learning, 2024

2023

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization.

Khashayar Gatmiry, Zhiyuan Li, Ching-Yao Chuang, Sashank J. Reddi, Tengyu Ma, Stefanie Jegelka

CoRR, 2023

Depth Dependence of μP Learning Rates in ReLU MLPs.

Samy Jelassi, Boris Hanin, Ziwei Ji, Sashank J. Reddi, Srinadh Bhojanapalli, Sanjiv Kumar

CoRR, 2023

What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models.

Khashayar Gatmiry, Zhiyuan Li, Tengyu Ma, Sashank J. Reddi, Stefanie Jegelka, Ching-Yao Chuang

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Efficient Training of Language Models using Few-Shot Learning.

Sashank J. Reddi, Sobhan Miryoosefi, Stefani Karp, Shankar Krishnan, Satyen Kale, Seungyeon Kim, Sanjiv Kumar

Proceedings of the International Conference on Machine Learning, 2023

The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers.

Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Differentially Private Adaptive Optimization with Delayed Preconditioners.

Tian Li, Manzil Zaheer, Ken Liu, Sashank J. Reddi, Hugh Brendan McMahan, Virginia Smith

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

On the Algorithmic Stability and Generalization of Adaptive Optimization Methods.

Han Nguyen, Hai Pham, Sashank J. Reddi, Barnabás Póczos

CoRR, 2022

Large Models are Parsimonious Learners: Activation Sparsity in Trained Transformers.

Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar

CoRR, 2022

FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients.

Jianyu Wang, Hang Qi, Ankit Singh Rawat, Sashank J. Reddi, Sagar Waghmare, Felix X. Yu, Gauri Joshi

CoRR, 2022

In defense of dual-encoders for neural ranking.

Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the International Conference on Machine Learning, 2022

Private Adaptive Optimization with Side information.

Tian Li, Manzil Zaheer, Sashank J. Reddi, Virginia Smith

Proceedings of the International Conference on Machine Learning, 2022

Robust Training of Neural Networks Using Scale Invariant Architectures.

Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the International Conference on Machine Learning, 2022

2021

A Field Guide to Federated Optimization.

Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Agüera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas N. Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horváth, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecný, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtárik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake E. Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu

CoRR, 2021

Distilling Double Descent.

Andrew Cotter, Aditya Krishna Menon, Harikrishna Narasimhan, Ankit Singh Rawat, Sashank J. Reddi, Yichen Zhou

CoRR, 2021

Efficient Training of Retrieval Models using Negative Cache.

Erik Lindgren, Sashank J. Reddi, Ruiqi Guo, Sanjiv Kumar

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Breaking the centralized barrier for cross-device federated learning.

Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Federated Composite Optimization.

Honglin Yuan, Manzil Zaheer, Sashank J. Reddi

Proceedings of the 38th International Conference on Machine Learning, 2021

Disentangling Sampling and Labeling Bias for Learning in Large-output Spaces.

Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the 38th International Conference on Machine Learning, 2021

A statistical perspective on distillation.

Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar

Proceedings of the 38th International Conference on Machine Learning, 2021

Adaptive Federated Optimization.

Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecný, Sanjiv Kumar, Hugh Brendan McMahan

Proceedings of the 9th International Conference on Learning Representations, 2021

RankDistil: Knowledge Distillation for Ranking.

Sashank J. Reddi, Rama Kumar Pasumarthi, Aditya Krishna Menon, Ankit Singh Rawat, Felix X. Yu, Seungyeon Kim, Andreas Veit, Sanjiv Kumar

Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 2021

2020

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning.

Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh

CoRR, 2020

Why distillation helps: a statistical perspective.

Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar

CoRR, 2020

Doubly-stochastic mining for heterogeneous retrieval.

Ankit Singh Rawat, Aditya Krishna Menon, Andreas Veit, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar

CoRR, 2020

Adaptive Sampling Distributed Stochastic Variance Reduced Gradient for Heterogeneous Distributed Datasets.

Ilqar Ramazanli, Han Nguyen, Hai Pham, Sashank J. Reddi, Barnabás Póczos

CoRR, 2020

Why are Adaptive Methods Good for Attention Models?

Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, Suvrit Sra

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers.

Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning.

Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh

Proceedings of the 37th International Conference on Machine Learning, 2020

Low-Rank Bottleneck in Multi-head Attention Models.

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the 37th International Conference on Machine Learning, 2020

Are Transformers universal approximators of sequence-to-sequence functions?

Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the 8th International Conference on Learning Representations, 2020

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes.

Yang You, Jing Li, Sashank J. Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, Cho-Jui Hsieh

Proceedings of the 8th International Conference on Learning Representations, 2020

Learning to Learn by Zeroth-Order Oracle.

Yangjun Ruan, Yuanhao Xiong, Sashank J. Reddi, Sanjiv Kumar, Cho-Jui Hsieh

Proceedings of the 8th International Conference on Learning Representations, 2020

Can gradient clipping mitigate label noise?

Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Why ADAM Beats SGD for Attention Models.

Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, Suvrit Sra

CoRR, 2019

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning.

Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh

CoRR, 2019

AdaCliP: Adaptive Clipping for Private SGD.

Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar

CoRR, 2019

Breaking the Glass Ceiling for Embedding-Based Classifiers for Large Output Spaces.

Chuan Guo, Ali Mousavi, Xiang Wu, Daniel Niels Holtmann-Rice, Satyen Kale, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Escaping Saddle Points with Adaptive Gradient Methods.

Matthew Staib, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar, Suvrit Sra

Proceedings of the 36th International Conference on Machine Learning, 2019

Stochastic Negative Mining for Learning with Large Output Spaces.

Sashank J. Reddi, Satyen Kale, Felix X. Yu, Daniel Niels Holtmann-Rice, Jiecao Chen, Sanjiv Kumar

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

2018

Adaptive Methods for Nonconvex Optimization.

Manzil Zaheer, Sashank J. Reddi, Devendra Singh Sachan, Satyen Kale, Sanjiv Kumar

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

On the Convergence of Adam and Beyond.

Sashank J. Reddi, Satyen Kale, Sanjiv Kumar

Proceedings of the 6th International Conference on Learning Representations, 2018

A Generic Approach for Escaping Saddle points.

Sashank J. Reddi, Manzil Zaheer, Suvrit Sra, Barnabás Póczos, Francis R. Bach, Ruslan Salakhutdinov, Alexander J. Smola

Proceedings of the International Conference on Artificial Intelligence and Statistics, 2018

2017

New Optimization Methods for Modern Machine Learning.

Sashank J. Reddi

PhD thesis, 2017

2016

Fast stochastic optimization on Riemannian manifolds.

Hongyi Zhang, Sashank J. Reddi, Suvrit Sra

CoRR, 2016

Fast Stochastic Methods for Nonsmooth Nonconvex Optimization.

Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

CoRR, 2016

Fast Incremental Method for Nonconvex Optimization.

Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

CoRR, 2016

AIDE: Fast and Communication Efficient Distributed Optimization.

Sashank J. Reddi, Jakub Konecný, Peter Richtárik, Barnabás Póczos, Alexander J. Smola

CoRR, 2016

Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds.

Hongyi Zhang, Sashank J. Reddi, Suvrit Sra

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization.

Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Variance Reduction in Stochastic Gradient Langevin Dynamics.

Kumar Avinava Dubey, Sashank J. Reddi, Sinead A. Williamson, Barnabás Póczos, Alexander J. Smola, Eric P. Xing

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Stochastic Variance Reduction for Nonconvex Optimization.

Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

Proceedings of the 33nd International Conference on Machine Learning, 2016

Fast incremental method for smooth nonconvex optimization.

Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

Proceedings of the 55th IEEE Conference on Decision and Control, 2016

Stochastic Frank-Wolfe methods for nonconvex optimization.

Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

Proceedings of the 54th Annual Allerton Conference on Communication, 2016

2015

Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing.

Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, Larry A. Wasserman

CoRR, 2015

Communication Efficient Coresets for Empirical Loss Minimization.

Sashank J. Reddi, Barnabás Póczos, Alexander J. Smola

Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, 2015

Large-scale randomized-coordinate descent methods with non-separable linear constraints.

Sashank J. Reddi, Ahmed Hefny, Carlton Downey, Avinava Dubey, Suvrit Sra

Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, 2015

On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants.

Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

On the High Dimensional Power of a Linear-Time Two Sample Test under Mean-shift Alternatives.

Sashank J. Reddi, Aaditya Ramdas, Barnabás Póczos, Aarti Singh, Larry A. Wasserman

Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 2015

Doubly Robust Covariate Shift Correction.

Sashank Jakkam Reddi, Barnabás Póczos, Alexander J. Smola

Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015

On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions.

Aaditya Ramdas, Sashank Jakkam Reddi, Barnabás Póczos, Aarti Singh, Larry A. Wasserman

Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015

2014

Kernel MMD, the Median Heuristic and Distance Correlation in High Dimensions.

Sashank J. Reddi, Aaditya Ramdas, Barnabás Póczos, Aarti Singh, Larry A. Wasserman

CoRR, 2014

On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives.

Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, Larry A. Wasserman

CoRR, 2014

k-NN Regression on Functional Data with Incomplete Observations.

Sashank J. Reddi, Barnabás Póczos

Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, 2014

2013

Scale Invariant Conditional Dependence Measures.

Sashank J. Reddi, Barnabás Póczos

Proceedings of the 30th International Conference on Machine Learning, 2013

2012

Incentive Decision Processes.

Sashank Jakkam Reddi, Emma Brunskill

Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, 2012

A Maximum Likelihood Approach For Selecting Sets of Alternatives.

Ariel D. Procaccia, Sashank Jakkam Reddi, Nisarg Shah

Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, 2012

2010

MAP estimation in Binary MRFs via Bipartite Multi-cuts.

Sashank Jakkam Reddi, Sunita Sarawagi, Sundar Vishwanathan

Proceedings of the Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Proceedings of a meeting held 6-9 December 2010, 2010