Preetum Nakkiran

According to our database, Preetum Nakkiran authored at least 48 papers between 2014 and 2024.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of three.

2024

A Formal Framework for Understanding Length Generalization in Transformers.

Xinting Huang, Andy Yang, Satwik Bhattamishra, Yash Sarrof, Andreas Krebs, Hattie Zhou, Preetum Nakkiran, Michael Hahn

CoRR, 2024

Classifier-Free Guidance is a Predictor-Corrector.

Arwen Bradley, Preetum Nakkiran

CoRR, 2024

How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks.

Etai Littwin, Omid Saremi, Madhu Advani, Vimal Thilak, Preetum Nakkiran, Chen Huang, Joshua M. Susskind

CoRR, 2024

Step-by-Step Diffusion: An Elementary Tutorial.

Preetum Nakkiran, Arwen Bradley, Hattie Zhou, Madhu Advani

CoRR, 2024

When is Multicalibration Post-Processing Necessary?

Dutch Hansen, Siddartha Devic, Preetum Nakkiran, Vatsal Sharan

CoRR, 2024

Loss Minimization Yields Multicalibration for Large Neural Networks.

Jaroslaw Blasiok, Parikshit Gopalan, Lunjia Hu, Adam Tauman Kalai, Preetum Nakkiran

Proceedings of the 15th Innovations in Theoretical Computer Science Conference, 2024

What Algorithms can Transformers Learn? A Study in Length Generalization.

Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Joshua M. Susskind, Samy Bengio, Preetum Nakkiran

Proceedings of the Twelfth International Conference on Learning Representations, 2024

LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures.

Vimal Thilak, Chen Huang, Omid Saremi, Laurent Dinh, Hanlin Goh, Preetum Nakkiran, Joshua M. Susskind, Etai Littwin

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Vanishing Gradients in Reinforcement Finetuning of Language Models.

Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua M. Susskind, Etai Littwin

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing.

Jaroslaw Blasiok, Preetum Nakkiran

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Empirical Limitations of the NTK for Understanding Scaling Laws in Deep Learning.

Nikhil Vyas, Yamini Bansal, Preetum Nakkiran

Trans. Mach. Learn. Res., 2023

Perspectives on the State and Future of Deep Learning - 2023.

Micah Goldblum, Anima Anandkumar, Richard G. Baraniuk, Tom Goldstein, Kyunghyun Cho, Zachary C. Lipton, Melanie Mitchell, Preetum Nakkiran, Max Welling, Andrew Gordon Wilson

CoRR, 2023

A Unifying Theory of Distance from Calibration.

Jaroslaw Blasiok, Parikshit Gopalan, Lunjia Hu, Preetum Nakkiran

Proceedings of the 55th Annual ACM Symposium on Theory of Computing, 2023

When Does Optimizing a Proper Loss Yield Calibration?

Jaroslaw Blasiok, Parikshit Gopalan, Lunjia Hu, Preetum Nakkiran

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Deconstructing Distributions: A Pointwise Framework of Learning.

Gal Kaplun, Nikhil Ghosh, Saurabh Garg, Boaz Barak, Preetum Nakkiran

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

Near-Optimal NP-Hardness of Approximating Max k-CSP_R.

Pasin Manurangsi, Preetum Nakkiran, Luca Trevisan

Adv. Math. Commun., 2022

APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representations.

Elan Rosenfeld, Preetum Nakkiran, Hadi Pouransari, Oncel Tuzel, Fartash Faghri

CoRR, 2022

The Calibration Generalization Gap.

A. Michael Carrell, Neil Mallinar, James Lucas, Preetum Nakkiran

CoRR, 2022

Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting.

Neil Mallinar, James B. Simon, Amirhesam Abedsoltan, Parthe Pandit, Mikhail Belkin, Preetum Nakkiran

CoRR, 2022

Limitations of the NTK for Understanding Generalization in Deep Learning.

Nikhil Vyas, Yamini Bansal, Preetum Nakkiran

CoRR, 2022

What You See is What You Get: Distributional Generalization for Algorithm Design in Deep Learning.

Bogdan Kulynych, Yao-Yuan Yang, Yaodong Yu, Jaroslaw Blasiok, Preetum Nakkiran

CoRR, 2022

Limitations of Neural Collapse for Understanding Generalization in Deep Learning.

Like Hui, Mikhail Belkin, Preetum Nakkiran

CoRR, 2022

Benign, Tempered, or Catastrophic: Toward a Refined Taxonomy of Overfitting.

Neil Mallinar, James B. Simon, Amirhesam Abedsoltan, Parthe Pandit, Misha Belkin, Preetum Nakkiran

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

What You See is What You Get: Principled Deep Learning via Distributional Generalization.

Bogdan Kulynych, Yao-Yuan Yang, Yaodong Yu, Jaroslaw Blasiok, Preetum Nakkiran

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Knowledge Distillation: Bad Models Can Be Good Role Models.

Gal Kaplun, Eran Malach, Preetum Nakkiran, Shai Shalev-Shwartz

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

2021

Turing-Universal Learners with Optimal Scaling Laws.

Preetum Nakkiran

CoRR, 2021

Revisiting Model Stitching to Compare Neural Representations.

Yamini Bansal, Preetum Nakkiran, Boaz Barak

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Optimal Regularization can Mitigate Double Descent.

Preetum Nakkiran, Prayaag Venkat, Sham M. Kakade, Tengyu Ma

Proceedings of the 9th International Conference on Learning Representations, 2021

The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers.

Preetum Nakkiran, Behnam Neyshabur, Hanie Sedghi

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

The Deep Bootstrap: Good Online Learners are Good Offline Generalizers.

Preetum Nakkiran, Behnam Neyshabur, Hanie Sedghi

CoRR, 2020

Distributional Generalization: A New Kind of Generalization.

Preetum Nakkiran, Yamini Bansal

CoRR, 2020

Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems.

Preetum Nakkiran

CoRR, 2020

Deep Double Descent: Where Bigger Models and More Data Hurt.

Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

More Data Can Hurt for Linear Regression: Sample-wise Double Descent.

Preetum Nakkiran

CoRR, 2019

Adversarial Robustness May Be at Odds With Simplicity.

Preetum Nakkiran

CoRR, 2019

SGD on Neural Networks Learns Functions of Increasing Complexity.

Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, Boaz Barak

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Algorithmic Polarization for Hidden Markov Models.

Venkatesan Guruswami, Preetum Nakkiran, Madhu Sudan

Proceedings of the 10th Innovations in Theoretical Computer Science Conference, 2019

Computational Limitations in Robust Classification and Win-Win Results.

Akshay Degwekar, Preetum Nakkiran, Vinod Vaikuntanathan

Proceedings of the Conference on Learning Theory, 2019

Tracking the l₂ Norm with Constant Update Time.

Chi-Ning Chou, Zhixian Lei, Preetum Nakkiran

Proceedings of the Approximation, 2019

2018

General Strong Polarization.

Jaroslaw Blasiok, Venkatesan Guruswami, Preetum Nakkiran, Atri Rudra, Madhu Sudan

Electron. Colloquium Comput. Complex., 2018

The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science.

Preetum Nakkiran, Jaroslaw Blasiok

CoRR, 2018

2017

Predicting Positive and Negative Links with Noisy Queries: Theory & Practice.

Charalampos E. Tsourakakis, Michael Mitzenmacher, Jaroslaw Blasiok, Ben Lawson, Preetum Nakkiran, Vasileios Nakos

CoRR, 2017

2016

Optimal systematic distributed storage codes with fast encoding.

Preetum Nakkiran, K. V. Rashmi, Kannan Ramchandran

Proceedings of the IEEE International Symposium on Information Theory, 2016

Near-Optimal UGC-hardness of Approximating Max k-CSP_R.

Pasin Manurangsi, Preetum Nakkiran, Luca Trevisan

Proceedings of the Approximation, 2016

2015

Compressing deep neural networks using a rank-constrained topology.

Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada

Proceedings of the 16th Annual Conference of the International Speech Communication Association, 2015

Automatic gain control and multi-style training for robust small-footprint keyword spotting with deep neural networks.

Rohit Prabhavalkar, Raziel Alvarez, Carolina Parada, Preetum Nakkiran, Tara N. Sainath

Proceedings of the 2015 IEEE International Conference on Acoustics, 2015

Having Your Cake and Eating It Too: Jointly Optimal Erasure Codes for I/O, Storage, and Network-bandwidth.

K. V. Rashmi, Preetum Nakkiran, Jingyan Wang, Nihar B. Shah, Kannan Ramchandran

Proceedings of the 13th USENIX Conference on File and Storage Technologies, 2015

2014

Fundamental limits on communication for oblivious updates in storage networks.

Preetum Nakkiran, Nihar B. Shah, K. V. Rashmi

Proceedings of the IEEE Global Communications Conference, 2014