Omid Saremi

According to our database, Omid Saremi authored at least 13 papers between 2015 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

The Slingshot Effect: A Late-Stage Optimization Anomaly in Adaptive Gradient Methods.

Vimal Thilak, Etai Littwin, Shuangfei Zhai, Omid Saremi, Roni Paiss, Joshua M. Susskind

Trans. Mach. Learn. Res., 2024

How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks.

Etai Littwin, Omid Saremi, Madhu Advani, Vimal Thilak, Preetum Nakkiran, Chen Huang, Joshua M. Susskind

CoRR, 2024

How Far Can Transformers Reason? The Locality Barrier and Inductive Scratchpad.

Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Colin Sandon, Omid Saremi

CoRR, 2024

What Algorithms can Transformers Learn? A Study in Length Generalization.

Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Joshua M. Susskind, Samy Bengio, Preetum Nakkiran

Proceedings of the Twelfth International Conference on Learning Representations, 2024

LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures.

Vimal Thilak, Chen Huang, Omid Saremi, Laurent Dinh, Hanlin Goh, Preetum Nakkiran, Joshua M. Susskind, Etai Littwin

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Vanishing Gradients in Reinforcement Finetuning of Language Models.

Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua M. Susskind, Etai Littwin

Proceedings of the Twelfth International Conference on Learning Representations, 2024

When can transformers reason with abstract symbols?

Enric Boix-Adserà, Omid Saremi, Emmanuel Abbe, Samy Bengio, Etai Littwin, Joshua M. Susskind

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Adaptivity and Modularity for Efficient Generalization Over Task Complexity.

Samira Abnar, Omid Saremi, Laurent Dinh, Shantel Wilson, Miguel Ángel Bautista, Chen Huang, Vimal Thilak, Etai Littwin, Jiatao Gu, Josh M. Susskind, Samy Bengio

CoRR, 2023

2022

The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon.

Vimal Thilak, Etai Littwin, Shuangfei Zhai, Omid Saremi, Roni Paiss, Joshua M. Susskind

CoRR, 2022

2021

Implicit Greedy Rank Learning in Autoencoders via Overparameterized Linear Networks.

Shih-Yu Sun, Vimal Thilak, Etai Littwin, Omid Saremi, Joshua M. Susskind

CoRR, 2021

Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottlenecks.

Etai Littwin, Omid Saremi, Shuangfei Zhai, Vimal Thilak, Hanlin Goh, Joshua M. Susskind, Greg Yang

CoRR, 2021

2015

An Improved Continuous-Action Extended Classifier Systems for Function Approximation.

Omid Saremi, Masoud Shariat Panahi, Amin Sabzehzar

Proceedings of the Complex Adaptive Systems 2015 Conference, San Jose, 2015

An Improved eXtended Classifier System for the Real-time-input Real-time-output (XCSRR) Stability Control of a Biped Robot.

Amin Sabzehzar, W. L. Shan, M. Shariat Panahi, Omid Saremi

Proceedings of the Complex Adaptive Systems 2015 Conference, San Jose, 2015