Sidak Pal Singh

According to our database, Sidak Pal Singh authored at least 24 papers between 2016 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis.

Weronika Ormaniec, Felix Dangel, Sidak Pal Singh

CoRR, 2024

Local vs Global continual learning.

Giulia Lanzillotta, Sidak Pal Singh, Benjamin F. Grewe, Thomas Hofmann

CoRR, 2024

Landscaping Linear Mode Connectivity.

Sidak Pal Singh, Linara Adilova, Michael Kamp, Asja Fischer, Bernhard Schölkopf, Thomas Hofmann

CoRR, 2024

Hallmarks of Optimization Trajectories in Neural Networks and LLMs: The Lengths, Bends, and Dead Ends.

Sidak Pal Singh, Bobby He, Thomas Hofmann, Bernhard Schölkopf

CoRR, 2024

Towards Meta-Pruning via Optimal Transport.

Alexander Theus, Olin Geimer, Friedrich Wicke, Thomas Hofmann, Sotiris Anagnostidis, Sidak Pal Singh

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Some Fundamental Aspects about Lipschitz Continuity of Neural Networks.

Grigory Khromov, Sidak Pal Singh

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Transformer Fusion with Optimal Transport.

Moritz Imfeld, Jacopo Graldi, Marco Giordano, Thomas Hofmann, Sotiris Anagnostidis, Sidak Pal Singh

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract).

Danilo Dordevic, Vukasin Bozic, Joseph Thommes, Daniele Coppola, Sidak Pal Singh

Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

2023

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers.

Vukasin Bozic, Danilo Dordevic, Daniele Coppola, Joseph Thommes, Sidak Pal Singh

CoRR, 2023

Towards guarantees for parameter isolation in continual learning.

Giulia Lanzillotta, Sidak Pal Singh, Benjamin F. Grewe, Thomas Hofmann

CoRR, 2023

On the curvature of the loss landscape.

Alison Pouplin, Hrittik Roy, Sidak Pal Singh, Georgios Arvanitidis

CoRR, 2023

Some Fundamental Aspects about Lipschitz Continuity of Neural Network Functions.

Grigory Khromov, Sidak Pal Singh

CoRR, 2023

The Hessian perspective into the Nature of Convolutional Neural Networks.

Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf

Proceedings of the International Conference on Machine Learning, 2023

2022

Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse.

Lorenzo Noci, Sotiris Anagnostidis, Luca Biggio, Antonio Orvieto, Sidak Pal Singh, Aurélien Lucchi

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Phenomenology of Double Descent in Finite-Width Neural Networks.

Sidak Pal Singh, Aurélien Lucchi, Thomas Hofmann, Bernhard Schölkopf

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

Analytic Insights into Structure and Rank of Neural Network Hessian Maps.

Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

2020

WoodFisher: Efficient second-order approximations for model compression.

Sidak Pal Singh, Dan Alistarh

CoRR, 2020

Model Fusion via Optimal Transport.

Sidak Pal Singh, Martin Jaggi

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

WoodFisher: Efficient Second-Order Approximation for Neural Network Compression.

Sidak Pal Singh, Dan Alistarh

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations.

Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, Martin Jaggi

Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 2020

2019

GLOSS: Generative Latent Optimization of Sentence Representations.

Sidak Pal Singh, Angela Fan, Michael Auli

CoRR, 2019

2018

Wasserstein is all you need.

Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, Martin Jaggi

CoRR, 2018

2017

RaaS and Hierarchical Aggregation Revisited.

Rohit Ranchal, Sidak Pal Singh, Pelin Angin, Ajay Mohindra, Hui Lei, Bharat K. Bhargava

Proceedings of the 2017 IEEE International Conference on Web Services, 2017

2016

SL - FII: Syntactic and Lexical Constraints with Frequency based Iterative Improvement for Disease Mention Recognition in News Headlines.

Sidak Pal Singh, Sopan Khosla, Sajal Rustagi, Manisha Patel, Dhaval Patel

Proceedings of the Workshop on Advances in Bioinformatics and Artificial Intelligence: Bridging the Gap co-located with 25th International Joint Conference on Artificial Intelligence (IJCAI 2016), 2016