Soufiane Hayou

According to our database, Soufiane Hayou authored at least 26 papers between 2018 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory.

Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Réda Alami, Ahmed Alzubaidi, Hakim Hacid

CoRR, 2024

Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map.

Yoonsoo Nam, Chris Mingard, Seok Hyeong Lee, Soufiane Hayou, Ard A. Louis

CoRR, 2024

The Impact of Initialization on LoRA Finetuning Dynamics.

Soufiane Hayou, Nikhil Ghosh, Bin Yu

CoRR, 2024

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse.

Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, Mérouane Debbah

CoRR, 2024

LoRA+: Efficient Low Rank Adaptation of Large Models.

Soufiane Hayou, Nikhil Ghosh, Bin Yu

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks.

Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Leave-one-out Distinguishability in Machine Learning.

Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

On the infinite-depth limit of finite-width neural networks.

Soufiane Hayou

Trans. Mach. Learn. Res., 2023

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms.

Fadhel Ayed, Soufiane Hayou

Trans. Mach. Learn. Res., 2023

Commutative Width and Depth Scaling in Deep Neural Networks.

Soufiane Hayou

CoRR, 2023

On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks.

Soufiane Hayou

CoRR, 2023

Width and Depth Limits Commute in Residual Networks.

Soufiane Hayou, Greg Yang

Proceedings of the International Conference on Machine Learning, 2023

2022

From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality.

Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li

Trans. Mach. Learn. Res., 2022

Connecting Optimization and Generalization via Gradient Flow Path Length.

Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li

CoRR, 2022

Feature Learning and Signal Propagation in Deep Neural Networks.

Yizhang Lou, Chris E. Mingard, Soufiane Hayou

Proceedings of the International Conference on Machine Learning, 2022

2021

Wide deep neural networks.

Soufiane Hayou

PhD thesis, 2021

Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning.

Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite

CoRR, 2021

The Equilibrium Hypothesis: Rethinking implicit regularization in Deep Neural Networks.

Yizhang Lou, Chris Mingard, Soufiane Hayou

CoRR, 2021

Regularization in ResNet with Stochastic Depth.

Soufiane Hayou, Fadhel Ayed

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Robust Pruning at Initialization.

Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh

Proceedings of the 9th International Conference on Learning Representations, 2021

The Curse of Depth in Kernel Regime.

Soufiane Hayou, Arnaud Doucet, Judith Rousseau

Proceedings of the I (Still) Can't Believe It's Not Better! Workshop at NeurIPS 2021, 2021

Stable ResNet.

Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau

Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 2021

2020

Pruning untrained neural networks: Principles and Analysis.

Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh

CoRR, 2020

2019

Training Dynamics of Deep Networks using Stochastic Gradient Descent via Neural Tangent Kernel.

Soufiane Hayou, Arnaud Doucet, Judith Rousseau

CoRR, 2019

On the Impact of the Activation function on Deep Neural Networks Training.

Soufiane Hayou, Arnaud Doucet, Judith Rousseau

Proceedings of the 36th International Conference on Machine Learning, 2019

2018

On the Selection of Initialization and Activation Function for Deep Neural Networks.

Soufiane Hayou, Arnaud Doucet, Judith Rousseau

CoRR, 2018