Amirkeivan Mohtashami

According to our database, Amirkeivan Mohtashami authored at least 15 papers between 2021 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs.

Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman

CoRR, 2024

DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging.

Matteo Pagliardini, Amirkeivan Mohtashami, François Fleuret, Martin Jaggi

CoRR, 2024

2023

The splay-list: a distribution-adaptive concurrent skip-list.

Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami

Distributed Comput., September, 2023

Social Learning: Towards Collaborative Learning with Large Language Models.

Amirkeivan Mohtashami, Florian Hartmann, Sian Gooding, Lukas Zilka, Matthew Sharifi, Blaise Agüera y Arcas

CoRR, 2023

MEDITRON-70B: Scaling Medical Pretraining for Large Language Models.

Zeming Chen, Alejandro Hernández-Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexandre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, Antoine Bosselut

CoRR, 2023

CoTFormer: More Tokens With Attention Make Up For Less Depth.

Amirkeivan Mohtashami, Matteo Pagliardini, Martin Jaggi

CoRR, 2023

Landmark Attention: Random-Access Infinite Context Length for Transformers.

Amirkeivan Mohtashami, Martin Jaggi

CoRR, 2023

Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models.

Amirkeivan Mohtashami, Mauro Verzetti, Paul K. Rubenstein

CoRR, 2023

Random-Access Infinite Context Length for Transformers.

Amirkeivan Mohtashami, Martin Jaggi

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Special Properties of Gradient Descent with Large Learning Rates.

Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich

Proceedings of the International Conference on Machine Learning, 2023

2022

On Avoiding Local Minima Using Gradient Descent With Large Learning Rates.

Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich

CoRR, 2022

Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods.

Amirkeivan Mohtashami, Sebastian U. Stich, Martin Jaggi

CoRR, 2022

Masked Training of Neural Networks with Partial Gradients.

Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich

Proceedings of the International Conference on Artificial Intelligence and Statistics, 2022

2021

Simultaneous Training of Partially Masked Neural Networks.

Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich

CoRR, 2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates.

Sebastian U. Stich, Amirkeivan Mohtashami, Martin Jaggi

Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 2021