Varun Gumma

According to our database, Varun Gumma authored at least 17 papers between 2021 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Contamination Report for Multilingual Benchmarks.

Sanchit Ahuja, Varun Gumma, Sunayana Sitaram

CoRR, 2024

HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings.

Varun Gumma, Anandhita Raghunath, Mohit Jain, Sunayana Sitaram

CoRR, 2024

On the Interchangeability of Positional Embeddings in Multilingual Neural Machine Translation Models.

Varun Gumma, Pranjal A. Chitale, Kalika Bali

CoRR, 2024

Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios.

Millicent Ochieng, Varun Gumma, Sunayana Sitaram, Jindong Wang, Vishrav Chaudhary, Keshet Ronen, Kalika Bali, Jacki O'Neill

CoRR, 2024

MunTTS: A Text-to-Speech System for Mundari.

Varun Gumma, Rishav Hada, Aditya Yadavalli, Pamir Gogoi, Ishani Mondal, Vivek Seshadri, Kalika Bali

CoRR, 2024

An Empirical Analysis of In-context Learning Abilities of LLMs for MT.

Pranjal A. Chitale, Jay P. Gala, Varun Gumma, Mitesh M. Khapra, Raj Dabre

CoRR, 2024

METAL: Towards Multilingual Meta-Evaluation.

Rishav Hada, Varun Gumma, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram

Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024, 2024

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks.

Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram

Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024

Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology.

Rishav Hada, Safiya Husain, Varun Gumma, Harshita Diddee, Aditya Yadavalli, Agrima Seth, Nidhi Kulkarni, Ujwal Gadiraju, Aditya Vashistha, Vivek Seshadri, Kalika Bali

Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, 2024

PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data.

Ishaan Watts, Varun Gumma, Aditya Yadavalli, Vivek Seshadri, Manohar Swaminathan, Sunayana Sitaram

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

MAFIA: Multi-Adapter Fused Inclusive Language Models.

Prachi Jain, Ashutosh Sathe, Varun Gumma, Kabir Ahuja, Sunayana Sitaram

Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?

Rishav Hada, Varun Gumma, Adrian de Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram

Proceedings of the Findings of the Association for Computational Linguistics: EACL 2024, 2024

2023

IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages.

Jay P. Gala, Pranjal A. Chitale, Raghavan AK, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar M., Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, Anoop Kunchukuttan

Trans. Mach. Learn. Res., 2023

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks.

Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Maxamed Axmed, Kalika Bali, Sunayana Sitaram

CoRR, 2023

An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models.

Varun Gumma, Raj Dabre, Pratyush Kumar

Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023

2022

PAMMELA: Policy Administration Methodology using Machine Learning.

Varun Gumma, Barsha Mitra, Soumyadeep Dey, Pratik Shashikantbhai Patel, Sourabh Suman, Saptarshi Das, Jaideep Vaidya

Proceedings of the 19th International Conference on Security and Cryptography, 2022

2021

PAMMELA: Policy Administration Methodology using Machine Learning.

Varun Gumma, Barsha Mitra, Soumyadeep Dey, Pratik Shashikantbhai Patel, Sourabh Suman, Saptarshi Das

CoRR, 2021