Ivan Stelmakh

According to our database, Ivan Stelmakh authored at least 19 papers between 2019 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

A Randomized Controlled Trial on Anonymizing Reviewers to Each Other in Peer Review Discussions.

Charvi Rastogi, Xiangchen Song, Zhijing Jin, Ivan Stelmakh, Hal Daumé III, Kun Zhang, Nihar B. Shah

CoRR, 2024

2023

Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experiments.

Alexander Goldberg, Ivan Stelmakh, Kyunghyun Cho, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, Nihar B. Shah

CoRR, 2023

A Gold Standard Dataset for the Reviewer Assignment Problem.

Ivan Stelmakh, John Wieting, Graham Neubig, Nihar B. Shah

CoRR, 2023

2022

How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?

Charvi Rastogi, Ivan Stelmakh, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, Jennifer Wortman Vaughan, Zhenyu Xue, Hal Daumé III, Emma Pierson, Nihar B. Shah

CoRR, 2022

To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online.

Charvi Rastogi, Ivan Stelmakh, Xinwei Shen, Marina Meila, Federico Echenique, Shuchi Chawla, Nihar B. Shah

CoRR, 2022

Cite-seeing and Reviewing: A Study on Citation Bias in Peer Review.

Ivan Stelmakh, Charvi Rastogi, Ryan Liu, Shuchi Chawla, Federico Echenique, Nihar B. Shah

CoRR, 2022

No Rose for MLE: Inadmissibility of MLE for Evaluation Aggregation Under Levels of Expertise.

Charvi Rastogi, Ivan Stelmakh, Nihar B. Shah, Sivaraman Balakrishnan

Proceedings of the IEEE International Symposium on Information Theory, 2022

ASQA: Factoid Questions Meet Long-Form Answers.

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, Ming-Wei Chang

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

2021

Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review.

Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Proc. ACM Hum. Comput. Interact., 2021

PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review.

Ivan Stelmakh, Nihar B. Shah, Aarti Singh

J. Mach. Learn. Res., 2021

Vox Populi, Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription.

Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov

CoRR, 2021

CrowdSpeech and Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription.

Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov

Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, 2021

VLDB 2021 Crowd Science Challenge on Aggregating Crowdsourced Audio Transcriptions.

Dmitry Ustalov, Nikita Pavlichenko, Ivan Stelmakh, Dmitriy Kuznetsov

Proceedings of the 2nd Crowd Science Workshop: Trust, 2021

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences.

Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment.

Ivan Stelmakh, Nihar B. Shah, Aarti Singh

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

Towards Fair, Equitable, and Efficient Peer Review.

Ivan Stelmakh

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

Debiasing Evaluations That Are Biased by Evaluations.

Jingyan Wang, Ivan Stelmakh, Yuting Wei, Nihar B. Shah

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

2020

A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions.

Ivan Stelmakh, Charvi Rastogi, Nihar B. Shah, Aarti Singh, Hal Daumé III

CoRR, 2020

2019

On Testing for Biases in Peer Review.

Ivan Stelmakh, Nihar B. Shah, Aarti Singh

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019